1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping away profits

Discussion in 'CSO' started by RSS, Oct 1, 2015.

  1. RSS

    RSS New Member Member

    At technology company Graphiq, web-scraping bots were becoming more than just a nuisance. They were impacting its bottom line.

    The company collects and interprets billions of data points from thousands of online data sources and turns them into visual graphs that website visitors can use for free. Scrapers were extracting data from hundreds of millions of these pages and building duplicate sites.

    “We don’t want people to reuse [our data] commercially for free because there is a cost associated with creating that content,” says Ivan Bercovich, vice president of engineering. “It also undermines the value of our content” and steals traffic away from its site. Then there are the operational costs associated with blocking those web-scraping attempts. “We may have months where we block 5% to 6% of all requests,” Bercovich says. “For a site of our volume – about 30 million visitors a month -- that’s a lot of wasted requests.”

    To read this article in full or to leave a comment, please click here

    Continue reading...

Share This Page