Categories
Monetize

My Experience in Choosing a Web Scraping Service

 Recently I decided to outsource a web scraping project to another company. I typed “web scraping service” in Google, chose six services from the first two search result pages and sent the project specifications to all of them to get quotes. Eventually I decided to go another way and did not order the services, but my experience may be useful for others who want to entrust web scraping jobs to third party services.

If you are interested in price comparisons only and not ready to read the whole story just scroll down.

A list of web scraping services I sent my project to:

  1. www.datahen.com – Canadian web scraping service with nice web design
  2. webdata-scraping.com – Indian service by Keval Kothari
  3. www.iwebscraping.com – India based web scraping company (same as www.3idatascraping.com)
  4. scrapinghub.com – A scraping service founded by creators of Scrapy
  5. web-scraper.com – Yet another web scraping service
  6. grepsr.com – A scraping service that we already reviewed two years ago

Sending the request

All the services except scrapinghub.com have quite simple forms for the description of the project requirements. Basically, you just need to give your contact details and a project description in any form. Some of them are pretty (like datahen.com), some of them are more ascetic (like web-scraper.com), but all of them allow you to send your requirements to developers.

Scrapinghub.com has a quite long form, but most of the fields are optional and all the questions are quite natural. If you really know what you need, then it won’t be hard to answer all of them; moreover they rather help you to describe your need in detail.

Note, that in the context of the project I didn’t make a request for a scraper itself. I asked to receive data on a weekly basis only.

Getting responses

Since I sent my request on Sunday it would have been ok not to receive responses the same day, but I got the first response in 3 hrs! It was from web-scraper.com and stated that this project will cost me $250 monthly. Simple and clear. Thank you, Thang!

Right after that, I received the second response. This time it was Keval from webdata-scraping.com. He had some questions regarding the project. Then after two days he wrote me that it would be hard to scrape some of my data with the software he uses, and that he will try to use a custom scraper. After that he disappeared… ((

Then on Monday I received Cost & ETAT details from datahen.com. It looked quite professional and contained not only price, but also time estimation. They were ready to create such a scraper in 3-4 days for $249 and then maintain it for just $65/month.

On the same day I received a quote from iwebscraping.com. It was $60 per week. Everything is fine, but I’d like to mention that it wasn’t the last letter from them. After I replied to them (right after receiving the quote), I received a reminder letter from them every other day for about a week. So be ready for aggressive marketing if you ask them for a quote )).

Finally in two days after requesting a quote I got a response from scrapinghub.com. Paul Tremberth wrote me that they were ready to build a scraper for $1200 and then maintain it for $300/month.

It is interesting that I have never received an answer from grepsr.com! Two years ago it was the first web scraping service we faced on the web, but now they simply ignored my request! Or perhaps they didn’t receive it somehow? Anyway I had no time for investigation.

So what?

Let us put everything together. Out of six web scraping  services I received four quotes with the following prices:

Service Setup fee Monthly fee
web-scraper.com $250
datahen.com $249 $65
iwebscraping.com $240
scrapinghub.com $1200 $300

 

From this table you can see that  scrapinghub.com appears to be the most expensive service among those compared.

EDIT: These $300/month gives you as much support and development needed to fix a 5M multi-site web crawler, for example. If you need a cheaper solution you can use their Autoscraping tool, which is free, and would have costed around $2/month to crawl at my requested rates.

The average cost of monthly scraping is about $250, but from a long term perspective datahen.com may save you money due to their low monthly fee.

That’s it! If I had enough money available it would be interesting to compare all these services in operation and provide you a more complete report, but this is all I have for now.

If you have anything to share about your experience in using similar services, please contribute to this post by commenting on it below. Cheers!

30 replies on “My Experience in Choosing a Web Scraping Service”

Micheal: Will it be possible to share some details on what you were trying to scrape (you can email me if you can see my email through this comment). I have worked quite extensively in the past in this area. I am having some plans about developing scraping tool and offering it as a service. I just wanted to get a feel for what kinds of sites people are trying to scrape from. So, any info you can provide can be helpful to me. In case, you cannot share this info., that’s fine too. Just wanted to check 🙂

Jeff, what do you mean a scraper for my business?
Do you want to offer scraping service based on scraping software that you owns?
OR you want to use a scraper for harvesting web data to be used in your business?

Michael,

Our apologies for not getting back to you on the project from Grepsr. We get a lot of requests, and have a filtering system. Perhaps it got filtered out – either way that is no excuse, we will do better next time.

Thanks!

Hello-

We are in need of scraping services for several different search terms. The company that we have been using in India takes too long to respond to our emails. We have several businesses and need scraping done for all of them. If anyone is looking for ongoing business, please contact me at harbormasterusa@gmail.com….Thanks!…Jeff

Hey, thanks for providing with the names of the good web scraping companies. I’ll mention one more data scraping comapny: Web Parsing (www.web-parsing.com); it provides its clients with the quality and accurate data and it’s affordable too!

This is the Very Good Analysis About the Web Scraping Service.

I get the Good Knowledge about the Web Scraping service.

i think it help me a lot .

Thank you.

Thanks. This is very helpful analysis about several data scrapper. I am now working with a guy in Philippine. He seems pretty exhaustive although the speed is not up to my satisfaction.

Will share more info about my satisfaction after having some more work with him.

Hi, I’m James, Chief Geek at The Data Scraping Group (www.data-scraping.com.au)

We provide enterprise quality scraped data for mission critical applications.

Just about anybody can scrape data these days, but to do so with ultra-high quality, and reliability, day-in and day-out is extremely challenging. In our experience, this can only be achieved with a significant investment in quality and spider management systems.

We are not the cheapest provider but our customers claim that we provide a cost effective, value-for-money service. And judging from our reviews, our customers love us, as we will do whatever it takes to ensure we meet their scraping goals.

I wish you all the best with your projects, and if it does not work out, feel free to give us a call.

Hey, thanks for providing with the names of the good web scraping companies. I’ll mention one more data scraping company: Free Web Scraping Services (freewebscrapingservices.com); it provides its clients with the quality and accurate data and it’s cheap too!

It’s quite a nice comparison but it’s missing out “http://scraping.services” which is relatively new.
However there is a big difference, your evaluated offers all do scraping mainly as a task of “development” and scraping.services is scraping only supported websites.
For now it’s Bing and Google

Hey, thanks for providing with the names of the good web scraping companies. I’ll mention one more data scraping company: Datatornedo (www.datatornedo.com); it provides its clients with the quality and accurate data and it’s cheap too!

I have used multiple services for scraping and I feel scraphero.com is the best I’ve user by far. It is affordable and the data quality is really good.

Hey, thanks for providing with the names of the good web scraping companies. I’ll mention one more data scraping company: DataCrops Softaware Pvt. Ltd. (https://datacrops.com)it provides its clients with the quality and accurate data.
DataCrops is a scalable software platform that intelligently extracts information from multiple websites and complex data sources using a robust self-enhanced technology.

Hello,
thanks for providing the names of the good web scraping companies. I’ll mention one more data scraping company: HIR INFOTECH PVT LTD. we provide high-quality structured data to enhance business outcomes and enable intelligent decision making.

We are a leader in the web scraping – data extraction industry. We provide high-quality, priority delivery on web data scraping, email scraping, product scraping, web searching, contact scraping, business directory extraction, and screen scraping requirements. We provide its clients with quality and accurate data and it’s affordable too!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.