Recently I received a question on insurance leads:
Is this a legal method of acquiring insurance leads [from the web]? Are there any agent testimonials on the efficiency of this type of service?
Legality issue in web scraping
With the matter of legality in web scraping, there should be a clear approach – it depends on the website and its privacy policy. There could be at least 2 cases:
- Public info (prices, inventory info, public offers), i.e. everything that is not protected by copyright and available for scraping.
- The copyright protected info – website Terms of Use or Terms of Service restrictions make copying and therefore web scraping illegal.
So far I have no insurance agent testimonies on the efficiency of any insurance lead scrape service. The web sites I searched [on the insurance leads] have given me the impression that the customer info they gather is highly secured (not viewable). I doubt that any sites are going to expose insurance leads. In most of them the leads are available by paid subscription plans.
If there are any such websites like insurance leads directories (public insurance quotes), we might develop a scraper that consistently grabs fresh or new info for further analysis. It does save the agent’s time for re-searching, re-visiting and so on. One scraper might work with multiple directory pages for scrape.
You might find it interesting to read about web page change tracking if you only need to see updates (no data storing applied).