Categories
Challenge SaaS

Web Scraper IDE to scrape tough websites

Recently we encountered a new powerful scraping service called Web Scraper IDE [of Bright Data]. The life-test and thorough drill-in are coming soon. Yet now we want to highlight its main features that has badly (in positive sense, strongly) impressed us.

Categories
Development

Backconnect Proxy Service with authorization in JAVA

Working with a Backconnect proxy service (Oxylab.io) we spent a long time looking for a way to authorize it. Originally we used JSoup to get the web pages’ content. The proxy() method can be used there when setting up the connection, yet it only accepts the host and port, no authentication is possible. One of the options that we found, was the following:

 

Categories
Review

Oxylabs.io at a glance

Oxylabs Logo VerticalOxylabs.io is an experienced player in the proxy market. In the past few years, they have significantly expanded their proxy pool.

Right now they have a residential proxy pool with over 60M IPs and over 2M datacenter proxies. Their residential proxies cover every country in the world (!) and offer city-level targeting. Oxylabs datacenter proxies come from 82 locations and feature 7850 subnets.

Oxylabs is mainly focused on businesses and it is reflected in their product subscription plans. But recently they have introduced a Fast-Checkout feature, where customers can purchase residential proxies in a few clicks. Together with a recently added smaller plan ($300/month for 20GB of traffic) Oxylabs becomes much more attractive for smaller customers as well.

Categories
Review

NetNut.io Review

Netnut.io logo

The most successful enterprises are always the ones which manage to stay a step ahead of their rivals. And to remain ahead, you have to be able to access the industry information faster and more consistently than anybody else. This is especially true for e-commerce and online retail industries, where the pricing contest is extremely fierce. Thus, the smallest developments in information processes can result in large changes in the outcomes.

Categories
Review

Choosing affordable residential proxies for web scraping

Proxies are an integrated part of most major web scraping and data mining projects. Without them, data collection becomes sloppy and biased. This is why it’s essential to know how to find the best affordable proxies for any web scraping project.

One of the best proxy types you could use for scraping is residential proxies. In this post, you’ll learn what they are, how they are priced and what to look for before committing your project’s budget.

Categories
Uncategorized

Smartproxy Review

Getting precise and localized data is becoming difficult. Advanced proxy networks are the only thing that is keeping some companies running intense data gathering operations.

Residential proxies are in extremely high demand, and there are only a few networks available that can offer millions of IP addresses around the world. 

Smartproxy is one of those networks, rapidly growing to offer the best product in residential and data center proxies.

Categories
Development

Scraping with free or paid proxies – what is the difference?

Anything free always sounds appealing. And we are often ready to go an extra mile to avoid expenses if we can. But is it a good idea to choose the free option when it comes to using proxies for data scraping? Or should you stick to the paid ones for better results?

Let’s weigh all the pros and cons to see why you should consider using residential IP providers like Infatica, Bright Data, NetNut, Geosurf and others.

Categories
Development Guest posting

Bright Data Proxy Manager with built-in scraping features

Web Data Extraction is critical to the online operations of companies across the globe. With more data being scraped daily, websites implement techniques to block extraction efforts.

Categories
Miscellaneous

Bright Data exclusive residential proxies to reach Linkedin in Russia

For some of our readers from Russia, it’s a new challenge to get to www.linkedin.com, which has been officially blocked in Russia.

On 4 August 2016, a Moscow court ruled that Linkedin must be blocked in Russia because it stores the user data of Russian citizens outside of the country, in violation of the new data retention law. The law requires all companies doing business in the country to store their users’ data locally.

Categories
Challenge Development Web Scraping Software

Brigth Data residential proxy for extracting from a data aggregator

In this post I’d like to share my experience with scraping data aggregator/business directory using the residential proxy of the Bright Data proxy provider in conjuction with its proxy manager.