As we mentioned before, it’s often necessary to use proxy server when you gather infromation from the web. In this tutorial I’ll show you how to tune Visual Web Ripper to run the web requests through proxy servers.

1. Creating a test project

Let’s create a test project that scrapes a page from our testing ground, extracts the IP address and saves it as in the resulting table:

2. Configuring proxy servers

Then we need to add proxies and let our scrape go through them. To do this, click Project>Project Options in the menu. Then in the opened window select the Proxies tab. Although Visual Web Ripper offers a large set of options here, let’s select Proxy List and Rotate per page load:

Now it’s time to add several proxies to our list. You can get a list of free open proxies at http://www.freeproxylists.net/ for instance, but if you need reliable private proxies you can get them at BestProxyAndVPN service for a little payment. To add the proxies click Add/Edit Proxy List button:

Besides all other options, here the program allows you to test your proxies right away. You can do so by pressing the Test All Proxies button. After the test, all good proxies will be marked with a green box and bad ones, with a red box (also you’ll be asked for bad proxy removal).

3. Running the project

As soon as we have configured our project, let’s run it and see how it works. To run, click Project>Run Project in the menu. In my case I got the following execution log:

As is seen from this log, first Visual Web Ripper switched the proxy and then requested the web page. Let’s see now what IP address has been scraped from the testing page (click View Data button):

This IP address should differ from your machine IP address. If it is, then the proxy works!

Happy scraping!