Categories
Development

SQL Dump Splitter

This nice free program saved my day when I had to transfer a big database from one MySql Server to another. If you need to restore a big database using phpMyAdmin you will find it usefull as well.

Categories
Web Scraping Software

How to use proxy in Visual Web Ripper

As we mentioned before, it’s often necessary to use proxy server when you gather infromation from the web. In this tutorial I’ll show you how to tune Visual Web Ripper to run the web requests through proxy servers.

Categories
Web Scraping Software

Does HideMyAss’s “Scheduled IP Change” feature really work?

HideMyAss Scheduled IP ChangeHideMyAss proxy service has a wonderful feature called “Scheduled IP Change” that automatically changes your IP address at set time intervals. This may help you greatly if you are trying to scrape a website that may block the IP address you use for scraping. But does this feature work as good as it is stated? Recently we have got the following testimony of one of our visitors:

Categories
Development

Using Regex Lookaround for HTML element extraction

Yes, I’m aware that using regex for HTML parsing is not the best idea. But still when I need to quickly extract some small portion of a web page I find myself applying regex more often than executing an XPath query, and its lookahead and lookbehind constructions may be quite helpful.

Categories
Uncategorized

About Proxy Servers

It’s frequently required to have your actual IP address hidden when doing web scraping or, alternately, to access the website from different counties. That’s why we have anonymizers, also called anonymous proxies. These days, it is possible to find an abundance of proxy software and services. Following is a general summary of the fundamentals of proxy:

Categories
Web Scraping Software

CyberGhost: a nice freemium proxy

As you scrape information from websites, it’s often necessary to keep your real IP hidden, quickly change your IP or simply access a website from a country that differs from your own. All these tasks are achieved by means of proxies, mediators between you and the target website. Though there are plenty of companies offering such services on the market today, in this post I’ll introduce you to CyberGhost, an affordable and nice looking proxy.

Categories
Development

5 Best XPath Cheat Sheets and Quick References

XPath Cheat Sheets I always love a good cheat sheet hanging on my corkboard when I’m working, and XPath is one of the fields where I often refer to it. If you’re looking for a good XPath cheat sheet you will probably find something useful in this post.

Categories
Web Scraping Software

Knowledge Walls: manipulation with JSON, XML, CSV and more

Personally, I prefer using online tools for performing quick manipulation on different data formats like JSON, XML, CSV and so on. They’re platform independent and always within reach of my hand (since I mainly work in a browser). After we published an article about 7 best JSON viewers, I was told about Knowledge Walls, a similar service containing many tools for text data manipulation.

Categories
Web Scraping Software

A simple way to turn a website into JSON

Recently, while surfing the web I stumbled upon an simple web scraping service named Web Scrape Master. It is a kind of RESTful web service that extracts data from a specified web site and returns it to you in JSON format.

Categories
Web Scraping Software

Quick Scraping with Yahoo Pipes

Yahoo PipesAs we are talking about web scraping, it would be a pity not to mention Yahoo Pipes, an exciting service provided by Yahoo!. This tool provides users with an intuitive graphical interface to assist them in organizing their favorite feeds and webpages into a single stream of content.