Pros and Cons of using Selenium WebDriver for Website Scraping

Since Selenium WebDriver is created for browser automation, it can be easily used for scraping data from the web. In this post we will consider some advantages and drawbacks of using WebDriver for web scraping.

How do I get pass dynamic “load more” btn?

Recently I’ve got a question:

How do I get pass the dynamic “load more” button using a Python web scraper?

Solve ReCaptcha with Selenium (python)

breaked by seleniumI’ve already written about how the new No CAPTCHA ReCaptcha works, and even had some success breaking it with an iMacros’ browser automation. But, the latest scraping tools are – for most part – driven by Python, so now I want to try the same experiment with Selenium + Python.

A Simple Code that Extracts a Hotel List from

In this post I will show you how easy it is to write a Python code that extracts hotel list from The simplicity of this code is achieved with the help of Selenium Web Driver which acts as the main data extraction means here.

How to change WebDriver’s IP address

I have already written several articles on how to use WebDriver for web scraping, but I have never touched on the topic of changing WebDriver’s IP address. Nevertheless, this topic is quite crucial when you come to web scraping, and here I’d like to show you an example of using proxies with WebDriver in Python (and you can easily convert it into your language API).

A simple LinkedIn Group Submitter

LinkedInLinkedIn API doesn’t allow you to publish into groups if you are not their administrator. That was done in order to eliminate spamming, but if you are a member of several groups of a similar topic and you want to share some interesting information with all of those groups, you have to do it manually group by group and eventually it becomes tedious. In this post I’ll show you a simple way to automate this process in C# using Selenium WebDriver.

Tutorial: How to use Headless Firefox for Scraping in Linux

I have already written several articles on how to use Selenium WebDriver for web scraping and all those examples were for Windows. But what about if you want to run your WebDriver-based scraper somewhere on a headless Linux server? For example on a Virtual Private Server with SSH-only access. Here I will show you how to do it in several simple steps.

An Example of Captcha Solver in Java

java_captcha Recently I published an article on how to solve captcha in C# using DeathByCaptcha service, and I promised to offer you an example in other languages as well. In this post I’ll offer a Java project that does the same thing.

How to Write a Captcha Solver that uses DeathByCaptcha service

Let’s look at a practical example on how to solve CAPTCHAs using the DeathByCaptcha service. This example is written in C#, but you can get it in Java as well.

6 Tips for Using WebDriver with Java

In this article I’ll share with you 6 usefult tips that may help you when you work with Selenium WebDriver in Java.