Pros and Cons of using Selenium WebDriver for Website Scraping

Since Selenium WebDriver is created for browser automation, it can be easily used for scraping data from the web. In this post we will consider some advantages and drawbacks of using WebDriver for web scraping.

Example of Scraping with Selenium WebDriver in C#

In this article I will show you how it is easy to scrape a web site using Selenium WebDriver. I will guide you through a sample project which is written in C# and uses WebDriver in conjunction with the Chrome browser to login on the testing page and scrape the text from the private area of the website.

What is Selenium WebDriver?

If you are interested in browser automation or web application testing you may have already heard of Selenium. Since there is a lot of terminology related to this framework, it is easy for you to get lost, especially if you come to Selenium for the first time. In this article I want to save your day by providing a short and clear explanation of what is what in the Selenium project.

What is Selenium?

Selenium is a web application testing framework that allows you to write tests in many programming languages like  Java, C#, Groovy, Perl, PHP, Python and Ruby. Selenium deploys on Windows, Linux, and MAC OS.

It is an open-source project, released under the Apache 2.0 license, so you can download and use it without charge.