Author: admin

Categories
Miscellaneous

Death By Captcha Updated API clients

Death By Captcha is a reputable CAPTCHA solving service with more than 7 years in the Captcha Solving business. They have recently updated all their API clients, so users can experience maximum efficiency and faster solving times.

They enthusiastically recommend that users and software developers visit the API page and update their DBC API implementation in order to get the most out of it (the API and docs are available for registered users only).  The free credits are provided for users to test or implement the new client API!
[box style=’info blue’]If you tell them you saw this info through the scraping.pro blog, they’ll give you a 1K free CAPTCHAs additional credit![/box]
For further info, you may contact them directly.

Categories
Development

Charles CA certificate with OpenSSL in Windows

Today I needed to enable a Charles proxy on my Windows PC. Later I have managed the Genymotion virtual device to be monitored by the Charles proxy.

Continue reading “Charles CA certificate with OpenSSL in Windows”
Categories
Guest posting Web Scraping Software

UiPath PDF Data Extraction

UiPath, one of the big providers of robotic process automation software, has some very interesting positioning. Unlike the other players on the market, they provide a free and fully featured community edition of their product for anybody to test and develop. The tool automates any application and is packed with all the web scraping and screen scraping capabilities for both desktop and web.  The platform also has a lively community forum featuring jobs, automation contests and knowledge-sharing between UiPath users: www.forum.uipath.com.

Continue reading “UiPath PDF Data Extraction”
Categories
Development Miscellaneous

SSH connection in terminal for Linux

Given:

For SSH access in a terminal type:

$ ssh igor_user@lx567.certain.com

then enter the password (testPass) at a password prompt.

Categories
Development

Python requests vs urllib2 for JS-stuffed website scrape

Question:

The Python requests library is a useful library having tons of advantages compared to other similar libraries. However, as I was trying to retrieve the Wikipedia page, requests.get() retrieved it only partially:

Continue reading “Python requests vs urllib2 for JS-stuffed website scrape”
Categories
Development

Headless browser python scraper at pythonanywhere

Recently I decided to work with pythonanywhere.com for running python scripts on JS stuffed websites.

Originally I tried to leverage the dryscrape library, but I failed to do it, and a nice support explained to me: “…unfortunately dryscrape depends on WebKit, and WebKit doesn’t work with our virtualisation system.”

Continue reading “Headless browser python scraper at pythonanywhere”
Categories
Development

Find XPath using web developer tools

Often for the purpose of scraping, one needs to find certain elements’ XPath on a webpage. How can one do that with browser Web developer tools, aka Web inspector? A picture is worth of thousand words.

Continue reading “Find XPath using web developer tools”
Categories
Miscellaneous

Dexi.io – October 2016 release

dexi-miniDexi.io has put out a new October 2016 release. It includes the following feature improvements:

Continue reading “Dexi.io – October 2016 release”
Categories
Development

New reCaptcha testing-ground

We want to share with our readers about a new testing-ground with reCaptcha v2.0. Since we do R&D of how to solve reCaptcha by web scripts and by captcha breaking services, it’s vital to have a reCaptcha testing ground.

This testing ground is designed according to the How to insert and configure reCaptcha post.

Continue reading “New reCaptcha testing-ground”
Categories
Web Scraping Software

Mozenda web scraping and publishing of data to cloud storage

Mozenda is a cloud web scraping service (SaaS), and we’ve already reviewed it. Since our last review, Mozenda has provided more useful utility features for data extraction. Besides multi-threaded extraction & smart data aggregation, Mozenda allows users to publish extracted data to cloud storage such as Dropbox, Amazon, and Microsoft Azure. In this post we will try to explain the new Mozenda extraction and integration capabilities.

Continue reading “Mozenda web scraping and publishing of data to cloud storage”