Today I needed to enable a Charles proxy on my Windows PC. Later I have managed the Genymotion virtual device to be monitored by the Charles proxy.
UiPath, one of the big providers of robotic process automation software, has some very interesting positioning. Unlike the other players on the market, they provide a free and fully featured community edition of their product for anybody to test and develop. The tool automates any application and is packed with all the web scraping and screen scraping capabilities for both desktop and web. The platform also has a lively community forum featuring jobs, automation contests and knowledge-sharing between UiPath users: www.forum.uipath.com.
Given:
- host: lx567.certain.com (SFTP)
- user: igor_user
- password: testPass
For SSH access in a terminal type:
$ ssh igor_user@lx567.certain.com
then enter the password (testPass) at a password prompt.
Question:
The Python requests library is a useful library having tons of advantages compared to other similar libraries. However, as I was trying to retrieve the Wikipedia page, requests.get() retrieved it only partially:
Recently I decided to work with pythonanywhere.com for running python scripts on JS stuffed websites.
Originally I tried to leverage the dryscrape library, but I failed to do it, and a nice support explained to me: “…unfortunately dryscrape depends on WebKit, and WebKit doesn’t work with our virtualisation system.”
Categories
Find XPath using web developer tools
Often for the purpose of scraping, one needs to find certain elements’ XPath on a webpage. How can one do that with browser Web developer tools, aka Web inspector? A picture is worth of thousand words. 
Categories
Dexi.io – October 2016 release
Dexi.io has put out a new October 2016 release. It includes the following feature improvements:
Categories
New reCaptcha testing-ground
We want to share with our readers about a new testing-ground with reCaptcha v2.0. Since we do R&D of how to solve reCaptcha by web scripts and by captcha breaking services, it’s vital to have a reCaptcha testing ground.
This testing ground is designed according to the How to insert and configure reCaptcha post.
Mozenda is a cloud web scraping service (SaaS), and we’ve already reviewed it. Since our last review, Mozenda has provided more useful utility features for data extraction. Besides multi-threaded extraction & smart data aggregation, Mozenda allows users to publish extracted data to cloud storage such as Dropbox, Amazon, and Microsoft Azure. In this post we will try to explain the new Mozenda extraction and integration capabilities.
In this post we want to show you the code for an automatic connection to 2captcha service for solving google reCaptcha v2.0. Not long ago, google drastically complicated the user-behavior reCaptcha (v2.0). This online service provides a method for solving it.