Scrape YouTube comments using Python & Selenium

Scraping youtube comments has become crucial if you are working on some sentiment analysis project. The comments section will give you an overview of the public sentiment toward any election or sports results, scams, wars, etc. Comments reflect an overall feeling of a person. What according to them is right and wrong is mentioned in the comments.

Challenge Development

CodeStar connection full working policy at AWS

The original AWSCodeStarFullAccess policy is full [to provide access to  AWS CodeStar via the AWS Management Console] yet it still does not grant enough access to create a Code connection for IAM at CodePipeline. So we had to manage to create a custom policy based on AWS tutorial suggestions.


Puppeteer async scraper with browsers number to be tuned based on CPU capacity

Recently we’ve got a tricky website of dynamic content to scrape. The data are loaded thru XHRs into each part of the DOM (HTML markup). So, the task was to develop an effective scraper that does async while using reasonable CPU recourses.


Choosing reliable [rotating] residential proxies

Our experience with the residential proxy services. The table info is in progress.

ResidentialCost/monthTraffic/month$ per GBRotatingIP whitelistingPerformance and moreNotes
$ 30025 GB9 - 12

"pay-as-you-go" - 15
yesyes- 30K requests
- 1.3 GB of data
- 5K pages crawled
Not allowing to scrape some of grey zone targets, incl. Linkedin. yes$ 4from 1 GB4yes
from $ 96from 10 GB3.5 - 12
yesyesOver 95% success
*Bans from Cloudflare are also few, less than 5%.
No "pay-as-you-go" plan
Charity Engine -docsyes-- starting from 3.6
CPU computing
- from $0.01 per avg CPU core-hour
- from $0.10 per GPU-hour - source.
failed to connect so far
"pay-as-you-go" - 12.5
yesNot allowing to scrape some of grey zone targets, incl. Linkedin.

See more exensive and detailed comparison in here.

Undetectable’s proxy partners.


MERN Stack – Build a Film Hall Application

Handle Your Project in Details

What is MERN?

The MERN stack is a set of frameworks and tools used for developing a software product. They are very specifically chosen to work together in creating a well-functioning software (see a MERN app code at the post bottom).


A Full Stack Todo App

In this post we are presenting you a Full-Stack Todo App, which is built in React.js, Next.js and Sequelize Sqlite3, and it is responsively implemented according to different screen sizes (see the code at the post bottom).

Challenge Development

AirTable scrape challenge

I need to get info from AirTable, see a table example.

The problem is that data are loaded highly dynamically. Html contains only the information that you currently see on the browser screen.

enter image description here

If there are a lot of records then it is difficult to collect such a table. One of the possible ways is to calculate the size of the screen and rows in the table. Then using the browser automation and use to make a script that will scroll through it bit by bit and collect data.

Is there any other feasible way to get data of a table? For example there is a HTTP requests coding way to get dynamic data.

JS infinite scroll does not work for AirTable either.

Please comment down here if having some tips , hints.


Google Sheets or MS Excel to scrape business directories ?

We’ve already stated some Tips and Tricks of scraping business directories or data aggregators sites. Yet recently someone has asked us to do aggregators’ scraping in the context of Google Sheets and/or MS Excel.

Challenge Development

Yelp scraping for high quality B2B leads

Recently we’ve performed the Yelp business directory scrape for acquiring high quality B2B leads (company + CEO info). This forced us to apply many techniques like proxying, external company site scrape, email verification and more.

Development Guest posting

Scrape ‘Ticketmaster’ using Selenium with Python

We’ve got some code provided by Akash D. working on He automates browser (Chrome as well as Edge) using Selenium with Python. The rotating authenticated proxies are leveraged to keep undetected. Yet, the site is protected with Distil network.