Companies like Facebook owner Meta, e-commerce behemoth Amazon and ride-hailing firm Lyft have announced job cuts in recent weeks as the US tech industry deals with an uncertain economic climate.
DW, 01.12.2022

Data Set

So, we’ve got lists (as a data directory) of those laid off tech professionals (from open sources). If someone is interesting in the data, we would gladly share it for a cost (starting from $5 per entry). The filtering by role, expertise and location is available.

Sample

Take a look at a data sample (no emails) below.

2022-laid-off-employees-sample Download

Please enquire me at: igor [dot] savinkin [at] gmail [dot] com for a customized data set.

Post author By Igor Savinkin
Post date December 2, 2022

Development

Node.js to automate a browser XHR (Ajax)

Lately I needed to scrape some data that are dynamically loaded by “Load more” button. A website JavaScript invokes XHR (or Ajax request) to fetch a next data portion. So, the need was to re-run those XHR with some POST parameters as variables.

So, how to make it in Node.js?

Post author By Igor Savinkin
Post date September 23, 2023

Development

Choosing the Best Proxies for Web Scraping

From eCommerce and market research to competitive analysis and more, web scraping has become an integral part of data collection. And for some, it’s the secret sauce for success.

But with great scraping power comes great responsibility.

Web scraping can result in IP bans and other harsh restrictions. To avoid these issues, many turn to proxies, which act as intermediaries between your requests and the target website. In this article, we’ll explore the top 3 proxy types for web scraping and focus on the key benefits of each proxy. Let’s go!

Post author By Igor Savinkin
Post date September 21, 2023

Guest posting

Bright Data’s Business Capabilities

Bright Data offers its customers a full suite of real-time data collection tools that help them gain and maintain a competitive market edge. BrightData prides itself on its ethical and 100% legally compliant approach.

Post author By Igor Savinkin
Post date September 9, 2023

Challenge Development

Infinite scroll for getting group members in Linkedin

Recently i was challenged with getting Linkedin group memebers info. The challenge made me to seek some ways to

Here is a video where I try to catch all Linkedin group members through endless scroll**, I having to be a member of the group.
The real speed is ~1 person/second though.

Sometimes you need to click on the “Scroll more results” button or even just hover the mouse over this button.

Automate scroll

**The post on how to start infinite scrol in the browser… But this JS code from the post does not help to replenish the page… Only when there is a focus on the browser tab/page, the script continues loading data again.

Post author By Igor Savinkin
Post date September 6, 2023

Review

A Comprehensive Examination of NetNut Proxy Services

When traversing the digital landscape for high-quality proxy services, the name NetNut often emerges as a frontrunner. This review aims to delve into the facets of NetNut’s offerings, emphasizing the distinctive aspects that set it apart in an intensely competitive environment.

An Introduction to NetNut

In the world of proxies, NetNut.io has built a reputation for speed, reliability, and robustness. Leveraging its partnership with a major US data carrier, NetNut offers a residential proxy network that is both unique and high-performing. The company’s network is expansive, with a coverage that spans millions of IPs globally, presenting an impressive degree of scalability.

Post author By Igor Savinkin
Post date September 5, 2023

Development

Static Residential Proxies

Static Residential Proxies do not rotate randomly but rather on-demand. They are good for web scraping to make a spider maintain its session at single website. And this session memory might last for life.

How are they different from dynamic/rotating residential proxies?

Greater stability
Faster speed
Website sessions keeping

Post author By Igor Savinkin
Post date June 9, 2023

Challenge Development

Node.js & Privacy Pass application for Cloudflare scrape solution

Over 7.59 million of websites use Cloudflare protection, 26% of
them are among the top 100K website worldwide. As Cloudflare
establishes itself as the norm regarding service protection, chances are, the site you want to scrape is more likely to use it than not.

When it comes to scrapping websites, captchas and other type of
protections were always the main obstacle in providing reliable data collection solutions. And most often this would lead to consider bypass services which aren’t always free.

Post author By Igor Savinkin
Post date May 2, 2023

Development

MobaXterm, better Putty alternative

No Comments on MobaXterm, better Putty alternative

MobaXterm is a server connectivity software (Windows) and it’s much better than Putty. It’s branded as “Enhanced terminal for Windows with X11 server, tabbed SSH client, network tools and much more”.

Post author By Igor Savinkin
Post date April 21, 2023

Uncategorized

Github AI pair programmer

GitHub Copilot · Your AI pair programmer

The copilot is integratable into 4 major IDEs

Visual Studio

Neovim

VS Code

JetBrains IDEs

Post author By Igor Savinkin
Post date April 14, 2023

Challenge Development

Undetected ChromeDriver in Python Selenium

Selenium comes with a default WebDriver that often fails to bypass scraping anti-bots. Yet you can complement it with Undetected ChromeDriver, a third-party WebDriver tool that will do a better job.

In this tutorial, you’ll learn how to use Undetected ChromeDriver with Selenium in Python and solve the most common errors.

Post author By Igor Savinkin
Post date April 11, 2023