We share with you how we’ve bypassed Akamai protected site.

Solution

The initial HTTP request scrape worked at first, but the site is very large and the requests delays started to be longer… It turned out to be too long to collect. So we had to apply browser automation and eventually the browser in headful mode, otherwise some kind of protection is triggered there (a request goes on indefinitely until the connection is broken).

• Browser automation (Selenium, Playwright)
• [Browser] being in headful mode

Local enviroment check

If we’re able to scrape some data at a local env. (desktop, laptop), then it’s clear that the bot block issue is not in a scraper spider but in IP.

Interested to see other bot protection cases ? Visit the post.