Categories
Challenge Development

AirTable scrape challenge

I need to get info from AirTable, see a table example.

The problem is that data are loaded highly dynamically. HTML contains only the information that you currently see on the browser screen.

enter image description here

If there are a lot of records then it is difficult to collect such a table. One of the possible ways is to calculate the size of the screen and rows in the table. Then using the browser automation and use to make a script that will scroll through it bit by bit and collect data.

Is there any other feasible way to get data of a table? For example there is a HTTP requests coding way to get dynamic data.

JS infinite scroll does not work for AirTable either.

Please comment down here if having some tips, hints.

One reply on “AirTable scrape challenge”

@IgorSavinkin, Look at the requests like this and that JSON.

You can find them in the Network tab of Developer Tools (F12).

Then you scroll down the table, find the requests and extract the pattern of them.

Then you can find such requests when you scrolling down the table:

https://airtable.com/icons/icon_definitions.svg?v=350094b58ca651c8c1337a5da69b93bc

It returns data for the icons you mentioned in the question of that post and then it takes all the images one by one.

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.

This site uses Akismet to reduce spam. Learn how your comment data is processed.