We recently applied Sequentum Cloud to a highly protected scrape target and got quite successful results. An initial Agent run had some errors, failing data capture of some of URLs. After a support consultation we were pointed to a built-in feature of Sequentum Cloud to rerun Agent to gather missing data.
Below we outline steps for rescraping missing [pagination] URLs.
Steps to “Retry Errors and Continue”
- Go to an Agent Runs page tab
2. Click on the Content Menu
3. Select the option Retry Errors [and Continue]. This will initiate a process of retrying any failed URLs or tasks.
4. Confirm Retry Errors and Continue button to proceed.
Voilà. Now your data is100% in place (in CSV 😉 ) with 3 clicks. The scrape logs are appended (not recreated) with rerun log records.