In the post we wanna share with you what data sources to scrape to find info best routes within Europe or worldwide. The routes might include walking, biking, driving, or public transport.
Public Transport APIs
Many cities and countries have public transport APIs that provide information on routes, schedules, and fares. Examples include the following:
- TransportAPI (UK)
- Citymapper API , eg. Citymapper API at Postman
- Google Maps Directions API
Octoparse: how to extract GPS coordinates from Google Maps
Travel Aggregators
Websites like Skyscanner, Kayak, and Rome2rio aggregate various transport options and can provide pricing information for different modes of travel.
Most popular web scraping targets and how to scrape them
Biking and Walking Apps
Platforms like Strava, Komoot, or MapMyRun can provide data on bike routes and walking paths, including user-generated content on distances and difficulties.
Ride-Sharing and Carpooling Services
Scraping data from services like BlaBlaCar or Uber can provide insights into carpooling options and costs.
Local Government Transport Websites
Many cities have official transport websites that provide information on public transport routes, schedules, real time maps and pricing.
Travel Blogs and Forums
Websites like TripAdvisor, Lonely Planet, or Reddit can have user-generated content discussing the cheapest ways to travel between destinations.
Scraping a Javascript-dependent website with Puppeteer, incl. TripAdvisor
OpenStreetMap
This collaborative mapping project can provide detailed information on roads, paths, and cycling routes.
Cost Comparison Websites
Websites that specialize in comparing travel costs, such as GoEuro (now Omio), can provide insights into the cheapest travel options.
When scraping data, ensure you comply with the terms of service of each website and consider the ethical implications of data collection. Read more in the legal section.