Flight Pricing at Scale: Opodo Scraped with Full Filter Logic

The client ordered a dataset on flight tickets from the booking website Opodo.com.

Websites for Scraping: opodo.com

An additional requirement was the application of multiple custom filters, so that the web server itself would handle the filtering and return only the flights with the unique parameters the client needed for their analytics pipeline.

This project implemented what was likely the most complex automation flow of user interactions we’ve built so far, including slider control—something rarely seen in automation projects.

Search and Filtering Rules:

Search for all flight tickets between specified departure and arrival airports for each of the next N days.
Filter results to include only flights where both the outbound and return trips are operated by the same airline.
Select the best-priced option for each airline on each departure date.
To optimize the number of requests, the scraper manipulated user filters on the webpage using an automated browser.

Setup & Launch Time: 4 working days

Data Volume: The daily dataset contains approximately 13,000 rows.

Talk to us to find out how we can help you

Let us take your work with data to the next level and outrank your competitors.

How does it Work?

1. Make a request

You tell us which website(s) to scrape, what data to capture, how often to repeat etc.

2. Analysis

An expert analyzes the specs and proposes a lowest cost solution that fits your budget.

3. Work in progress

We configure, deploy and maintain jobs in our cloud to extract data with highest quality. Then we sample the data and send it to you for review.

4. You check the sample

If you are satisfied with the quality of the dataset sample, we finish the data collection and send you the final result.

Get in Touch with Us

Tell us more about you and your project information.