Flight Pricing at Scale: Opodo Scraped with Full Filter Logic
Automated scraping of filtered flight ticket data from Opodo.com, including complex on-page interactions for airline and pricing selection.
Learn More
This is a managed service. You set the scope - which markets, which hotels or destination regions, which departure airports, durations and board types - and we build the crawler, keep it working when the front end changes, and deliver the output.
Delivery is CSV, JSON, XLSX or an API your systems call directly, pushed to S3, Google Cloud Storage, SFTP or a database you name. Schedules run daily, several times a day, or on a cadence matched to how fast your segment moves; hotel metadata changes far more slowly than offers, so the two normally run as separate jobs. Runs are monitored, and a front-end change is our problem to fix rather than a broken file in your inbox.
Extraction runs on two levels. The hotel record is stable, the offer is not, and both are delivered in one dataset joined on giataId.
A single crawl of TUI is a snapshot that is already ageing. Offers are not durable records: each one is keyed by a hash rather than a stable ID, and the booking engine re-checks the selected offer through /api/bookingVacancy - a Verfügbarkeitsprüfung, or availability re-check - before it lets a booking proceed. An offer that resolved in the morning can fail that check by the afternoon, and /api/alternativeFlights then swaps the flight legs underneath the same hotel and week.
Scheduled crawling turns that volatility into signal. Repeat the same hotel, week, duration and board type on a fixed cadence and the movement shows: the Katalogpreis (brochure list price) holding still while the live Aktionspreis (promotional price) falls, Frühbucher (early-booker) rates giving way to Last Minute and Restplätze (remaining places), a room type dropping out of the offer set and returning at a different price, an operator code switching from TUID to XTUI on the same product.
Structure moves too. Slugs go stale after a rebrand - tui.co.uk still serves the renamed Tivoli La Caleta property under a sheraton-la-caleta-resort-and-spa.html path - so a crawl keyed on slugs quietly drifts while a crawl keyed on GIATA IDs does not. Hotels also leave the sitemap and come back. Only a history tells you which.
TUI sells Pauschalreisen (package holidays) - flight, hotel and transfer priced as one product - next to hotel-only stays and seat-only flights. On tui.com the unit of record is the hotel, and every hotel page is addressed as /hotels/{hotel-slug}-{giataId}/hotelinformation/. The trailing number is the GIATA ID, the identifier the travel industry uses to match properties between systems, and the page states it outright: on /hotels/riu-bravo-20410/hotelinformation/ an inline script assigns giataId the value 20410. A pass over the CloudFront-hosted hotel sitemaps at the end of August 2026 returned roughly 190,000 unique hotel URLs, all of them ending in /hotelinformation/.
Search and booking sit in a separate application. The IBE (Internet Booking Engine) is an AngularJS single-page app mounted as an app-ibe element on /suchen/hotels/ and /suchen/angebote/, running on a TYPO3 stack behind Varnish. /suchen/, /book/ and /checkout/ are all IBE routes with no server-rendered offers. Result sets page by offset rather than by number: requests carry skip and fetchSize, responses return nextSkip alongside hotelCount and offerCount, and the client falls back to batches of ten.
Prices load client-side. Each hotel page ships a tui-hotel-best-offer component that fetches offers and emits hotelOffer objects with totalPrice and currency for two scopes, HOTEL for room-only and PACKAGE for flight plus hotel. Reviews, room text and sustainability labels are declarative Shadow DOM components keyed on giata-id, hotel-id, tenant and locale, so light-DOM selectors miss them; tui-customer-reviews is configured for DE, EN and FR at five reviews per page.
Get a QuoteDevelopers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Package holiday pricing in Europe is set in a market with few large operators, and TUI is the reference the others price against. A daily feed of TUI package prices by departure airport (Abflughafen) and destination region (Zielgebiet, carried in the search as regionId) lets a competing operator see where it sits per route and per week, instead of per brochure.
Hotel groups use the same records in reverse. Chains with inventory on tui.com - RIU, Grecotel, Iberotel, Atlantica, Grupotel, and the TUI-run brands TUI BLUE, TUI MAGIC LIFE, ROBINSON, TUI SUNEO and TUI Sensimar - can check how their properties are presented, which board types get promoted, and what a package price implies about the net rate a distributor is working from.
Revenue teams model the board spread: what All Inclusive costs over Halbpension (half board) for the same room, week and departure airport. Sector analysts watch which Veranstalter (tour operator) code fulfils which market. Comparison and affiliate sites build listings from hotel metadata, ratings and imagery. UK-side work usually covers tui.co.uk, First Choice, Marella Cruises and TUI Airways together, since those brands share customers but not URL structures. Excursions on TUI Musement, holiday homes on TUI Villas and rentals on TUI Cars run on separate stacks and are scoped as separate jobs.
Automated scraping of filtered flight ticket data from Opodo.com, including complex on-page interactions for airline and pricing selection.
Learn More
Daily scraping of Booking.com services - hotels, flights, car rentals, and attractions - with best-price selection across global destinations.
Learn MoreLearn how to use web scraping to solve data problems for your organization
If you work in travel tech, an OTA, a hotel chain, or at an airport, you are in a price-and-availability arms race. Fares change by the hour, room inventory disappears in minutes, and competitors test new bundles and ancillaries constantly.
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
ScrapeIt builds and operates crawlers for teams that need the data, not the maintenance. Travel sites are the awkward end of that work: the record you want sits behind a booking engine, the price is assembled in the browser, and every country site is a different application. We have shipped this shape of project before, we work within what a site's robots file and terms allow, and we stay responsible for keeping the feed alive instead of handing over a script.
Not a public one. The booking engine has internal endpoints under {languagePrefix}/api/ - hotel, offer, region, referenceData and booking, plus bookingVacancy, alternativeFlights, hotelAvailability, productcontent and hotelLookup - but they are not open to third parties. A plain HTTP request to /api/region or /api/offer comes back as HTTP 403 from Varnish with a page titled 'crawler/spider lock down', asking crawlers to contact TUI operations instead. There is no key to apply for and no feed to subscribe to. What we deliver is an extraction service built on the public pages, scoped and scheduled to what you actually need.
We work from publicly accessible pages, read and follow robots.txt, and stay out of anything behind a login. On tui.com robots.txt is permissive - User-agent: * with Allow: / - and a sitemap index is published, so the hotel pages themselves are open to crawlers. The booking engine endpoints are a separate matter: they are edge-restricted and explicitly ask crawlers to make contact, and we treat that as a boundary rather than an obstacle. Prices reach the public hotel page through a component rendered in the browser, and the rendered public page is what we work from. We keep request rates modest and collect factual product data, not editorial text or personal data. Legal advice for your own jurisdiction and use case has to come from your counsel.
Different infrastructure per country. tui.com, tui.at and tui.ch answer plain HTTP clients from a Varnish edge that identifies itself as plcache.edge. tui.co.uk, tui.nl, tui.be, tui.se, tui.dk, tui.no and tui.fi sit behind Akamai - the response header reads AkamaiGHost - and a non-browser client is refused with 403 Access Denied. tui.pl is a different stack again, a Next.js app on Vercel, and firstchoice.co.uk runs on CloudFront. We do not sell a way around bot management. We run a real browser environment, keep request rates polite, and tell you at the scoping stage which markets are cheap to cover and which are not.
GIATA IDs are industry-standard hotel identifiers, and tui.com carries one in every hotel URL, typically four to seven digits. The same ID is reused across the German-language tenants: hreflang on a tui.com hotel page points at tui.at and at tui.ch with the same number, and tui.ch adds a language prefix such as /de/ or /fr/ that the booking engine mirrors as its API prefix. Tenant is derived from the hostname as TUICOM, TUIAT or TUICH. Other markets share nothing. tui.nl uses a root-level slug with an eight-digit number, tui.pl uses destination-code hotel codes such as bcn10291 or tfs36003 that reappear in its image CDN paths, tui.co.uk uses slug-only .html paths, tui.se uses a pure slug hierarchy and tui.be prefixes the locale and appends ISO3 and IATA codes. Matching those markets is entity resolution against GIATA using name, address, coordinates and category, and we build that step into the pipeline.
Yes. Those are first-class search parameters - adults, boardType, destination, duration, searchScope, startDate and giataId among them, with the client switching to minified keys (a, bT, d, du, sS, sD) once a built URL passes 2083 characters. A scope such as All Inclusive, seven nights, Mallorca, departing DUS or HAJ, over the next six months is an ordinary request, and the matching offer deep link is /suchen/angebote/{Hotel-Name}/{giataId}/offer/. Flights run alongside packages: TUI publishes route pages such as /flug/duesseldorf-dus-mallorca-pmi/, fares load into a tui-flight-offer-list component carrying origin, destination and trip type, and the deep link is /flug/flugauswahl/ (flight selection) with origin, destination, dates and journeyType. The search bar declares market X3DE, X3 being the TUI fly carrier code. Output is CSV, JSON, XLSX or an API on your schedule; price depends on scope and refresh rate, so we quote after a short scoping call.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582