One freight quote tells you nothing. The same lane quoted every week for a year tells you what the market did, and that is a grid, not a lookup.
ScrapeIt builds and runs the collection as a managed service. You define the lanes, modes and load profiles; we build the grid, run it on a fixed schedule so the series stays comparable, and hand back CSV, JSON, Excel or a push into your warehouse with prices decomposed, transit times attached and validity recorded.
The schedule is deliberately regular rather than frequent. A grid quoted every week at the same profiles produces a comparable series; a grid quoted whenever somebody remembers produces noise. Regularity matters more than frequency on this source.
We collect what the site publishes openly through its rate tools, honour published crawl rules and pace requests. Marketplace quotes reflect marketplace pricing rather than contracted enterprise rates, and we state that limit alongside the data rather than letting it be assumed away. Bring the use case to your own counsel before the project starts.
Each quote record carries origin and destination as ports or locations, the mode, the load type such as full container, less than container load, air or express, the container type or the dimensional and weight profile, and the incoterms basis where stated.
Price is decomposed rather than delivered as a single figure. Base freight, surcharges, origin and destination charges, and the total are kept apart, because a base rate that looks competitive and a total that does not is the most common pattern in freight quoting and it is invisible in a single number.
Transit time is captured as the estimated days in transit plus, where published, the routing including transhipment points. Direct and transhipped routings on the same lane are different products and should not be averaged together.
Validity is recorded. Freight quotes expire, and a rate without the date it was quoted for and the window it is valid in is not a usable observation. Every row carries the quote date, the requested shipping date and the validity period where given.
Currency is stored as quoted, with the conversion rate and date recorded separately if a client wants a common currency, so that exchange movement never gets mistaken for freight market movement.
Lane design is where a freight data project succeeds or fails, and it happens before any collection. A useful grid covers the lanes a client actually ships, plus the alternatives they might switch to, at the load profiles they actually use. Twenty well chosen lanes quoted consistently beat two hundred quoted occasionally, and the cost difference is substantial.
Benchmarking against contracted rates is the most common application. A client's negotiated rate on one side, the observed market on the same lane and profile on the other, tracked over time. The gap is the negotiating position, and it is the number that pays for the dataset in the first quarter of most engagements.
Seasonality becomes visible only with history. Freight has strong seasonal patterns and they shift year to year, so a series needs to run long enough to distinguish a seasonal move from a structural one. We are direct with clients that the first three months of a freight grid are setup and the value starts accumulating after that.
Multi source comparison is worth considering for anything that will inform a large procurement decision. A marketplace shows marketplace pricing, which is not identical to contracted enterprise pricing, and stating that limit is part of delivering the data honestly.
Freightos is an online freight marketplace where forwarders quote against shipment requirements and shippers compare. Its significance for data work is simple: international freight pricing has traditionally been opaque, negotiated and unpublished, and a marketplace makes it observable.
The site carries rate calculation tools alongside its booking marketplace, and both respond. A quote is a function of the lane, the mode, the load type, the dimensions and weight, and the date, which is what makes this a grid problem rather than a page scraping problem.
Modes matter. Ocean full container, ocean less than container load, air freight and express behave differently in price, volatility and transit time, and mixing them into one rate column produces a series that means nothing. The same applies to container types: a forty foot high cube and a twenty foot standard are separate markets.
Transit estimates arrive alongside price and are the second half of the answer. A rate that is twenty per cent cheaper on a route that takes twelve days longer is not cheaper in any sense a planner cares about, and any comparison that reports price without transit is incomplete.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
A single quote is an anecdote. Freight prices move sharply and seasonally, and a rate captured once tells you what one forwarder offered on one day for one profile. Nobody can plan on that, and nobody should benchmark on it.
What produces insight is the grid: a defined set of lanes, modes and load profiles, quoted on a fixed schedule, held over time. That turns freight pricing into a series with direction and volatility, and it answers the questions procurement actually has. Is our contracted rate above or below market on this lane. Which lanes are tightening. How far ahead of a seasonal peak do prices start moving.
The requirement for consistency is why this must be a grid rather than ad hoc lookups. Comparing a full container quote from March with a less than container load quote from July on a slightly different lane is not a comparison. The profile has to be held constant so that the only thing changing is the market.
The second reason is the decomposition. Surcharges are where freight pricing hides, and a lane where the base rate fell while surcharges rose has not become cheaper. Keeping the components apart is what makes the total explicable rather than merely reported.
The third is transit alongside price. Cheaper and slower is a trade a planner may or may not want, and the data has to present both halves for that decision to be possible at all.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt is a managed extraction company, not a tool you have to learn. Our team designs the grid with you, runs it on schedule, watches it as the site changes, and repairs it before a gap appears in a series whose whole value is continuity.
You see a sample first, in your format, on your own lanes and load profiles, with prices decomposed so you can see where the surcharges sit before committing to a series.
You can, and it will not help you much. A single quote is one forwarder's offer on one day for one profile. The value here is the series: the same lanes and load profiles quoted on a fixed schedule, held over time, so you can see direction and volatility. That is what answers whether your contracted rate is above or below market.
Because that is where freight pricing hides. A lane where the base rate fell while surcharges rose has not become cheaper, and a single total conceals which happened. We keep base freight, surcharges, origin and destination charges and the total apart so the movement is explicable rather than just reported.
Yes, along with the routing including transhipment points where published. A rate twenty per cent cheaper on a route twelve days slower is not cheaper in any sense a planner cares about, and direct and transhipped routings on the same lane are different products that should never be averaged together.
Fewer than you first think, chosen well. Twenty lanes quoted consistently at your real load profiles beat two hundred quoted occasionally, and cost far less. We design the grid with you before any collection starts, covering the lanes you ship plus the alternatives you might switch to.
No, and we say so rather than letting it be assumed. A marketplace shows marketplace pricing; large shippers negotiate enterprise rates that are not published anywhere. The grid is an excellent benchmark and a directional market signal, and it is not a substitute for your own contracted rate card.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
