The robots file closes the individual vessel pages. That is not a hurdle to route around, it is the scope, and the honest answer is a different source for the deep detail.
ScrapeIt builds and runs the pipeline as a managed service. We collect the permitted surfaces here, integrate licensed AIS where your use case needs positional history, join both to vessel particulars and to your own bookings, and hand back CSV, JSON, Excel or a push into your warehouse with provenance recorded per row.
We honour the crawl rules this site publishes, which means the individual vessel view and ship photo paths are out of scope. If a vendor has offered you per vessel tracking scraped from those pages, that offer is built on ignoring the rules the site states, and it is worth asking them about it.
Vessel movement data is commercial information about ships and their operators rather than personal data, so the sensitivity is contractual rather than privacy driven. Licence terms differ between AIS providers and matter for redistribution, so bring the use case to your own counsel before the project starts, particularly if the output will be resold.
From the permitted surfaces we collect vessel identity and particulars as listed: name, IMO number, MMSI, vessel type, flag, gross tonnage and deadweight where published, year built, and current status where shown in a list view.
Port and movement information covers port name and country, arrivals and departures as listed, and the vessels associated with a port in the index views, each with the timestamp the observation was made.
Every record carries its source surface explicitly. When a client asks later where a field came from, the answer has to be precise, particularly on a source where some paths are permitted and others are not.
Where a project also uses licensed AIS, that data joins on IMO and MMSI and carries its own provenance and licence fields, so a downstream user can always tell which rows came from which source and what each licence permits. Mixing licensed and public data into one undifferentiated table is how a compliance problem gets built by accident.
Positional history, sustained per vessel tracks and voyage reconstruction come from the licensed side rather than from this site, and we label them accordingly.
Licensed AIS is the substrate for any serious ocean visibility product, and integrating it is ordinary data engineering rather than scraping: ingest, normalise, gap fill, join to vessel particulars and to your own bookings. We build that pipeline as readily as we build a collector, and on this vertical it is more often the right recommendation.
Port congestion analysis is one of the more valuable outputs and it is derived rather than collected. Arrivals, waiting time at anchorage and berth turnaround produce a congestion measure per port over time, which is a leading indicator for schedule reliability on the lanes that call there.
Identifier hygiene deserves attention. A vessel has an IMO number that persists for its life, an MMSI that can change, and a name that changes on sale, sometimes several times. Keying on the name produces a dataset where one ship becomes three. We key on IMO, keep the name history, and record the MMSI as an attribute rather than an identity.
Joining to shipment data is the endpoint most clients want: your container on a vessel, that vessel's position and estimated arrival, and the port's current congestion, on one row. That is a join across three sources with three provenances, and keeping them labelled is what makes it maintainable.
VesselFinder tracks ships using AIS, the automatic identification system vessels broadcast, and presents positions, movements, port calls and vessel particulars. It is one of the better known public windows onto ocean freight, which is why it is asked about so often.
Before scoping anything on this source it is worth reading its crawl rules, because they are specific. The robots file disallows the individual vessel view pages and the ship photo paths, while leaving list and index surfaces open. That is a meaningful boundary: the deepest per ship detail sits behind exactly the paths that are closed.
We treat that as scope rather than as an obstacle. What we collect here are the permitted surfaces, which still support fleet and port level work. For anything requiring sustained per vessel position history, the correct route is licensed AIS data from a provider whose terms permit it, and we will tell a client that rather than quietly crawling a disallowed path and calling it a service.
This is a case where the honest scoping conversation happens first. Plenty of vendors will promise per vessel tracking from public web pages. What that actually means is ignoring the rules the site publishes, and it is not a foundation for a product anyone intends to run for years.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Ocean visibility is a genuine commercial need. Knowing where a vessel is, when it will berth and how congested a port has become drives real decisions in supply chain planning, commodity trading and freight procurement. The demand is not in question.
What is in question is where the data legitimately comes from. AIS is broadcast openly by ships, but the networks that receive, aggregate and clean it invest in doing so and license the result. A public website built on that data is not a free tap, and its crawl rules say so. Ignoring them produces a pipeline that works until it does not, on a foundation nobody can defend in a procurement review.
So we do the boring thing. Permitted surfaces are collected from here and used for what they support: fleet composition, port association and index level movement observation. Everything requiring sustained positional history is bought from a licensed AIS provider and joined on vessel identifiers, with provenance recorded per row.
The second reason this matters is that the licensed route is usually better anyway. Continuous position history, cleaned and gap filled, is not something a web page ever contained, and reconstructing voyages from intermittent public observations produces tracks with holes that look like the vessel teleported.
The third is that a compliant design survives. Products built on disallowed paths get cut off, and the moment that happens is never convenient.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt is a managed extraction company, not a tool you have to learn. Our team builds the collection and the licensed data integration, watches both as sources change, and tells you plainly when the right answer is a licence rather than a collector.
You see a sample first, in your format, over the ports and vessels you actually care about, with provenance on every row so you can see exactly which source each field came from.
No. The site's robots file disallows the vessel view paths and we honour that. For sustained positional history the correct route is licensed AIS from a provider whose terms permit your use, which we will integrate for you. Any vendor offering per vessel tracking scraped from those pages is offering to ignore the rules the site publishes.
The permitted list and index surfaces: vessel identity and particulars as listed, port and movement information from index views, and vessel to port association, each with an observation timestamp. That supports fleet composition and port level work. It does not support continuous vessel tracks, and we say which is which.
Because continuous, cleaned, gap filled position history is not something a web page ever contained. Reconstructing voyages from intermittent public observations produces tracks with holes that look like the vessel teleported across an ocean. The licensed route gives you the underlying feed rather than a sampled view of somebody's rendering of it.
By keying on the IMO number, which persists for a vessel's life. Names change on sale, sometimes repeatedly, and MMSI can change too. Keying on the name turns one ship into three in your dataset. We keep the name history and record MMSI as an attribute rather than as identity.
Yes, as a derived measure rather than a collected field: arrivals, waiting time at anchorage and berth turnaround, tracked over time per port. It is a leading indicator for schedule reliability on the lanes calling there, and it works considerably better on licensed AIS than on intermittent public observation.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
