Google News holds no articles of its own. What it holds is the ranking, and that is the only place to see whose version of a story most people actually read.
ScrapeIt runs the Google News collector as a managed service. You name the topics, search terms, countries and languages; we build the collection, run it on a sustainable cadence and hand back CSV, JSON, Excel or a push into your warehouse. Destination URLs arrive resolved, so the dataset joins straight onto any direct publisher collection you already have.
We are straightforward about the constraints. There is no article text on this service, historical ranking cannot be recovered after the fact, and the request rate has to stay moderate for access to stay stable. Anyone promising you full text or a backfilled ranking history from Google News is describing something else.
We collect what the service renders publicly, honour the crawl rules it publishes and pace requests deliberately. The headlines belong to their publishers and remain copyrighted, so the dataset is built for measurement rather than republication. Bring the use case to your own counsel before the project starts.
Each item carries the headline as displayed, the publisher name, the publication timestamp as reported, the destination URL, and the thumbnail URL where one is shown. Alongside that we record the query context: topic or search term, country edition, language, and the exact time the view was collected.
Position is the field that makes the dataset worth having. Every item records where it appeared in the result, and every collection pass is its own observation. A publisher that held the lead slot for six hours and one that appeared at position forty once are different facts, and only a positional record separates them.
Story clusters are kept as structure rather than flattened. The lead item, the related items grouped under it, the number of outlets in the cluster and the spread of their timestamps all survive into the export. That gives you the syndication group for free: one event, the outlets that covered it, and who was first.
Destination URLs need care. Google News links out through its own redirect layer, so the raw href is not the publisher URL. We resolve to the final destination and keep both, because the resolved URL is what joins this dataset to any direct collection you run against the publishers themselves.
Every row carries the collection timestamp and the edition it was collected for. Without both, a ranking row means nothing.
Country and language editions are the main axis of any serious project on this source. The same topic returns a different set of outlets and a different lead in each edition, and a brand tracked in eight markets needs eight collections rather than one with a language filter bolted on. We set them up as separate scheduled views, each tagged with its edition on every row.
Topic pages are more stable than search for long running monitoring, because a topic address stays put while a search phrase drifts with the language people use. For a fixed subject we prefer topics and keep search as a supplement for terms that have no topic of their own.
Pacing matters more here than on a publisher site. Google is aggressive about automated traffic, and the only sustainable approach is a measured request rate spread across the schedule rather than a burst. This is a case where collecting less often and continuously beats collecting hard and getting cut off; we set the cadence with that trade in mind and say so up front rather than promising a rate we cannot hold.
Historical depth is limited by the service itself. Google News is a view of what is current, and it does not offer an archive to walk backwards through. A ranking history exists only if somebody was collecting at the time, which is the argument for starting a positional feed before you need it rather than after.
Google News is an index, not a publisher. Every item on it belongs to somebody else, and the page carries only what an index needs: the headline as the publisher wrote it, the publisher name, a relative timestamp, sometimes a thumbnail, and a link out. There is no article body anywhere on the service. Anyone selling you Google News article text is selling you something else.
That sounds like a limitation and is actually the point. The value here is ordering. For a given topic, in a given country and language, Google News decides which outlets appear, which story leads, and which versions are grouped together as full coverage of the same event. That decision is invisible on the publishers own sites and it is a large part of what the public actually reads.
The service is organised into topics, each with its own stable address, plus search results and a home feed. Every view is parameterised by country and language, and the same topic returns visibly different results between editions. A story leading in one market can be absent from another, which is exactly the comparison most international monitoring briefs are trying to make.
Story grouping is the other structural feature worth having. Google clusters coverage of one event and shows a lead item with related items beneath it. Collected properly, that cluster is a ready made syndication group: one event, many outlets, in an order somebody else assigned.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Most media monitoring counts publications. Somebody wrote about us, that is one. The trouble is that a wire story republished by forty sites produces forty rows, and a piece that led the technology topic for an afternoon produces one. The count is not just imprecise, it is inverted.
Google News fixes the ordering problem because ordering is what it does. Collecting a topic on a schedule tells you which outlets Google surfaced, in what order, for how long, and which version it chose to lead with. For a communications team that is closer to the real question than any count of mentions: not how many people wrote about this, but whose version people were shown.
The clustering is the second reason. Deduplicating syndicated coverage is the hardest part of building a news dataset from scratch, and here somebody has already grouped it. We take the cluster as published, keep it as structure, and use it to cross check the deduplication we run over direct collection from the publishers.
The third is geography. Every view is parameterised by country and language, so the same topic can be collected across a dozen editions in one pass. Watching a story appear in one market before another is a genuinely early signal, and it is not available anywhere else in one place.
What this dataset is not is a source of text. There is no body here to collect, and if you need article content it has to come from the publishers, which is a separate job we usually run alongside this one.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt is a managed extraction company, not a tool you have to learn. Our team builds the collection, watches it as the service changes, and repairs it before your dashboard goes quiet.
You see a sample first, in your format, over the topics and editions you actually track. Approve it and the full run follows on the cadence we agree is sustainable.
No, and nobody can, because it is not there. Google News is an index: headline, publisher, timestamp, thumbnail and a link out. If you need article bodies they have to come from the publishers themselves, which is a separate collection we often run alongside this one and join on the resolved destination URL.
For each collection pass, which items appeared for a topic or search in a given country and language, in what order, and at what time. Repeated over a day that becomes a history: which outlet held the lead slot, for how long, and who never surfaced at all. It answers whose version of a story readers were shown, which a count of mentions cannot.
Yes, and for most briefs you should. Every view is parameterised by country and language, and results differ substantially between editions. We set each edition up as its own scheduled collection and tag every row with the edition it came from, so cross market comparison is a filter rather than a project.
No. The service shows what is current and offers no archive of past ranking, so a positional history only exists if someone was collecting at the time. This is the one source where starting early genuinely matters, and it is worth beginning the feed before the campaign rather than after it.
Google restricts automated access in its terms and is active about enforcement, so we run this at a deliberately moderate rate, honour the crawl rules the service publishes and spread requests across the schedule rather than bursting. We would rather quote a slower cadence that holds than a fast one that gets cut off in a fortnight. The headlines remain the property of their publishers, so the dataset is for measurement rather than republication, and we ask every client to have their own counsel approve the use case first.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
