US government data websites publish records that companies act on: a contract notice with a response date, a recall with a campaign number, a facility's quarter in violation, a trademark that went dead. Each site shows them through its own search form, with its own identifiers and dates.
Below: eight sources we scrape - seven federal agency sites and CourtListener, a nonprofit archive of US court records - with fields, best uses and how to join them. Other countries' registers are on our government data scraping service page.
Teams that buy federal records as data: government-contracting sales and capture teams, compliance and risk departments, legal and IP researchers, insurers, automotive analysts and ESG researchers. Each entry gives what the site is, the fields of a row, what only it adds and its best use.
All eight publish in English; amounts, where shown, are in US dollars. Four things need a rule first:
The order is usefulness to a data buyer, not a verdict on the sites: records that feed a commercial decision and join to a company come first - procurement, then rules and enforcement, patents and courts, vehicle safety, and last the catalog.
SAM.gov, run by the U.S. General Services Administration, is where federal contracting offices post solicitation, award and sole source notices. A notice row carries the Notice ID, usually the solicitation number, notice type, Department, Sub-tier and Office, set-aside, Product Service Code, NAICS code, place of performance and Date Offers Due with its time zone; award notices add the Contractor Awarded Name, its UEI and the Total Contract Value. What sets SAM.gov apart: notices change after posting, and the Updated Date carries a revision count, so each pass is compared with the last. The Exclusions domain lists firms barred from federal awards with UEI and CAGE code, a screening list for checks like those in how big data improves AML and KYC processes. Best for: bid pipelines and subcontractor screening. See our SAM.gov scraper page.
USAspending.gov, run by the Treasury's Bureau of the Fiscal Service, is the public record of federal spending: SAM.gov shows what the government intends to buy, USAspending the money once committed and paid. An award row holds award type, PIID or FAIN, Unique Award Key, awarding and funding agency, recipient name, UEI and congressional district, obligated and outlayed amounts, potential award amount, end dates, NAICS, PSC and place of performance. Only USAspending adds the chain behind a number: Transaction History lists each modification with action date, amount and action type, and recipient profiles group Child Recipients under a Parent Recipient. Best for: competitor maps by agency and code, and recompete lists from potential end dates. See our USAspending scraper page.
The Federal Register is the daily journal of US federal agencies: proposed and final rules, notices and presidential documents, and the place where public comment begins. A row carries document number, title, type, publication and effective dates, issuing agencies, regulation identifier, topics, abstract, full text, the comment opening and closing dates and where comments are submitted. Agencies come as rows, because documents are often issued jointly. The regulation identifier links the documents of one rulemaking from proposal to final rule, where matching on titles fails once an agency rewords one between stages. Best for: comment deadline monitoring for regulatory affairs teams, agency activity over time and rulemaking lifecycles. See our Federal Register scraper page.
ECHO, Enforcement and Compliance History Online, is the US Environmental Protection Agency's record of inspections, violations and enforcement under the Clean Air Act, the Clean Water Act's NPDES permits, RCRA and the Safe Drinking Water Act, with federal and state actions side by side. Its Detailed Facility Report is keyed on the FRS ID that ties a site's programme IDs together: name, address, coordinates, NAICS and SIC codes, compliance status per statute, quarters in noncompliance, inspections, formal and informal enforcement actions with lead agency, penalties assessed and collected, and Toxics Release Inventory releases. The compliance grid shows a rolling twelve quarters, so only scheduled snapshots keep a longer history. Best for: supplier and ESG screening, due diligence on plants and permit renewal leads from NPDES expiration dates. See our EPA ECHO scraper page.
The United States Patent and Trademark Office, part of the Department of Commerce, keeps its records in separate systems: Patent Public Search for patents, Trademark Search and TSDR for trademarks. A patent row carries patent number and kind code, application number, filing date, continuity data, every CPC code marked inventive or additional, abstract, numbered claims and the assignee company. A trademark row carries serial and registration numbers, literal elements, trademark status with its date, goods and services by international class, filing basis, company owner and the dated prosecution history. Inventor and attorney names stay out by default. Repeated runs catch a mark published for opposition or a registration turned dead after a missed Section 8 declaration. Best for: trademark watching, portfolio audits and patent data by assignee and CPC subclass. See our USPTO scraper page.
CourtListener is not a government website: it is a free legal research site run by Free Law Project, a US nonprofit, and the court-records source of this list. An opinion row carries case name, court, date filed, precedential status, docket number, citations and full text, with the authorities it cites and its Cited by count. A docket row adds nature of suit, cause, assigned judge and every entry with document number, date and description. CourtListener's own id and the court's docket number join the records of one case. Dockets in its RECAP Archive grow as users contribute documents, so one is complete and the next sparse, and revisits date each new entry. Private individuals named as parties stay out by default; companies stay in. Best for: litigation analytics, docket watches and company exposure checks. See our CourtListener scraper page.
The National Highway Traffic Safety Administration, part of the US Department of Transportation, sets vehicle safety standards and requires manufacturers to recall vehicles and equipment with safety defects. Each vehicle page, addressed by model year, make, model, body style and drive, counts recalls, investigations, complaints and manufacturer communications. A recall carries campaign number, manufacturer, components, potential number of units affected and remedy; a complaint carries crash, fire, injury and death fields, component and narrative, published without the filer's identity and with a partly masked VIN. A recall campaign lands on many vehicle pages, so rows are keyed on the campaign number to count it once. 5-Star Safety Ratings add frontal, side and rollover results. Best for: fleet recall checks, parts planning and insurance risk models; for asking prices of the same models see our list of US car websites to scrape in 2026. See our NHTSA scraper page.
Data.gov is the US government's catalog of datasets, run by the General Services Administration. It is a metadata catalog: states, counties and cities publish beside federal agencies, so City of Chicago crime records sit next to NOAA entries. A row carries title, catalog slug, the publisher's identifier, organization and its type, tags, its resources, Dataset Last Updated, Catalog Last Checked, Accrual Periodicity and access level. When a project needs the tables behind the records, we collect them, map each publisher's columns to one schema and keep the catalog record on every row. Repeated runs give an organization's release log of new, edited and removed datasets. Best for: following an agency's releases, freshness checks and merging one topic across publishers. See our Data.gov scraper page.
We extract the data you need from any website to satisfy all your business requirements with 100% accuracy.
One company appears under different keys across these sites, and one record can repeat within a site. Three rules:
Neither project was on a site of this list, but both problems recur on federal sites.
A US property-technology startup wanted its own copy of a public employer directory readable only through a search form, with a captcha on each new query. The run covered 17,576 three-letter searches from AAA to ZZZ and found 1,376,889 rows; merging rows with the same name and employer ID left 988,996 unique employers. A second file kept the search, page number and UTC collection time behind every row, so any record can be checked against the live search. Capped ECHO results and overlapping SAM.gov keyword searches need the same treatment: merge on the source's own id and keep the query that found each row. Case: 988,996 employer records from a captcha-gated search.
For a client contacting marketplace sellers one by one, we collected 72,961 bol.com seller shops in the Netherlands and Belgium, 63,611 of them with a Dutch KvK number. Those rows carried 55,170 distinct KvK numbers, so shops grouped by company rather than by storefront - the role the UEI plays for federal contractors. Case: bol.com sellers with KvK, VAT and address.
ScrapeIt is a managed service: we build, run and repair the crawlers, with anti-bot handling - pacing, proxy rotation, CAPTCHA handling and rendering - included. A free project assessment comes first, a sample can be ordered, field documentation comes with the dataset, and runs are one-off or on your schedule. Names of inventors, attorneys, contracting staff and private parties to cases are personal data: we handle them under GDPR and leave them out by default.
Formats: CSV, Excel (XLSX), JSON, JSONLines, XML. Channels: our API (an endpoint we host), a JSON feed, FTP, SFTP, Amazon S3, Google Cloud Storage, Dropbox, Google Drive, e-mail or a load into your database; an MCP server on request. Plans start from EUR 169 a month, or EUR 199 for a one-time extract. See also the public registers we scrape and US sites we scrape.
It depends on the decision. Bid teams start with SAM.gov and USAspending, compliance and ESG teams with the Federal Register and EPA ECHO, IP and legal research with the USPTO and CourtListener, insurers and automotive analysts with NHTSA.
Yes. We collect only publicly available data - everything a visitor can see on the site - and we collect it legally. Private persons named in these records are personal data, left out by default.
Look for a managed service that keys records on the identifiers the sources print and keeps a change history between runs. ScrapeIt builds, runs and repairs the crawlers for all eight sites above, after a free project assessment.
Yes. SAM.gov and USAspending both print the UEI, so contractors join on it. ECHO keys facilities on the FRS ID, so they join by company name and address.
Daily for SAM.gov notices, NHTSA recall watches, trademark watch lists and tracked dockets. Weekly for USAspending awards, patent portfolios and the Data.gov catalog; weekly or monthly for ECHO compliance histories.
More on getting data out of the sources we work with
Five German job sites worth scraping - the Arbeitsagentur Jobsuche, Stepstone, XING, stellenanzeigen.de and Jobware - with the fields each shows, what it adds to a dataset and how to merge boards without counting one vacancy twice.
Five UK job sites worth scraping in 2026 - Totaljobs, CV-Library, Reed, Adzuna and the DWP's Find a job: what each ad shows, what only that site adds, and how to merge them without counting one vacancy twice.
Lead scraping is a reliable way to get the right customer contacts to market your product, saves organizations time, and helps you better understand the audience you want to attract. Contacts include emails, phone numbers, or social media profiles.
Let us take your work with data to the next level and outrank your competitors.
You tell us which website(s) to scrape, what data to capture, how often to repeat etc.
An expert analyzes the specs and proposes a lowest cost solution that fits your budget.
We configure, deploy and maintain jobs in our cloud to extract data with highest quality. Then we sample the data and send it to you for review.
If you are satisfied with the quality of the dataset sample, we finish the data collection and send you the final result.
Tell us more about you and your project information.
