FDA publishes far more than a browser can carry: thirty openFDA datasets on four different hosts, each on its own clock, none of them joined for you. We take the whole load.
A scoping call answers two questions. Which sets do you need - approvals and the Orange Book, the label corpus, recall enforcement reports, the device registries, the event streams, the warning letters and inspection results that never reached the API - and what do you want them joined on: application number, NDC, set id, product code, establishment identifier or firm name. From there we set a cadence per set rather than one schedule for all, backfill from the bulk files and keep it current from the live layer.
The editorial and legacy hosts refuse plain automated clients, so anti-bot handling, proxy rotation and CAPTCHA solving are part of the service rather than a surcharge. We extract FDA data only from pages and endpoints open to any visitor, at polite rates and inside the published crawl delay. Delivery is CSV, JSON, XLSX, a database drop or an FDA API of your own, with a run timestamp and source dataset on every row.
Approvals. A Drugs@FDA record is an application: the application number carrying its NDA, ANDA or BLA prefix, the sponsor name, an array of products and an array of submissions. Each product has a product number, brand name, active ingredients as name and strength pairs, dosage form, route, marketing status and a therapeutic equivalence code, plus reference drug and reference standard flags. Each submission has a type of ORIG or SUPPL, a number, a status and status date, a review priority, a class code such as Labeling or Efficacy, property codes such as Orphan, and application documents pointing at the approval letter and label PDFs. The Orange Book repeats the product with its equivalence codes, application type A or N and the approval date.
Labeling. A label record is a Structured Product Label: a set id that survives every revision, an id and version for the specific submission, an effective time, and named sections rather than free prose - boxed warning, indications and usage, dosage and administration, contraindications, warnings and cautions, adverse reactions, drug interactions, clinical pharmacology, use in specific populations, how supplied, the medication guide and the principal display panel. Consumer products carry their own set: purpose, do not use, stop use, keep out of reach of children.
Recalls. An enforcement report carries a recall number, an event id grouping products in one action, a classification of Class I, II or III, a status, the recalling firm and its full address, the reason for recall, the product description, code info such as lot and expiry, product quantity, distribution pattern, and four dates: initiation, classification, report and termination.
Devices. A cleared 510(k) is a K number with applicant, address, country code, device name, three-letter product code, advisory committee, clearance type, dates received and decided, and a decision code. A PMA adds supplement number, type and reason. A unique device identifier record is the deepest: primary and secondary identifiers with their issuing agency, brand name, version or model, labeler DUNS, GMDN terms with code and definition, product codes carrying regulation number and device class, sterilization and MRI safety, and flags for prescription, over the counter, kit and single use.
The enforcement record is the half that never made it into the API. Warning letters sit in a filtered table with posted date, letter issue date, company name, issuing office, subject and links to the response and closeout letters; the table ran to roughly 3,700 rows in September 2026. Each letter page adds a MARCS-CMS number, a reference number such as 320-26-120, the delivery method, the recipient name and title, the ten-digit establishment identifier of the inspected site, the inspection dates and the cited parts of 21 CFR. Trade secrets are blanked in place as (b)(4), so the text is usable but gapped.
Inspection outcomes sit on the compliance dashboard, refreshed weekly and limited to final actions. Each row is a project area within an inspection, classified No Action Indicated, Voluntary Action Indicated or Official Action Indicated, so one visit can produce several classifications. Citations are published separately, keyed to the inspection and the establishment identifier, alongside Form 483 observations, compliance actions such as injunctions and seizures, and import refusals by country, port and product code. That dashboard has its own API, but it is a POST service behind an issued authorization key, capped at five thousand rows per response.
Around the edges sit the sets people forget: complete response letters as OCR text of scanned PDFs, drug shortage entries with availability status, the national drug code directory and its NSDE package companion, UNII and substance records, tobacco product problem reports, cosmetic and food adverse events from CAERS, and animal drug events.
FDA is the United States Food and Drug Administration, and its public record lives on four hosts that behave nothing alike. www.fda.gov is the editorial site, organized by eight regulated product areas - food, drugs, medical devices, radiation-emitting products, vaccines blood and biologics, animal and veterinary, cosmetics, tobacco - and carrying the warning letter and recall listings as filtered tables with an XLSX export. accessdata.fda.gov holds the older search applications: Drugs@FDA, the Orange Book, the 510(k) and PMA databases, device classification, registration and listing, MAUDE. open.fda.gov and its api.fda.gov endpoints are the machine-readable layer, an Elasticsearch service publishing thirty datasets in nine families. datadashboard.fda.gov is the compliance layer: inspections, citations, compliance actions, import refusals.
Addressing differs by host, and that shapes any FDA scraper. A warning letter is the company slug, the MARCS-CMS case number and the issue date, as in /warning-letters/jabil-inc-731037-08272026. accessdata pages are ColdFusion scripts driven by form parameters - KNumber, ProductCode, Applicant, DecisionDateFrom, Panel, Decision - so a device query is a form shape, not a path. openFDA is one endpoint per dataset, https://api.fda.gov/drug/label.json or /device/510k.json, with search, sort, count, limit and skip as the only verbs.
Nothing sits behind a login. robots.txt sets a crawl delay of thirty seconds for every agent except the two the agency names for its own index, closes /search/, /node/, /file/, /taxonomy/ and the account paths, and points at a sitemap index fanning out into dozens of child files. The terms attached to openFDA put the data in the public domain under a CC0 dedication, commercial use included.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Nothing here is locked, which is exactly why it gets underestimated. The public API answers 240 requests a minute from one address and stops at a thousand a day; a free key raises the daily ceiling to one hundred and twenty thousand. A single response is capped at a thousand records, skip stops at twenty-five thousand, and together they cap paging at twenty-six thousand hits. Past that you either follow the search_after cursor in the Link header, a page at a time, or take the bulk files - about 2,400 zipped JSON parts, roughly 134 GB compressed.
The clocks are the second trap. Approvals and the drug directory refresh on working days, labels, recalls and device event reports weekly, clearances and classifications monthly, and the drug, food and animal event sets quarterly with a stated lag of three months or more. Scrape FDA sources on one schedule and half the feed is stale while the other half re-downloads unchanged rows. Coverage windows differ too: approvals reach back to 1939, device clearances to 1976, recalls and drug adverse events to 2004, unique device identifiers to 2013.
Then the joins. The harmonized openfda block carrying the application number, NDC codes, UNII, RxCUI and pharmacologic class is attached only where the agency could match it: about a third of label records have it, and fewer than one drug recall in five. Everything else has to be matched on firm name, product description and lot text. There is no FDA price field anywhere in these sets either - money is a dimension you bring yourself, keyed on NDC or application number.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt runs the whole job as a managed service. We design the collectors, schedule them per dataset, monitor the runs, and repair them when a table, a form or a field name changes. Your side receives files or an endpoint and keeps no scraping engineers of its own.
We are not affiliated with FDA or any part of the Department of Health and Human Services. Nothing behind a login is touched. Adverse event reports describe patients only in de-identified form - age, sex, outcome, coded reaction - and we work with those public sets exactly as published and never try to re-identify anyone.
Yes, and it is genuinely open: openFDA serves thirty datasets in nine families as JSON, no key needed to start, and the same data is published as zipped bulk files. What it does not carry is the enforcement narrative. Warning letters, Form 483 observations, inspection classifications, citations, import refusals, guidance documents and import alerts are not openFDA endpoints - they live on the editorial site and on the compliance dashboard, whose own API is a POST service behind an issued authorization key. We cover both layers and hand them over on one schema.
Each set has its own clock. Approvals and the national drug code directory refresh on working days; labels, recall enforcement reports, device adverse events, device recalls and unique device identifiers weekly; 510(k) clearances, premarket approvals, classifications, registration and listing and the Orange Book monthly; drug, food and animal adverse events quarterly, with the agency itself warning that the drug event set can lag a quarter or more. We poll each one on its own cadence and stamp every row with the export date it came from, so a diff between two runs is real movement and not re-delivery.
Identifiers first, because they are what make the sets one graph: application number, product and package NDC, SPL set id and version, UNII and RxCUI, product code, K number, PMA number, device identifier with its issuing agency, recall number and event id, establishment identifier. Then the payload - ingredients and strengths, marketing status, therapeutic equivalence codes, label sections, recall classification and reason, decision codes and dates, GMDN terms. Where the harmonised block is missing, and on labels and drug recalls it usually is, we match on firm, product description and lot text and flag the confidence.
Usually you want both. The bulk export is the sane way to build history - roughly 134 GB of compressed JSON in about 2,400 parts, quarterly slices for the event sets and fixed-size slabs for the registries - and the live endpoints are the sane way to keep it current inside the daily quota. We normalize both into one schema and deliver CSV, JSON, XLSX, Parquet, a database drop or a hosted endpoint. Nested arrays such as products, submissions, reactions and identifiers can be shipped nested or flattened into child tables, whichever your warehouse wants.
The agency states that its openFDA content is public domain under a CC0 dedication and may be copied, modified and used commercially without permission, with attribution requested rather than required. Two caveats we always raise: the GMDN device terms inside the identifier set are licensed from a third party and need their own licence for commercial reuse or model training, and the agency asks that nothing imply its endorsement. On people: the public event sets are de-identified by design, we take them as published, and we do not attempt to re-identify anyone.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
