DuckDuckGo stitches Bing links, its own DuckDuckBot crawl and Instant Answers from over 100 sources into a page that tracks nobody. We turn that page into rows by region and keyword.
ScrapeIt builds the DuckDuckGo scraper, hosts it and answers for its output. DuckDuckGo scraping fails quietly: the site screens incoming traffic for malicious bots, and a collector that only asks whether a page loaded will happily store result pages with the organic block missing. We check every capture for substance instead: organic rows present, the right kl region on the request, Instant Answer, Search Assist and ad blocks marked present or absent. Anti-bot handling, proxy rotation and CAPTCHA solving are included, with egress in the country each region belongs to.
The first delivery is a sample for your sign-off. Production runs follow on the cadence you choose, from hourly checks during a launch to a monthly market sweep.
We extract DuckDuckGo data block by block, because each block has its own supplier and its own shape.
Every row also carries the query, kl region, safe search level, date filter and capture time, since those inputs decide what the page shows.
The one official API DuckDuckGo has offered is the Instant Answer API on api.duckduckgo.com, and its documentation called it an API for some Instant Answers, not for full search results. It returned a topic summary with its source, a definition, a direct answer with its type, related topics, official site links, a type code and the redirect for bang queries, as JSON or XML. Organic links, ads, news, images and videos were never in it, and most deep queries came back blank. The DuckDuckGo API documentation page has since been taken down; it gave no number for a DuckDuckGo API rate limit, only automatic throttling and higher limits on request.
What passes for a DuckDuckGo search API Python package is duckduckgo_search, a DuckDuckGo scraper GitHub project that says it is not affiliated with DuckDuckGo and is for educational use only. Renamed ddgs, it is now a metasearch library whose default auto backend queries several engines and re-ranks the merged list itself, and that is what LangChain's DuckDuckGo tool runs on.
DuckDuckGo bangs change what a query is. A query carrying a site bang such as !w or !a never produces a DuckDuckGo result page; it redirects to that site's own search, adjusted to the chosen region, and a leading backslash jumps straight to the first result. The directory held 13,566 bangs at the end of September 2026, so keyword lists are screened for them before a run.
DuckDuckGo is a private search engine run by Duck Duck Go, Inc., an independent company majority-owned by founder Gabriel Weinberg and team members. A DuckDuckGo scraper has to follow how the company itself describes the page, because the page is assembled from several suppliers. Traditional links and images are largely sourced from Bing. DuckDuckGo also runs its own crawler, DuckDuckBot, maintains many indexes, and fills DuckDuckGo Instant Answers from over 100 sources, Sportradar for scores and Wikipedia for topic summaries among them. Apple supplies maps, Microsoft Advertising most ads, Tripadvisor hotel ads and Yelp local business ads. Google is not a source.
DuckDuckGo describes a result page as three sections: Instant Answers, usually on top; organic links with image, news and video modules woven in; and sponsored links. Search Assist, an AI answer that cites one or two sources, appears as often as the visitor's setting allows. The main page runs on JavaScript: its HTML carries no organic links, and a script on links.duckduckgo.com fetches them with a per query token, vqd, passing the region on as a Bing market code.
For browsers without JavaScript DuckDuckGo keeps two lighter versions, DuckDuckGo HTML and DuckDuckGo Lite, with fewer features. DuckDuckGo's robots.txt disallows both paths on duckduckgo.com, /html and /lite, and query strings on links.duckduckgo.com, where the JavaScript page gets its links; the html and lite subdomains stay open to crawlers so that their noindex tags are respected. A DuckDuckGo HTML scraper does not step around any of this. Two more hosts change the page: noai.duckduckgo.com drops Search Assist, Duck.ai prompts and AI images, and safe.duckduckgo.com locks safe search to strict.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Put DuckDuckGo results vs Google side by side and the inputs differ before the ranks do. DuckDuckGo keeps no search history and bases ads on the page being viewed rather than on a profile, so one capture for a query, region and safe search setting is close to what every visitor there sees, shifted only by the approximate location guessed from the connection. That is a clean baseline for share of voice.
Supply is the second difference. Most organic links come from Bing, so a DuckDuckGo position reads best beside the Bing position in the matching market, and the gap between the two is the signal: Instant Answers on top, DuckDuckGo's own indexes, its news quality factor and spam demotion all move links. DuckDuckGo is also literal. It says it tries to be very exact where other engines rewrite the query, so misspellings and variants need rows of their own. It prints no result count either, so competition is measured from the rows, not from an estimate.
Third, nothing else reports it. DuckDuckGo passes no search terms to the sites people click, offers no webmaster console and sends even removal requests to Bing. Search Assist adds a second layer: DuckAssistBot fetches its sources in real time, and a publisher can block it without any effect on organic rank, so a brand can hold the first organic slot and still be missing from the answer. A DuckDuckGo search scraper is the only record of either. Personal data stays out: no names, contact details or anything else that points to a private individual.
Learn how to use web scraping to solve data problems for your organization
The use of sentiment analysis tools in business benefits not only companies but also their customers by allowing them to improve products and services, identify the strengths and weaknesses of competitors' products, and create targeted advertising.
Using web scraping to extract Google Maps data will help you quickly and efficiently find businesses in any industry, city, state, or region. And the extracted contact information, such as phone or email address, social networks, or links to web pages, will help to contact them.
Lead scraping is a reliable way to get the right customer contacts to market your product, saves organizations time, and helps you better understand the audience you want to attract. Contacts include emails, phone numbers, or social media profiles.
ScrapeIt is a managed web scraping agency. Our engineers write each crawler, host it, fix it when DuckDuckGo changes a layout, and deliver files your analysts can open on day one. There is no code for you to maintain and no proxy pool for you to rent.
Because DuckDuckGo's organic links are largely Bing's, we tell you up front when a Bing feed in the same market already covers the question, and when only the DuckDuckGo page, with its Instant Answers and its own ad load, will do.
Not for search results. There is no DuckDuckGo search engine results API, free or paid: DuckDuckGo wrote in its API documentation that the way it builds results leaves it without the rights to syndicate them. Its one official API, the Instant Answer API, was free for non-commercial use with attribution, needed no DuckDuckGo API key and returned only topic summaries, definitions, direct answers, related topics, official site links and bang redirects, never organic links, ads, news, images or videos. There is no DuckDuckGo API pricing either, because there is nothing to buy.
Mostly Bing, plus its own. DuckDuckGo states that most organic links and images are sourced from Bing, that it runs its own crawler, DuckDuckBot, and many indexes, and that Instant Answers come from over 100 specialized sources. It does not use Google; YouTube videos reach the page through indirect sources. Apple supplies maps and Microsoft most ads. In practice a removal granted by Bing flows through to DuckDuckGo, and DuckDuckGo and Bing positions for the same market are worth tracking together.
Region is set by the kl parameter, written country first and language second: us-en, uk-en, de-de, fr-fr, and wt-wt for all regions. The codes are DuckDuckGo's own, uk rather than gb for Britain and xa-ar for Saudi Arabia, and several countries split by language, such as be-fr and be-nl or us-en and us-es. On the organic request each region travels as a Bing market code, de-de as de-DE and uk-en as en-GB, while wt-wt still asks for en-US. We take codes from the live region picker, which already differs from the older help page list, and run each region as its own job.
We load the page as a visitor in that region would, with egress in the matching country, because DuckDuckGo guesses location from the connection for local results and ads. Each keyword runs per kl region, safe search level and device, with ads left on and the Search Assist setting recorded. Files arrive as CSV, JSON or XLSX, or through an endpoint you query, landing in S3, on SFTP, behind a webhook or in your warehouse. The default table has one row per block element, keyed by capture time, keyword, region and rank.
That depends on where you operate and what you do with the data, and we are not lawyers. DuckDuckGo's robots.txt disallows /html and /lite, its no-JavaScript result pages, and allows /search.html, which is only the embeddable search box. Its terms tie access to authorized use, its URL parameters are meant for individual use, its acceptable use policy bars reselling any portion of the service, framing it or disrupting it, and DuckDuckGo has said it lacks the rights to syndicate or frame its results. We collect only what any visitor sees and exclude personal data, and the scope is agreed with your counsel before the first run.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
