Dice data scraping, Indeed job search, Dice data extraction, Dice automation, website scraping, web scraping, job data, labor market data, employment data, recruiting, staffing
You give us the scope: keyword and location hubs, specific advertisers, an employerType or contract-type filter, or a list of job-detail guids you already hold. We build the crawler, run it and keep it working when Dice changes its markup.
Output arrives as CSV, JSON, XLSX or a hosted API, on a schedule you set. Daily runs suit contract inventory, where postings turn over quickly; weekly is usually enough for direct-hire tracking. Runs are delivered with new, changed and disappeared records marked, so you can load deltas instead of rebuilding a table, and every row keeps its guid as a stable join key across runs.
Search responses and job-detail pages expose different amounts of the record, so we crawl both. The search layer returns a summary truncated at roughly 500 characters; the full description and the skills array exist only on the detail page, which also carries a schema.org JobPosting block.
Skills are the field that makes Dice worth its own pipeline. A poster types skills as a comma-separated list and can also accept skills that Dice suggests from the job description, and Dice weights the skills field above skills mentioned only in the body text. The result is an array attached to each posting, rendered under a Skills heading on the detail page.
It is not a clean controlled vocabulary. The array mixes canonical expansions such as "Cascading Style Sheets" and "Amazon Web Services" with raw poster strings such as "CRM SME". Abbreviated and spelled-out forms of the same technology can appear in one array, and terms such as "Immigration" ride along from legal boilerplate. We deliver the array verbatim plus a deduplicated, mapped version so counts are usable.
Company profile pages are rendered in the browser rather than in the HTML, so they need a different collection path. Dice also publishes an annual Tech Salary Report built from a survey of its registered users, with cuts by occupation, skill and metro; that is survey data, separate from the per-posting salary string.
On personal data: staffing postings on Dice regularly carry a named recruiter and a contact address inside the description. Our output is limited to commercial listing information, and named contact details can be excluded at the field level on request. Several US states, California among them, regulate personal information under their own statutes, and we leave the legal assessment of your use case to your counsel.
Dice is a United States job board for technology and engineering roles, operated by DHI Group (NYSE: DHX). DHI also runs ClearanceJobs, its security-cleared sibling brand, and in 2021 transferred majority ownership of eFinancialCareers to that unit's management team. Dice stays narrow: the listings are technical, the readership is technologists, and the site is organized around skills as much as job titles.
Two URL shapes matter for collection. A posting lives at dice.com/job-detail/{guid}, where the guid is a 36-character UUID and the only identifier present in the public URL. Search hubs live at /jobs/q-{Title}-jobs and /jobs/q-{Title}-l-{City,+ST}-jobs, and employer pages at /company-profile/{uuid}. At the time of writing the title sitemap listed 5,132 title hubs and the two title-plus-location sitemaps listed just under 65,000 more. No sitemap lists individual job URLs, so postings have to be discovered through search.
The advertiser mix is the part buyers usually underestimate. IT staffing and consulting firms carry a heavy share of Dice inventory, and the site records this in a field named employerType whose values are Direct Hire, Recruiter and Other. That single field separates a company hiring for itself from an agency filling a client requirement, which is why Dice is the practical place to measure US contract tech demand.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Because contract work is a first-class field rather than a keyword guess, Dice supports questions other US boards cannot answer cleanly. You can size the contract-versus-permanent split for a given skill and metro, then split the contract half again into W2, corp-to-corp and independent using the contract types array.
The employerType field turns the staffing market into a measurable one. Filtering to Recruiter isolates agency inventory, which is how bench sales teams find live requirements, how staffing firms watch competitors, and how vendor-management buyers see which suppliers are working which accounts. Filtering to Direct Hire leaves the companies hiring for themselves.
Repeated crawls make the movement visible. postedDate and modifiedDate together separate a genuinely new requirement from a refreshed one, which matters on Dice because agencies re-post aggressively and the same role can surface under several advertisers. Tracking skill strings over months gives demand curves per technology, and pairing skills with the salary text gives rate benchmarks by stack rather than by job title alone.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt is a managed web scraping agency. We are not a library or a self-serve tool: we scope the job with you, build and host the crawlers, monitor them, and hand over clean data on an agreed schedule. Maintenance is part of the arrangement, so a layout change on the source site is our problem to fix rather than yours to debug. Send the fields and the coverage you need and we will confirm what is collectable before any work starts.
Yes. The skills array sits on the job-detail page under a Skills heading, not in the search results, so we visit every posting individually. We return the array exactly as Dice publishes it and a cleaned version beside it, because the raw list mixes canonical names like "Cascading Style Sheets" with poster-typed strings and with terms pulled out of the description boilerplate.
Dice carries a field called employerType with three values: Direct Hire, Recruiter and Other. It is also a search facet, so we can either collect it as a column on every row or restrict the crawl to agency postings from the start. It is the most reliable agency marker available, though the company name and the description still help where an advertiser is set up as Other.
Yes, and it comes from a real field rather than keyword matching. Dice stores employmentDetail.contract.types with values CONTRACT_W2, CONTRACT_CORP_TO_CORP and CONTRACT_INDEPENDENT, and a posting can list more than one. The same object holds a free-text term such as "06 Months" or "12 Months", which is null when the advertiser left the contract length blank.
Dice used to publish a jobs API, but it was shut down in October 2017 and the old endpoint returns 404 today. The integrations Dice documents now run the other way: they let employers and ATS vendors push postings into Dice, not pull listings out. Collection is therefore a crawl, and the crawl has a shape. One search query is capped. Dice fixes the page size at 30 regardless of what you request, and asking for a page beyond 25 returns page 25 again, so a query yields at most 750 records even when the result header reports several thousand. We work around it by splitting the scope into narrower slices - by skill, by city, by posted-date window, by employment type - and deduplicating on guid afterwards.
The URL keeps returning HTTP 200 with a page titled "Job Not Found" rather than a 404, so status codes alone will not tell you a role has closed. We detect the not-found state in the body and mark the record as gone, with the last successful capture retained. The JobPosting markup also carries validThrough, which gives an expected expiry date while the posting is still live.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
