Indeed is a job board and an aggregator at once, so one real vacancy can appear several times. Our Indeed scraper collects the postings, groups the duplicates and returns clean rows.
Plans from €169/month · Free project assessment · Reply within 1 business day
Recruitment agencies and staffing firms use deduplicated Indeed jobs to find employers who are hiring now and to see which roles they cannot fill. HR and compensation teams use employer-stated pay, filtered away from Indeed estimates, to benchmark offers by title and metro. Market researchers read posting volume by occupation as a demand indicator. Sales teams treat a burst of hiring in a function - engineering, warehouse operations, field service - as a buying signal and build territory lists from it.
Each of these needs a different shape of the same crawl. Benchmarking wants employer-stated pay only. Demand tracking wants the deduplicated count. Lead generation wants the raw employer list with contact-free company detail. We agree the shape before we start scraping.
Every posting we pull carries the fields you would expect, plus one that most datasets leave out: a flag recording where the salary came from.
The salary flag matters more than it looks. Indeed shows an employer-stated figure when the employer supplies one, and its own estimated range when they do not. The estimate is derived from self-reported pay and from comparable roles in the same location, and it disappears once the employer adds real pay information. Averaging the two together yields a number that describes neither the employer nor the market, so we keep them in separate columns and let you decide which population you are studying.
Company pages are a separate object from postings. Indeed maintains an employer profile with a rating, a review count, salary pages and question threads, and it sits under its own /cmp/ path rather than inside the job record. We can collect employer profiles alongside postings and join them on the employer, but the two move on different clocks - a rating shifts slowly, a posting list changes daily - so we schedule them as separate runs.
Postings expire, and that is easy to miss. An expired posting generally remains addressable by its job key and returns a notice in place of the description, so a crawl that only follows live search results quietly loses the record of what closed. We keep a register of every key we have seen and revisit it, which turns a vanished row into a close date you can actually analyse.
Over several months the useful movement appears: posting volume by metro and by occupation, how long a title stays open before it disappears, how often an employer reposts the same role, whether employer-stated pay drifts, and the seasonal shape of hiring in retail, logistics, healthcare and education. None of that is visible in a single snapshot, which is why most briefs here end up on a schedule rather than as one export.
The delivery stays inside posting and employer data. We do not collect candidate resumes or job seeker profiles. Where an employer has typed a recruiter name or an email into a description, those details can be excluded on request.
Indeed runs two businesses on one surface. Employers post jobs into it directly, and it also indexes postings from employer career sites, applicant tracking systems and other boards. That second channel is why one real vacancy can appear several times under slightly different titles.
A posting lives at a viewjob URL keyed by the jk parameter, a hexadecimal job key. The key identifies one Indeed record, not one vacancy, so a role arriving both as a direct post and as an ATS feed carries more than one key. Search runs at /jobs with the query in q and the location in l, and pages advance through a start offset in steps of ten. Paging is bounded, so a broad query cannot be exhausted by turning pages; wide briefs get split by title, location and date window instead.
Indeed also operates national sites - indeed.de, indeed.fr, indeed.co.in among them. These are separate result sets with their own locale, currency and employer mix, not translations of one index, so a multi-country brief is several crawls and the merge afterwards is the real work. Indeed was founded in 2004, is part of Recruit Holdings, and carries listings in more than 60 countries and over two dozen languages.
Indeed is defensive about automated traffic, so pacing, rendering and proxy rotation are built into every crawl we run - that handling is our job, not yours. What comes out is every public posting field: title, employer, location, salary, job type, description, posting date and the jk key.
Get a Quote
Developers
Customers worldwide
Pages extracted
Hours saved for our clients
€199 / one-time
setup fee - included
€169 / mo
setup fee €499
€229 / mo
setup fee €499
€349 / mo
setup fee €499
€549 / mo
setup fee €799
Delivery: CSV, JSON or XLSX files, our API or a JSON feed. MCP server on request.
Full plan details →Ask Indeed for warehouse roles in a metro area and you get a list. Count the list and you have counted records, not openings. The same job arrives at the index directly from the employer and again through an ATS feed or a partner board, and each arrival receives its own key.
Matching on URL does not solve this. Two records for one vacancy hold two different jk values, so a URL comparison calls them distinct and the posting count inflates. The workable key is the employer, the normalised job title, the location and the first-seen posting date. Titles need cleaning before that comparison works - requisition numbers, appended city names, punctuation, seniority words in inconsistent order. Locations need the same treatment, because one employer writes the metro and another writes the suburb.
We keep both layers in the output. Every raw record survives with its own key, and a group id marks the records we judged to be one vacancy. You can count openings, count listings, or look at the gap between the two, which is itself a signal about how an employer distributes its jobs.
Result order is not a second signal. Sponsored placement is bought and sits above organic listings for the same query, so rank reflects bids rather than demand. We record whether a posting was sponsored and where it sat, and we do not treat the ordering as evidence about the labour market.
Learn how to use web scraping to solve data problems for your organization
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
If you specialize in machine learning, you need to feed large amounts of data to the algorithms. Web scraping is the easiest and the most efficient method of collecting the data from all over the Internet.
ScrapeIt is a managed service, not a tool you install. You describe the roles, locations and countries you care about; we build the crawler, run it on your schedule, watch it when Indeed changes its markup, and deliver CSV, JSON, XLSX or an API endpoint.
You see a sample before the full run. Field names, the dedupe rule and the salary split are agreed on that sample, so what arrives in month three matches what you approved in week one. We also handle the rework when a layout change breaks a selector.
Yes. The ScrapeIt Indeed API returns the jk job key, job title, employer, location, salary with its salary_source flag and posting date as JSON from an endpoint we host, refreshed on the schedule you set. The same data also comes as CSV or XLSX files or straight into your database.
Yes. We can extract Indeed data filtered by industry, role, region, employer or keyword, and we capture everything visible in a listing - the full job description, employer name, posting date and the surrounding metadata. Filters are applied at crawl time where Indeed supports them and in post-processing where it does not, so you receive the slice you asked for rather than a whole country dump you have to cut down yourself.
Both are a matter of schedule. Runs can be one-off, monthly, weekly, daily or several times a day, and each row carries the timestamp of the run that produced it, so you always know how current a figure is. For employer monitoring we hold a watchlist of companies and revisit them on that cadence, reporting new postings, closed postings, changes to job descriptions and shifts in hiring volume. Because we keep the job keys we have already seen, closures show up as a dated event rather than as a row that silently disappears.
No. Output is limited to posting and employer data. Candidate resumes and job seeker profiles sit behind a login, so we do not collect them and we do not offer them. Where an employer has typed a recruiter name or an email address into a job description, those personal contact details can be stripped from the delivery on request. Posting and employer records are business information, but if any personal data does end up in scope for your use case, the GDPR applies to how you process it and we will configure the extraction to match the basis you are relying on.
Other sites too. The same pipeline runs against eCommerce catalogues, real estate portals, automotive marketplaces and company directories. Indeed sits at the harder end of that range because of the duplication and the sponsored placement, so the machinery around it - change detection, scheduled reruns, field-level validation - carries over well to simpler sources.
Step 1 - Make a Request
You share your needs, expectations, and desired timeframe. We’ll suggest the best solution based on your request and budget.
Step 2 - Configuring Custom Web Crawlers
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Step 3 - Collect and Deliver
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Step 4 - Maintain and Support
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Indeed data from €169/month. Free project assessment, reply within 1 business day.
Scrapeit Sp. z o.o.
10/208 Legionowa str., 15-099, Bialystok, Poland
NIP: 5423457175
REGON: 523384582
