Careerjet Salary Data, Duplicates and the Ninety-Day Clock
Be honest about salary before you buy it. Careerjet computes no pay estimate of its own; it republishes whatever the source ad stated. So the salary column is filled on a minority of rows, and there it can be a single figure, a range, or a rate per hour, day, week, month or year. The structured data behind each ad splits that into a currency code, a minimum, a maximum and a unit, which is what makes Careerjet salary data comparable at all - but an average from it describes advertisers who publish pay, not the market. We deliver the parsed figure, the raw string and a flag for ads that carry nothing.
Duplication here is structural. One vacancy reaches the index from the employer site, again through the applicant tracking system, and again through a board that syndicated it, and each arrival becomes its own record with its own id and source host. Matching on the ad address never collapses them. What works is employer and title after normalisation, location, posting date and a fingerprint of the description body, which the source supplies almost verbatim. We keep every raw record and add a group id, so you can count ads, count vacancies, or study the gap.
The validity date is a trap worth naming: it sits exactly ninety days after the posting timestamp, a fixed window rather than a closing date supplied by the employer. Real disappearance is measured by revisiting ids already seen. Careerjet company pages are filtered job lists with a count and a follow button, not employer profiles - no ratings, reviews or pay pages. Recruiter names and contact details are personal data and not part of a standard delivery.