A data.gov.sg dataset page opens on the table itself, with column names, data types and the most common values. Our data.gov.sg scraper turns those agency tables into one scheduled feed.
Plans from €169 a month · Free project assessment · Reply within 1 business day
CSV · Excel · JSON · XML · FTP · SFTP · Amazon S3 · Google Cloud
API, JSON feed or CSV, XLSX and JSON files
ScheduleOne-time, monthly, weekly, daily or 3 times a day
data.gov.sg pages, filters, fields and a schedule
Built, monitored and repaired by our team
API, JSON feed or CSV, XLSX and JSON files
You pick the agencies, topics or single datasets - HDB for housing, URA for planning layers, NEA for hawker centres - and we build the crawler, run it on your schedule and fix it when data.gov.sg changes its pages; search results only render in a browser, so page rendering is part of the job. How to scrape data.gov.sg well is mostly a question of keys: every dataset has an id of the form d_ plus a long hexadecimal string, every collection a number, and our rows keep both, so a table stays the same table between runs. Proxy rotation and pacing are included.
Deliveries go out once, daily, weekly or on demand, as CSV, XLSX, JSON or XML, to Dropbox, Amazon S3 or your database, with field documentation.
A data.gov.sg dataset page shows the catalogue record and a preview of the table on one screen. We collect both: the metadata row and, when the project calls for it, the records themselves under the column names the agency publishes.
In the HDB resale flat prices table each row is one transaction: month, town, flat type, block, street name, storey range, floor area in square metres, flat model, lease commence date, remaining lease and resale price - eleven columns, each kept under the agency's own column name such as flat_type or remaining_lease. Other agencies follow the same pattern with their own legends, and the field documentation we ship repeats the agency's column descriptions next to ours.
Collections are where one dataset stops being enough. On data.gov.sg related datasets of a single agency form a collection, and HDB's Resale Flat Prices collection holds five tables split by period: 1990 to 1999, 2000 to February 2012, March 2012 to December 2014, January 2015 to December 2016, and January 2017 onwards. Until March 2012 transactions are dated by approval, after that by registration. Remaining Lease appears only from 2015, so the three older tables have ten columns, and floor_area_sqm is typed Numeric there but Text in the current table. We stitch the five into one HDB resale prices history with a basis column and consistent types.
Search coverage follows the site's facets - Topics, Columns, Period, Formats and Agencies. A listing card shows format, column count, agency and coverage, and collections appear as cards of their own, such as Sold and Rented HDB Properties and Facilities.
Repeated runs read Last updated and the Update history, which by the site's own note records data ingestion times but not metadata edits such as a new title, so we compare titles and descriptions between passes ourselves. Singapore company records from ACRA have a page of their own: our ACRA scraper.
Singapore's national data portal, data.gov.sg, is published by Open Government Products. Datasets are published by agencies under their own names - Housing & Development Board, Urban Redevelopment Authority, National Environment Agency, Council For Estate Agencies, Ministry of Health, Ministry of Education and dozens more - and filed under topics: Arts & Culture, Education, Economy, Environment, Geospatial, Housing, Health, Social and Transport.
Three levels organise the catalogue. An agency publishes datasets; related datasets of the same agency form a collection; each dataset is addressed by its id, while its title and coverage can change. Unlike a catalogue of links, the page shows the table itself: a data explorer with paging, a column legend and, on some datasets, a chart such as HDB's comparison of resale prices by flat type.
Cadence is stated per dataset and differs widely. HDB resale transactions are updated daily and CEA Salesperson Information three times a day; NEA's Dates of Hawker Centres Closure lists a year of quarterly cleaning dates and is still revised during the year it covers, while URA publishes amendments to the Master Plan 2019 as separate layers. That spread is why a refresh plan for data.gov.sg is set dataset by dataset.
Delivery: CSV, JSON or XLSX files, our API or a JSON feed. MCP server on request.
Full plan detailsOn data.gov.sg agencies publish tables rather than reports, and the tables are granular: one row per resale transaction, one row per registered salesperson, one row per hawker centre.
Learn how to use web scraping to solve data problems for your organization
Data scraping is the wide term that includes documents, scans and old screens; web scraping is the part that works on websites. We compare sources, daily work, output and cost drivers, give a briefing checklist and walk through two of our projects.
Leveraging advances in technology, the AI-powered web scraper has skyrocketed in demand and is helping to expand capabilities by automating tedious daily tasks and speeding up data collection from thousands of websites several times over.
Web scraping is a method of obtaining web data by extracting it from pages of web resources with the help of a program, that is, in automatic mode. It is used to syntactically convert web pages into more usable forms.
ScrapeIt is a managed scraping service. We build the crawlers, run them on the schedule you set and repair them when a source changes, then deliver the dataset with every field documented. With a portal like data.gov.sg the work sits in the joins - period-split collections, columns renamed between versions, ids that must survive a retitled dataset - and that is the part we take over. Ask for a sample on the datasets you care about. Plans start from EUR 169 a month, or EUR 199 for a one-time extract.
Still unsure? Write to info@scrapeit.io
Yes. The ScrapeIt data.gov.sg API returns dataset id, agency, coverage period, Last updated and the table rows - for HDB resales month, town, flat type and resale price - as JSON from an endpoint we host, refreshed on your schedule. Access is by a key we issue, and the same data also comes as CSV or Excel files or a JSON feed.
Yes. Dataset pages, collections, column legends and table previews on data.gov.sg are visible without logging in. We collect only publicly available data - everything a visitor can see on data.gov.sg - and we collect it legally. Where a table names individuals, as CEA Salesperson Information does, those columns are left out by default.
Yes. We combine the five period tables into one, align column titles that differ between files, such as Lease Commencement Date in the 2015 to 2016 table, add Remaining Lease where it exists and keep a column for the approval or registration basis. New months are appended on each run, since the current table is updated daily.
Each dataset states its own cadence. HDB resale prices are updated daily, CEA Salesperson Information at 7:00AM, 11:00AM and 5:00PM, and others only when the agency revises them, as with URA's Master Plan layers. We match the run to that cadence and record the Update history timestamps in SGT, so every delivery shows which ingestion it reflects.
Any agency on data.gov.sg and any of its topics, from Housing and Transport to Health and Economy. A brief can name agencies such as HDB, URA or NEA, single datasets by id, or whole collections, and map layers arrive as GeoJSON with their attributes.
You share your needs, expectations, and desired timeframe. We'll suggest the best solution based on your request and budget.
Our specialists configure the crawlers and extract a sample dataset for your review before proceeding with the full-scale extraction.
Once you approve the sample, we launch the project and start full data collection. We gather, filter, and structure the data for easy use, delivering it on time in your preferred format.
Our team manages ongoing processes, monitors website changes, and supports all data extraction cycles. We can also help integrate data into your systems or create dashboards to simplify analysis.
Tell us which sites, which fields and how often. A couple of lines is enough to start.
data.gov.sg data from €169/month. Free project assessment, reply within 1 business day.