The Entire Iherb Supplements Catalog, Captured End-to-End in 3 Days

A client came to us needing a complete dataset of the Supplements category on iHerb.com. Not a sample and not a subset - the full catalog, with every unique attribute a product page exposes and the description text in full, including the details that distinguish one packaging size from another. The scope was defined by category boundary rather than by row count, which changes how the project has to be planned from the first day.

Sources: iherb.com
Coverage: all products in the Supplements category
Volume: 250,000 rows
Setup and launch: 3 working days

Why "all unique details" is the hard part of a supplements catalog

The client's requirement was explicitly to capture what makes each entry unique, plus the description in its entirety rather than a truncated preview. That rules out the common shortcut of taking whichever fields are easiest to reach and calling the result a catalog.

Full description text also has to survive extraction intact. Capturing the first paragraph and dropping the rest produces a file that looks complete in a spreadsheet preview and turns out to be useless the moment someone tries to work with it.

Packaging variations are separate products, not a footnote

The most consequential decision in this project was how to treat packaging sizes. The same supplement exists in several packaging sizes, and each of those variants is a distinct thing on the shelf: its own identity within the catalog, its own place in the assortment.

The tempting simplification is to collapse all variants into one row per product and store the sizes as a list in a single cell. It reduces row count and makes the file look tidier. It also destroys the dataset. Once variants are flattened, you can no longer compare like with like, and any downstream work has to reverse-engineer the collapse from a text field.

The client asked specifically for information on the different packaging sizes, so the structure was built around variants from the start rather than retrofitted afterward.

What three working days to launch actually means

Setup and launch took 3 working days. It is worth being precise about what that figure covers, because it is easy to misread. It is the time from the start of the project to a running scraper producing the agreed structure.

The point is speed of reaching volume, not the volume itself. A quarter of a million rows is a substantial dataset, but the size alone is not the achievement. The relevant question for a client is how long they wait between approving a spec and having something they can use. Compressing the setup phase is what makes a catalog-wide scope practical.

What the client ended up with

The result is a dataset of 250,000 rows covering the entire Supplements category on iHerb.com, with the unique details of each product, the full description, and packaging variations covered rather than merged away. The client received a catalog they can work with directly - complete against the category boundary they defined, and delivered from a scraper that was set up and launched within 3 working days.