Extract UNIQLO catalogue, pricing and inventory data

Fields extracted from UNIQLO

Fields keep UNIQLO's own naming, so the output maps back to what the storefront returned, not to a generic apparel schema. The set below is the usual starting point.

• Identifiers - productId, priceGroup, l1Id for the listing row, l2Id for each colour-size SKU, and the communicationCode in goods-colour-size-length form.
• Taxonomy - name and breadcrumbs split into gender, class, category and subcategory, mapped to the category IDs that storefront uses. Men is 22211 on /us/en, 37268 on /uk/en and 1071 on /jp/ja, so each market needs its own map.
• Pricing - prices.base and prices.promo, the currency object with code and symbol, and taxPolicy - EXCLUSIVE on the US site, INCLUSIVE on the UK and Japan sites. Listing responses return base equal to promo, so the pre-discount reference comes from the per-product l2s call.
• Variants - colors with COLxx code and displayCode, sizes with SMAxxx, and PTBxxx length codes where a garment carries them.
• Availability - stocks.statusCode and quantity per colour-size row, so a sold-out size inside a live colour stays visible rather than collapsing into one flag.
• Flags - flags.productFlags for badges such as Limited Offer and Online Exclusive, and flags.priceFlags for discount, Multi-Buy, appmemberLimited and colorSizeLimitedPrice, each with effectiveTime stamps.
• Garment detail - composition, washingInformation / careInstruction, description copy and image sets per colour - the fields that carry weight when comparing HEATTECH, AIRism, Ultra Light Down, UT Graphic Tees and Uniqlo U.
• Reviews - rating.average, rating.count and rateCount, plus review records with the Bazaarvoice ID, fit score, purchasedSize, purchasedColorName, heightRange, weightRange, ageRange and a Unix createDate.
• Store level - Find in Store / Click & Collect availability from the store and storeStock branches, where a storefront exposes it. StyleHint and StylingBook content carries the same codes and can be added.

What scheduled crawling adds

One export of uniqlo.com is a catalogue. The commercial value sits in the difference between two of them.

UNIQLO reissues goods numbers between seasons, so an E-code is not a stable key across years. A retired item does not disappear: /us/en/products/E445392-000/00 resolves to /us/en/products/E473944-000/00, and the details response marks the old record with productLifeCycleData.isDiscontinuedProduct and names the replacement in nextModelProducts. There is a query parameter that suppresses that redirect, but it is Disallowed in robots.txt, so we read the chain from the redirect itself. Stored run after run, that chain keeps one continuous price and stock history for a garment instead of a new product appearing every autumn.

Repetition also exposes timing. Discount windows carry effectiveTime start and end stamps, so scheduled runs record when a promotion opened, how deep it went, which colour-size rows it covered and whether it came back. Stock deltas between runs show sell-through, restocks on sizes that were dark for weeks, and the point where a colour is quietly withdrawn. Coverage moves too: UNIQLO's own US sitemap listed 1,935 product URLs on a recent read, and a share of those already answer with the API's 'Product states or representative L2 states is invalid' error, while codes that never existed return a real 404. Telling a withdrawn line apart from one that never launched needs history.