Yandex Market Scraping: Cards, Offers and Prices

Fields a Yandex Market scraper returns

A row is useful only when it says which level it describes, so we extract Yandex Market data with every identifier kept on the row and every price labelled with its type.

• Identifiers - modelId, marketSku, oskuId and wareId, the canonical card and model URLs, and the numeric hid and nid of the category the item sits in.
• Merchant - businessId and businessName, shopId, supplierId and supplierType, plus shopSku, the article the merchant sends in its own feed, a plain number or a UUID.
• Titles and taxonomy - item name, brand with its vendorId, and the whole breadcrumb path, for example Home appliances, Small kitchen appliances, Toasters, Toasters Tefal.
• Price - the base price and the additionalPrices array, whose priceType values include yaBank, the Yandex Pay card price printed in large type, and withDiscount; plus the struck-through old price, the discount percent and any promo code with its minimum order.
• Stock - isAvailable and lastStock.stockItemCount, the source of the 3 left line on a card.
• Delivery - deliveryType, either COURIER or PICKUP, with deliveryDateFrom, deliveryDateTo, deliveryDays, the precise flag and calculatedFor, which ties the quote to one address and one offer.
• Service flags and badges - isExpress, isUltima, isOnDemand, isCrossBorder, isComfortPlus, the sponsored flag on advertised tiles, and featureBadges such as Original or a promo code token.
• Ratings - the unrounded rating value, numGrades, numReviews, salesNumber and the whose field, which says whether the stars belong to the model or to the merchant.
• Specs - name and value pairs grouped the way each category groups them: a fridge carries Dimensions and volume, a toaster Number of toasts.
• Reviews and questions - grade, the separate pros, cons and comment fields, helpfulness votes, photo and video flags, and questions with the merchant reply and its date.
• Media - the gallery on avatars.mds.yandex.net, where one asset is addressable at full size or as a fixed thumbnail.

Reviewer names, buyer profiles and anything behind a login stay out of scope: we collect what a signed-out visitor is shown.

Where a naive Yandex Market crawler goes wrong

The first trap is the identifier in the address bar. On a card the structured Product block publishes an sku that is not the number in the URL: the URL carries the oskuId, the structured block carries the marketSku, and the model behind both has a third id again. Keep only what the URL hands you and your history breaks the first time one merchant's card is swapped for another on the same model.

The second is the headline price. The large figure on a tile is the Yandex Pay card price out of additionalPrices, not the plain price; the plain one lives in the price field and is what gets struck through. Read the rendered number and you record a discount only some buyers get. Signed-in accounts are shown lower prices again, and the site says so on the page - we do not log in, so we report the signed-out price and label it that way.

The third is rating scope. A card can show stars whose owner is the merchant rather than the model, and the same model reads sixty-three grades on one view and several hundred on another. The displayed value is rounded to one decimal while the payload carries it to nine.

The last is coverage. A category listing stops handing back tiles long before the category is exhausted, so covering a category means slicing by brand node, price band, item condition and delivery window rather than paging to the end. Spec facets and the internal endpoints are closed in robots, so slices get planned around what the file allows.