France 24 Scraper for Articles, Video and Shows in Four Languages

A France 24 scraper has to treat one domain as four newsrooms. France 24, the international news channel of France Medias Monde, went on air on 6 December 2006 in French and English, added Arabic in April 2007 and a Spanish service in 2017. All four live on www.france24.com as path prefixes: /fr/, /en/, /ar/ and /es/. France 24 is free to read and watch, with no paywall, no meter and no registration wall, so every article renders in full.

What the desks share is the frame. The four fronts point to each other with hreflang, show brands such as Focus, Reporters and The Observers run in several languages, and one CMS builds every page with the same markup. What they write separately is the journalism. Articles carry no hreflang to a counterpart, each desk tags with its own taxonomy (the tag Israel has one ID in English and another in Spanish), and the section trees differ: the Spanish desk splits America Latina from the US and Canada, France 24 Arabic runs a Maghreb section, the English desk a single Americas section.

Addresses follow the language. French and Spanish slugs keep their accents, Arabic slugs are written in Arabic script and arrive percent-encoded, and inLanguage reads fr, en, ar or es-419, since the Spanish service writes for Latin America. The section in an article URL is not always the slug of the section page, so sections are read from the markup, not from the path.

France 24 was built to carry a French perspective on world affairs, and each of its four desks chooses what its own audience hears. One company, one CMS and one data layer sit behind all four, so differences between desks are editorial, not technical.

• Agenda comparisonCompare what France 24 Arabic leads with for the Maghreb and the Middle East, what the Spanish desk tells Latin America and what the French desk gives French-speaking Africa. Sections and tags are counted per desk without translating a word.
• Headline boardsA board of France 24 headlines today polls the four fronts, the Newsfeed and each France 24 news sitemap every few minutes and reads each new page once.
• Wire versus own reportingThe France 24 Newsfeed carries AFP dispatches, labelled AFP in listings and listed without keywords or images in the news sitemap, while desk stories close with a credit line and some Newsfeed addresses end up holding a full desk story. Splitting the two shows what the channel reports itself.
• Verification researchTruth or Fake, the France 24 Observers and the Fight the Fake hub publish dated, tagged debunks of viral images and claims, a ready corpus for disinformation studies.
• Attention and sponsorshipThe Most read box, captured hourly, ranks what readers pick in each language, and sponsored pieces can be counted by sponsor.