HelloFresh.de — Recipe Scraping via SSR Hydration

HelloFresh.de serves recipe content as Next.js SSR pages with recipe data embedded in __NEXT_DATA__ hydration scripts. No public API is exposed; extraction requires HTML parsing of the dehydrated React Query state.

Architecture

Next.js + TanStack Query. Recipe data fetched server-side and serialized into __NEXT_DATA__.props.pageProps.ssrPayload.dehydratedState.queries[].

Key query keys

Query key	Data
`['recipe.search', {...}]`	Recipe list for category page
`['foodContentHubRecipe.byId', {'id': '...'}]`	Full recipe detail

Category Page (Discovery)

import urllib.request, json, re

def hf_fetch_category(category: str, skip=0, take=50) -> tuple[list[dict], int]:
    url = f"https://www.hellofresh.de/recipes/{category}?skip={skip}&take={take}"
    headers = {
        "User-Agent": "Mozilla/5.0...", "Accept": "text/html...",
        "Accept-Language": "de-DE,de;q=0.9", "Referer": "https://www.hellofresh.de/",
    }
    req = urllib.request.Request(url, headers=headers)
    with urllib.request.urlopen(req, timeout=15) as resp:
        html = resp.read().decode("utf-8", errors="ignore")
        match = re.search(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', html, re.DOTALL)
        data = json.loads(match.group(1))
        ssr = data['props']['pageProps']['ssrPayload']
        for q in ssr.get('dehydratedState', {}).get('queries', []):
            key = q.get('queryKey', [])
            if len(key) > 0 and key[0] == 'recipe.search':
                d = q['state']['data']
                return d.get('items', []), d.get('total', 0)
        return [], 0

Category slugs (62 total): amerikanische-rezepte, asiatische-rezepte, beliebteste-rezepte, burger, chinesische-rezepte, deutsche-rezepte, einfache-rezepte, eintopf-rezepte, familien-rezepte, flammkuchen-rezepte, fleisch-rezepte, franzosische-rezepte, fusions-rezepte, gesunde-rezepte, glutenfreie-rezepte, griechische-rezepte, grillrezepte, indische-rezepte, italienische-rezepte, japanische-rezepte, kalorienreduzierte-rezepte, karibische-rezepte, koreanische-rezepte, low-carb-rezepte, marokkanische-rezepte, mediterrane-rezepte, meeresfruechte-rezepte, mexikanische-rezepte, orientalische-rezepte, pasta-rezepte, pizza, proteinreiche-rezepte, rezepte-naher-osten, salat-rezepte, sandwich-rezepte, schnelle-gerichte, schnitzel, skandinavische-rezepte, spanische-rezepte, suppen-rezepte, thailandische-rezepte, tofu, tuerkische-rezepte, vegane-rezepte, vegetarische-rezepte, vietnamesische-rezepte, weihnachtsessen, zitrone, etc.

Recipe Detail

def hf_fetch_detail(recipe_id: str, slug: str) -> dict | None:
    url = f"https://www.hellofresh.de/recipes/{slug}-{recipe_id}"
    req = urllib.request.Request(url, headers=headers)
    with urllib.request.urlopen(req, timeout=15) as resp:
        html = resp.read().decode("utf-8", errors="ignore")
        match = re.search(r'<script id="__NEXT_DATA__"[^>]*>(.*?)</script>', html, re.DOTALL)
        data = json.loads(match.group(1))
        ssr = data['props']['pageProps']['ssrPayload']
        for q in ssr.get('dehydratedState', {}).get('queries', []):
            key = q.get('queryKey', [])
            if len(key) > 0 and key[0] == 'foodContentHubRecipe.byId':
                return q['state']['data'].get('recipe')
        return None

Normalization

Field	Source	Notes
id	`recipe.id`	Strip `-de-DE` suffix
title	`recipe.name`
description	`recipe.description` or `recipe.headline`
ingredients	`yields[0].ingredients` + `ingredients` lookup	Two parallel arrays
instructions	`steps[].instructionsMarkdown` -> strip HTML
prep_time	`recipe.prepTime` ISO 8601	Parse PT30M -> 30
servings	`yields[0].yields`	Default 2
calories	`nutrition[]` where name contains "kcal"
tags	`recipe.tags[].name`	Skip "SEO"
category	`recipe.cuisines`	May be str or dict with `name`

Ingredient assembly

def assemble_ingredients(recipe):
    ing_lookup = {ing['id']: ing['name'] for ing in recipe.get('ingredients', [])}
    ingredients = []
    for ying in recipe.get('yields', [{}])[0].get('ingredients', []):
        name = ing_lookup.get(ying.get('id', ''), '')
        amount = ying.get('amount', 0)
        unit = ying.get('unit', '')
        if name and amount is not None:
            if unit and unit != 'nach Geschmack':
                ingredients.append(f"{amount} {unit} {name}")
            elif unit == 'nach Geschmack':
                ingredients.append(f"{name} (nach Geschmack)")
            else:
                ingredients.append(f"{amount} {name}")
        elif name:
            ingredients.append(name)
    return ingredients

Performance

Category page: ~0.5-1.0s
Detail page: ~0.5-1.5s
Throughput: ~10-20 recipes/min with 8 workers
Total public recipes: ~5,000-10,000

Pitfalls

No public API - HTML hydration only; may break on Next.js upgrade
cuisines polymorphic - list[str] or list[dict]; check isinstance
Amount of 0 means "nach Geschmank" or optional
Deduplicate by id - recipes appear across categories
Keep polite - 8 workers max, 0.5s between batches

5.2 KiB Raw Permalink Blame History