hermes-skills/food-nutrition/meal-planning/references/multi-source-recipe-scraper/chefkoch-api-v2-recipes.md

Chefkoch.de Internal API v2 — Discovery & Detail

Discovered during v5.2 scraper development. This endpoint is public and unauthenticated (as of 2026-06) and returns structured JSON for 384,252+ recipes.

Base URL

https://api.chefkoch.de/v2/recipes

No API key, no auth headers, no cookies required.

Endpoints

1. Listing (Discovery)

GET /v2/recipes?offset={N}&limit={limit}

Parameters

• offset — integer, start index (pagination)
• limit — integer, max 100 per request (default 30)

Response shape

{
  "count": 384252,
  "queryId": "",
  "results": [
    {
      "recipe": {
        "id": "4073161636112189",
        "title": "Flammkuchen mit Kartoffeln und Schwarzwälder Schinken",
        "slug": "Flammkuchen-mit-Kartoffeln-und-Schwarzwaelder-Schinken",
        "siteUrl": "https://www.chefkoch.de/rezepte/4073161636112189/Flammkuchen-mit-Kartoffeln-und-Schwarzwaelder-Schinken.html",
        "difficulty": 1,
        "preparationTime": 15,
        "rating": {"rating": 4.75, "numVotes": 20},
        "hasImage": true,
        "isPremium": true,
        "createdAt": "2021-11-05T13:36:51+01:00"
      },
      "score": 384252
    }
  ]
}

Discovery-only fields (useful for filtering before detail fetch):

• id — recipe ID (string)
• title / slug / siteUrl — human metadata
• preparationTime — minutes (int)
• difficulty — 1=easy, 3=hard
• rating.rating / rating.numVotes
• isPremium / isPlus — premium flag

2. Detail (Full Recipe)

GET /v2/recipes/{id}

Response shape (key fields)

{
  "id": "4073161636112189",
  "title": "Flammkuchen…",
  "slug": "Flammkuchen-mit-Kartoffeln-und-Schwarzwaelder-Schinken",
  "siteUrl": "https://www.chefkoch.de/rezepte/...",
  "subtitle": "Originalrezept von Viki Fuchs",
  "additionalDescription": "",
  "instructions": "Den Ofen auf 220 °C Umluft vorheizen…",
  "ingredientGroups": [
    {
      "header": "Für den Teig: ",
      "ingredients": [
        {
          "id": "8891376",
          "name": "Öl",
          "unit": "EL",
          "unitId": "13",
          "amount": 4.0,
          "isBasic": false,
          "usageInfo": "",
          "foodId": "184"
        }
      ]
    }
  ],
  "tags": ["Schnell", "einfach", "Backen", "Frankreich"],
  "rating": {"rating": 4.75, "numVotes": 20},
  "preparationTime": 15,
  "kCalories": 0,
  "servings": 2,
  "nutrition": null,
  "difficulty": 1
}

Structured Ingredients (API vs JSON-LD)

Aspect	JSON-LD (recipeIngredient)	API (ingredientGroups)
Format	raw strings: "4 EL Öl"	objects: {amount: 4, unit: "EL", name: "Öl"}
Groups	flat list	grouped by header (e.g. "Für den Teig:")
Parsing	requires regex heuristics	already parsed
Accuracy	~95% (edge cases in parentheses)	~99%

Normalization recipe (from API objects to flat strings):

ingredients = []
for group in data.get("ingredientGroups", []):
    header = group.get("header", "").strip()
    if header:
        ingredients.append(f"--- {header} ---")
    for ing in group.get("ingredients", []):
        parts = [str(p) for p in [ing.get("amount"), ing.get("unit"), ing.get("name")] if p]
        if parts:
            ingredients.append(" ".join(parts))

Rate Limits & Politeness

Observed behavior:

• No explicit rate limiting (no 429 responses observed at 16 concurrent requests)
• Server responds reliably with application/json
• Total count fluctuates slightly (cache propagation)

Recommended politeness:

• 0.3s between listing batches
• 16 concurrent detail fetches (ThreadPoolExecutor)
• No backoff needed for normal operation

Comparison: API vs Playwright

Metric	Playwright v4/v5	API v5.2
Discovery	Playwright scroll + DOM parse	1 HTTP GET per 100 recipes
Detail fetch	urllib.requests + JSON-LD regex	1 HTTP GET per recipe
Avg per recipe	~3–5s	~0.2s
Browser dependency	Yes (chromium binary)	No
Headless issues	Cookie banners, JS timing	None
Max recipes reachable	~25k (search exhaustion)	~384k (full catalog)
Bot detection risk	Moderate (requires stealth)	Very low (same API as site frontend)

Migration from Playwright to API

# old (v5.1)
urls = await _fetch_links(page, seed_url)  # Playwright
recipe = await _scrape_one(page, url)       # requests + JSON-LD

# new (v5.2)
recipes_meta = _fetch_recipe_list(offset, limit=100)  # API listing
recipe = _fetch_recipe_detail(meta["id"])              # API detail

Listing API: Offset Ceiling Reality

The unfiltered listing endpoint (/v2/recipes?offset=N&limit=100) appears to return the same global list regardless of any sort=, order=, with=, or difficulty= parameters tested (2026-06-22). The only way to get diversity is to advance offset.

CRITICAL: Offset has a practical ceiling. In testing:

• offset=0 returns 100 results ✅
• offset=2000 returns 0 results ❌ (server silently drops)

This means the unfiltered listing CANNOT be used to reach the full 384k catalog by offset alone. For deep scraping, use the search query strategy below.

Search Queries via ?query=

The endpoint supports text search with real filtering:

GET /v2/recipes?query={term}&offset=0&limit=100

Verified search queries (with approximate counts as of 2026-06):

Query	Count	Usable offsets
Hauptspeise	~158,000	0–~1000
Kuchen	~58,000	0–~1000
Vegan	~28,000	0–~1000
Italienisch	~7,000	0–~1000
(blank)	~384,000	0–~2000 (barely)

Search offset ceiling: ~1000 results (10 pages of 100). Beyond that, the search returns empty.

Deep Scraper Strategy for 25k+ Recipes (v5.2)

Because no single query reaches beyond ~1000–2000 offsets, the scraper must rotate queries:

1. Maintain a list of 50+ search terms (categories, ingredients, cuisines)
2. Each run: pick a rotation window (e.g., 20 terms)
3. For each term: random offset between 0 and ~100, fetch listing
4. Filter known IDs, fetch details in parallel
5. Rotate to next term

This prevents seed exhaustion while staying inside each query's usable window.

QUERIES = [
    "Hauptspeise", "Vorspeise", "Nachtisch", "Beilage", "Fruehstueck",
    "Backen", "Grillen", "Salat", "Suppe", "Eintopf", "Pasta", "Dessert",
    "Kuchen", "Brot", "Getraenk", "Fingerfood", "Curry", "Wok", "Auflauf",
    "Gratin", "Risotto", "Lasagne", "Lachs", "Haehnchen", "Rind", "Schwein",
    "Fisch", "Gemuese", "Kartoffeln", "Reis", "Linsen", "Vegan", "Vegetarisch",
    "Low-Carb", "Schnell", "Einfach", "Italienisch", "Asiatisch", "Tacos",
    "Bowl", "Wrap", "Nudeln", "Haehnchenbrust", "Hackfleisch", "Burger",
    "Pizza", "Pfannkuchen", "Quiche", "Muffins", "Smoothie",
]

def _fetch_list(query, offset, limit=100):
    url = f"https://api.chefkoch.de/v2/recipes?query={query}&offset={offset}&limit={limit}"
    ...

Performance observed (2026-06-22):

Initial projection under-estimated actual throughput significantly.

With 12 workers: ~10–15 recipes/sec (throughput ceiling is detail-fetch latency, not listing) With 16 workers: ~800–900 recipes/minute (~13–15 recipes/sec) With 32 workers: ~750 recipes/minute sustained (diminishing returns from context switching)

Key insight: The listing fetch is negligible latency (~100ms). Detail fetch dominates (~200ms per recipe at 32 concurrent). 16 concurrent detail fetches saturates the network + server response pipe. More workers add no value.

Recommended configuration:

• 16 ThreadPool workers (sweet spot)
• 0.3s delay between listing batches
• limit=100 per listing request
• Each run rotates to next query when hitting the offset ceiling

State Management for API Scraper

State file format

{
  "query_index": 0,
  "api_offset": 0,
  "source_counts": {"chefkoch": 25020},
  "new_this_run": 0,
  "failed_listings": 0
}

Field semantics:

• query_index — position in QUERIES list (rotates when exhausted)
• api_offset — current offset within the active query's pagination window
• source_counts — total recipes scraped per source
• new_this_run — deduplicated count since last state write
• failed_listings — consecutive failed listing requests (used for fast-exit)

CRITICAL: Offset increment must be += limit, not += 1

The scraper fetches batches of limit recipes per listing call. State must advance api_offset by limit (e.g., 100), not by 1. Incrementing by 1 causes the same 100-recipe batch to be re-fetched on every subsequent run, producing zero new recipes while appearing to "run successfully".

# CORRECT — advances to next batch
def main():
    state = _load_state()
    offset = state.get("api_offset", 0)
    limit = 100
    # ... fetch listing at offset ...
    state["api_offset"] = offset + limit  # 0 → 100 → 200
    _save_state(state)

# WRONG — re-fetches same batch every time
def main():
    state = _load_state()
    offset = state.get("api_offset", 0)
    # ... fetch listing at offset ...
    state["api_offset"] = offset + 1  # 0 → 1; next run: fetches offset 1 (same as offset 0 in effect)

The same batch always returns the same 100 recipe IDs. Deduplication drops everything, new_this_run stays at 0, but the state increments by 1 each run, crawling nowhere at ~1 recipe offset per launch.

Pitfalls

1. Do not use api.chefkoch.de/v2/recipes/{id}/ingredients — returns resource_not_found (404)
2. Ingredients live inside the detail response under ingredientGroups — there is no separate endpoint
3. Tag list may contain empty strings — filter with .strip() before storage
4. Count is approximate — the count field drifts slightly; do not hardcode it as a loop bound
5. Offset-based pagination with a hard ceiling — unfiltered listing exhausts at ~2000; search queries at ~1000
6. Query parameter filtering is NOT supported on the listing endpoint — sort=, difficulty=, etc. are ignored
7. Do not cache the count field as a loop bound — use query rotation instead

Headers Required

Minimal headers suffice; full mimicry not required:

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
    "Accept": "application/json",
    "Accept-Language": "de-DE,de;q=0.9,en;q=0.8",
    "Referer": "https://www.chefkoch.de/",
}