# Chefkoch.de: Hybrid Scraper (Playwright + Requests + JSON-LD)

Chefkoch.de (380k+ Rezepte) blockiert **direkte `requests` auf Suchseiten** (liefert JS-Shell statt Inhalt), **aber** einzelne Rezept-Seiten können über **Requests + JSON-LD** geparst werden. Die optimale Architektur ist ein **Zwei-Phasen-Ansatz**.

## Warum der alte Ansatz scheitert

**Falsch:** `recipe_scrapers.scrape_html(page.content(), url)` auf Chefkoch-Suchseiten — Chefkoch erkennt nicht-JS-Clients und liefert leere DOM-Shells.

**Falsch:** Playwright pro Rezept — zu langsam (5s+ pro Rezept) und überfordert bei 100+ Rezepten.

## Korrekte Architektur: Zwei Phasen

### Phase 1: Link-Sammlung via Playwright (Browser nötig)

Nur für die **Suchseite** — Chefkoch rendernt Ergebnisse client-side.

```python
from playwright.async_api import async_playwright

async with async_playwright() as p:
    browser = await p.chromium.launch(headless=True)
    page = await browser.new_page(
        user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
    )
    await page.goto("https://www.chefkoch.de/rs/s0/{query}/Rezepte.html",
                    wait_until="networkidle", timeout=30000)

    # Scroll to load more results
    for _ in range(15):
        links = await page.query_selector_all("a[href*='rezepte/']")
        for el in links:
            href = await el.get_attribute("href")
            if href and "/rezepte/" in href:
                if not any(skip in href for skip in ["was-koche", "was-backe", "alle-rezepte", "/rs/s"]):
                    recipe_links.add(href.split("?")[0])
        await page.evaluate("window.scrollBy(0, 500)")
        await asyncio.sleep(0.5)

    await browser.close()
```

**Pitfall:** Keine `.recipe-list` oder `.bi-recipe-item` Selektoren verwenden — diese Klassenamen ändern sich. Stattdessen: `a[href*='rezepte/']` mit URL-Pattern-Filtern.

### Phase 2: Rezept-Daten via Requests + JSON-LD (kein Browser)

**Schnell** (~300ms statt 5s pro Rezept) und **zuverlässiger** als DOM-Scraping.

```python
import urllib.request, re, json

def fetch_recipe_jsonld(url: str) -> dict:
    req = urllib.request.Request(url, headers={
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
    })
    with urllib.request.urlopen(req, timeout=15) as resp:
        html = resp.read().decode("utf-8", errors="ignore")

    # Extract JSON-LD blocks
    blocks = re.findall(r'<script type="application/ld\+json">(.*?)</script>', html, re.DOTALL)

    for block in blocks:
        try:
            data = json.loads(block.strip())
            if isinstance(data, list):
                for item in data:
                    if item.get("@type") == "Recipe":
                        return item
            elif data.get("@type") == "Recipe":
                return data
        except:
            continue
    return None

# Usage
recipe = fetch_recipe_jsonld("https://www.chefkoch.de/rezepte/...")
name = recipe["name"]
ingredients = recipe.get("recipeIngredient", [])  # List of strings like "4 EL Sojasauce"
prep_time = recipe.get("prepTime", "")  # ISO 8601: "PT25M"
rating = recipe.get("aggregateRating", {}).get("ratingValue", "n/a")
```

## Wichtige JSON-LD Felder

| Feld | Typ | Beispiel |
|------|-----|----------|
| `name` | str | `"Chicken Teriyaki - der japanische Klassiker"` |
| `recipeIngredient` | list[str] | `["4 EL Sojasauce", "250 g Hähnchenbrust"]` |
| `recipeInstructions` | list[dict/text] | `[{"@type": "HowToStep", "text": "..."}]` |
| `prepTime` / `cookTime` / `totalTime` | str (ISO 8601) | `"PT25M"`, `"PT1H30M"` |
| `aggregateRating.ratingValue` | float | `4.79` |
| `aggregateRating.ratingCount` | int | `251` |

## Zutaten-String Parsen

`recipeIngredient` liefert rote Strings. Parser-Heuristik:

```python
def parse_ingredient(s: str) -> dict:
    s = re.sub(r"[\t\n]+", " ", s.strip())
    parts = s.split(None, 2)  # maxsplit=2

    # Default
    amount, unit, name = 1.0, "Stk", s

    # Try: <number> [unit] <name>
    m = re.match(r"([\d.,/]+(?:\s*-\s*[\d.,/]+)?)", parts[0])
    if m:
        num_str = m.group(1).replace(",", ".")
        # Parse number (handles fractions, ranges)
        if "-" in num_str:
            a, b = [float(x.strip()) for x in num_str.split("-")]
            amount = (a + b) / 2
        elif "/" in num_str:
            a, b = [float(x.strip()) for x in num_str.split("/")]
            amount = a / b
        else:
            amount = float(num_str.strip())

        # Check if second part is known unit
        KNOWN_UNITS = {"el", "tl", "g", "kg", "ml", "l", "cl", "dl",
                       "stk", "stück", "packung", "pck", "bund", "zweig",
                       "dose", "glas", "tasse", "prise", "scheibe"}
        if len(parts) >= 2 and parts[1].lower().rstrip(".") in KNOWN_UNITS:
            unit = parts[1]
            name = parts[2] if len(parts) > 2 else ""
        else:
            name = " ".join(parts[1:])

    return {"item": name.strip(), "amount": amount, "unit": unit}
```

**Pitfall:** regex mit `re.match(r"...", first)` und `split(None, 2)` statt komplexem Gesamt-regex — vermeidet "unbalanced parenthesis" Fehler bei verschachtelten Klammern in Zutaten-Namen wie "Reiswein (Mirin oder Sake)".

## Playwright Install

```bash
pip install playwright
playwright install chromium
```

**Verify:**
```python
python3 -c "from playwright.sync_api import sync_playwright; \
  p = sync_playwright().start(); \
  b = p.chromium.launch(); b.close(); p.stop(); \
  print('Chromium OK')"
```

**Fehler ohne Installation:**
```
Error: Executable doesn't exist at ~/.cache/ms-playwright/chromium-1071/...
Please run: playwright install
```

**Im Hintergrund-Cronjob:** Immer `async_playwright` verwenden, nie `sync_playwright` — blockiert den Event Loop.

## Performance-Vergleich

| Ansatz | Zeit pro Rezept | 100 Rezepte |
|--------|----------------|-------------|
| Playwright pro Rezept | 5–8s | 8–13 Min |
| **Hybrid (recommended)** | **0.3s (HTTP) + 3s Suchseite** | **~3 Min** |

## Dynamische Queries aus Interview-Daten

| Interview-Input | Generierte Query |
|-----------------|-----------------|
| 27°C + asiatisch | "asiatisch leicht huhn" |
| Sarah + low-carb | "low carb spargel fisch" |
| Regen + Familie | "einfach familie warm" |

## Familien-Fit-Score

- Asiatisch → +2.0 (Dominik)
- Low-Carb/Salat/Fisch → +1.5 (Sarah)
- Nudeln/Huhn/Eier → +1.0 (Cleo)
- Bewertung >4.5 → +ratingValue × 0.5
- Gemüse-Vielfalt → +0.3 pro Art