hermes-skills/food-nutrition/meal-planning/references/semantic-search-implementation.md

Semantic Search + Preference Learning — Implementation

Date: 2026-06-28 Status: Implemented and tested

Embedding Model

Model: vllm/harrier-oss-v1-0.6b (noris)

• 1024 dimensions, multilingual (German-capable)
• Dedicated embedding model (NOT an LLM), 0.6B params
• Endpoint: POST https://ai.noris.de/v1/embeddings
• Auth: Same noris API key from config.yaml
• Batch: Up to 64 inputs per call, ~31 recipes/sec throughput

Discovery

Initial investigation found that norris does NOT serve embeddings via standard LLM models (glm, gemma, qwen all return 404). The model list included vllm/harrier-oss-v1-0.6b which was the only embedding model that actually works. Other embedding names (bge-m3, e5-large, gte-large) are recognized by the Bifrost gateway but not activated on any API key.

Quality Validation

Cosine similarity matrix (higher = more similar):

	Curry	Salat	Eintopf	Hähnchen	Wintergericht
Curry	1.000	0.647	0.677	0.812	0.660
Salat	0.647	1.000	0.715	0.667	0.661
Eintopf	0.677	0.715	1.000	0.696	0.829
Hähnchen	0.812	0.667	0.696	1.000	0.694
Wintergericht	0.660	0.661	0.829	0.694	1.000

Curry↔Hähnchen highest (both chicken dishes), Eintopf↔Wintergericht high (both warm/hearty), Salat↔Eintopf lower (different concepts). Working as expected.

Database Schema

recipes.embedding (JSON column)

Added via ALTER TABLE recipes ADD COLUMN embedding JSON NULL. Stores 1024-dim embedding as JSON array: [0.023, -0.054, ...].

meal_decisions table

CREATE TABLE meal_decisions (
  id INT AUTO_INCREMENT PRIMARY KEY,
  decision_date DATE NOT NULL,
  recipe_id VARCHAR(255) NOT NULL,
  recipe_title VARCHAR(500),
  recipe_category VARCHAR(200),
  accepted BOOLEAN NOT NULL DEFAULT FALSE,
  context_temp FLOAT NULL,           -- outdoor temp °C
  context_season VARCHAR(20) NULL,   -- spring/summer/autumn/winter
  context_weekday VARCHAR(20) NULL,
  context_last_meals TEXT NULL,       -- JSON array of recent meal categories
  reason TEXT NULL,                   -- free-text rejection reason
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

preference_rules table

CREATE TABLE preference_rules (
  id INT AUTO_INCREMENT PRIMARY KEY,
  rule_type ENUM('context_category','context_tag','variety','semantic') NOT NULL,
  condition_json TEXT NOT NULL,  -- {"temp_min":20, "temp_max":99}
  effect_json TEXT NOT NULL,     -- {"penalty_category":"Eintopf"}
  weight FLOAT DEFAULT 1.0,      -- grows with confirmations (capped at 10)
  confirmations INT DEFAULT 1,
  rejections INT DEFAULT 0,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

Scripts

All in profiles/nutrition-coach/scripts/:

generate_embeddings.py

• Batch-generates embeddings for recipes where embedding IS NULL
• Builds embedding text from: title + category + tags[:10] + ingredients[:20] + description[:200] (capped at 1000 chars)
• Batch size: 64 recipes per API call
• Rate limit: 0.2s between calls
• Run: python3 scripts/generate_embeddings.py
• Should be run after scraper fills new recipes (could be chained in cronjob)

semantic_search.py

Core module with these functions:

semantic_search(query, filters, top_k, min_rating)

• Gets query embedding from norris
• Loads up to 5000 candidate recipes from DB (ordered by rating DESC)
• Computes cosine similarity in Python
• Blends similarity with rating: score = similarity * (1 + rating/10)
• Supports filters: category, source, exclude_categories, max_calories

record_decision(recipe_id, accepted, context)

• Logs to meal_decisions table
• Auto-creates preference rule if rejection reason mentions weather/temperature
• Example: record_decision("hf_123", False, {"temp": 28, "season": "summer", "reason": "zu heiß für Eintopf"})
• Auto-rule: temp>20°C → penalize category from the rejected recipe

get_preference_adjustments(context)

• Returns {boost_categories, penalty_categories, boost_tags, penalty_tags} based on matching rules
• Each entry includes the rule's weight

rank_with_preferences(recipes, context)

• Re-ranks a list of recipe dicts by applying:
  • Category penalties/boosts from learned rules (-0.1 × weight per match)
  • Tag penalties/boosts (-0.05 × weight per match)
  • Variety penalty: -0.15 if same category as recent meals
• Checks category + title + tags concatenated (not just category field — see pitfall below)

mine_rules(min_confidence=2)

• Analyzes meal_decisions for statistical patterns
• Groups by (category, temperature_bucket) and (category, season)
• If rejection rate > 50% with enough samples → creates/upgrades rule
• Should run periodically as decisions accumulate

Tested Examples

Semantic Search

Query: "schnelles Hähnchengericht"
→ Hähnchenbrust in Joghurt-Parmesan-Salbei-Panade (sim: 0.622, ★5.0)
→ Hähnchenschnitzel auf Muttis Art (sim: 0.626, ★4.9)
→ Hühnchenschnitzel (sim: 0.614, ★5.0)

Preference Learning

1. record_decision("ck_Erbsensuppe", False, {"temp": 28, "reason": "zu heiß für Eintopf"})
   → Auto-rule created: temp>20°C → penalize "Eintopf" (weight: 1.0)

2. semantic_search("Eintopf mit Kartoffeln", top_k=10)
   → rank_with_preferences(results, {"temp": 28, "season": "summer"})
   
Result: All Eintopf recipes get Δ=-0.2000 (two matching rules × 0.1 weight each)
  0.7365 (Δ-0.2000) | Möhren-Kartoffel-Eintopf (Kochen)
  0.7304 (Δ-0.2000) | Eintopf mit Spitzkohl, Hackfleisch (Kartoffeln)
  0.7226 (Δ-0.2000) | Möhreneintopf alla Isa (Eintopf)

Key Pitfalls

1. Category field mismatch: Most "Eintopf" recipes have category "Kochen" or "Kartoffeln", not "Eintopf". Rule matching must check title + tags + category as a concatenated lowercase string.

2. Decimal type from MySQL: rating comes back as decimal.Decimal, not float. Must cast with float(rating or 0) before arithmetic.

3. Auto-rule with non-existent recipe IDs: record_decision("test_reject_1", ...) creates a rule with empty category because the recipe doesn't exist in the DB. Always use real recipe IDs from the DB.

4. Duplicate rules from auto-creation + mining: Both _auto_create_rule() and mine_rules() can create the same rule. _upsert_rule() handles this with matching on (rule_type, condition_json, effect_json) and increments weight/confirmations.

Integration with Meal Planner

The meal planner (nutrition-plan-generator-v4.py) should be updated to:

1. Call semantic_search() instead of linear JSONL scan
2. Call rank_with_preferences() with weather context
3. Call record_decision() when user accepts/rejects proposals in Phase 2
4. Periodically call mine_rules() to discover new patterns

Context for decisions:

• Temperature: From HA weather sensor or Tibber API
• Season: Derived from date
• Weekday: From date
• Last meals: From previous week's finalized plan