109 lines
4.4 KiB
Markdown
109 lines
4.4 KiB
Markdown
# Semantic Search + Preference Learning Investigation
|
||
|
||
Date: 2026-06-28
|
||
|
||
## Goal
|
||
|
||
Add semantic recipe search ("etwas Leichtes für heiße Tage") and preference learning ("wenn warm dann kein Eintopf") to the meal-planning pipeline.
|
||
|
||
## Embedding Model Requirements
|
||
|
||
- **Dimensions:** 1024 (user requirement)
|
||
- **Language:** Multilingual / German-capable
|
||
- **Hosting:** Preferred on norris (same provider as LLM models)
|
||
|
||
## Norris Embedding Endpoint Investigation
|
||
|
||
Tested all 20 models listed at `https://ai.noris.de/v1/models` against the `/v1/embeddings` endpoint:
|
||
|
||
### Models tested via `/v1/embeddings`
|
||
| Model | Result |
|
||
|-------|--------|
|
||
| vllm/bge-reranker-v2-m3 | 404 — provider API error |
|
||
| vllm/glm-5-2-nvfp4 | 404 — provider API error |
|
||
| vllm/gemma-4-31b-it | 404 — provider API error |
|
||
| vllm/qwen3.6-27b-nvfp4 | 404 — provider API error |
|
||
|
||
### Additional embedding models tested by name
|
||
| Model | Result |
|
||
|-------|--------|
|
||
| vllm/bge-m3 | "no keys found that support model: bge-m3" |
|
||
| vllm/multilingual-e5-large | "no keys found that support model" |
|
||
| vllm/e5-mistral-7b-instruct | "no keys found that support model" |
|
||
| vllm/gte-large | "no keys found that support model" |
|
||
| vllm/NV-Embed-v2 | "no keys found that support model" |
|
||
| vllm/snowflake-arctic-embed-l-v2.0 | "no keys found that support model" |
|
||
|
||
### Conclusion
|
||
Norris recognizes embedding model names (gateway returns "no keys found" not "model not found"), meaning the Bifrost gateway knows about them but no API key has them activated. The `/v1/embeddings` endpoint returns 404 for all currently-available models (LLM models don't serve embeddings).
|
||
|
||
**Action needed:** User must request norris to activate an embedding model (bge-m3 recommended — 1024 dims, multilingual, strong German performance).
|
||
|
||
## Local vLLM Check
|
||
|
||
Checked for local vLLM instances that might serve embeddings:
|
||
- `10.0.30.97:8000` (ComfyUI CT) — No route to host (SSH and HTTP both unreachable)
|
||
- `10.0.30.102:8000` (OpenWebUI CT) — No response
|
||
- `10.0.30.99:8000` (Docker host) — No response
|
||
- `localhost:8000/8001` — No response
|
||
|
||
No local vLLM embedding server found. ComfyUI CT (10.0.30.97) has GPU but was unreachable during investigation.
|
||
|
||
## Proposed Architecture (pending embedding model availability)
|
||
|
||
### Database Schema Additions
|
||
|
||
```sql
|
||
-- Add embedding column to recipes table
|
||
ALTER TABLE recipes ADD COLUMN embedding JSON NULL COMMENT '1024-dim vector as JSON array';
|
||
|
||
-- Decision logging
|
||
CREATE TABLE meal_decisions (
|
||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||
decided_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||
recipe_id VARCHAR(255) NOT NULL,
|
||
decision ENUM('accepted', 'rejected') NOT NULL,
|
||
context JSON COMMENT 'temperature, season, weekday, recent_meals',
|
||
INDEX idx_recipe (recipe_id),
|
||
INDEX idx_decided (decided_at)
|
||
);
|
||
|
||
-- Learned preference rules
|
||
CREATE TABLE preference_rules (
|
||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||
rule_pattern VARCHAR(255) NOT NULL COMMENT 'e.g. temp>20=>avoid:eintopf',
|
||
weight FLOAT DEFAULT 1.0 COMMENT 'confidence, grows with confirmations',
|
||
confirmed_count INT DEFAULT 0,
|
||
rejected_count INT DEFAULT 0,
|
||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||
UNIQUE KEY uniq_pattern (rule_pattern)
|
||
);
|
||
```
|
||
|
||
### Pipeline Flow
|
||
|
||
```
|
||
Meal Planner selects recipes:
|
||
1. SQL filter (rating > 4.0, not used this week)
|
||
2. Preference rules apply (boost/penalize by context)
|
||
3. Semantic search for "mood" query (light, hearty, asian...)
|
||
4. Top-K presented as proposal
|
||
|
||
User feedback ("nein, kein Eintopf bei dem Wetter"):
|
||
→ INSERT INTO meal_decisions (rejected, temp=28°C, category=eintopf)
|
||
→ Rule mining detects pattern → INSERT/UPDATE preference_rules
|
||
→ Next run: Eintopf auto-downweighted when warm
|
||
```
|
||
|
||
### Embedding Generation Strategy
|
||
|
||
- Batch-generate embeddings for existing 28k+ recipes (one-time)
|
||
- Incrementally embed new recipes as v7 scraper inserts them
|
||
- Store as JSON array in `embedding` column (simple, no pgvector/Qdrant dependency)
|
||
- Cosine similarity computed in Python at query time (28k × 1024 floats ≈ 115MB RAM, <1s)
|
||
- At 100k recipes: ~400MB RAM for full similarity matrix — still feasible in-process
|
||
|
||
### Alternative: MariaDB VECTOR Type
|
||
|
||
MariaDB 11.6+ supports native `VECTOR` type with `VEC_DISTANCE` functions. If the Galera cluster upgrades, switch from JSON-stored arrays to native vectors for indexed similarity search. Current MariaDB version on the cluster should be checked before relying on this.
|