# Vision Model Capabilities (vLLM / Noris)

Tested against `https://ai.noris.de/v1` — OpenAI-compatible vLLM endpoint.

## Multimodal (Accepts Images) ✅

| Model | Notes |
|-------|-------|
| `vllm/release/gemma-4-31b-it` | Works. Despite not being labeled "vision", accepts `image_url` content type and describes images accurately. Good for slide QA. |

## Text-Only (Rejects Images) ❌

| Model | Error | Notes |
|-------|-------|-------|
| `vllm/release/qwen3.6-27b` | `400: At most 0 image(s) may be provided in one prompt` | Explicitly rejects image input |
| `vllm/release/qwen3.6-35b-a3b` | `400: Input should be a valid string` | Rejects multimodal content format |
| `vllm/release/gpt-oss-120b` | Returns "I'm not able to view images" | Accepts the request but model itself can't process images |
| `vllm/release/glm-5-2` | Not tested for images | Primary chat model — assumed text-only |

## Testing Procedure

Send a single image with a simple "describe this image" prompt:

```python
import urllib.request, json, base64

with open("test.jpg", "rb") as f:
    img_b64 = base64.b64encode(f.read()).decode()

payload = {
    "model": "<model_to_test>",
    "messages": [{"role": "user", "content": [
        {"type": "text", "text": "Describe this image in 1 sentence."},
        {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{img_b64}"}}
    ]}],
    "max_tokens": 80
}

req = urllib.request.Request(
    "https://ai.noris.de/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Authorization": "Bearer <key>", "Content-Type": "application/json"}
)
resp = urllib.request.urlopen(req, timeout=30)
print(json.loads(resp.read())["choices"][0]["message"]["content"])
```

If the response describes the image content → multimodal. If it apologizes about not seeing images or returns HTTP 400 → text-only.

## Config Gotcha

The `auxiliary.vision` section in `~/.hermes/config.yaml` must have:
- Correct model name WITH prefix (e.g. `vllm/release/gemma-4-31b-it`, not `vllm/gemma-4-31b-it`)
- Non-empty `base_url` and `api_key`

If these are empty or wrong, `vision_analyze` fails with "no keys found that support model" — even if the top-level `vision.*` section is correct. Both sections must be consistent.