The original eviction logic blocked ASR eviction even when an LLM
genuinely needed all 16GB VRAM (e.g., gpt-oss-20b at 13GB). Now uses
two-pass eviction: first evicts lower/same priority, then cascades to
higher priority as last resort. Added tests for ASR-survives and
full-cascade scenarios.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tracks GPU VRAM usage (16GB) and handles model loading/unloading with
priority-based eviction: LLM (lowest) -> TTS -> ASR (highest, protected).
Uses asyncio Lock for concurrency safety.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements create_api_key_dependency() FastAPI dependency that validates
Bearer tokens against a configured list of ApiKey objects (401 on missing,
malformed, or unknown tokens). Includes 5 TDD tests covering all cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers project scaffolding, config, auth, VRAM manager, all four
backends, API routes, Dockerfile, deployment scripts, and four
phases of testing (integration, functional, VRAM, performance).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers architecture, model registry, VRAM management, API endpoints,
container setup, Open WebUI integration, Traefik routing, and
four-phase testing plan.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
rdmsandClaude Opus 4.6