tlg da35e94b16 fix: add triton kernels for MXFP4, fix GGUF KV cache quantization ...

- Add 'kernels' package to Dockerfile for native MXFP4 execution
  (fixes gpt-oss-20b OOM: 15.2GB→13.5GB)
- Reduce GGUF n_ctx from 8192 to 4096 and quantize KV cache to Q8_0
  to reduce VRAM usage
- Use GGML_TYPE_Q8_0 constant instead of string for type_k/type_v

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-05 22:49:16 +02:00

..

config

fix: Chatterbox uses separate classes per variant, remove turbo

2026-04-05 21:43:40 +02:00

docs/superpowers

Add llmux implementation plan (30 tasks)

2026-04-03 22:43:37 +02:00

llmux

fix: add triton kernels for MXFP4, fix GGUF KV cache quantization

2026-04-05 22:49:16 +02:00

scripts

fix: use LLMUX_SRC env var for Dockerfile path in pod creation script

2026-04-05 13:05:38 +02:00

tests

fix: Chatterbox uses separate classes per variant, remove turbo

2026-04-05 21:43:40 +02:00

.gitignore

chore: add .gitignore for venv, caches, and local dirs

2026-04-05 22:17:42 +02:00

Dockerfile

fix: add triton kernels for MXFP4, fix GGUF KV cache quantization

2026-04-05 22:49:16 +02:00

requirements.txt

fix: Dockerfile uses explicit pip install, skip pre-installed packages

2026-04-05 14:10:07 +02:00