rdmsandClaude Opus 4.6 7c4bbe0b29 feat: Jinja template thinking toggle, Qwen3.5-9B GGUF Q8_0 ...

- Thinking/Instruct toggle via Jinja template patching in llama-cpp
  backend: creates separate handlers for thinking-enabled and
  thinking-disabled modes
- Replace lovedheart/Qwen3.5-9B-FP8 (safetensors, 15.8GB OOM) with
  unsloth/Qwen3.5-9B-GGUF Q8_0 (9.2GB, fits)
- Enable flash_attn in llama-cpp for better performance
- GGUF path resolution falls back to flat gguf/ directory

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-06 09:44:02 +02:00

kischdle/llmux

feat: Jinja template thinking toggle, Qwen3.5-9B GGUF Q8_0

2026-04-06 09:44:02 +02:00

.gitignore

Initial commit with .gitignore

2026-03-31 17:58:54 +02:00

S

Description

No description provided

144 KiB

Languages

Python 91.2%

Shell 6%

Dockerfile 2.8%