Codeman maintainer
f60b63c005
feat(local-model): q8_0 K/V cache, flash attention and a bounded prompt cache for llama.cpp
...
runtimeEnv() now also sets OLLAMA_KV_CACHE_TYPE=q8_0, OLLAMA_FLASH_ATTENTION=1
and LLAMA_ARG_CACHE_RAM=1024. Read in the v0.40.15 source of the pinned
runtime: the first becomes --cache-type-k/-v of llama-server (17/32 of the
f16 cache), the second --flash-attn on (a quantized V cache needs it, and
"auto" or "off" would fail such a load), and llama-server inherits the third
from ollama serve's environment, which bounds its host prompt cache to
1 GiB instead of letting it grow by gigabytes in a session. The MLX runner
reads none of them.
The catalogue's llama.cpp rows are computed for these values, so the two
change together. Not yet measured on a Mac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com >
2026-11-08 09:18:51 +01:00
..
2026-04-03 16:23:54 +02:00
2026-05-25 23:31:50 +02:00
2026-10-15 18:17:34 +02:00
2026-06-09 04:35:29 +02:00
2026-07-28 10:57:14 +02:00
2026-11-08 09:01:00 +01:00
2026-11-08 09:00:59 +01:00
2026-11-06 23:55:02 +01:00
2026-10-17 18:28:03 +02:00
2026-11-08 09:18:51 +01:00
2026-10-10 02:54:39 +02:00
2026-08-08 22:31:38 +03:00
2026-08-10 11:19:51 +02:00
2026-09-24 01:48:26 +02:00
2026-11-06 23:55:02 +01:00
2026-10-22 19:32:17 +02:00
2026-06-29 11:35:56 +05:30
2026-08-13 13:54:47 +02:00
2026-10-22 19:23:17 +02:00
2026-11-04 17:31:48 +01:00
2026-11-06 22:16:38 +01:00
2026-08-25 03:08:15 +02:00
2026-10-27 01:21:56 +01:00
2026-11-04 17:28:52 +01:00
2026-07-19 15:38:53 +02:00
2026-11-06 05:36:20 +01:00
2026-11-06 04:31:26 +01:00
2026-09-23 11:40:03 +02:00
2026-10-21 04:11:28 +02:00
2026-09-14 14:54:41 +02:00
2026-08-23 08:39:03 +02:00
2026-11-06 06:05:53 +01:00
2026-09-21 03:03:31 +02:00
2026-05-09 03:23:34 +02:00
2026-11-08 09:01:00 +01:00
2026-10-15 19:10:38 +02:00
2026-09-14 15:53:22 +02:00
2026-10-24 14:32:36 +02:00
2026-11-06 06:05:53 +01:00
2026-10-09 02:40:04 +02:00
2026-10-14 21:23:58 +02:00
2026-03-21 07:20:18 +01:00
2026-03-21 07:20:18 +01:00
2026-11-06 02:35:41 +01:00
2026-11-06 06:05:53 +01:00
2026-08-09 05:29:45 +02:00
2026-03-01 06:05:26 +01:00
2026-02-26 16:44:34 +01:00
2026-08-10 11:15:47 +02:00
2026-08-13 19:40:54 +02:00
2026-11-06 13:11:22 +01:00
2026-10-21 03:51:31 +02:00
2026-11-06 07:40:07 +01:00
2026-02-18 12:51:35 +01:00
2026-10-08 14:55:02 +02:00
2026-08-07 04:33:15 +02:00
2026-11-06 23:55:02 +01:00
2026-08-14 10:03:10 +02:00
2026-07-27 14:00:26 +02:00
2026-10-09 08:20:02 +02:00
2026-10-26 03:13:25 -04:00
2026-08-08 23:00:36 +03:00
2026-10-08 14:55:02 +02:00
2026-10-10 06:33:00 +02:00
2026-08-22 14:13:58 +02:00
2026-10-24 14:45:34 +02:00
2026-06-15 08:40:05 +02:00
2026-10-11 12:36:59 -04:00
2026-11-06 22:16:38 +01:00
2026-08-06 01:28:17 +02:00
2026-08-06 01:28:17 +02:00
2026-08-04 15:02:46 +02:00
2026-09-14 23:38:54 +02:00
2026-08-15 06:38:26 +02:00