blog

hearth llm case study, rerun on a 2060

2026-10-14

verdict

i swapped gpus this week, so i reran the first hearth case study on the new card. hearth still runs granite4.2:3b.

  1. warm answers got 38% to 2.4x faster on three of the four models. raw generation speed only went up 8 to 15%, so most of the gain came from reading the prompt faster.
  2. the tev1 router got 4.7x faster per decision. it still sends the same question to the wrong tool, so it stays out.
  3. accuracy didn’t change in any way i can pin on the card. granite4.2:3b picked the right tool in all 45 answers and got the facts right in 44.
  4. both 8b models still stop at about 4.7 GB on the gpu and spill the rest onto the cpu. they stopped at the same point on the 1660 ti, so the limit comes from having 6 GB.

what changed

the homelab got an intel arc a750 for transcoding and photo ml, and the gtx 1660 ti came out. ollama now runs on the rtx 2060, which it shares only with an iptv transcoder that sat idle the whole run.

both cards are turing, and both have 6 GB. the 2060 has about 17% more memory bandwidth, and it has tensor cores, which the 1660 ti doesn’t.

ollama also changed: the first chat runs were on 0.33.3, and this run is on 0.40.3. the router was already on 0.40.3 the first time, so the router numbers are the only clean card-against-card pair.

method

the method is the same as last time, so the numbers line up:

  1. a frozen fixture, so every model saw identical tool output.
  2. 15 questions, 4 models and 3 interleaved runs (180 answers), with every model unloaded before each run.
  3. the speed probe, plus 3 interleaved runs of tev1 at both sizes.
  4. this time i also waited for a cpu-heavy encode job to finish first. the 8b models spill onto the cpu, so a busy cpu would have skewed exactly the numbers being compared.

the scorer was wrong again

i read every miss by hand, like last time. four were the scorer’s fault: “8 hours old” where it only accepted “8 h old”, “2.1 tb” where it wanted 2102 gb, and two more like that. i added those phrasings as alternatives and re-scored the first run with the new list, and its numbers didn’t move.

the vram measurement needed a fix too. the gpu exporter lags behind the card, and on this run the lag sometimes outlasted the 30 s the analysis skips at the start of each window, so one of granite4.2:3b’s three windows still showed the previous 8b model’s 4.7 GB. each model’s peak is now the median of its three per-run peaks. that drops the stale window and leaves the first run’s numbers unchanged.

speed

once loaded, three of the four models answer much faster. granite4.2:8b barely moves, because the quarter of it running on the cpu sets its pace.

warm median answer · seconds, lower is faster GTX 1660 Ti · 2026-10-11 RTX 2060 · 2026-10-14
warm median answer · seconds, lower is fastertwo bars per model, GTX 1660 Ti then RTX 2060, each labelled with its value, and the speed-up on the right of the model name. full values are in the table that follows.granite4.2:3b · GTX 1660 Ti 6.5 s · RTX 2060 4.7 s · +38% fastergranite4.2:3b+38% faster6.5 s4.7 sqwen3.5:4b · GTX 1660 Ti 5.3 s · RTX 2060 2.9 s · 1.8× fasterqwen3.5:4b1.8× faster5.3 s2.9 sgranite4.2:8b · GTX 1660 Ti 12.9 s · RTX 2060 12.4 s · +4% fastergranite4.2:8b+4% faster12.9 s12.4 slfm2.5:8b · GTX 1660 Ti 4.5 s · RTX 2060 1.9 s · 2.4× fasterlfm2.5:8b2.4× faster4.5 s1.9 s
warm median answer · seconds, lower is faster
modelGTX 1660 TiRTX 2060change
granite4.2:3b6.5 s4.7 s+38% faster
qwen3.5:4b5.3 s2.9 s1.8× faster
granite4.2:8b12.9 s12.4 s+4% faster
lfm2.5:8b4.5 s1.9 s2.4× faster

generation speed rose a lot less, which fits the bandwidth gap between the cards.

warm generation speed · tokens per second, higher is faster GTX 1660 Ti · 2026-10-11 RTX 2060 · 2026-10-14
warm generation speed · tokens per second, higher is fastertwo bars per model, GTX 1660 Ti then RTX 2060, each labelled with its value, and the speed-up on the right of the model name. full values are in the table that follows.granite4.2:3b · GTX 1660 Ti 90.1 tok/s · RTX 2060 103.6 tok/s · +15% fastergranite4.2:3b+15% faster90.1 tok/s103.6 tok/sqwen3.5:4b · GTX 1660 Ti 66.6 tok/s · RTX 2060 76.8 tok/s · +15% fasterqwen3.5:4b+15% faster66.6 tok/s76.8 tok/sgranite4.2:8b · GTX 1660 Ti 22.3 tok/s · RTX 2060 25.0 tok/s · +12% fastergranite4.2:8b+12% faster22.3 tok/s25.0 tok/slfm2.5:8b · GTX 1660 Ti 132.8 tok/s · RTX 2060 143.8 tok/s · +8% fasterlfm2.5:8b+8% faster132.8 tok/s143.8 tok/s
warm generation speed · tokens per second, higher is faster
modelGTX 1660 TiRTX 2060change
granite4.2:3b90.1 tok/s103.6 tok/s+15% faster
qwen3.5:4b66.6 tok/s76.8 tok/s+15% faster
granite4.2:8b22.3 tok/s25.0 tok/s+12% faster
lfm2.5:8b132.8 tok/s143.8 tok/s+8% faster

so most of the gain is in prompt processing. a hearth answer reads a long prompt (ten tool descriptions, then the tool’s output) and writes a short reply, so reading the prompt is a big share of every answer. tensor cores speed up that kind of matrix-heavy work. i didn’t time prompt processing on its own, and ollama also changed between the runs, so i can’t prove that’s the cause.

the router, again

this is the clean comparison, because the router ran on the same ollama version both times. tev1 makes one choice per question, so a routing call is almost all prompt processing, which is where the new card helps most.

tev1 routing decision · warm median seconds, lower is faster GTX 1660 Ti · 2026-10-11 RTX 2060 · 2026-10-14
tev1 routing decision · warm median seconds, lower is fastertwo bars per model, GTX 1660 Ti then RTX 2060, each labelled with its value, and the speed-up on the right of the model name. full values are in the table that follows.tev1:4b · GTX 1660 Ti 1.87 s · RTX 2060 0.40 s · 4.7× fastertev1:4b4.7× faster1.87 s0.40 stev1:0.8b · GTX 1660 Ti 0.36 s · RTX 2060 0.14 s · 2.6× fastertev1:0.8b2.6× faster0.36 s0.14 s
tev1 routing decision · warm median seconds, lower is faster
modelGTX 1660 TiRTX 2060change
tev1:4b1.87 s0.40 s4.7× faster
tev1:0.8b0.36 s0.14 s2.6× faster

tev1:4b now decides in 0.40 s, against 1.87 s on the 1660 ti. it gave the same answers as before, run after run, with nearly the same confidence. that includes sending the data-disk “what’s using the space” question to overall disk usage every time, so it’s still at 93.3% right tool. tev1:4b plus granite4.2:3b also still needs about 7.2 GB, which doesn’t fit a 6 GB card.

accuracy

accuracy · % of 45 answers per model granite4.2:3b · production other models
accuracy · % of 45 answers per modeltwo bars per model: share of answers that called the right tool, and share that contained the right facts. sorted by facts.granite4.2:3b is highlighted. full values are in the table that follows.granite4.2:3b · right tool 100.0% · right facts 97.8%granite4.2:3bTOOL100.0%FACTS97.8%granite4.2:8b · right tool 100.0% · right facts 97.8%granite4.2:8bTOOL100.0%FACTS97.8%qwen3.5:4b · right tool 100.0% · right facts 93.3%qwen3.5:4bTOOL100.0%FACTS93.3%lfm2.5:8b · right tool 82.2% · right facts 80.0%lfm2.5:8bTOOL82.2%FACTS80.0%
accuracy · % of 45 answers per model
modelright toolright facts
granite4.2:3b (production)100.0%97.8%
granite4.2:8b100.0%97.8%
qwen3.5:4b100.0%93.3%
lfm2.5:8b82.2%80.0%

the accuracy numbers moved a little in both directions, and nothing points to the card. hearth doesn’t fix a sampling seed, so answers vary between runs, and 45 answers per model is a small sample. last time lfm2.5:8b scored 12, 15 and 13 out of 15 on tools across three runs, and this time 13, 11 and 13.

the misses look like last time’s:

  1. lfm2.5:8b still calls the wrong tool on the “what’s using the space” questions, and it sent “is sonarr importing ok?” to the logs instead of the queue.
  2. qwen3.5:4b twice said sonarr was fine when the queue had stalled items, the worst kind of answer for an ops bot.
  3. granite4.2:3b’s one fact miss was running out of steps on the sonarr question.

vram

peak vram · RTX 2060 (6 GB) granite4.2:3b · production other models
peak vram · RTX 2060 (6 GB)one bar per model: peak vram during its runs against the whole 6 GB card (6,144 MiB). models ollama could not fit entirely on the gpu are marked spilled to cpu.granite4.2:3b is highlighted. full values are in the table that follows.6 GB CARDgranite4.2:3b · peak vram 2,501 MiB · on gpu 100%granite4.2:3b2,501 MiBqwen3.5:4b · peak vram 3,749 MiB · on gpu 100%qwen3.5:4b3,749 MiBgranite4.2:8b · peak vram 4,693 MiB · on gpu 78%granite4.2:8bSPILLED TO CPU4,693 MiBlfm2.5:8b · peak vram 4,693 MiB · on gpu 89%lfm2.5:8bSPILLED TO CPU4,693 MiB
peak vram · RTX 2060 (6 GB)
modelpeak vramon gpu
granite4.2:3b (production)2,501 MiB100%
qwen3.5:4b3,749 MiB100%
granite4.2:8b4,693 MiB78%
lfm2.5:8b4,693 MiB89%

the peaks are within 12 MiB of the 1660 ti’s. the 8b models stop at the same ~4.7 GB on both cards, with 78% and 89% of them on the gpu, so that limit comes from how ollama budgets a 6 GB card.

the decision

hearth stays on granite4.2:3b. the faster card helped every model, but the order didn’t change:

  1. granite4.2:8b tied the 3b on accuracy this time and still doesn’t fit on the card.
  2. lfm2.5:8b is now the fastest by far, at a 1.9 s median, and still picks the wrong tool on nearly 1 in 5 answers.
  3. qwen3.5:4b answers in 2.9 s and picks the right tool every time, but it’s the one that says things are fine when they aren’t.
  4. granite4.2:3b answers in 4.7 s, down from 6.5 s, fits in 2.5 GB, and when it missed, it ran out of steps instead of giving a wrong answer.

everything here is from the 2026-10-14 runs, scored with the same script as the first study.

← back