family member
Llama 3.2 logo

Llama 3.2 3B

225 tokens a second on Windows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM. About $1.14 per million tokens here, against OpenRouter's $0.33: the API is 3.5× cheaper. Run it here for privacy or to keep work on your own machines, not to save money.

13 builds of Llama 3.2 3B, 17 signed runs on real machines: different packagers, precisions and engines, each a different file with its own numbers. 3.2B parameters, as the vendor states.

225tok/s, fastest machine
$1.14per M tokens · $0.33 on an API
128Kcontext proved by recall
1.2 GBmemory, smallest build
5providers measured it
Run it on your gram Install a gram with one command. It works through the catalog on your own machine and signs every result. 5 providers have measured Llama 3.2 3B so far.Use it in your own room Ask Evo in your gramx and it runs on a machine that proved it, at the metered $1.14 per million tokens. On your own machines it costs you nothing.Compare every build Put it beside every other model in one table: speed, price, context and the API price, one row each.
best on ollama · 13 builds

trained 128K · proved 128K context, by unsloth Q4_K_M

225.2 tok/s peak, by unsloth IQ1_S

Speed against price, every machine

Up and left is faster and cheaper · the line joins machines no other beats on both · price on a log scale · dashed: OpenRouter

$1$10USD per million tokenstok/sWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 1-bit by unsloth · 225.19 tok/s · $1.1411/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 4-bit by unsloth · 205.06 tok/s · $1.2569/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 4-bit by unsloth · 203.94 tok/s · $1.2642/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 3-bit by unsloth · 199.51 tok/s · $1.2897/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 2-bit by unsloth · 124.17 tok/s · $2.0703/MWindows · GeForce RTX 3080 Ti 12 GB · 16 CPUs · 16 GB RAM · 4-bit by bartowski · 99.3 tok/s · $2.5805/MLinux · L4 23 GB · 12 CPUs · 53 GB RAM · 4-bit by unsloth · 97.41 tok/s · $2.2303/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 16-bit (full precision) by unsloth · 94.89 tok/s · $2.7532/MLinux · NVIDIA 15 GB · 4 CPUs · 31 GB RAM · 1-bit by unsloth · 94.46 tok/s · $1.4323/MLinux · NVIDIA 15 GB · 4 CPUs · 31 GB RAM · 4-bit by bartowski · 86.87 tok/s · $1.5492/MLinux · NVIDIA 15 GB · 4 CPUs · 31 GB RAM · 3-bit by bartowski · 78.82 tok/s · $1.7074/MWindows · GeForce RTX 3080 Ti 12 GB · 16 CPUs · 16 GB RAM · 4-bit by maziyarpanahi · 78.03 tok/s · $3.284/MWindows · NVIDIA 12 GB · 16 CPUs · 16 GB RAM · 8-bit by unsloth · 72.81 tok/s · $3.5559/MLinux · NVIDIA 15 GB · 4 CPUs · 31 GB RAM · 3-bit by bartowski · 71.97 tok/s · $1.87/MWindows · Intel(R) UHD Graphics 730 2 GB · 8 CPUs · 16 GB RAM · 4-bit by unsloth · 7.31 tok/s · $9.9205/M

Same build, different machines

Q4_K_M by unsloth on 3 machines: the fastest decodes 25× as fast as the slowest. One dot per machine, tokens per second.

  1. NVIDIA GeForce RTX 3080 Ti×1204 tok/s
  2. NVIDIA L4×197 tok/s
  3. Intel(R) UHD Graphics 730×18.1 tok/s

What it fits on

The smallest build needs about 1.2 GB; 17 of 18 kinds of machine on the network can hold it. Bars: each kind's memory (its card's, else RAM) · ticks: every build, at the size it held plus a fifth for the prompt.

  1. Linux, CPU only · 2 CPUs1 GB RAMnone
  2. NVIDIA GeForce RTX 3050 Laptop GPU4 GB on the card5 of 12
  3. NVIDIA GeForce GTX 16504 GB on the card5 of 12
  4. AMD Radeon(TM) Vega 8 Mobile Graphics7 GB RAM5 of 12
  5. Windows, CPU only · 12 CPUs7 GB RAM5 of 12
  6. Apple silicon8 GB RAM5 of 12
  7. Apple M18 GB RAM5 of 12
  8. Mac, CPU only · 8 CPUs8 GB RAM5 of 12
  9. NVIDIA GeForce RTX 3080 Ti12 GB on the card6 of 12
  10. Tesla T415 GB on the card6 of 12
  11. Intel(R) UHD Graphics 73016 GB RAM6 of 12
  12. Intel(R) Iris(R) Xe Graphics16 GB RAM6 of 12
  13. Apple M416 GB RAM6 of 12
  14. Apple M116 GB RAM6 of 12
  15. NVIDIA L422 GB on the card7 of 12
  16. 2× Tesla T429 GB on the cardevery build
  17. Linux, CPU only · 4 CPUs31 GB RAMevery build
  18. NVIDIA RTX PRO 6000 Blackwell Server Edition95 GB on the cardevery build

How far back it remembers, machine by machine

Filled: the needle came back at that prompt size · outlined: asked and lost · blank: not asked · trained for 128K. 1 walk ended because the model forgot, 2 because the machine could not hold the prompt.

machine · build4K8K16K32K64K128Kwhy it stopped
Gram ·dae679NVIDIA L4 · Q4_K_M by unsloth128K recalled at every size asked
Gram ·a5be462× Tesla T4 · Q4_K_M by bartowski128K recalled at every size asked
Gram ·a5be462× Tesla T4 · Q3_K_L by bartowski128K recalled at every size asked
Gram ·a5be462× Tesla T4 · IQ3_M by bartowski128K recalled at every size asked
Gram ·a5be462× Tesla T4 · Q8_0 by bartowski128K recalled at every size asked
Gram ·a5be462× Tesla T4 · F16 by bartowski128K recalled at every size asked
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · Q4_K_M by bartowski64K this machine could not serve at 128K
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · F16 by unsloth16K an older probe: where it stopped was not recorded
Gram ·9e3c79Intel(R) UHD Graphics 730 · Q4_K_M by unsloth4K this machine could not serve at 8K
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · IQ1_S by unsloth— an older probe: where it stopped was not recorded
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · Q4_0 by unsloth— an older probe: where it stopped was not recorded
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · Q4_K_M by unsloth— an older probe: where it stopped was not recorded
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · IQ3_XXS by unsloth— an older probe: where it stopped was not recorded
Gram ·84bf54NVIDIA GeForce RTX 3080 Ti · IQ2_XXS by unsloth— an older probe: where it stopped was not recorded

The 14 longest of 17 walks; every one is on its build's page.

Carbon per million tokens

Lowest build on each machine · 3 of 3 read off a power sensor, the rest modelled from the parts · fewer grams is better

  1. Gram ·dae679158 g1.20 J/tok · sensor
  2. Gram ·a5be46798 g7.49 J/tok · sensor
  3. Gram ·84bf541.2 kg6.26 J/tok · sensor

Carbon is location-based: signed energy × the yearly average of the grid where the machine signed its place (world grid average (Singapore not in the table) · United States grid average · India grid average; Ember Global Electricity Review 2025 (2024 generation), location-based). Facility overhead is not included. A machine on its own solar is not credited until it can prove it.

Measured by

The people whose machines signed these numbers

proved by at least one build: Tool calling · Tool loop finished · Faithful to tool results · Structured output · Sustained output · Writes code · Reasons about code · Synthesises sources · Follows instructions

Every build

Each its own file with its own page: the machines that ran it and their signatures.

buildenginewhose workmemorycontext provedbest tok/smachines
Q4_K_M by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q4_K_Mollamaquantizer · unsloth19.5 GB128K203.93
Q4_K_M by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q4_K_Mollamabartowski24.9 GB128K99.32
Q3_K_L by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q3_K_Lollamabartowski24.7 GB128K78.81
IQ3_M by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:IQ3_Mollamabartowski24.5 GB128K72.01
Q8_0 by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0ollamabartowski26.5 GB128K62.81
F16 by bartowskihf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:F16ollamabartowski26.9 GB128K38.31
F16 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:F16ollamaquantizer · unsloth9.9 GB16K94.91
IQ1_S by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ1_Sollamaquantizer · unsloth1.6 GB—225.22
Q4_0 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q4_0ollamaquantizer · unsloth2.1 GB—205.11
IQ3_XXS by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ3_XXSollamaquantizer · unsloth1.5 GB—199.51
IQ2_XXS by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:IQ2_XXSollamaquantizer · unsloth1.2 GB—124.21
Q8_0 by unslothhf.co/unsloth/Llama-3.2-3B-Instruct-GGUF:Q8_0ollamaquantizer · unsloth3.8 GB—72.81
IQ1_S by maziyarpanahihf.co/MaziyarPanahi/Llama-3.2-3B-Instruct-GGUF:IQ1_Sollamamaziyarpanahi——45.71
derivatives — somebody else's change to these weights

Fine-tunes, merges and abliterations of this member, each its own model with its own page. None of their numbers are counted above.

Llama-3.2-3B-Instruct-uncensored1 build measuredLlama-3.2-3B-Instruct-Q4_K_M-GGUFnot measuredLlama-3.2-3B-Instructnot measuredLlama-3.2-3B-Instruct-abliterated-GGUFnot measuredLlama-3.2-3B-Instructnot measured