Introducing PKaI Nano 1.2: Better Data, Stronger Results
Our third from-scratch Nano model, built on a broader and cleaner training mix, with gains over Nano 1.1 on seven of eight suite metrics and a seven-task accuracy average of 51.69.
The Next Nano
Today we are releasing PKaI Nano 1.2, the third model in the Nano series and the strongest checkpoint we have shipped.
Nano 1.2 is a smarter model, and it got there three ways: a considerably broader and higher-quality training mix, improved training techniques refined over two previous Nano releases, and a deliberate size bump over Nano 1.1. Like every Nano before it, it was trained from scratch on our own stack.
Nano 1.2 improves on Nano 1.1 on seven of the eight metrics in our evaluation suite.
What Improved
Across seven accuracy benchmarks, Nano 1.2 averages 51.69, up from 48.83 for Nano 1.1 and 43.98 for Nano 1.
The clearest gains are on the harder language modeling tasks, which have now improved across three consecutive releases. LAMBADA OpenAI moved from 23.33 on Nano 1 to 33.82 on Nano 1.1 and now 44.07. WikiText perplexity tells the same story, falling from 50.13 to 32.33 and now 24.13. Lower is better on perplexity, so that is a 25.4 percent reduction from Nano 1.1, and less than half of where Nano 1 started.
One honest caveat: ARC-Challenge dipped slightly, from 28.24 to 27.73, the only metric that moved in the wrong direction. We noticed it, and we are working on improvements in all areas, including this one, for future releases.
| Benchmark | PKaI Nano 1.2 | PKaI Nano 1.1 | PKaI Nano 1 |
|---|---|---|---|
| HellaSwag | 39.71 | 36.52 | 31.02 |
| SciQ | 81.50 | 80.10 | 73.00 |
| PIQA | 65.94 | 64.96 | 59.74 |
| WinoGrande | 52.96 | 52.72 | 52.57 |
| ARC-Easy | 49.92 | 45.45 | 43.35 |
| ARC-Challenge | 27.73 | 28.24 | 24.83 |
| LAMBADA OpenAI | 44.07 | 33.82 | 23.33 |
| WikiText Perplexity | 24.13 | 32.33 | 50.13 |
Better Data, Not Just More Data
The biggest change behind Nano 1.2 is the training mix. Earlier Nano models trained on a narrower slice of web and synthetic text. This run broadened that foundation considerably, with more reference-style writing, more structured technical content, and a healthier balance across sources.
Everything still goes through the same in-house cleaning, decontamination, and deduplication process before training. What changed is what we let in, not the standards we hold it to.
We think this is the right way to improve models at this scale. Raw size is cheap. Clean, well-chosen data is what actually moves the numbers, and the results above back that up.
What This Model Is
PKaI Nano 1.2 is a base language model, not a chat assistant. It continues text rather than following instructions, and it has not been instruction-tuned or safety-tuned.
Like any small base model, it can be inconsistent and factually unreliable. We release it as a research milestone and as a public record of how far the stack has come.
What Comes Next
Nano 1.2 is our strongest release yet, but it is not the ceiling. The next PKaI models are already in development.
They apply everything the Nano series taught us about data quality and training stability, and they will be smarter and more capable in ways that actually matter. More on those soon.
Credits and Notes
PKaI Nano 1.2 was trained from scratch by PowderKeg Intelligence.
The model uses the Mistral-7B-v0.1 tokenizer from Mistral AI, released under the Apache 2.0 license.
LAMBADA OpenAI refers to the OpenAI-preprocessed LAMBADA test variant used in our evaluation setup.
Benchmarks are reported from our own evaluation setup and should be read as comparative signals between our releases, not as absolute claims.
Model Files
PKaI Nano 1.2 is released for free under the Apache 2.0 license, with model files available on Hugging Face.