How good are small local models?
Our model, Claude Opus 4.5, and Claude Haiku 5.5 answered the same test items, and the same code scored them. The two Claude models received the task instructions and the allowed labels, data types, or field names.
Where does a small trained model lead?
We trained a separate small model for each task, and each one started from an open model.
Data privacy (Gretel PII)
Our model scored 88.1 percent. Claude Opus 4.5 scored 70.7 percent.
Added after the results.
Sample of 300 synthetic documents.
Everyday tasks (BANKING77)
Our model scored 81.5 percent. Claude Opus 4.5 scored 73.9 percent.
This BANKING77 score may be higher because some test messages closely match the examples used to train the local model.
Reading documents (CORD)
Our model scored 94.3 percent. Claude Opus 4.5 scored 89.9 percent.
Added after the results.
Development result, 96 receipts.
The score leaves out 4 test receipts, because their answer keys hold lists.
Where do big cloud models lead?
Big cloud models lead most on long tasks with many steps.
On many-turn tasks we score 15.0 and Claude Opus 4.5 scores 81.0.
The full results show each score and each loss.