§ 05 · Practice
Frontier-quality answers, laptop-sized models.
For narrow, well-defined tasks, a small model trained on the right data beats a large general one, and it runs on the MacBook already on your desk. We select, fine-tune, quantise and package models that work fully offline.
§ 05.1 / Scope
Capabilities.
Model selection
Baseline evaluations across open model families (Qwen, Llama, Gemma, Phi and others) on your actual task, with licensing checked.
Fine-tuning
Supervised and preference tuning with LoRA and full fine-tunes; distillation from frontier models when you lack labelled data.
Quantisation & packaging
4-bit and 8-bit quantisation for MLX, llama.cpp and Core ML. Accuracy loss measured, not assumed.
Native deployment
Signed macOS apps, command-line tools or a local API server. Works on a plane, in a clean room, or behind an air gap.
Profiling
Tokens per second, memory footprint, thermal and battery behaviour on the M-series chips your team actually uses.
Lifecycle
An evaluation and update pipeline so the model improves as your data does, without regressions.
§ 05.2 / Output
Deliverables.
- —Model weights, the training recipe and an evaluation report
- —A signed macOS app or CLI, or a local server
- —Fine-tuning and evaluation code you own
- —An update playbook for the next version
Tools & Standards.
nf-core and community standards where they exist, custom software where they do not.
- MLX
- llama.cpp
- Core ML
- PyTorch
- Hugging Face
- LoRA / QLoRA
- Unsloth
- Swift
- lm-eval
§ 05.3 / Questions
Frequently asked questions.
- Why a 3B model instead of a frontier API?
- Privacy, cost and latency. A 3B model in 4-bit fits in about 2 GB of memory and runs at tens of tokens per second on an M-series chip, with no per-token cost and no data leaving the device. For narrow tasks, fine-tuned small models match or beat general large ones.
- What tasks are a good fit?
- Classification and extraction, summarising reports, protocol Q&A, variant and literature triage, code and command generation for your tools. Open-ended research questions still belong with a frontier model.
- Which Macs are supported?
- Any Apple Silicon Mac. We target 8 GB machines as the floor and profile on M1 through M4 families.
§ Practice Scoping · 05
Scope a project in on-device ai models.
Tell us about your assay types, dataset scale, and timelines. We reply with a preliminary scoping memo within two working days.
Scope On-device AI Models →