MacBrains Labs

Where did it really run?

I build language models for the Neural Engine in my MacBook and write down what actually happened.

Where the work ran, whether the numbers came out right, and what I still can’t prove. Every number here names the exact model build it was measured on, and the failures stay in.

See what I foundHow I work

New to the Neural Engine? Start with the one-minute guide

That’s Notch, the lab dog. No sound, and Notch holds still if your device asks for reduced motion.

Where did it run? · Qwen2.5 1.5B, 28 layers, LUT4

Mostly assigned to the Neural Engine.

Before I say anything about the hardware, I check where the compiler actually put each operation.

2,147operations assigned to the Neural Engine
82operations assigned to the CPU
2,229 cells, one per operation: 2,147 assigned to the Neural Engine, 82 assigned to the CPU. 2,229 cells, one per operation: 2,147 assigned to the Neural Engine, 82 assigned to the CPU.
Shipped build, rerun on 30 October 2026, infer function, one cell per operation96.3% / 3.7%

What this shows

Apple’s MLComputePlan reports which device each operation is assigned to, in this exact compiled model. The 82 on the CPU are the argmax at the very end of the language-model head, the step that picks the output token. An earlier June 2026 build of the same model reported 2,090 ANE / 79 CPU, with the same argmax tail on the CPU. Both runs are on the evidence page.

ane_compute_plan.swift, lines 14 to 21
if let usage = plan.deviceUsage(for: op) {
    switch usage.preferred {
    case .cpu: name = "CPU"
    case .gpu: name = "GPU"
    case .neuralEngine: name = "ANE"
    @unknown default: name = "?"
    }
} else { name = "none" }
This is the part of my tool that asks Core ML where each operation goes. Note the word preferred. It’s the plan, not a trace. Full source

What it doesn’t prove

An assignment isn’t proof the work stayed on the Neural Engine while it ran. It also says nothing about speed, or about whether the model’s answers are any good.

See the placement evidence

Was the math right? · LFM2.5 1.2B, single layers and the full 16-layer chain

Every slice passed. The chain didn’t.

That failure was useful. It changed what I tested next.

What happened

Each layer slice on its own, and the final head, matched the reference within a 0.005 relative-L2 gate on the Neural Engine. Put all 16 layers together and the chain and its token logits didn’t.

What it doesn’t prove

Layers that pass one at a time don’t add up to a correct model. The whole chain has to be tested.

Read the chain investigation

Is it actually faster? · A18 Pro, decode speed, batch sweeps

I was wrong about the speed limit.

The measurements didn’t match my explanation, so the explanation had to go.

What I thought

A 57.3 tok/s ceiling set by memory bandwidth.

What the measurements showed

Smaller weights didn’t speed it up. More work per dispatch did.

How I got it wrong

I got that ceiling by dividing a compiler planning figure by the size of the weight file. Then smaller LUT2 weights gave no matching speedup, and putting more work in each dispatch went right past the supposed limit. So I retracted it. At the full 28-layer production depth, per-layer compute takes up more of each step than the fixed cost of a dispatch.

What it doesn’t prove

The batch sweep timed probe builds whose output quality wasn’t validated. A later perplexity check on the B=4 probe found its logits roughly 1000 times larger than the B=1 model’s, and the records don’t resolve that. A speed result also only holds for its own device, build and run conditions. Not for every model or workload.

Read the measurement notes

How I work

I keep what I built, what I saw and what I think it means in separate piles.

Raw output excerpt, shipped build, rerun on 30 October 2026
total ops: 6251
  none   4022  (64.3%)
  ANE   2147  (34.3%)
  CPU     82  (1.3%)

=== assigned-compute ops only (ANE+CPU+GPU = 2229) ===
  ANE  2147  (96.3% of compute)
  CPU    82  (3.7% of compute)
  GPU     0  (0.0% of compute)
What my compute-plan tool printed for the shipped build of qwen25_monolithic_full_lut4.mlmodelc, infer function. The 4022 “none” entries are 3808 constants, 213 constexpr_lut_to_dense weight expansions and 1 identity, so none of them gets a device. The whole file
Build the real thing

I start from the actual program and how it gets compiled. Native Swift, MIL I write by hand, and Apple’s public Core ML tools. That keeps every boundary out in the open. No entitlement bypasses and no private kernel access.

The build stack
Count every operation

I go through state reads and writes, projections, norms and the output tail. What the compiler says it planned stays separate from any evidence of what actually ran.

The placement audit
Keep the failures

I compare the numbers at every boundary I can expose. When a chain fails, an output tail lands somewhere I didn’t expect, or a hypothesis falls apart, it stays in the record next to what worked.

The parity case study

The whole notebook

Short notes on what passed, what drifted and which ideas didn’t hold up, each tied to its sources. Plus archived releases you can check yourself, without taking my word for it.

Notes

Publication archive

Full publication feed · Search the evidence explorer

Who runs this

See the full research record

I’m Nayte Wilson. MacBrains Labs is my independent research notebook on Apple Silicon: how programs get compiled, where the work gets assigned, and what the results can actually back up. The methods are open, and the limits are part of the work, not footnotes.

I’m not affiliated with Apple, and I don’t have any special access to the Neural Engine. No result here is universal. Each measurement is about the exact model build, backend, tensor shape and hardware it names.

My ORCID iD is 0009-0003-4026-8212. The public measurement summaries are archived on Zenodo under DOI 10.5281/zenodo.23798037. If you cite them, cite that version DOI. And to be clear, a DOI means it’s archived. It doesn’t mean it was peer reviewed.

I don’t do all of this by hand. AI agents do a lot of the building, running and first-pass checking, under my direction. When this site says “I ran” or “I measured”, that’s what it means.

The tools get the same treatment. Two security advisories document bugs that were found and fixed in my private research automation.