EdgeFit Atlas

Measured on-device inference. Every number reproducible; every failure recorded.

bart-base

hf:facebook/bart-base · embed

Graph fingerprint

81.9M graph parameters (as exported) · 350 graph nodes · attention: mha · norm: layernorm

Constant×88, Add×63, MatMul×48, Mul×28, Reshape×26, Transpose×24, Where×14, LayerNormalization×13

Graph size by weight dtype: fp32 350 nodes, int8 484 nodes, fp32 585 nodes. Quantization inserts quantize/dequantize pairs, so the node count grows even as the artifact shrinks.

Latency by recipe

CPUaccelerator
0100200p50 latency, msALL · sm87502.00ALL · sm8750: 2.00 — cv 3.8%ALL · sm86502.66ALL · sm8650: 2.66 — cv 2.6%ALL · sm85503.74ALL · sm8550: 3.74 — cv 1.7%ort-cpu-int8-dyn · Apple M211.36ort-cpu-int8-dyn · Apple M2: 11.36 — cv 0.2%, 78 MiBort-cpu-int8-perchan · Apple M211.53ort-cpu-int8-perchan · Apple M2: 11.53 — cv 2.6%, 79 MiBort-cpu-fp32 · Apple M231.96ort-cpu-fp32 · Apple M2: 31.96 — cv 0.1%, 313 MiBort-cpu-4thread · Apple M232.00ort-cpu-4thread · Apple M2: 32.00 — cv 0.1%, 313 MiBunknown · Apple M232.59unknown · Apple M2: 32.59 — cv 0.2%, 78 MiB, 8 repeats spanning 11.37–53.42 msort-cpu-dynamic · Apple M233.10ort-cpu-dynamic · Apple M2: 33.10 — cv 0.3%, 313 MiBort-coreml-fp32 · Apple M238.68ort-coreml-fp32 · Apple M2: 38.68 — cv 1.1%, 313 MiBort-coreml-ane · Apple M246.43ort-coreml-ane · Apple M2: 46.43 — cv 7.2%, 313 MiBort-coreml-dynamic · Apple M258.02ort-coreml-dynamic · Apple M2: 58.02 — cv 13.8%, 313 MiBALL · google-tensor-g272.75ALL · google-tensor-g2: 72.75 — cv 4.3%ALL · google-tensor-g493.00ALL · google-tensor-g4: 93.00 — cv 5.6%ALL · sm8350108.20ALL · sm8350: 108.20 — cv 12.2%ALL · samsung-exynos-1280224.50ALL · samsung-exynos-1280: 224.50 — cv 4.2%ALL · sdm845226.91ALL · sdm845: 226.91 — cv 4.8%
Median of 10–97 timed runs after 3 discarded warmups. Hover a bar for variance and artifact size.

Speed against numerics

CPUaccelerator
0.980.991204060p50 latency, ms · lower is bettercosine vs fp32 referenceort-cpu-int8-dyn · Apple M2 — cv 0.2%, 78 MiBort-cpu-int8-perchan · Apple M2 — cv 2.6%, 79 MiBort-cpu-fp32 · Apple M2 — cv 0.1%, 313 MiBort-cpu-4thread · Apple M2 — cv 0.1%, 313 MiBunknown · Apple M2 — cv 0.2%, 78 MiB, 8 repeats spanning 11.37–53.42 msort-cpu-dynamic · Apple M2 — cv 0.3%, 313 MiBort-coreml-fp32 · Apple M2 — cv 1.1%, 313 MiBort-coreml-ane · Apple M2 — cv 7.2%, 313 MiBort-coreml-dynamic · Apple M2 — cv 13.8%, 313 MiB
Marker area scales with artifact size. Cosine is a numerics check against the fp32 PyTorch reference, not task accuracy — a model can hold cosine 0.999 and still fail on the slice that matters.

Repeatability

Recipes measured more than once on this unit, in separate sessions. This is the weakest useful form of the check — the real one runs the same recipe on two physical units of the same SKU, and we have one machine. Agreement here cannot catch a defect in the methodology that both runs share; it can only catch drift.

Recipesessionsfastest msslowest msspread
unknown · Apple M2811.3753.42370.01%

Every recipe measured

RecipeTargetOutcomep50 mscvsize MiBcosineFLOP fb (auth)time fb (run)partitions
ALLALLsuccess2.003.8%———0.0%—
ALLALLsuccess2.662.6%———0.0%—
ALLALLsuccess3.741.7%———0.0%—
ort-cpu-int8-dynCPUsuccess11.360.2%780.98050.0%0.0%—
unknownunknownsuccess11.370.2%780.98050.0%0.0%—
unknownunknownsuccess11.460.2%790.99080.0%0.0%—
ort-cpu-int8-perchanCPUsuccess11.532.6%790.99080.0%0.0%—
ort-cpu-fp32CPUsuccess31.960.1%3131.00000.0%0.0%—
ort-cpu-4threadCPUsuccess32.000.1%3131.00000.0%0.0%—
unknownunknownsuccess32.050.1%3131.00000.0%0.0%—
unknownunknownsuccess32.110.1%3131.00000.0%0.0%—
unknownunknownsuccess33.070.2%3131.0000—0.0%—
ort-cpu-dynamicCPUsuccess33.100.3%3131.0000—0.0%—
ort-coreml-fp32CoreMLsuccess38.681.1%3131.000099.8%38.5%31
unknownunknownsuccess38.970.6%3131.000099.8%31.8%31
unknownunknownsuccess41.532.0%3131.000099.8%34.6%31
ort-coreml-aneCoreMLsuccess46.437.2%3131.000099.8%38.5%31
unknownunknownsuccess53.420.5%3131.0000—16.3%45
ort-coreml-dynamicCoreMLsuccess58.0213.8%3131.0000—17.2%45
ALLALLsuccess72.754.3%———100.0%—
ALLALLsuccess93.005.6%———100.0%—
ALLALLsuccess108.2012.2%———100.0%—
ALLALLsuccess224.504.2%———100.0%—
ALLALLsuccess226.914.8%———100.0%—
unknownunknownlowering failure——156————
ort-coreml-fp16CoreMLlowering failure——156————
unknownunknownlowering failure———————
ort-coreml-mlprogramCoreMLlowering failure———————
ort-cpu-fp16CPUlowering failure——156————
unknownunknownlowering failure——156————

Recorded failures

Kept rather than hidden. A recipe that cannot lower is a fact about the toolchain, and it is what teaches the static filter not to propose it again.

unknown lowering failure
session creation failed: [ONNXRuntimeError] : 1 : FAIL : Load model from artifacts/onnx/facebook__bart-base__qac2c417fcab07239/model.onnx failed:Type Error: Type (tensor(float16)) of output arg (/inner/Cast_output_0) of node (/inner/Cast) does not match expected type (tensor(float)).
ort-coreml-fp16 lowering failure
session creation failed: [ONNXRuntimeError] : 1 : FAIL : Load model from artifacts/onnx/facebook__bart-base__qe502041bfe932a0c/model.onnx failed:Type Error: Type (tensor(float16)) of output arg (/inner/Cast_output_0) of node (/inner/Cast) does not match expected type (tensor(float)).
unknown lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<768x768xf32>) -> tensor<1x128x768xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'
ort-coreml-mlprogram lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<768x768xf32>) -> tensor<1x128x768xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'
ort-cpu-fp16 lowering failure
session creation failed: [ONNXRuntimeError] : 1 : FAIL : Load model from artifacts/onnx/facebook__bart-base__qe502041bfe932a0c/model.onnx failed:Type Error: Type (tensor(float16)) of output arg (/inner/Cast_output_0) of node (/inner/Cast) does not match expected type (tensor(float)).
unknown lowering failure
session creation failed: [ONNXRuntimeError] : 1 : FAIL : Load model from artifacts/onnx/facebook__bart-base__qac2c417fcab07239/model.onnx failed:Type Error: Type (tensor(float16)) of output arg (/inner/Cast_output_0) of node (/inner/Cast) does not match expected type (tensor(float)).

Reproduce

Commands for every row on this page
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Snapdragon 8 Elite QRD" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Samsung Galaxy S24" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Samsung Galaxy S23" --compute-unit all
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_int8_dynamic.yaml
# recipe 8874109ac191f342 is no longer in the library
# recipe 0d234a98080811e0 is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_int8_perchannel.yaml
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_fp32.yaml
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_4thread.yaml
# recipe dfd2274bc3b8d084 is no longer in the library
# recipe 01ed12e61ab71105 is no longer in the library
# recipe de6f5f655dba97d1 is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_dynamic.yaml
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_coreml_fp32.yaml
# recipe bc78cddd81cdbb10 is no longer in the library
# recipe 12f53b312e9e575a is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_coreml_ane.yaml
# recipe 03c715d6c22ec9fb is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_coreml_dynamic.yaml
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Google Pixel 7" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Google Pixel 9" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Samsung Galaxy S21" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Samsung Galaxy A53 5G" --compute-unit all
uv run edgefit measure-remote --model hf:facebook/bart-base --device "Google Pixel 3" --compute-unit all
# recipe e281ff992cfddabd is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_coreml_fp16.yaml
# recipe 9fd7903b253cc021 is no longer in the library
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_coreml_mlprogram.yaml
uv run edgefit measure --model hf:facebook/bart-base --recipe recipes/ort_cpu_fp16.yaml
# recipe 48270d6297b00fc1 is no longer in the library