EdgeFit Atlas

Measured on-device inference. Every number reproducible; every failure recorded.

toxic-comment-model

hf:martin-ha/toxic-comment-model · classify

Graph fingerprint

67.0M graph parameters (as exported) · 564 graph nodes · attention: mha · norm: layernorm

Constant×183, Add×62, Unsqueeze×61, MatMul×48, Reshape×27, Concat×26, Mul×26, Transpose×24

Graph size by weight dtype: fp32 342 nodes, fp16 343 nodes, int8 486 nodes, fp32 564 nodes. Quantization inserts quantize/dequantize pairs, so the node count grows even as the artifact shrinks.

Latency by recipe

CPUaccelerator
0100200p50 latency, msALL · sm87502.26ALL · sm8750: 2.26 — cv 3.5%ALL · sm86502.62ALL · sm8650: 2.62 — cv 1.5%ALL · sm85503.74ALL · sm8550: 3.74 — cv 1.1%ort-cpu-int8-dyn · Apple M211.35ort-cpu-int8-dyn · Apple M2: 11.35 — cv 0.3%, 64 MiBort-cpu-int8-perchan · Apple M211.43ort-cpu-int8-perchan · Apple M2: 11.43 — cv 0.1%, 64 MiBort-cpu-fp32 · Apple M231.97ort-cpu-fp32 · Apple M2: 31.97 — cv 0.7%, 255 MiBort-cpu-4thread · Apple M232.00ort-cpu-4thread · Apple M2: 32.00 — cv 0.3%, 255 MiBort-cpu-dynamic · Apple M233.16ort-cpu-dynamic · Apple M2: 33.16 — cv 0.2%, 256 MiBunknown · Apple M234.46unknown · Apple M2: 34.46 — cv 0.1%, 64 MiB, 10 repeats spanning 11.37–53.93 msort-coreml-fp16 · Apple M235.50ort-coreml-fp16 · Apple M2: 35.50 — cv 0.2%, 128 MiBort-cpu-fp16 · Apple M235.55ort-cpu-fp16 · Apple M2: 35.55 — cv 0.3%, 128 MiBort-coreml-ane · Apple M237.92ort-coreml-ane · Apple M2: 37.92 — cv 4.2%, 255 MiBort-coreml-fp32 · Apple M238.45ort-coreml-fp32 · Apple M2: 38.45 — cv 4.7%, 255 MiBort-coreml-dynamic · Apple M251.75ort-coreml-dynamic · Apple M2: 51.75 — cv 0.5%, 256 MiBALL · google-tensor-g291.83ALL · google-tensor-g2: 91.83 — cv 9.1%ALL · google-tensor-g491.88ALL · google-tensor-g4: 91.88 — cv 6.3%ALL · sm8350109.25ALL · sm8350: 109.25 — cv 11.2%ALL · sdm845220.64ALL · sdm845: 220.64 — cv 5.0%ALL · samsung-exynos-1280236.29ALL · samsung-exynos-1280: 236.29 — cv 6.3%
Median of 10–97 timed runs after 3 discarded warmups. Hover a bar for variance and artifact size.

Speed against numerics

CPUaccelerator
0.9999940.9999960.99999812040p50 latency, ms · lower is bettercosine vs fp32 referenceort-cpu-int8-dyn · Apple M2 — cv 0.3%, 64 MiBort-cpu-int8-perchan · Apple M2 — cv 0.1%, 64 MiBort-cpu-fp32 · Apple M2 — cv 0.7%, 255 MiBort-cpu-4thread · Apple M2 — cv 0.3%, 255 MiBort-cpu-dynamic · Apple M2 — cv 0.2%, 256 MiBunknown · Apple M2 — cv 0.1%, 64 MiB, 10 repeats spanning 11.37–53.93 msort-coreml-fp16 · Apple M2 — cv 0.2%, 128 MiBort-cpu-fp16 · Apple M2 — cv 0.3%, 128 MiBort-coreml-ane · Apple M2 — cv 4.2%, 255 MiBort-coreml-fp32 · Apple M2 — cv 4.7%, 255 MiBort-coreml-dynamic · Apple M2 — cv 0.5%, 256 MiB
Marker area scales with artifact size. Cosine is a numerics check against the fp32 PyTorch reference, not task accuracy — a model can hold cosine 0.999 and still fail on the slice that matters.

Repeatability

Recipes measured more than once on this unit, in separate sessions. This is the weakest useful form of the check — the real one runs the same recipe on two physical units of the same SKU, and we have one machine. Agreement here cannot catch a defect in the methodology that both runs share; it can only catch drift.

Recipesessionsfastest msslowest msspread
unknown · Apple M21011.3753.93374.36%

Every recipe measured

RecipeTargetOutcomep50 mscvsize MiBcosineFLOP fb (auth)time fb (run)partitions
ALLALLsuccess2.263.5%———0.0%—
ALLALLsuccess2.621.5%———0.0%—
ALLALLsuccess3.741.1%———0.0%—
ort-cpu-int8-dynCPUsuccess11.350.3%641.00000.0%0.0%—
unknownunknownsuccess11.370.1%641.00000.0%0.0%—
ort-cpu-int8-perchanCPUsuccess11.430.1%641.00000.0%0.0%—
unknownunknownsuccess11.520.2%641.00000.0%0.0%—
ort-cpu-fp32CPUsuccess31.970.7%2551.0000—0.0%—
unknownunknownsuccess32.000.3%2551.0000—0.0%—
ort-cpu-4threadCPUsuccess32.000.3%2551.0000—0.0%—
unknownunknownsuccess32.090.1%2551.0000—0.0%—
ort-cpu-dynamicCPUsuccess33.160.2%2561.0000—0.0%—
unknownunknownsuccess33.370.2%2561.0000—0.0%—
ort-coreml-fp16CoreMLsuccess35.500.2%1281.0000—100.0%—
unknownunknownsuccess35.540.2%1281.0000—0.0%—
ort-cpu-fp16CPUsuccess35.550.3%1281.0000—0.0%—
unknownunknownsuccess35.650.2%1281.0000—100.0%—
ort-coreml-aneCoreMLsuccess37.924.2%2551.0000—39.6%32
ort-coreml-fp32CoreMLsuccess38.454.7%2551.0000—40.3%32
unknownunknownsuccess39.513.6%2551.0000—37.1%32
unknownunknownsuccess41.234.4%2551.0000—38.8%32
ort-coreml-dynamicCoreMLsuccess51.750.5%2561.0000—19.3%45
unknownunknownsuccess53.930.8%2561.0000—22.8%45
ALLALLsuccess91.839.1%———100.0%—
ALLALLsuccess91.886.3%———100.0%—
ALLALLsuccess109.2511.2%———100.0%—
ALLALLsuccess220.645.0%———100.0%—
ALLALLsuccess236.296.3%———100.0%—
unknownunknownlowering failure———————
ort-coreml-mlprogramCoreMLlowering failure———————

Recorded failures

Kept rather than hidden. A recipe that cannot lower is a fact about the toolchain, and it is what teaches the static filter not to propose it again.

unknown lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<768x768xf32>) -> tensor<1x128x768xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'
ort-coreml-mlprogram lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<768x768xf32>) -> tensor<1x128x768xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'

Reproduce

Commands for every row on this page
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Snapdragon 8 Elite QRD" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Samsung Galaxy S24" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Samsung Galaxy S23" --compute-unit all
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_int8_dynamic.yaml
# recipe 8c11df0af5628081 is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_int8_perchannel.yaml
# recipe a9cbb3efc7d5779f is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_fp32.yaml
# recipe 5d4307e8d8b29be9 is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_4thread.yaml
# recipe b65bc764aaf66e4a is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_dynamic.yaml
# recipe 9d160f2de995f9e0 is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_coreml_fp16.yaml
# recipe d4d7059281332866 is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_cpu_fp16.yaml
# recipe caf2e243f674fe7f is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_coreml_ane.yaml
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_coreml_fp32.yaml
# recipe 9c96e8c6f44bf3fc is no longer in the library
# recipe df96524e1d1b528a is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_coreml_dynamic.yaml
# recipe 332688f680191a2b is no longer in the library
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Google Pixel 7" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Google Pixel 9" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Samsung Galaxy S21" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Google Pixel 3" --compute-unit all
uv run edgefit measure-remote --model hf:martin-ha/toxic-comment-model --device "Samsung Galaxy A53 5G" --compute-unit all
# recipe fdcfa8bf22c61340 is no longer in the library
uv run edgefit measure --model hf:martin-ha/toxic-comment-model --recipe recipes/ort_coreml_mlprogram.yaml