EdgeFit Atlas

Measured on-device inference. Every number reproducible; every failure recorded.

all-MiniLM-L6-v2

hf:sentence-transformers/all-MiniLM-L6-v2 · embed

Graph fingerprint

22.6M graph parameters (as exported) · 561 graph nodes · attention: mha · norm: layernorm

Constant×182, Add×63, Unsqueeze×61, MatMul×48, Reshape×27, Concat×26, Mul×26, Transpose×24

Graph size by weight dtype: fp32 339 nodes, fp16 340 nodes, int8 474 nodes, fp32 561 nodes. Quantization inserts quantize/dequantize pairs, so the node count grows even as the artifact shrinks.

Latency by recipe

CPUaccelerator
0204060p50 latency, msALL · sc8480xp0.93ALL · sc8480xp: 0.93 — cv 3.6%ALL · sm87501.23ALL · sm8750: 1.23 — cv 14.0%ALL · sm86501.52ALL · sm8650: 1.52 — cv 1.9%ALL · sm86501.52ALL · sm8650: 1.52 — cv 2.7%ALL · sm85502.18ALL · sm8550: 2.18 — cv 3.7%ALL · sc8340xp2.19ALL · sc8340xp: 2.19 — cv 1.3%ALL · sc8380xp2.24ALL · sc8380xp: 2.24 — cv 1.7%ALL · qcs90752.53ALL · qcs9075: 2.53 — cv 0.7%ALL · qualcomm-sa8775p2.77ALL · qualcomm-sa8775p: 2.77 — cv 6.5%ALL · qualcomm-sa8295p3.64ALL · qualcomm-sa8295p: 3.64 — cv 1.1%ort-cpu-int8-dyn · Apple M24.46ort-cpu-int8-dyn · Apple M2: 4.46 — cv 0.3%, 22 MiBort-cpu-int8-perchan · Apple M24.49ort-cpu-int8-perchan · Apple M2: 4.49 — cv 0.3%, 22 MiBort-cpu-4thread · Apple M29.59ort-cpu-4thread · Apple M2: 9.59 — cv 0.7%, 86 MiBort-cpu-fp32 · Apple M29.59ort-cpu-fp32 · Apple M2: 9.59 — cv 0.2%, 86 MiBort-cpu-dynamic · Apple M210.38ort-cpu-dynamic · Apple M2: 10.38 — cv 0.3%, 86 MiBunknown · Apple M210.89unknown · Apple M2: 10.89 — cv 0.3%, 22 MiB, 10 repeats spanning 4.47–21.81 msort-cpu-fp16 · Apple M211.37ort-cpu-fp16 · Apple M2: 11.37 — cv 0.4%, 43 MiBort-coreml-fp16 · Apple M211.37ort-coreml-fp16 · Apple M2: 11.37 — cv 0.1%, 43 MiBort-coreml-fp32 · Apple M218.99ort-coreml-fp32 · Apple M2: 18.99 — cv 4.5%, 86 MiBort-coreml-ane · Apple M220.16ort-coreml-ane · Apple M2: 20.16 — cv 1.2%, 86 MiBALL · google-tensor-g220.84ALL · google-tensor-g2: 20.84 — cv 4.6%ALL · google-tensor-g521.14ALL · google-tensor-g5: 21.14 — cv 5.8%ort-coreml-dynamic · Apple M221.44ort-coreml-dynamic · Apple M2: 21.44 — cv 0.6%, 86 MiBALL · google-tensor-g422.73ALL · google-tensor-g4: 22.73 — cv 8.2%ALL · sm732529.67ALL · sm7325: 29.67 — cv 3.4%ALL · sm835030.00ALL · sm8350: 30.00 — cv 5.9%ALL · sm8250-ab30.94ALL · sm8250-ab: 30.94 — cv 26.3%ALL · sm775031.49ALL · sm7750: 31.49 — cv 4.4%ALL · sm815033.69ALL · sm8150: 33.69 — cv 4.7%ALL · samsung-exynos-133060.42ALL · samsung-exynos-1330: 60.42 — cv 7.6%ALL · sdm84562.63ALL · sdm845: 62.63 — cv 5.4%ALL · qcs649064.04ALL · qcs6490: 64.04 — cv 4.0%ALL · sm6150-ac64.40ALL · sm6150-ac: 64.40 — cv 7.5%ALL · samsung-exynos-128065.03ALL · samsung-exynos-1280: 65.03 — cv 17.0%
Median of 10–97 timed runs after 3 discarded warmups. Hover a bar for variance and artifact size.

Speed against numerics

CPUaccelerator
0.980.9911020p50 latency, ms · lower is bettercosine vs fp32 referenceort-cpu-int8-dyn · Apple M2 — cv 0.3%, 22 MiBort-cpu-int8-perchan · Apple M2 — cv 0.3%, 22 MiBort-cpu-4thread · Apple M2 — cv 0.7%, 86 MiBort-cpu-fp32 · Apple M2 — cv 0.2%, 86 MiBort-cpu-dynamic · Apple M2 — cv 0.3%, 86 MiBunknown · Apple M2 — cv 0.3%, 22 MiB, 10 repeats spanning 4.47–21.81 msort-cpu-fp16 · Apple M2 — cv 0.4%, 43 MiBort-coreml-fp16 · Apple M2 — cv 0.1%, 43 MiBort-coreml-fp32 · Apple M2 — cv 4.5%, 86 MiBort-coreml-ane · Apple M2 — cv 1.2%, 86 MiBort-coreml-dynamic · Apple M2 — cv 0.6%, 86 MiB
Marker area scales with artifact size. Cosine is a numerics check against the fp32 PyTorch reference, not task accuracy — a model can hold cosine 0.999 and still fail on the slice that matters.

Repeatability

Recipes measured more than once on this unit, in separate sessions. This is the weakest useful form of the check — the real one runs the same recipe on two physical units of the same SKU, and we have one machine. Agreement here cannot catch a defect in the methodology that both runs share; it can only catch drift.

Recipesessionsfastest msslowest msspread
unknown · Apple M2104.4721.81388.07%

Every recipe measured

RecipeTargetOutcomep50 mscvsize MiBcosineFLOP fb (auth)time fb (run)partitions
ALLALLsuccess0.933.6%———0.0%—
ALLALLsuccess1.2314.0%———0.0%—
ALLALLsuccess1.521.9%———0.0%—
ALLALLsuccess1.522.7%———0.0%—
ALLALLsuccess2.183.7%———0.0%—
ALLALLsuccess2.191.3%———0.0%—
ALLALLsuccess2.241.7%———0.0%—
ALLALLsuccess2.530.7%———0.0%—
ALLALLsuccess2.776.5%———0.0%—
ALLALLsuccess3.641.1%———0.0%—
ort-cpu-int8-dynCPUsuccess4.460.3%220.97850.0%0.0%—
unknownunknownsuccess4.470.3%220.97850.0%0.0%—
ort-cpu-int8-perchanCPUsuccess4.490.3%220.99240.0%0.0%—
unknownunknownsuccess4.490.2%220.99240.0%0.0%—
ort-cpu-4threadCPUsuccess9.590.7%861.00000.0%0.0%—
ort-cpu-fp32CPUsuccess9.590.2%861.00000.0%0.0%—
unknownunknownsuccess9.620.2%861.00000.0%0.0%—
unknownunknownsuccess9.905.8%861.00000.0%0.0%—
ort-cpu-dynamicCPUsuccess10.380.3%861.0000—0.0%—
unknownunknownsuccess10.380.2%861.0000—0.0%—
ort-cpu-fp16CPUsuccess11.370.4%431.00000.0%0.0%—
ort-coreml-fp16CoreMLsuccess11.370.1%431.000099.8%100.0%—
unknownunknownsuccess11.400.1%431.000099.8%100.0%—
unknownunknownsuccess11.733.0%431.00000.0%0.0%—
ort-coreml-fp32CoreMLsuccess18.994.5%861.000099.5%28.8%31
unknownunknownsuccess20.110.6%861.000099.5%29.5%31
ort-coreml-aneCoreMLsuccess20.161.2%861.000099.5%31.1%31
ALLALLsuccess20.844.6%———100.0%—
unknownunknownsuccess20.940.3%861.000099.5%30.0%31
ALLALLsuccess21.145.8%———100.0%—
ort-coreml-dynamicCoreMLsuccess21.440.6%861.0000—12.4%44
unknownunknownsuccess21.810.2%861.0000—10.4%44
ALLALLsuccess22.738.2%———100.0%—
ALLALLsuccess29.673.4%———100.0%—
ALLALLsuccess30.005.9%———100.0%—
ALLALLsuccess30.9426.3%———100.0%—
ALLALLsuccess31.494.4%———100.0%—
ALLALLsuccess33.694.7%———100.0%—
ALLALLsuccess60.427.6%———100.0%—
ALLALLsuccess62.635.4%———100.0%—
ALLALLsuccess64.044.0%———100.0%—
ALLALLsuccess64.407.5%———100.0%—
ALLALLsuccess65.0317.0%———100.0%—
unknownunknownlowering failure———————
ort-coreml-mlprogramCoreMLlowering failure———————

Recorded failures

Kept rather than hidden. A recipe that cannot lower is a fact about the toolchain, and it is what teaches the static filter not to propose it again.

unknown lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<384x384xf32>) -> tensor<1x128x384xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'
ort-coreml-mlprogram lowering failure
the runtime aborted the process (SIGABRT). This is a delegate-level crash, not a Python exception. Last output: l"(%arg1, %0) <{transpose_lhs = false, transpose_rhs = false}> : (tensor<1x128x1xf32>, tensor<384x384xf32>) -> tensor<1x128x384xf32>
/AppleInternal/Library/BuildRoots/b11baf73-9ee0-11ef-b7b4-7aebe1f78c73/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphExecutable.mm:975: failed assertion `original module failed verification'

Reproduce

Commands for every row on this page
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Snapdragon X2 Elite CRD" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Snapdragon 8 Elite QRD" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy S24" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy S24 (Family)" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy S23" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Snapdragon X Plus 8-Core CRD" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Snapdragon X Elite CRD" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Dragonwing IQ-9075 EVK" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "SA8775P ADP" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "SA8295P ADP" --compute-unit all
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_int8_dynamic.yaml
# recipe f333a2e98449cb02 is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_int8_perchannel.yaml
# recipe 10c4537dc938a9fe is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_4thread.yaml
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_fp32.yaml
# recipe 4bab06fc0db54913 is no longer in the library
# recipe 5b132a9c116f87d4 is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_dynamic.yaml
# recipe 1b1ca31c7b9104b6 is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_cpu_fp16.yaml
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_coreml_fp16.yaml
# recipe 55285c71caf058a9 is no longer in the library
# recipe bc751dafe2d65dff is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_coreml_fp32.yaml
# recipe 3fa721a1cc8c6c5f is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_coreml_ane.yaml
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Google Pixel 7" --compute-unit all
# recipe d05d6eaea9b34557 is no longer in the library
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Google Pixel 10" --compute-unit all
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_coreml_dynamic.yaml
# recipe eec1afcc75571e38 is no longer in the library
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Google Pixel 9" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy A73 5G" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy S21" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy Tab S7" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Snapdragon 7 Gen 4 QRD" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Google Pixel 4" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy A14 5G" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Google Pixel 3" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Dragonwing RB3 Gen 2 Vision Kit" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Xiaomi Redmi Note 10 5G" --compute-unit all
uv run edgefit measure-remote --model hf:sentence-transformers/all-MiniLM-L6-v2 --device "Samsung Galaxy A53 5G" --compute-unit all
# recipe 711b5aaaa429f03c is no longer in the library
uv run edgefit measure --model hf:sentence-transformers/all-MiniLM-L6-v2 --recipe recipes/ort_coreml_mlprogram.yaml