EdgeFit Atlas

Measured on-device inference. Every number reproducible; every failure recorded.

229measurements
210succeeded
19failures recorded
9models
13recipes
29devices

The matrix

Every measurement in the corpus, successes and failures together. Sort by any column; a missing value always sorts last, because “not measured” is not a small number.

Generative rows carry TTFT and tok/s instead of a single latency, and their p50 ms cell is deliberately empty: prefill and decode are different workloads, and one averaged number would describe neither. See methodology.

Model Recipe Target Outcome p50 ms TTFT ms tok/s cv size MiB peak RSS first run ms cosine fallback (run)
distilbert-base-uncasedunknownApple M2 unknownsuccess11.37——0.2%64187—0.97210.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess11.47——0.2%64188—0.99320.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess32.12——0.1%253488—1.00000.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess32.12——0.1%253488—1.00000.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess33.38——0.2%253488—1.00000.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess35.58——0.2%127494—1.0000100.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess35.65——0.2%127518—1.00000.0%
distilbert-base-uncasedunknownApple M2 unknownsuccess36.66——0.2%2531174—1.000039.5%
distilbert-base-uncasedunknownApple M2 unknownsuccess39.41——1.5%2531177—1.000033.7%
distilbert-base-uncasedunknownApple M2 unknownsuccess51.66——0.2%2531308—1.000022.0%
distilbert-base-uncasedunknownApple M2 unknownlowering failure—————————
bart-baseALLsm8750 ALL 3Psuccess2.00——3.8%—205——0.0%
bart-baseALLsm8650 ALL 3Psuccess2.66——2.6%—128——0.0%
bart-baseALLsm8550 ALL 3Psuccess3.74——1.7%—93——0.0%
bart-baseort-cpu-int8-dynApple M2 CPUsuccess11.36——0.2%78216120.98050.0%
bart-baseunknownApple M2 unknownsuccess11.37——0.2%78216—0.98050.0%
bart-baseunknownApple M2 unknownsuccess11.46——0.2%79218—0.99080.0%
bart-baseort-cpu-int8-perchanApple M2 CPUsuccess11.53——2.6%79218120.99080.0%
bart-baseort-cpu-fp32Apple M2 CPUsuccess31.96——0.1%313682331.00000.0%
bart-baseort-cpu-4threadApple M2 CPUsuccess32.00——0.1%313681331.00000.0%
bart-baseunknownApple M2 unknownsuccess32.05——0.1%313681—1.00000.0%
bart-baseunknownApple M2 unknownsuccess32.11——0.1%313681—1.00000.0%
bart-baseunknownApple M2 unknownsuccess33.07——0.2%313681—1.00000.0%
bart-baseort-cpu-dynamicApple M2 CPUsuccess33.10——0.3%313681341.00000.0%
bart-baseort-coreml-fp32Apple M2 CoreMLsuccess38.68——1.1%3131582481.000038.5%
bart-baseunknownApple M2 unknownsuccess38.97——0.6%3131620—1.000031.8%
bart-baseunknownApple M2 unknownsuccess41.53——2.0%3131583—1.000034.6%
bart-baseort-coreml-aneApple M2 CoreMLsuccess46.43——7.2%3131580501.000038.5%
bart-baseunknownApple M2 unknownsuccess53.42——0.5%3131581—1.000016.3%
bart-baseort-coreml-dynamicApple M2 CoreMLsuccess58.02——13.8%31315811971.000017.2%
bart-baseALLgoogle-tensor-g2 ALL 3Psuccess72.75——4.3%—243——100.0%
bart-baseALLgoogle-tensor-g4 ALL 3Psuccess93.00——5.6%—265——100.0%
bart-baseALLsm8350 ALL 3Psuccess108.20——12.2%—284——100.0%
bart-baseALLsamsung-exynos-1280 ALL 3Psuccess224.50——4.2%—283——100.0%
bart-baseALLsdm845 ALL 3Psuccess226.91——4.8%—244——100.0%
bart-baseunknownApple M2 unknownlowering failure————156————
bart-baseort-coreml-fp16Apple M2 CoreMLlowering failure————156————
bart-baseunknownApple M2 unknownlowering failure—————————
bart-baseort-coreml-mlprogramApple M2 CoreMLlowering failure—————————
bart-baseort-cpu-fp16Apple M2 CPUlowering failure————156————
bart-baseunknownApple M2 unknownlowering failure————156————
mobilenet_v2_1.0_224ALLsm8650 ALL 3Psuccess0.35——7.9%—160716—0.0%
mobilenet_v2_1.0_224ALLsm8650 ALL 3Psuccess0.38——13.1%—142553—0.0%
mobilenet_v2_1.0_224ALLgoogle-tensor-g4 ALL 3Psuccess4.71——29.2%—1511115—0.0%
mobilenet_v2_1.0_224ALLgoogle-tensor-g4 ALL 3Psuccess6.61——14.2%—1531275—0.0%
vit-base-patch16-224-in21kALLsc8480xp ALL 3Psuccess4.42——1.3%—3411678—0.0%
vit-base-patch16-224-in21kALLsm8750 ALL 3Psuccess6.26——1.6%—2305753—0.0%
vit-base-patch16-224-in21kunknownsm8750 unknown 3Psuccess6.26——1.5%—214——0.0%
vit-base-patch16-224-in21kALLsm8650 ALL 3Psuccess6.72——0.6%—2653538—0.0%
vit-base-patch16-224-in21kunknownsm8650 unknown 3Psuccess7.68——0.4%—164——0.0%
vit-base-patch16-224-in21kunknownsm8650 unknown 3Psuccess7.68——0.7%—163——0.0%
vit-base-patch16-224-in21kALLsm8650 ALL 3Psuccess7.68——0.5%—1657728—0.0%
vit-base-patch16-224-in21kunknownsm8650 unknown 3Psuccess7.73——0.5%—164——0.0%
vit-base-patch16-224-in21kunknownsm8650 unknown 3Psuccess7.75——0.5%—167——0.0%
vit-base-patch16-224-in21kALLsc8340xp ALL 3Psuccess10.70——0.8%—19611453—0.0%
vit-base-patch16-224-in21kunknownsm8550 unknown 3Psuccess10.94——0.9%—253——0.0%
vit-base-patch16-224-in21kunknownsm8550 unknown 3Psuccess10.94——0.9%—260——0.0%
vit-base-patch16-224-in21kALLsm8550 ALL 3Psuccess11.01——1.2%—2497381—0.0%
vit-base-patch16-224-in21kALLsc8380xp ALL 3Psuccess11.51——3.3%—19712710—0.0%
vit-base-patch16-224-in21kALLqualcomm-sa8775p ALL 3Psuccess13.75——2.9%—10028509—0.0%
vit-base-patch16-224-in21kALLqualcomm-sa8295p ALL 3Psuccess16.72——1.5%—9915170—0.0%
vit-base-patch16-224-in21kALLqcs9075 ALL 3Psuccess17.28——2.2%—1610010—0.0%
vit-base-patch16-224-in21kort-cpu-int8-dynApple M2 CPUsuccess42.70——0.2%83200440.98420.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess42.84——0.1%83200—0.98420.0%
vit-base-patch16-224-in21kort-cpu-int8-perchanApple M2 CPUsuccess43.04——0.2%83200440.99450.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess43.11——0.1%83200—0.99450.0%
vit-base-patch16-224-in21kALLsm8650 ALL 3Psuccess56.17——0.4%—45743438—0.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess71.54——2.5%327671—1.000020.4%
vit-base-patch16-224-in21kort-coreml-aneApple M2 CoreMLsuccess71.90——2.8%327686891.000021.9%
vit-base-patch16-224-in21kort-coreml-fp32Apple M2 CoreMLsuccess73.60——1.6%327670881.000022.8%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess75.63——3.7%327604—1.000024.4%
vit-base-patch16-224-in21kort-cpu-dynamicApple M2 CPUsuccess104.59——0.4%3274321051.00000.0%
vit-base-patch16-224-in21kort-cpu-fp32Apple M2 CPUsuccess104.77——0.1%3274451061.00000.0%
vit-base-patch16-224-in21kort-cpu-4threadApple M2 CPUsuccess104.80——0.3%3274451061.00000.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess105.53——0.1%327465—1.00000.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess105.67——0.1%327444—1.00000.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess105.87——0.1%327431—1.00000.0%
vit-base-patch16-224-in21kort-coreml-fp16Apple M2 CoreMLsuccess109.80——0.2%1647721121.0000100.0%
vit-base-patch16-224-in21kort-cpu-fp16Apple M2 CPUsuccess109.80——0.1%1647711121.00000.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess110.14——0.1%164720—1.00000.0%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess110.42——0.1%164749—1.0000100.0%
vit-base-patch16-224-in21kALLgoogle-tensor-g4 ALL 3Psuccess146.70——9.6%—2541319—94.8%
vit-base-patch16-224-in21kunknownApple M2 unknownsuccess179.36——0.9%327969—1.000046.2%
vit-base-patch16-224-in21kort-coreml-dynamicApple M2 CoreMLsuccess181.08——0.5%3279562341.000046.9%
vit-base-patch16-224-in21kALLgoogle-tensor-g5 ALL 3Psuccess264.26——4.6%—447776—100.0%
vit-base-patch16-224-in21kunknownsm8650 unknown 3Psuccess267.62——9.7%—453——0.0%
vit-base-patch16-224-in21kunknowngoogle-tensor-g4 unknown 3Psuccess303.76——2.9%—446——100.0%
vit-base-patch16-224-in21kALLgoogle-tensor-g4 ALL 3Psuccess306.09——9.0%—424155—100.0%
vit-base-patch16-224-in21kALLgoogle-tensor-g4 ALL 3Psuccess308.80——5.2%—443613—100.0%
vit-base-patch16-224-in21kunknowngoogle-tensor-g2 unknown 3Psuccess328.49——18.5%—430——100.0%
vit-base-patch16-224-in21kunknownsamsung-exynos-990 unknown 3Psuccess351.44——6.1%—462——100.0%
vit-base-patch16-224-in21kALLsm7750 ALL 3Psuccess354.83——0.6%—461450—100.0%
vit-base-patch16-224-in21kunknownsm8350 unknown 3Psuccess380.07——6.7%—458——100.0%
vit-base-patch16-224-in21kALLsm7325 ALL 3Psuccess390.35——9.8%—4421075—100.0%
vit-base-patch16-224-in21kunknowngoogle-tensor-g3 unknown 3Psuccess444.12——16.4%—442——100.0%
vit-base-patch16-224-in21kALLsm8250-ab ALL 3Psuccess463.10——7.5%—4861344—100.0%
vit-base-patch16-224-in21kALLsm8150 ALL 3Psuccess465.26——4.8%—4381040—100.0%
vit-base-patch16-224-in21kALLqcs6490 ALL 3Psuccess708.96——0.5%—464667—100.0%
vit-base-patch16-224-in21kunknownsm7250 unknown 3Psuccess750.98——2.2%—420——100.0%
vit-base-patch16-224-in21kunknownsamsung-exynos-1280 unknown 3Psuccess780.79——2.1%—451——100.0%
vit-base-patch16-224-in21kunknownsdm845 unknown 3Psuccess820.37——4.2%—429——100.0%
vit-base-patch16-224-in21kALLsm6150-ac ALL 3Psuccess894.80——6.3%—463890—100.0%
vit-base-patch16-224-in21kALLsamsung-exynos-1330 ALL 3Psuccess987.58——12.1%—4681669—100.0%
vit-base-patch16-224-in21kort-coreml-mlprogramApple M2 CoreMLlowering failure————327————
vit-base-patch16-224-in21kunknownApple M2 unknownlowering failure————327————
toxic-comment-modelALLsm8750 ALL 3Psuccess2.26——3.5%—233——0.0%
toxic-comment-modelALLsm8650 ALL 3Psuccess2.62——1.5%—129——0.0%
toxic-comment-modelALLsm8550 ALL 3Psuccess3.74——1.1%—184——0.0%
toxic-comment-modelort-cpu-int8-dynApple M2 CPUsuccess11.35——0.3%64187121.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess11.37——0.1%64188—1.00000.0%
toxic-comment-modelort-cpu-int8-perchanApple M2 CPUsuccess11.43——0.1%64188121.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess11.52——0.2%64188—1.00000.0%
toxic-comment-modelort-cpu-fp32Apple M2 CPUsuccess31.97——0.7%255492331.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess32.00——0.3%255491—1.00000.0%
toxic-comment-modelort-cpu-4threadApple M2 CPUsuccess32.00——0.3%255491331.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess32.09——0.1%255490—1.00000.0%
toxic-comment-modelort-cpu-dynamicApple M2 CPUsuccess33.16——0.2%256491341.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess33.37——0.2%256491—1.00000.0%
toxic-comment-modelort-coreml-fp16Apple M2 CoreMLsuccess35.50——0.2%128470361.0000100.0%
toxic-comment-modelunknownApple M2 unknownsuccess35.54——0.2%128523—1.00000.0%
toxic-comment-modelort-cpu-fp16Apple M2 CPUsuccess35.55——0.3%128498361.00000.0%
toxic-comment-modelunknownApple M2 unknownsuccess35.65——0.2%128495—1.0000100.0%
toxic-comment-modelort-coreml-aneApple M2 CoreMLsuccess37.92——4.2%2551183471.000039.6%
toxic-comment-modelort-coreml-fp32Apple M2 CoreMLsuccess38.45——4.7%2551185451.000040.3%
toxic-comment-modelunknownApple M2 unknownsuccess39.51——3.6%2551186—1.000037.1%
toxic-comment-modelunknownApple M2 unknownsuccess41.23——4.4%2551184—1.000038.8%
toxic-comment-modelort-coreml-dynamicApple M2 CoreMLsuccess51.75——0.5%25613161471.000019.3%
toxic-comment-modelunknownApple M2 unknownsuccess53.93——0.8%2561267—1.000022.8%
toxic-comment-modelALLgoogle-tensor-g2 ALL 3Psuccess91.83——9.1%—255——100.0%
toxic-comment-modelALLgoogle-tensor-g4 ALL 3Psuccess91.88——6.3%—261——100.0%
toxic-comment-modelALLsm8350 ALL 3Psuccess109.25——11.2%—264——100.0%
toxic-comment-modelALLsdm845 ALL 3Psuccess220.64——5.0%—329——100.0%
toxic-comment-modelALLsamsung-exynos-1280 ALL 3Psuccess236.29——6.3%—282——100.0%
toxic-comment-modelunknownApple M2 unknownlowering failure—————————
toxic-comment-modelort-coreml-mlprogramApple M2 CoreMLlowering failure—————————
Llama-3.2-1B-InstructunknownApple M2 unknownsuccess—118.312.485.7%57175810——0.0%
Llama-3.2-1B-Instructort-cpu-dynamicApple M2 CPUsuccess—124.112.0929.7%57175810——0.0%
Llama-3.2-1B-Instructort-coreml-dynamicApple M2 CoreMLsuccess—473.05.2620.4%57177253——16.4%
Llama-3.2-1B-InstructunknownApple M2 unknownsuccess—390.06.393.4%57178000——15.0%
resnet-50ALLsm8650 ALL 3Psuccess0.65——6.7%—257652—0.0%
resnet-50ALLsm8650 ALL 3Psuccess1.43——2.7%—961824—0.0%
resnet-50ALLsm8650 ALL 3Psuccess1.46——4.5%—1371338—0.0%
resnet-50ALLsc8380xp ALL 3Psuccess1.87——8.9%—775239—0.0%
resnet-50ALLgoogle-tensor-g5 ALL 3Psuccess62.36——4.7%—207147—100.0%
resnet-50ALLgoogle-tensor-g4 ALL 3Psuccess78.22——4.0%—246118—100.0%
resnet-50ALLsamsung-exynos-1330 ALL 3Psuccess234.65——0.8%—340379—100.0%
clip-vit-base-patch32unknownsm8750 unknown 3Psuccess2.42——4.7%—223——0.0%
clip-vit-base-patch32ALLsm8750 ALL 3Psuccess2.43——4.3%—2263209—0.0%
clip-vit-base-patch32ALLsm8650 ALL 3Psuccess2.79——1.9%—1154641—0.0%
clip-vit-base-patch32unknownsm8650 unknown 3Psuccess2.79——2.7%—113——0.0%
clip-vit-base-patch32unknownsm8650 unknown 3Psuccess2.80——1.9%—112——0.0%
clip-vit-base-patch32unknownsm8550 unknown 3Psuccess3.63——1.3%—248——0.0%
clip-vit-base-patch32ALLsm8550 ALL 3Psuccess3.64——1.3%—2474631—0.0%
clip-vit-base-patch32ort-cpu-int8-dynApple M2 CPUsuccess12.02——0.3%84179130.99230.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess12.02——0.1%84262—0.99230.0%
clip-vit-base-patch32ort-cpu-int8-perchanApple M2 CPUsuccess12.09——0.2%84180130.99100.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess12.12——0.3%84181—0.99100.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess18.03——3.5%334642—0.99998.2%
clip-vit-base-patch32unknownApple M2 unknownsuccess18.37——2.2%334612—0.99999.6%
clip-vit-base-patch32ort-coreml-fp32Apple M2 CoreMLsuccess18.40——3.4%334624250.99998.4%
clip-vit-base-patch32ort-coreml-aneApple M2 CoreMLsuccess20.07——5.8%334592260.99998.3%
clip-vit-base-patch32ort-cpu-4threadApple M2 CPUsuccess28.29——0.3%334436291.00000.0%
clip-vit-base-patch32ort-cpu-fp32Apple M2 CPUsuccess28.32——0.2%334436291.00000.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess28.35——0.1%334436—1.00000.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess28.35——0.1%334436—1.00000.0%
clip-vit-base-patch32ort-cpu-dynamicApple M2 CPUsuccess28.52——0.6%334434291.00000.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess28.60——0.1%334474—1.00000.0%
clip-vit-base-patch32ort-coreml-dynamicApple M2 CoreMLsuccess45.84——8.6%334688870.999936.0%
clip-vit-base-patch32unknownApple M2 unknownsuccess46.24——5.7%334680—0.999931.7%
clip-vit-base-patch32unknowngoogle-tensor-g2 unknown 3Psuccess64.59——2.2%—422——100.0%
clip-vit-base-patch32ALLgoogle-tensor-g4 ALL 3Psuccess76.50——4.7%—428584—100.0%
clip-vit-base-patch32unknowngoogle-tensor-g4 unknown 3Psuccess84.76——6.1%—435——100.0%
clip-vit-base-patch32unknownsamsung-exynos-990 unknown 3Psuccess93.95——10.5%—443——100.0%
clip-vit-base-patch32unknowngoogle-tensor-g3 unknown 3Psuccess96.61——5.2%—425——100.0%
clip-vit-base-patch32unknownsm8350 unknown 3Psuccess96.83——14.1%—441——100.0%
clip-vit-base-patch32unknownsamsung-exynos-1280 unknown 3Psuccess178.66——7.2%—434——100.0%
clip-vit-base-patch32unknownsm7250 unknown 3Psuccess188.76——1.3%—403——100.0%
clip-vit-base-patch32unknownsdm845 unknown 3Psuccess204.64——4.3%—440——100.0%
clip-vit-base-patch32ort-coreml-mlprogramApple M2 CoreMLlowering failure————334————
clip-vit-base-patch32ort-coreml-fp16Apple M2 CoreMLlowering failure————167————
clip-vit-base-patch32ort-cpu-fp16Apple M2 CPUlowering failure————167————
clip-vit-base-patch32unknownApple M2 unknownlowering failure————167————
clip-vit-base-patch32unknownApple M2 unknownlowering failure————167————
clip-vit-base-patch32unknownApple M2 unknownlowering failure————334————
all-MiniLM-L6-v2ALLsc8480xp ALL 3Psuccess0.93——3.6%—302927—0.0%
all-MiniLM-L6-v2ALLsm8750 ALL 3Psuccess1.23——14.0%—1991090—0.0%
all-MiniLM-L6-v2ALLsm8650 ALL 3Psuccess1.52——1.9%—1621879—0.0%
all-MiniLM-L6-v2ALLsm8650 ALL 3Psuccess1.52——2.7%—159——0.0%
all-MiniLM-L6-v2ALLsm8550 ALL 3Psuccess2.18——3.7%—1011710—0.0%
all-MiniLM-L6-v2ALLsc8340xp ALL 3Psuccess2.19——1.3%—483783—0.0%
all-MiniLM-L6-v2ALLsc8380xp ALL 3Psuccess2.24——1.7%—483819—0.0%
all-MiniLM-L6-v2ALLqcs9075 ALL 3Psuccess2.53——0.7%—122484—0.0%
all-MiniLM-L6-v2ALLqualcomm-sa8775p ALL 3Psuccess2.77——6.5%—886093—0.0%
all-MiniLM-L6-v2ALLqualcomm-sa8295p ALL 3Psuccess3.64——1.1%—1433473—0.0%
all-MiniLM-L6-v2ort-cpu-int8-dynApple M2 CPUsuccess4.46——0.3%2212450.97850.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess4.47——0.3%22124—0.97850.0%
all-MiniLM-L6-v2ort-cpu-int8-perchanApple M2 CPUsuccess4.49——0.3%2212450.99240.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess4.49——0.2%22126—0.99240.0%
all-MiniLM-L6-v2ort-cpu-4threadApple M2 CPUsuccess9.59——0.7%86228101.00000.0%
all-MiniLM-L6-v2ort-cpu-fp32Apple M2 CPUsuccess9.59——0.2%86228101.00000.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess9.62——0.2%86228—1.00000.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess9.90——5.8%86233—1.00000.0%
all-MiniLM-L6-v2ort-cpu-dynamicApple M2 CPUsuccess10.38——0.3%86223111.00000.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess10.38——0.2%86222—1.00000.0%
all-MiniLM-L6-v2ort-cpu-fp16Apple M2 CPUsuccess11.37——0.4%43202121.00000.0%
all-MiniLM-L6-v2ort-coreml-fp16Apple M2 CoreMLsuccess11.37——0.1%43217121.0000100.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess11.40——0.1%43230—1.0000100.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess11.73——3.0%43241—1.00000.0%
all-MiniLM-L6-v2ort-coreml-fp32Apple M2 CoreMLsuccess18.99——4.5%86630241.000028.8%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess20.11——0.6%86631—1.000029.5%
all-MiniLM-L6-v2ort-coreml-aneApple M2 CoreMLsuccess20.16——1.2%86600251.000031.1%
all-MiniLM-L6-v2ALLgoogle-tensor-g2 ALL 3Psuccess20.84——4.6%—150——100.0%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess20.94——0.3%86598—1.000030.0%
all-MiniLM-L6-v2ALLgoogle-tensor-g5 ALL 3Psuccess21.14——5.8%—132179—100.0%
all-MiniLM-L6-v2ort-coreml-dynamicApple M2 CoreMLsuccess21.44——0.6%86627871.000012.4%
all-MiniLM-L6-v2unknownApple M2 unknownsuccess21.81——0.2%86626—1.000010.4%
all-MiniLM-L6-v2ALLgoogle-tensor-g4 ALL 3Psuccess22.73——8.2%—136170—100.0%
all-MiniLM-L6-v2ALLsm7325 ALL 3Psuccess29.67——3.4%—133372—100.0%
all-MiniLM-L6-v2ALLsm8350 ALL 3Psuccess30.00——5.9%—148——100.0%
all-MiniLM-L6-v2ALLsm8250-ab ALL 3Psuccess30.94——26.3%—173345—100.0%
all-MiniLM-L6-v2ALLsm7750 ALL 3Psuccess31.49——4.4%—207160—100.0%
all-MiniLM-L6-v2ALLsm8150 ALL 3Psuccess33.69——4.7%—107244—100.0%
all-MiniLM-L6-v2ALLsamsung-exynos-1330 ALL 3Psuccess60.42——7.6%—184562—100.0%
all-MiniLM-L6-v2ALLsdm845 ALL 3Psuccess62.63——5.4%—146——100.0%
all-MiniLM-L6-v2ALLqcs6490 ALL 3Psuccess64.04——4.0%—85198—100.0%
all-MiniLM-L6-v2ALLsm6150-ac ALL 3Psuccess64.40——7.5%—227328—100.0%
all-MiniLM-L6-v2ALLsamsung-exynos-1280 ALL 3Psuccess65.03——17.0%—141——100.0%
all-MiniLM-L6-v2unknownApple M2 unknownlowering failure—————————
all-MiniLM-L6-v2ort-coreml-mlprogramApple M2 CoreMLlowering failure—————————

What the first run costs

Every latency figure above describes the second run onward. The first one also loads the model, initialises the delegate and compiles kernels — and a user opening an app pays that, once, every cold start.

The ratio is not a constant offset that could be subtracted. On the same phone, forcing a model onto the CPU loaded far faster than letting it reach the NPU, so part of an accelerator's headline win is bought with startup cost the headline never shows.

ModelDevicecold load msfirst run mssteady msratio
all-MiniLM-L6-v2sc8480xp 3P2927—0.933,157×
resnet-50sc8380xp 3P5239—1.872,807×
vit-base-patch16-224-in21ksc8480xp 3P11678—4.422,639×
all-MiniLM-L6-v2qualcomm-sa8775p 3P6093—2.772,196×
vit-base-patch16-224-in21kqualcomm-sa8775p 3P28509—13.752,073×
mobilenet_v2_1.0_224sm8650 3P716—0.352,018×
all-MiniLM-L6-v2sc8340xp 3P3783—2.191,730×
all-MiniLM-L6-v2sc8380xp 3P3819—2.241,706×
clip-vit-base-patch32sm8650 3P4641—2.791,665×
mobilenet_v2_1.0_224sm8650 3P553—0.381,475×
clip-vit-base-patch32sm8750 3P3209—2.431,321×
resnet-50sm8650 3P1824—1.431,278×

Does the accelerator actually help?

The question a delegate never answers for you. Enabling an accelerator can make a model slower with no error and no warning — the silent-fallback failure mode. Both fallback columns are shown because they disagree, and the disagreement is the finding: see methodology.

Each row compares an accelerator against the CPU of the same device. Comparing across devices would measure the two devices, not the delegate.

ModelTaskDeviceCPU fp32acceleratedverdictFLOP fb (as authored)time fb (as run)
toxic-comment-modelclassifyApple M231.9711.372.81× faster0.0%0.0%
bart-baseembedApple M231.9611.372.81× faster0.0%0.0%
vit-base-patch16-224-in21kvisionApple M2104.5942.842.44× faster0.0%0.0%
clip-vit-base-patch32unknownApple M228.2912.022.35× faster0.0%0.0%
all-MiniLM-L6-v2embedApple M29.594.472.15× faster0.0%0.0%