A field-tested path to running int8 YOLOv5 on a SpacemiT K1 with the SpaceMIT ONNX Runtime execution provider, including a measured 13x speedup over CPU and the three pitfalls that stop most people first.
Running an int8 model on the SpacemiT K1 NPU is supposed to be a 20-minute job. In practice most people stall on the same three things, and a recent round of forum threads documents all three with measurements. Here is the consolidated path.
The target: a YOLOv5n detection model at 640x640 on a K1 board, run through onnxruntime with the SpaceMITExecutionProvider.| Model | CPU | SpaceMIT EP 2.0.5 | Speedup |
|---|---|---|---|
| fp32 | 1142 ms | 504 ms | 2.3x |
| int8 (xslim quantized) | 2277 ms | 175 ms | 13x |
pip install xslim --index-url https://git.spacemit.com/api/v4/projects/33/packages/pypi/simple
There is a dependency trap here too. xslim declares nvidia-pyindex, which fails to build in many environments and is not needed for SpacemiT quantization at all. Skip it and install the real dependencies explicitly: pip install --no-deps xslim --index-url https://git.spacemit.com/api/v4/projects/33/packages/pypi/simple pip install loguru numpy onnx onnx-graphsurgeon onnxconverter-common onnxslim opencv-python pandas protobuf sympy tabulate torch torchvision tqdm onnxruntimeUsage is unchanged from the xquant era, and quantization is best done on an x86 machine:
python -m xslim -i model.onnx -o model.q.onnx # dynamic int8 python -m xslim --fp16 -i model.onnx -o model.fp16.onnx python -m xslim -c config.json -i model.onnx -o model.q.onnxCalibration config templates live in the official ModelZoo archive, which ships paired fp and quantized YOLOv5/v6/v7 models together with their xquant_config.json files - the fastest way to validate the pipeline end to end before quantizing your own network.
The Debian package repository ships spacemit-ort 2.0.3. On YOLO models that version fails during session initialization with "SpaceMIT EP does not support Compile". EP 2.0.5 fixes it, and it is available from the same private PyPI:
pip3 download spacemit-ort==2.0.5 --no-deps --index-url https://git.spacemit.com/api/v4/projects/33/packages/pypi/simpleA wheel is a zip. Extract it to get the spacemit_ort directory containing libspacemit_ep.so.2.0.5.
Now the part that wastes the most time. If you pip-install that wheel into a virtualenv alongside a pip-installed onnxruntime, you get:undefined symbol: _ZTIN11onnxruntime18IExecutionProviderE
The working arrangement is to keep the system apt onnxruntime (1.24.x) and point PYTHONPATH at the extracted directory:PYTHONPATH=/path/to/extracted python3 your_script.py
And the import order is the opposite of what several threads claim: onnxruntime must be imported first, then spacemit_ort. Loading the execution provider before onnxruntime fails because the symbols are not resident yet. import onnxruntime as ort # must come first import spacemit_ort # then the EPA minimal smoke test before touching your real model:
import onnxruntime as ort import spacemit_ort import numpy as np print(ort.get_available_providers()) model = "/usr/lib/python3.12/dist-packages/onnxruntime/datasets/logreg_iris.onnx" x = np.random.rand(3, 2).astype(np.float32) sess = ort.InferenceSession(model, providers=["SpaceMITExecutionProvider"]) print(sess.run(None, {"float_input": x}))One more version-specific note: the spacemit-tcm 3.0.1 deb on the spine-runtime release page depends on linux-image-6.18.3-generic and will not install on a 6.6 kernel (Bianbu 2.2 or 2.3). On those systems, installing python3-spacemit-ort and spacemit-tcm from the normal apt repository is sufficient - the /dev/tcm_sync_mem node is optional.
The SpaceMIT EP does not accept every int8 ONNX you can produce. Models exported by ultralytics or onnxruntime.quantization with default settings frequently hit the "does not support" initialization failure even on EP 2.0.5. Symmetric quantization produced by xslim is the format the EP expects. If you want a known-good reference before debugging your own model, pull a quantized YOLOv5 from the ModelZoo archive and confirm it runs - this separates "my model is wrong" from "my runtime setup is wrong" in one step.
These numbers are single-model, single-board measurements from community testing, not vendor benchmarks. Your speedup depends on operator coverage in the EP, on input resolution, and on whether your preprocessing runs on CPU. And the SpaceMIT EP operator implementations themselves are not open source yet - the vendor has said the source release is expected toward the end of the year, and that meanwhile llama.cpp's SpacemiT backend is the most useful public reference for what the hardware likes. If you are building on the larger K3 with its 8 AI cores, the same ONNX Runtime discipline applies; for hand-written matrix code on K1, see our notes on the IME vmadot path, which is what the EP ultimately drives.