Programming the SpacemiT K1 IME Matrix Extension: A vmadot Guide

What is the SpacemiT IME Extension?

The SpacemiT K1 ships with an Intelligent Matrix Extension (IME) on top of its RISC-V vector implementation. One of the key instructions is vmadot, which performs a dot-product-like accumulation on 8-bit signed integer elements. The extension is aimed at accelerating small matrix multiplications used in quantized neural networks, making the K1's 2 TOPS AI claim usable from hand-written or compiler-generated code.

In the SpacemiT forum, a developer asked how to reach the 2x2x4[x2] MAC unit shown in the micro-architecture diagram instead of falling back to the larger 4x4x8 path. The official reply points to the reference examples in the RISC-V IME extension specification repository.

Reference Code and Toolchain

SpacemiT publishes example code at:

https://github.com/spacemit-com/riscv-ime-extension-spec/tree/master/example

The repository contains a minimal GEMM demo using vmadot. To build and run it:

# Download the SpacemiT toolchain wget https://archive.spacemit.com/toolchain/spacemit-toolchain-linux-glibc-x86_64-v1.1.2.tar.xz tar -xvf spacemit-toolchain-linux-glibc-x86_64-v1.1.2.tar.xz # Build the demo riscv64-unknown-linux-gnu-gcc -march=rv64gcv vmadot-gemm-demo.c -o gemm-vmadot-4x8x4 # Test on QEMU wget https://archive.spacemit.com/spacemit-ai/qemu/jdsk-qemu-v10.0.2.tar.gz tar -xzvf jdsk-qemu-v10.0.2.tar.gz qemu-riscv64 -cpu max,vlen=256 gemm-vmadot-4x8x4 # Run on K1 hardware scp gemm-vmadot-4x8x4 root@<k1-board>:/tmp/ ssh root@<k1-board> /tmp/gemm-vmadot-4x8x4

Inline Assembly Example

The forum user contributed the following snippet to exercise the IME unit. The key is to set the vector length (vsetvli) and element width correctly before calling vmadot:

size_t vmadot_gemm_2x2x4x2_avl128(const int8_t* A_copies, const int8_t* packedB, int32_t* raw_c) { size_t configured_vl; __asm__ volatile( "vsetvli t0, zero, e32, m2 " "vxor.vv v28, v28, v28 " "vsetvli t0, zero, e8, m1 " "vxor.vv v0, v0, v0 " "vxor.vv v1, v1, v1 " "vle8.v v0, (%[SRC_A]) " "vle8.v v1, (%[SRC_B]) " "li t0, 128 " "vsetvli t0, t0, e8, m1, ta, ma " "vmadot v28, v0, v1 " "mv %[VL], t0 " "vsetvli t0, zero, e32, m2 " "vse32.v v28, (%[DST_C]) " : [VL] "=&r"(configured_vl) : [SRC_A] "r"(A_copies), [SRC_B] "r"(packedB), [DST_C] "r"(raw_c) : "cc", "t0", "v0", "v1", "v28", "v29", "memory" ); return configured_vl; }

Important Details

• vsetvli configuration matters. The first vsetvli zeros the 32-bit accumulator vector v28. The second loads 8-bit elements into v0 and v1. The third sets AVL=128 before vmadot so the hardware schedules the desired MAC shape.
• Operand layout. A_copies and packedB must be prepared in the format the hardware expects. The GitHub demo shows how to pack matrices for the 4x8x4 kernel; adapting it to 2x2x4[x2] requires matching the lane grouping documented in the IME spec.
• Use ta, ma tail/mask policies. These are required for deterministic behavior when the vector length does not evenly divide the matrix tile.
• Verify on QEMU first. Running under jdsk-qemu lets you check correctness before moving to physical K1 hardware where vector and IME implementation details may differ slightly from the simulator.

When to Use IME on K1

The K1 (8-core X60 at 1.6 GHz, 50K DMIPS, 2 TOPS) does not have a dedicated NPU. Instead, the IME extends the vector unit to accelerate int8 dot products. This is ideal for:

• Quantized ONNX Runtime models via the MatMulNBits path
• Custom llama.cpp kernels for small models
• Embedded vision pipelines such as YOLO on K1

For larger models, consider the K3 (8x X100 at 2.4 GHz + 8x A100 AI cores = 60 TOPS), which supports 30B parameter local inference and has more mature Triton and llama.cpp support.

WCH MCU Companion

For sensor acquisition or low-level control around a K1-based edge node, the WCH CH32V307 ($3, gigabit Ethernet MAC) or CH32V317 ($4, 480Mbps USB) can act as protocol converters. The CH32V003 ($0.1, RV32EC) is sufficient for simple GPIO or watchdog tasks.

Source: SpacemiT Forum - How to call the IME matrix extension on K1? Source: SpacemiT RISC-V IME Extension Examples