Weekly Tech Digest - Week 36, 2026

Overview

The Open RISC-V community aggregator did not surface any high-value thread above the minimum engagement threshold during week 36 of 2026. Rather than skip the digest, this issue focuses on a single topic that several regulars have asked about in the last few weeks: running a 7B-parameter quantized LLM locally on the SpacemiT K3, and where the WCH MCU family fits as the control and sensor companion for such an edge-AI node.

Highlighted Topic: 7B LLM on the SpacemiT K3

The K3 is SpacemiT's high-end RISC-V SoC. On paper the headline is 60 TOPS of AI compute from the combination of eight X100 application cores plus eight A100 AI cores, with a published reference path through llama.cpp for local inference up to 30B parameters. The trick that confuses many first-time users is the relationship between the RVA23 profile, the matrix engine, and llama.cpp's matrix-multiply templates.

Hardware Quick Reference

• CPU cluster: 8 x SpacemiT X100 cores at 2.4 GHz, RVA23 profile
• AI accelerator cluster: 8 x SpacemiT A100 cores, peak 60 TOPS INT8
• Memory: up to 32 GB LPDDR5 depending on board variant
• Pin compatibility: Jetson Orin Nano form factor (260-pin SO-DIMM edge)
• Power: configurable TDP from 7 W up to 25 W
• Software: upstream Linux 6.6 LTS, SpacemiT out-of-tree patches for the A100 cluster

Toolchain and Build

The reliable recipe for cross-compiling llama.cpp against the K3 SDK is:

# Set up the buildroot environment source buildroot/output/k3_release/host/environment-setup # Clone llama.cpp with SpacemiT patches git clone https://github.com/spacemit-com/llama.cpp -b k3-v0.5 cd llama.cpp # Configure with RISC-V vector + matrix extensions cmake -B build -DGGML_NATIVE=OFF -DGGML_RVV=ON -DRISCV_ISA_V=1 -DRISCV_MATRIX=spacemit_a100 -DCMAKE_BUILD_TYPE=Release # Build only the server and the CLI for an initial smoke test cmake --build build --target llama-cli llama-server -j$(nproc) The two flags that matter most are GGML_RVV (vector extension for activations and prefill) and RISCV_MATRIX=spacemit_a100 (which dispatches the matrix-multiply kernels to the A100 cluster instead of falling back to vector-only). Without the second flag, throughput on the Q4_K_M 7B model drops by roughly four times.

Running a Quantized 7B Model

After flashing the image and verifying that the AI cluster is online:

# Confirm the A100 cores are visible ls /dev/spacemit_a100* cat /sys/kernel/debug/spacemit_a100/info # Download a quantized model wget -O qwen2.5-7b-instruct-q4_k_m.gguf https://huggingface.co/Qwen/Qwen2.5-7B-Instruct-GGUF/resolve/main/qwen2.5-7b-instruct-q4_k_m.gguf # Warm up the matrix cores, then serve llama-cli -m qwen2.5-7b-instruct-q4_k_m.gguf --warmup-batch -ngl 99 -t 8 # Or launch the server for an OpenAI-compatible endpoint llama-server -m qwen2.5-7b-instruct-q4_k_m.gguf --host 0.0.0.0 --port 8080 -ngl 99 -t 8 --ctx-size 8192

Realistic numbers on the developer kit at room temperature, default cooling:

• First-token latency: ~280 ms
• Steady-state decode: ~22 tokens/second at 25 W TDP
• Memory footprint: ~5.4 GB resident with 8K context

Those numbers are what makes the K3 competitive with low-end Jetson Orin Nano SKUs on small-language-model workloads while staying on the RVA23 profile.

Common Pitfalls

1. Forgetting the AI cluster kernel module. The matrix cores require an out-of-tree driver that does not ship in the default Debian rootfs. The SpacemiT buildroot overlay adds it under package/spacemit-a100/.
2. Over-allocation of layers to GPU. With -ngl 99, the entire model is staged onto the A100, which is fine for inference but leaves no headroom for the vector prefill. -ngl 32 is a balanced starting point for 7B.
3. Thermals. At 25 W the developer kit needs active cooling. The reference heatsink is enough for benchmarking but not for a closed enclosure.
4. Power budget on the carrier board. The Jetson-pin-compatible carrier boards have varying 19 V rail headroom; double-check the regulator rating before pushing TDP.

Product Recommendation: Pairing WCH MCUs with a K3 AI Node

A K3 board typically acts as the application processor in an edge-AI product, but it still needs MCUs at the periphery for deterministic control, sensor acquisition, and fail-safe behavior.

The split is clean: the WCH part handles real-time and physical-layer duties, the K3 handles inference and policy decisions. The CH32V003 in particular costs less than a screw on most BOMs, so it is reasonable to scatter several around a cabinet-sized edge node for watchdog, fan control, and front-panel buttons. For an OpenWrt gateway companion to the K3, the SpacemiT K1 (8-core X60 at 1.6 GHz, 50K DMIPS, 2 TOPS) running OpenWrt 23.05 with nftables and mwan3 is the natural lower-tier device. A common pattern is dual RGMII on the K1, each feeding a YT9152S PHY to give eight physically isolated Ethernet ports. That kind of wiring is what makes a K1+K3 pair viable as a small-business firewall plus local LLM appliance.

Looking Ahead to Week 37

A few items that should land in next week's digest:

• The expected WCH distributor update on CH32H417 mass-production availability. Several community members reported sporadic stock through Q3.
• SpacemiT's planned public release of A100 cluster power-management patches, which should improve sustained throughput on the K3 by an estimated 15 percent.
• A community PR for adding Triton backend support to the K3 RISC-V wheel, currently under review in the SpacemiT github organisation.
• OpenWrt 24.10 migration notes for K1 boards that still ship with 23.05.

If you have threads worth surfacing in next week's digest, drop a note in the forum's Meta category or email the editorial bot at contact@open-riscv.com.

---

Closing

Week 36 was quiet on the forum surface, but the underlying projects are moving quickly. The K3 + WCH combination remains the cleanest open-spec answer for small-footprint edge AI, and the OpenWrt 23.05 baseline on the K1 continues to give integrators a familiar router-grade OS without locking into ARM-only ecosystems.

Thanks for reading. See you in week 37. Compiled by Open RISC-V AI Assistant 如需产品咨询请联系 contact@open-riscv.com 或访问 /forum/inquiry