Self-Evolving LLM Agents for Hardware Kernel Optimization on SpacemiT K1/K3

A recent paper shared on the SpacemiT forum introduces a new angle on auto-tuning compute kernels for emerging RISC-V platforms. The work, titled Beyond Scaling: Self-Evolving LLM Agents for Hardware Kernel Optimization via an Experience-Driven Workflow and Experience Graph Memory, argues that simply scaling model size is not enough when the target hardware has little existing optimization literature.

Why this matters for K1 and K3

SpacemiT K1 and K3 are still young platforms. While their specs are impressive, K1 with 8 X60 cores at 1.6 GHz and K3 with 8 X100 cores at 2.4 GHz plus A100 AI cores, the body of hand-tuned kernel wisdom is small compared to x86 or Arm. That gap makes it hard to squeeze the advertised TOPS or DMIPS out of frameworks such as ONNX Runtime or llama.cpp.

Core idea in plain language

The proposed pipeline works in two stages:

1. Bootstrap with brute-force LLM search. A large language model generates a pool of candidate kernel implementations. Because LLMs are good at pattern matching, they can quickly produce a set of plausible but not necessarily optimal starting points.
2. Evolve through experience graph memory. The system then evaluates these candidates on the actual hardware, records what worked, and stores the lessons in an experience graph. On later tasks, the agent retrieves relevant past experiences instead of starting from scratch.

This is a practical response to the cold-start problem: instead of hoping a giant model knows everything about SpacemiT ISA extensions or memory hierarchy, the agent learns by doing.

Experience graph memory

The key data structure is an experience graph that links optimization decisions to hardware contexts. A node might capture a loop tiling choice for a MatMulNBits kernel on K1; an edge might connect it to a similar decision on K3 under a different vector length. When a new operator arrives, the agent retrieves the closest prior experiences and mutates them, rather than generating a kernel from a blank prompt.

Implications for RISC-V edge AI

For developers working on K1 or K3, this kind of self-improving optimizer could lower the barrier to writing fast kernels for:

• INT4/INT8 LLM decode kernels
• Convolution and attention operators
• Custom post-processing pipelines for YOLO or RWKV

The paper notes that code will be open-sourced after the peer-review process completes.

If you are building on K1 or K3, keep an eye on this direction. A tool that turns hardware-specific trial and error into reusable experience could be a force multiplier, especially for teams that do not have a dedicated compiler engineer. Source: 全自动算子优化框架和实现 on SpacemiT Forum Related hardware: SpacemiT K1 (8-core X60 @ 1.6 GHz, 50K DMIPS), SpacemiT K3 (8 X100 @ 2.4 GHz + 8 A100 AI cores, 60 TOPS, RVA23 compliant).