K3 Kernel/Hardware Bug: Stale Code Execution with WD Black SN7100 NVMe SSD

A recently updated thread on the SpacemiT forum (Topic #1474) highlights a serious reliability issue encountered by the Zig Software Foundation when deploying four Milk-V Jupiter 2 boards (SpacemiT K3) into their continuous integration cluster. The problem is notable because it points to stale or incorrect code being executed by the CPU, rather than a conventional application crash.

Hardware and Software Context

• Boards: Milk-V Jupiter 2 (SpacemiT K3 Pico ITX form factor)
• Kernel: 6.18.3-generic #1.0.2.4 PREEMPT(voluntary)
• Storage: SanDisk WD Black SN7100 1TB PCIe 4.0 M.2 2280 NVMe SSD
• Compiler: Zig self-hosted compiler built from source

The boards replaced an earlier fleet of Milk-V Jupiter (M1) boards and are intended to run Zig's own build system under heavy parallel load.

Symptom: The CPU Runs Different Code Than Memory Contains

The failure manifests as a SIGSEGV (signal 11) during the Zig build process. What makes the crash unusual is the evidence in the kernel trap log:

CPU: 4 UID: 1004 PID: 229411 Comm: zig1 epc : 000000000121bab6 status: 8000000200006020 badaddr: 0000000000000009 cause: 000000000000000f # store/AMO page fault Code: eb63 68bb 058a 95ea 418c 8582 040e 9406 1402 9001 (943a) 400c

At the reported program counter, GDB disassembly shows:

=> 0x121bab6 <f2688+270>: add s0,s0,a4

The instruction is a register-only add, which cannot trigger a store/AMO page fault. The same bytes were confirmed on disk, in memory via GDB, and in dmesg. The conclusion drawn by the reporter is that the CPU executed code different from what was actually loaded into memory.

Reproduction Constraints

The issue is highly specific:

• Reproduces only on the M.2 NVMe SSD, not on the on-board UFS.
• Reproduces only on the X100 application cores, not on the A100 AI cores.
• Occurs on all four boards tested, ruling out a single defective unit.
• Affects the WD Black SN7100; other SSD models such as the Samsung 970 EVO Plus do not show the problem.

A Key Diagnostic Clue: Inode-Bound Behavior

The same executable binary behaves differently depending on its file path, even though SHA-256 hashes match exactly:

$ cp build-release/zig1 build-release/zig1.cp $ ./zig1.cp ... # exit 0, normal $ ./zig1 ... # SIGSEGV $ sha256sum zig1 zig1.cp 5bf0342502b90f61510239534eaa13bf08d6261394ce17db8a404c5c9c82472c zig1 5bf0342502b90f61510239534eaa13bf08d6261394ce17db8a404c5c9c82472c zig1.cp

Forcing the kernel to drop its page cache immediately resolves the failure:

$ echo 1 | sudo tee /proc/sys/vm/drop_caches $ ./zig1 ... # exit 0, normal

This strongly suggests a page-cache or instruction-cache coherency edge case tied to how the NVMe controller or its driver interacts with the K3 memory subsystem.

Official Response and Current Status

SpacemiT engineers acknowledged the report and attempted to reproduce it on Pico ITX boards booted from other NVMe models (SN570, KC3000). They were initially unable to reproduce the failure. After the reporter confirmed that a Samsung 970 EVO Plus does not trigger the bug, SpacemiT ordered a WD Black SN7100 for compatibility testing. As of the latest update on September 11, 2026, the drive had not yet arrived, and a firmware or kernel fix is expected no sooner than the following month.

Practical Workarounds

If you are operating K3-based boards under heavy compiler or CI workloads, two workarounds are known today:

1. Avoid the WD Black SN7100 for the boot/root filesystem until a fix is released. Verified alternatives include the Samsung 970 EVO Plus, WD Blue SN570, and Kingston KC3000.
2. When stale-code symptoms appear, drop the page cache as a temporary relief:

echo 1 | sudo tee /proc/sys/vm/drop_caches

Why It Matters for RISC-V CI Infrastructure

This case is a reminder that RISC-V Linux platforms are still maturing at the board-support and storage-controller interface layers. A compiler CI fleet is an especially demanding workload: it repeatedly maps many large executables, executes them once, and then discards them. That pattern can expose page-cache coherency bugs that are invisible in lighter desktop use. For teams planning to deploy K3 or similar RISC-V boards at scale, qualifying the NVMe SSD model should be part of the bring-up checklist.

At open-riscv.com we track the SpacemiT K1 and K3 families closely because they are among the first RVA23-class boards suitable for both application and AI workloads. We will update this note once SpacemiT releases a fix or root-cause analysis. Source: Possible kernel or hardware bug on K3 leading to stale/incorrect code running