Performance Optimizations for AMD Radeon R9700

New performance benchmarks released by independent developers suggest significant speed improvements for users running large language models (LLMs) on single AMD Radeon R9700 graphics cards. According to reports shared on Reddit, recent optimizations have reportedly doubled the performance across various workloads when running Unsloth’s Qwen3.8 27b NVFP4 model.

The reported data, which has not been independently verified by third-party testing labs, indicates that the single-card setup can achieve a median decode speed of 153 tokens per second (tok/s) for JSON tasks. Other category-specific decode speeds include 120.5 tok/s for coding, 140.0 tok/s for mathematics, and 67.1 tok/s for general chat.

Benchmarking and Concurrency Results

The performance metrics provided by the developer, available via GitHub and Codeberg repositories, highlight the hardware’s capability under different operational loads. In terms of prefill performance—the initial processing phase of a prompt—the R9700 reached a peak of 3,619 tok/s at a depth of 16,000 tokens. As the prefill depth increased to 64,000 tokens, the speed decreased to 3,192 tok/s.

The benchmarks also detailed the impact of concurrency on throughput. When managing multiple simultaneous requests, the system demonstrated the following median speeds:

  • 1 concurrent request: 120 tok/s
  • 4 concurrent requests: 322 tok/s
  • 8 concurrent requests: 471 tok/s

Technical Implementation and Reliability Concerns

The optimizations appear to be linked to the use of the NVFP4 datatype. While the performance gains are notable, some technical discussions on community forums have raised questions regarding data conversion efficiency, specifically whether casting from NVFP4 to MXFP4 before reaching FP16 or BF16 formats is necessary.

Furthermore, some users have reported reliability issues when utilizing specific configurations. In a reported test involving agent-based workloads, a setup utilizing “DFlash + prefix caching + GDN align” resulted in corrupted or empty outputs and malformed tool calls during long, multi-turn contexts. One user noted that in a 49-request test, they encountered four timeouts and six corrupted responses.

The report suggests that disabling prefix caching may mitigate these corruption issues, allowing for a more stable environment with a context window of up to 131K. Additionally, users noted that certain features, such as Multi-Token Prediction (MTP), currently fail to start in the Radiance 0.9.3 build.

While these results offer a glimpse into the potential of the AMD Radeon R9700 for local LLM execution, the community is advised to approach the benchmarks with caution due to the reported stability issues in complex, long-context workloads.