Performance Benchmarks for Qwen3.6 Models on AMD Radeon AI Pro R9700
Technical benchmarks shared on Reddit indicate significant performance capabilities for the Qwen3.6 series of large language models when deployed on a single AMD Radeon AI Pro R9700 (gfx1201) GPU. The reported data, provided by a user optimizing local AI setups, details the performance of both the 27B dense and 35B Mixture-of-Experts (MoE) versions of the models using the vLLM inference engine.
The testing utilized the stilldeadcode/vllm-radiance:0.5.8 Docker image via Podman. While the reference configuration for this image is optimized for dual-card setups using FP8 weights, the reported results focus on a single-card configuration using INT4 quantization (specifically the Avesed/Qwen3.6-{27B,35B}-INT4-W4A16 weights).
Qwen3.6 35B MoE Performance Metrics
The 35B model, which utilizes a Mixture-of-Experts architecture (A3B), demonstrated high throughput during testing. According to the user’s report, the model achieved a decoding speed of approximately 61.4 tokens per second (tok/s) at a 4k context length. As the context length increased, decoding speeds showed a gradual decline, reaching 49.5 tok/s at a 150k context length.
Prefill speeds for the 35B model remained relatively stable across various depths, starting at approximately 7,800 tok/s at 4k context and decreasing to 3,690 tok/s at 150k context. The user noted that the 35B model at FP8 precision would not fit on a single 32GB card at a useful context length, necessitating the use of INT4 quantization.
Qwen3.6 27B Dense Model and Speculative Decoding
The 27B dense model was tested with Multi-Token Prediction (MTP) enabled and speculative decoding set to four tokens. The user reported that using four speculative tokens provided a performance increase of 17% to 48% over eight tokens across various test depths.
The 27B model maintained a decoding speed of approximately 59.6 tok/s at a 4k context length, which decreased to 53.7 tok/s at a 100k context length. The mean accepted length for speculative tokens was recorded between 4.4 and 4.6.
Technical Configuration and Observations
The reported setup required specific adjustments to the default vLLM configuration to accommodate a single-card environment. Key modifications included setting the tensor parallel size to 1 and increasing the GPU memory utilization to 0.98.
The user identified a technical issue in the Avesed INT4 repository regarding the tokenizer.json file, noting that hardcoded truncation and padding limits (512) could interfere with vision tasks above 672px. The user recommended setting these values to null to resolve the issue.
While these benchmarks provide detailed insight into the R9700’s capabilities for running large-scale models, it should be noted that these results are based on individual user testing and have not been independently verified by third-party hardware reviewers.