Microsoft Documentation Suggests Looped Transformer Architecture in OpenAI’s GPT-6 Series

Microsoft has confirmed through a publicly accessible web page that OpenAI has utilized “Looped Transformers” within its GPT-6 series. The disclosure, which was subsequently removed from the site following the leak, appears to validate previous reporting by The Information regarding the architectural direction of OpenAI’s next-generation models.

Technical Specifications and Inference Passes

According to the leaked documentation, the GPT-6 series employs a recurrent approach to inference. Specifically, the documentation noted that GPT-6.1 Sol utilizes two inference passes per token, whereas a previous iteration, GPT-6 Sol, reportedly utilized three.

Technical analysis of these findings suggests that the “Sol” and “Astra” designations may refer to different configurations of the same underlying pre-trained base model. In this architecture, the model performs multiple forward passes through the same weights to refine the vector representation of a token before outputting it. While this increases the effective depth of the model and improves reasoning capabilities, it necessitates a trade-off in generation speed. For example, if a model performs two passes per token, it effectively produces half the number of output tokens for the same amount of compute compared to a standard single-pass transformer.

Performance Benchmarks and Speculative Architectures

Unverified data circulating in developer communities has attempted to correlate these architectural shifts with observed inference speeds. Data points suggest a correlation between the number of inference passes and tokens per second (tok/s):

  • GPT-6 Sol and 5.6 Sol: Approximately 100 tok/s
  • GPT-6.1 Sol and GPT-6 Astra: Approximately 60 tok/s

The drop in speed from 100 tok/s to 60 tok/s aligns with the transition from a single-pass or low-loop configuration to a multi-pass system.

Some independent observers have speculated that OpenAI may be employing a “nested” or “mixture-of-experts” style architecture to manage costs. This theory suggests that different versions of the model—such as “Luna,” “Sol,” and “Astra”—may share the same base weights but vary in the number of active parameters or inference loops used. This would allow the provider to offer lower-cost, faster versions of the model (like Luna) for general use, while reserving more compute-intensive, multi-pass versions (like Astra) for high-reasoning tasks.

Industry Implications

If these architectural shifts are confirmed at scale, it may signal a move toward recurrent neural networks that prioritize computational efficiency and reasoning depth over raw token throughput. Such developments could influence the research directions of other major AI players and hardware manufacturers, including NVIDIA CORP, as the industry seeks to optimize how large-scale models utilize compute during inference.

At present, while the Microsoft documentation provides concrete evidence of the use of looped transformers, the specific relationship between the various GPT-6 model tiers remains unverified.