Technical Demonstrations in Virtual Environments and Sparse Memory Research

Two distinct technical projects have recently emerged from independent research, showcasing advancements in browser-based 3D environments and experimental large language model (LLM) architectures.

Browser-Based O’Neill Cylinder Simulation

A developer has released a browser-based simulation of a walkable O’Neill cylinder, a theoretical space habitat. According to a report on Reddit, the digital environment features a 20-mile-long habitat that users can explore from a street-level perspective.

The project aims to visualize the scale of such structures, which are often discussed in science fiction, such as the Citadel in the Mass Effect series. While the simulation provides a visual representation of the habitat’s interior, it remains a digital model intended for exploration rather than a scientific tool for atmospheric or structural analysis.

Research into Sparse Memory in Small Language Models

In the field of artificial intelligence, a researcher has published findings regarding the use of large lookup tables to augment small language models. The project, hosted on GitHub under the name “sparse-memory-lm,” explores whether a small model can achieve performance parity with a much larger model by utilizing a massive external memory source.

The researcher reported that a 21-million-parameter model, when paired with a 6.4-billion-parameter lookup table containing 16.8 million rows, performed at a level comparable to a 114-million-parameter dense model trained on the same 500 million Wikipedia tokens.

Key technical observations from the study include:

  • Hardware Efficiency: The researcher noted that the lookup table does not require Video RAM (VRAM). By using 4-bit memory-mapping from an NVMe SSD, the model achieved a generation speed of approximately 140 tokens per second on an RX 9070 GPU, utilizing only 0.4 GB of VRAM.
  • Latency Constraints: The study noted that reading long prompts from an SSD remains a bottleneck, as every missed row incurs a 4 KB page cost.
  • Kernel Compatibility: The researcher developed Triton kernels for the architecture, which are reportedly compatible with various hardware, including Radeon, MI350X, and H100/H200 GPUs.
  • Limitations: The researcher cautioned that the model’s output consists of fluent English but contains “made-up facts.” Furthermore, attempts to bolt a lookup table onto a pre-existing, finished model (Qwen3.5-0.8B) did not yield improved results compared to a standard small dense add-on.

The research was conducted using a combination of personal hardware and cloud computing via Runpod. While the results provide a proof-of-concept for sparse memory architectures, the findings are currently limited to a small scale and have not undergone independent peer review. The researcher expressed interest in testing the architecture at a 1-billion-parameter scale.