#llamacpp

New uncensored GGUF LLMs unveiled: “Heretic” and “Ultra Heretic” reduce refusal rates, 69B LongCat‑Flash‑Sparse handles 1M tokens, plus Qwen‑based models with vision and MoE support.

Discover how Qwen 3.8 now runs on mid‑range Android phones and high‑end desktops, delivering 22‑30 tok/s with top‑tier German & Spanish translation quality.

New llama.cpp updates aim to boost CPU and hybrid inference speed, adding AVX‑512, ARM NEON, and new quantization formats for faster, low‑memory LLM inference.

Qwen 3.8 Flash Next: Apple Silicon and high‑memory GPUs now deliver faster, more efficient inference with custom Metal‑optimized sparse attention, MTP, and Q4 quantization—unlocking 200+ tps prefill and 22 tps decode speeds.

Explore Qwen 3.8’s cutting‑edge GSQ & RCO quantization and SSD offloading—compress models to 2‑3 bpw and cut RAM usage by 26 GB while keeping top‑tier performance.

Explore a lean, 700‑line C implementation of Gemini‑4 E2B that runs LLM inference on CPUs, delivering 640+ tok/s and revealing every math step.

Unsloth expands local AI with GLM‑5.3 Flash GGUF, advanced UD 3.0 quantization, and community‑focused releases for GPU‑friendly LLM deployment.

Explore Qwen3.8‑Flash‑Next: hardware needs, quantization tricks, and Engrams tech for local deployment, plus llama.cpp support and performance benchmarks.

Is Nvidia buying HuggingFace? Explore the potential impact on llama.cpp, open‑source licensing, and the future of local AI inference.