Tencent has released AuK-Flash, a distilled version of its AuK foundation model designed for high-speed speech generation and editing. The model is part of the broader AuK project, a 1.5B parameter foundation model trained on millions of hours of diverse audio data.

Technical Specifications and Capabilities

The AuK-Flash variant is optimized for efficiency, utilizing a 4-step inference process to accelerate audio processing. According to documentation provided via GitHub and Hugging Face, the model operates through a unified natural-language instruction interface. This allows users to perform a wide range of audio tasks using text-based commands.

The model’s capabilities are categorized into several functional areas:

  • Speech Generation: Includes zero-shot Text-to-Speech (TTS), where the model replicates a reference voice, and “Instruct TTS,” which generates speech based solely on a textual description of a voice.
  • Content and Lyric Editing: Enables users to rewrite spoken content (inserting, replacing, or removing text) and modify lyrics in singing recordings while attempting to preserve the original melody and voice.
  • Acoustic and Paralinguistic Editing: Supports adjustments to pitch, speaking rate, and volume. It also allows for changes in emotion, timbre, and the removal of regional accents. Users can also add or remove nonverbal sounds, such as breaths or laughter, or convert speech into a whisper.
  • Enhancement and Separation: Provides tools for speech denoising and dereverberation, as well as the separation of specific speakers from a mix or the extraction of vocals from music.

Availability and Implementation

The official weights for AuK-Flash have been made available on Hugging Face and ModelScope. Developers can access the model through a provided “Cookbook” on GitHub, which includes instruction templates and examples in Python and CLI formats.

Initial Reception and Observations

Early feedback from the developer community on platforms such as Reddit has been mixed. Some users noted the model’s impressive versatility given its relatively small size compared to large language models. However, some users reported that the “Instruct TTS” feature did not meet expectations during initial testing.

While project demonstrations show high-quality audio, some observers noted that minor artifacts occasionally appear in tasks such as speaker separation. Additionally, there has been speculation regarding the model’s linguistic scope, with some users questioning whether its primary effectiveness is limited to English and Chinese.

As of the release, the model is hosted under the Tencent-Hunyuan repository.