Open-source VoiceStudio offers local voice cloning, AI dubbing, transcription and audiobook creation across 646 languages without subscriptions or API dependency.
Imphal, Sept. 12: An open-source project called VoiceStudio is bringing a broad set of AI-powered voice and audio production tools to users who want to run speech generation locally rather than rely entirely on cloud-based services.
Developed by GitHub user debpalash, VoiceStudio describes itself as a fully local alternative to hosted voice-generation platforms. The project combines voice cloning, voice design, video dubbing, transcription, dictation, audiobook production and batch audio processing in a single application. Its repository currently lists support for 16 text-to-speech engines, 11 automatic speech recognition engines and a catalogue covering 646 TTS languages, although the actual language coverage and quality vary by engine.
The project is currently marked as an active beta, with developers advising users who need stable production work to use the latest release rather than the continuously changing `main` branch.
Local alternative to cloud voice platforms
The central idea behind VoiceStudio is to move much of the speech-production workflow from remote servers to the user's own computer.
The repository says its default local workflow does not require an account, API key, subscription or usage meter. Voices, projects, settings and generated outputs remain on the machine by default. Remote workers and external speech-recognition services are available, but they are presented as explicit opt-in features rather than being required for the core workflow.
This approach gives users greater control over recordings and generated material, while shifting the responsibility for hardware, model downloads, storage and performance from a cloud provider to the user.
VoiceStudio supports Windows 10/11 on x64, macOS 13.3 and later on Apple Silicon, and Linux x86_64 systems with glibc 2.39 or newer. It can also be deployed through Docker. The project supports NVIDIA CUDA, Apple Silicon MPS/MLX, Linux ROCm and CPU-based processing, with optional remote workers for additional compute.
Voice cloning without model training
One of the project's prominent features is zero-shot voice cloning.
According to the project's documentation, users can provide a short reference recording, select a language and enter the text they want generated. A clean recording of around three seconds can work, while the project recommends five to 15 seconds for better results. The reference clip acts as a prompt rather than conventional training data.
VoiceStudio also includes a separate voice-design workflow. Rather than copying an existing speaker, users can describe characteristics such as age, accent, pitch, style and delivery to create a voice matching those instructions.
The distinction is significant for users working with narration, fictional characters, audiobooks or other synthetic voices where reproducing a particular real person's voice is not necessarily the objective.
Video dubbing is built into the workflow
VoiceStudio goes beyond text-to-speech by providing a video-dubbing pipeline.
The repository describes a workflow that can transcribe a video, translate the transcript, preserve speaker identities, synthesise the translated speech and export the resulting video. Users can also work with captions, timing information, glossaries and individual transcript segments. Completed dubbing projects can flag timing issues for review before export.
The project provides examples of multilingual dubbing outputs in Spanish, French, Japanese and Chinese, demonstrating the intended use of the system for multilingual audio production.
Speaker diarization is supported through tools including Pyannote and WhisperX, allowing the system to distinguish speakers in source material. Vocal isolation using Demucs is also included for separating speech from background audio.
Audiobooks, stories and batch production
VoiceStudio also targets long-form audio production.
Its features include multi-voice scripts, EPUB and PDF importing, chapter rendering and M4B audiobook export. A casting interface allows different voices to be assigned to speakers, while batch queues can process multiple audio or video jobs and monitor a local folder for new videos.
This makes the project broader than a conventional text-to-speech application. Its design attempts to bring several stages of an audio-production workflow into one environment, from source transcription and voice selection to synthesis, editing and final export.
Multiple AI engines instead of a single model
Another notable feature is the project's engine architecture.
VoiceStudio does not depend on one speech model. Its model catalogue allows users to install, remove, select and route different TTS, ASR and LLM models. The repository currently lists engines including OmniVoice, CosyVoice 3, GPT-SoVITS, VoxCPM2, MOSS-TTS, KittenTTS, MLX-Audio, Sherpa-ONNX, IndexTTS and others.
The available engines have different language coverage, hardware requirements, cloning capabilities and licences. The project therefore warns users to check engine-specific licensing and resource requirements rather than treating all models as interchangeable.
For speech recognition, VoiceStudio supports engines including WhisperX, Faster-Whisper, MLX Whisper, PyTorch Whisper, Parakeet, Moonshine, FunASR and Sherpa-ONNX. The project says WhisperX is the default ASR engine and is particularly suited to dubbing, subtitles and word-level timing.
Hardware requirements remain an important factor
The local approach does not eliminate computing requirements.
The project's listed minimum configuration includes 8 GB of RAM and 10 GB of free disk space. A GPU is optional because CPU processing is supported, but the project recommends 8 GB or more of RAM and an SSD with at least 20 GB of free space for a more comfortable setup. Four GB of VRAM is listed as a minimum for GPU-accelerated work, while 8 GB or more is recommended for the default multi-stage workflow. Larger optional models can require considerably more memory.
The repository provides different engine recommendations depending on hardware. Apple Silicon users can use MLX-based models and OmniVoice, while systems equipped with NVIDIA GPUs can use CUDA-based engines. Lower-powered or CPU-only systems can use models such as PocketTTS, Sherpa-ONNX and KittenTTS for speech generation, with Moonshine or Faster-Whisper available for speech recognition.
The trade-off is therefore straightforward: users avoid recurring cloud processing costs and gain local control, but they must provide the computing resources necessary to run the selected models.
Built around a local API
VoiceStudio is also designed as a platform rather than only a desktop application.
Its architecture combines a Tauri v2 desktop shell with a React and Vite interface and a FastAPI backend. Communication between the desktop interface and backend takes place through HTTP, server-sent events and WebSockets on a local port. The backend contains separate registries for TTS and ASR engines, along with dubbing, audio and long-form processing pipelines.
The project exposes an OpenAI-compatible audio API, allowing software already built around compatible audio endpoints to communicate with a local VoiceStudio installation instead of a cloud provider.
The local API provides speech-generation and transcription endpoints, including support for multiple audio formats and streaming transcription. The repository gives examples of configuring the OpenAI Python client to use `localhost:3900` as the base URL.
That could make VoiceStudio useful not only to end users but also to developers building applications that require speech generation or transcription while keeping the processing environment under local control.
Integration with AI agents
The project also has a connection to the growing ecosystem of AI agents.
VoiceStudio provides an MCP server that can be used by compatible clients such as Claude Desktop, Cursor and other AI tools. The available MCP functions include speech generation, voice cloning and transcription.
The repository additionally provides skills compatible with tools such as Claude Code, Codex and Cursor through the skills.sh ecosystem. This allows developers to integrate local speech and transcription capabilities into agent-assisted workflows.
The combination of a local API and MCP interface means VoiceStudio can potentially function as a speech layer for other applications rather than being limited to its own graphical interface.
Privacy is a central design feature
The project's local-first architecture also addresses one of the key concerns surrounding voice AI: where recordings and generated speech are processed.
VoiceStudio says audio recordings, transcripts, voices and projects remain on the local machine in its default configuration. Analytics is disabled unless the user provides consent. If enabled, the project says it sends only allowlisted, content-free usage information and excludes text, audio, filenames, voices and projects.
The network boundary is also deliberately defined. The desktop application communicates with a loopback-only backend, while remote access requires additional authentication. Remote workers and OpenAI-compatible external ASR are opt-in features.
For organisations handling sensitive interviews, recordings, internal communications or unpublished media, this local-processing model could be an important consideration.
Watermarking and responsible-use safeguards
Voice cloning also introduces obvious risks around impersonation and unauthorised use.
The project explicitly advises users to obtain permission before cloning or synthesising another person's voice. It also integrates AudioSeal-based imperceptible audio watermarking by default to help detect and identify synthetic speech without altering its audible quality.
At the same time, the repository makes clear that the software's application licence does not automatically grant commercial rights to every model or model weight used through the platform. Individual engines and their weights can carry separate licensing conditions.
This is particularly relevant for commercial content creators, publishers and organisations considering VoiceStudio for production work.
Open-source project still in beta
VoiceStudio is released under the AGPL-3.0 application licence, while individual models and model components retain their own upstream licensing terms. The project is free and does not have a paid tier, according to its repository.
The repository remains under active development. It currently contains thousands of commits and lists ongoing issues and pull requests, while the project itself warns that the `main` branch can change between releases.
For users who want to test the system without installing it locally, the project also provides a Google Colab option. However, the developers note that Colab is remote computing, meaning uploaded audio and project data do not remain solely on the user's machine in that setup.
A different model for AI voice production
VoiceStudio's significance lies less in offering another text-to-speech interface and more in combining a range of speech technologies into a local production environment.
Voice cloning, voice design, transcription, diarization, dubbing, audiobook production, dictation, batch processing, local APIs and AI-agent integration are all presented as parts of the same platform. The project's architecture also allows users to choose among different speech engines according to language, hardware, quality and licensing requirements.
The result is a tool aimed at users willing to trade the convenience of managed cloud infrastructure for greater control over their data, models and computing environment. For developers and creators with capable hardware, VoiceStudio represents an increasingly practical route toward running sophisticated speech-generation workflows without sending every recording or text prompt to a third-party cloud service.
