Overview
3Deoskill's LipTalkAudioConverter is a free software written in Python, which can convert audio speech files into special *.json files.
These *.json files can be loaded into the plugin in Cinema 4D to create the lip syncs.
It supports up to 2500 languages.
The LipTalkAudioConverter is licensed under the GNU General Public License.
Requirements
Wave Files (Mono and Stereo)
Installation for Windows
For Windows users it is pretty simple.
Download the zip-file for Windows
Unpack it
Start the LipTalkAudioConverter.exe
Installation for MacOs
- Method
Download the zip-file for MacOs
Unpack it
It will contain a Unix-Executable
Start the LipTalkAudioConverter.exec
2. Method
Download the zip-file package for Windows with the exe file and unpack it
Download the free Software Wine (Wine can run Windows exe files directly in MacOs)
Try to run the LipTalkAudioConverter .exe via Wine
Convert
If you end up with the transcript - everything is fine
Download Wine Software: www.winehq.org/
Installation Win/macOs (Source Code Files)
Python is an high-level interpreted language, which means it requires Python to be installed on your Mac in order to run python scripts or develop python software. Python comes with its standard library, but some additional modules are needed for the LipTalkAudioConverter to work properly. You can install these modules by typing a simple command into the Windows CMD or the Terminal in MacOs. It will install all necessary packages into the site-packages folder of your python installation. After the installation is complete, you can run the LipAudioConverter.py which is included in the LipTalk zip-file for macOS by double-clicking it.
Download the LipTalk zip-file for macOS
Unpack it (3 Files - LipTalkAudioConverter.pyc, requirements.txt, bash.sc)
Download Python from python.org (Minimum Version Python 3.9)
Install Python (Install in Sys Path)
Win users open CMD, MacOs users open Terminal
Type in following commands
Win: pip install allosaurus
macOs: python3 pip install allosaurus
It installs now all packages which are necessary for liptalk to work fully
Run the LipAudioConverter.py by double-click
Convert Audio
If you end up with the transcript - everything is fine
Getting Started
Language
Phonem Emitter Strength
Advanced Mode (for speech files with longer than average vowels)
Load an audio file (wave-file, mono or stereo)
The path where you want to save the generated transcript file
Convert button to convert the audio file
Status / progress bar
From this menu you can choose the language of the audio file.
International is the standard setting and can convert up to 2500 languages.
You can also find a small selection of languages in the menu that offer special phonetic features to improve the accuracy of speech intelligibility. The best thing to do here is to test which setting gives you the best results.The Phoneme Emitter value is an important parameter for generating phonemes from speech signals. It determines how many phonemes should be recognized and output per unit of time and how precisely the speech analysis is carried out.
The default value is 1.0. If you are not satisfied with the phoneme recognition, you can simply increase the value. However, if too high a value has negative effects, for example if too many phonemes result in unnatural or irregular animation, you can lower the value slightly, for example to 0.8 or 0.9.The Advanced Mode checkbox is an option you can use when working with speech files that contain long vowels. If you enable this option, the amplitude values of the audio file will be included in the transcript file, resulting in a larger file size. For example, a 10-second audio file can be 20 MB in size.
If you convert in Advanced Mode, you will later have the option to switch between Normal and Advanced Mode in the LipTalk Plugin. If you do not convert in Advanced Mode, only Normal Mode is available.To convert an audio Wave-File, you can select a file here. The file can be stereo or mono, but the quality should be high. If the level is too low, the noise will increase. In addition, there should be no disturbing background noise that affects the pronunciation of the phonemes. The audio signal should be normalized, but this is not mandatory as the AudioConverter adjusts the amplitudes internally.
If you already have an audio file selected, the program will automatically create a transcript file with the same name and location as the audio file. But you can also browse your computer and define a custom name for the transcript file.
To begin the conversion process, you need to click on the Convert button. Make sure you have an active internet connection, as the program will download a pretrained model from a remote server.
The program displays a progress bar while it performs the conversion. After completing the calculation, it indicates the status Finished.
Tip: To speed up the conversion process, keep the converter running. The pretrained model is already downloaded, so it does not need to do it again.