Spoken Language Understanding
 Every language in Taiwan, with the exception of dominant Mandarin, is endangered. Since a language is the bearer of a culture, an incredible freight of human knowledge and experience and understanding--of epics, myths, nursery rhymes, proverbs, parables, ritual formulae, jokes, love-songs and dirges. Therefore, something must be done before it's too late. To take the first step, we are now working hard to implement a character-level sequence to sequence-based Chinese text to Taiwanese language interpreter for Taiwanese Speech Synthesis.
 Deep neural conversational modeling, especially the sequence to sequence method, is a new approach for spoken dialogue system research. Therefore, a character-level Chinese gossip chatbot, had been implemented to test the idea.
 Speech is the most natural, powerful and universal media for human-machine/computer communication. Our focus is to develop all necessary modules for Spoken Dialog System including robust speech, speaker and language recognition and natural speech synthesis.
 *Demo: Youtube
Android App with RealTime Multilingual (mixed Chinese/English) Speech Recognition Server based on:
1) Deep Neural Networks, 2) WebSocket/Http and 3) GStreamer.
 *Demo: Youtube or Google Drive
Automatic Broadcast Radio Subtitle Generation (audio to *.srt) using Deep Neural Networks. This is a real-time multilingual (mixed Chinese/English) speech recognizer. Our aim is to study semi-supervised/unsupervised training.
 To approach
the goal of establishing an End-to-End speech synthesis system, we propose to use
character-level recurrent neural networks (RNNs) to directly convert input character
sequences into latent linguistic feature vectors.
 Smart Home is a hot research and development (R&D) topic recently. Our focus is to build a spoken dialog-based Smart Housekeeper and make speech, speaker and language recognition more robust and useful for our daily life.
 We have developed a speech synthesis-enabled DAISY player. It is small, portable and designed for use by people with "print disabilities", including blindness, impaired vision, and dyslexia.
 (from http://diging.github.io/tethne)
Multimedia data (in Youtube, Live broadcast,...) are dramatically increasing recently. We therefore need a efficient way to browse, use and archive those speech data through speech/speaker recognition, story summarization and organization.
 Secondary language learning is more and more important today (global village). Therefore, we would like to build a computer-assisted language learning system that could simultaneously detect pronunciation errors, speech prosody deviations and dialogue act mistakes.
| Multimedia Signal Processing
 | Activities
Our team has been ranked the [3rd] out of 19 in the overall performance (average of the performance of all conditions in Cavg) ranking list, and the [4th], [2nd] and [2nd] out of 19 in the 1-, 3-second and full length utterance ranking list. Note: The submissions with a star after the team name is an extended submission. This means they should not be treated equally as the regular submissions (without a star).  http://cslt.riit.tsinghua.edu.cn/mediawiki/index.php/OLR_Challenge_2017
Our team NCTU-NTUT has be ranked the [4th] out of 19 with Mean-Ranked 6.5 out of 23 runs submissions.
Our Team received the "創新研發應用獎" (Technology Innovation Award) in the Broadcast Golden Bell Awards 2017 ceremony for Automatic Subtitling of Broadcast Radio Programs and Interactive Learning Systems. 
 Our deep LSTM-based ASR achieved overall performance Rank 2, English performance Rank 1 in the extended submission of OC16 Chinese-English MixASR Challenge
Machine Learning
 Our workhorse: Karas + Tensorflow + Ubuntu 16.04 on Tyan FT77C-B7079 4U 8 Bay + ASUS ROG Strix GeForce GTX 1080 * 8
 Our gated DNN system for the NIST 2015 language recognition i-vector machine learning challenge. It was designed to solve the language clustering and out-of-set detection issues simultaneously. It achieves a relative performance gain of up to 51%, compared to the baseline cosine distance scoring (CDS) system provided by organizer.
Example of FA-DNN outputs for speaker recognition: (a) original speaker i-vectors, (b) purified speaker i-vectors.
 Although, deep neural networks (DNNs) are very powerful but it still could be easily affected by noises. We have developed a new factor analysis DNN (FA-DNN) structure and training algorithm that can successfully separate wanted signals and noises.
Wireless Communication
 Indoor navigation and its application is a hot research topic recently. We want to combine Air-Beacon and internet information retrieval to built a in-door navigation system for Location-Based Service (LBS).
 Sound is an universal media for broadcasting or exchanging messages between different platforms, for example, between iOS/Android smart TVs, tablets and mobile phones.
|
