TalkSee

Communication is an important and unavoidable aspect of our lives. Communicating with different people having diverse cultures and linguistics requires an adequate understanding of their language. As one cannot acquire knowledge of every language, this creates a barrier in social interaction and effective communication among different people.

People used to hire translators with them, each for a specific language. But as the technology evolved, translating gadgets became available in the market which resolved this problem. And in today’s era when technology is progressing rapidly and at its peak, many translation applications are available online for free. These applications translate your language into desired languages just upon a click.

Our proposed system is also such an application but differs in features from the conventional translators. TalkSee is not just a translator, but a web-based chat application solely determined to be designed for providing real-time translation and communication between users through typing in their preferred languages. Furthermore, a model or avatar of the sender will read aloud the translated text message on the receiver’s end thus providing a more meaningful experience of in-person communication.

•   To provide users with an option of setting up their profiles and preferred language that will help users to communicate in their native language

•    Provision of effective translation of the text messages being exchanged between sender and receiver.

•   To build a virtual model from the user’s profile pictures in which the translated message will be fed and converted to speech then a video of that talking avatar with lip syncing will be shown to the receiver  to provide a similar experience of face-to-face communication

There are unlimited multilingual translation applications available that not only allows you to communicate with friend/family but provides an option to add new friends in your contact list to enhance your communication network.  

The related work or application for translation of language are given below:

• Chat Translator [1]   provides translation of text messages and support speech translation, but it is not a chat application (Chat Translator, 2014)

• NEEO, Free IM & Chat Translator [2] is also a text messenger that allows you to translate and send text and voice messages (Neeopal, 2020)

• TransFire [3]  A messaging app that translates text in the chat. It has simple interface and does not support voice translation (Olanoff, 2012)

• Mobile Language translator[4] is a mobile application that translates English language into Bahasa Malaysia, Bahasa Indonesia, Mandarin and Tamil. It provides translation of common phrases that are used in these languages and provides pronunciation of particular words  (Sim, Md Gapar Md Johar, Ilfaki, & Kevin Loo Teow Aik, 2012)

But none of the above is providing translation with an avatar speaking according to lip syncing in a video and providing a more realistic experience. The related work or application through which got an idea to introduce a talking Avatar are given below:

• Crazy Talk [5] is an AR application software that creates 2D avatars with body motions. However, the software requires a still image for the purpose instead of a real time object. (Cartoon Animator 4, 2020)

• ·SpeakPic [6] uses text-to-speech technique and creates a lip movement it requires a picture for that (SpeakPic, n.d.)

Our proposed system is a chat application with the user typing messages in his/her language. Possible extensions are,

•  Adding speech recognition feature or other desired features

• TalkSee as a mobile application

[1]      Chat Translator. (2014, June 19th). Retrieved 2020, from https://play.google.com/store/apps/details?id=com.lecode.chatranslator&hl=en

[2]      Neeopal. (2020, July 3rd). Retrieved from Neeopal: https://www.neeopal.com/

[3]      Olanoff, D. (2012, January 5th). TransFire. Retrieved December 2020, from thenextweb: https://thenextweb.com/apps/2012/01/04/this-chat-app-has-something-the-others-dont-real-time-language-translation/

[4]     Sim, L. F., Md Gapar Md Johar, Ilfaki, A. O., & Kevin Loo Teow Aik. (2012). Investigative Study towards the Development of Mobile Language. International Journal of Digital Content Technology and its Applications.

[5]      Cartoon Animator 4. (2020). Cartoon Animator 4. Retrieved 2020, from reallusion: https://www.reallusion.com/cartoon-animator/default.html

[6]      SpeakPic. (n.d.). Retrieved 2020, from SpeakPic: https://play.google.com/store/apps/details?id=com.speakpic&hl=en&gl=US

https://talkSee.com

1. TalkSee Frontend & Backend

2. GAN+Wav2Lip Model