Like human brain which understands both eyes and ears,
this AI can understands both!.
So this AI understands image and text together!
We augment Person with Disability with this AI
An AI copilot that reasons with both EYES & EARS
Imagine potential uses cases of this AI with person with disabilities
Source code?
To get the source of this, https://colab.research.google.com/drive/1DmJP4-XOOWoL_397G9DFYTHRquPgjgAY
Try it:
Multimodal AI for Empowering individuals with disability
Persona: Persons with Visual impariment or visual cognitive requirements
Try it:
Steps to try
Use Google Chrome web browser on a Windows 10/11 laptop with microphone
Sign into Google account such as any gmail account
Open https://colab.research.google.com/drive/1DmJP4-XOOWoL_397G9DFYTHRquPgjgAY
On the menu, navigate to "RunTime"
Click "RunTime", and then click "Run All"
The demo will start in 1 minute (Please wait for the AI model to load in the machine)
The AI will accept two inputs (YOUR IMAGE, YOUR SENTENCE)
Select any random image from the dropdown (This is YOUR IMAGE)
Ask any question (type your question) (This is YOUR SENTENCE)
The AI will reason over both IMAGE + SENSTENCE
AI will return answers to your question! Vola!
You can try out all 7 modules submitted as part of this single hackathon entry
