Architectural Design

Our current exploration of potential software components to be used.  ~Rough Draft Only~

User Input Devices: PC Workstation or Web browser first, them Smart Phone.

User Input channels: Typed-in text first, then spoken voice.

Device/Hardware Input Interfaces: Photo, video, other smart phone senors, various other special purpose devices.

Output Devices: The user chosen input device via text, then print to paper, then audio/image/video output.

Software Processing:

1. Chatbot Software Frameworks

   • 1. C++ language API

        1. Botpress??

        2. Rasa??

        3. Wit.Ai??

        4. AIML - Program O??

        5. BotLibre

        6. ChatterBot-cpp

        7. DialogFlow, c++, now google sells it

        8. 
           

2. Voice to Text and back.

   • 2. 2. 1. Snowboy??

3. AI/Expert System

   • Structuring the chat-bot's algorithmic flow.

   • Chat-bot overall framework:  

     1. Google: DialogFlow CX

     2. Microsoft offerings?

     3. LangChain  https://medium.com/data-and-beyond/langchain-basics-part-1-b62caf402514

     4. Oral to Text conversion running locally https://github.com/ggerganov/whisper.cpp

4. AI/ML 

• • Entity Extraction

  • Retrieval-Augmented Generation (RAG) paired with generalized LLM

  • LLM Health Specific Models: 

    • Google:

      • Vertex AI API

      • MedLM Model: -medium, -large, 20231213

        • • "MedLM Chest Xray" available now.

          • Built on Med-PaLM2, 

          • Available on:

            • • Vertex AI (API??)

              • Gemini soon also (API??)

          • https://cloud.google.com/vertex-ai/generative-ai/docs/medlm/overview  & https://cloud.google.com/static/vertex-ai/generative-ai/docs/medlm/MedLM-model-card.pdf

      • Articulate Medical Intelligence Explorer (AMIE): expert-level differential diagnosis - chat-bot? or API available? 20240112

        • • https://blog.research.google/2024/01/amie-research-ai-system-for-diagnostic_12.html & https://arxiv.org/abs/2401.05654

      • Google / Fitbit & Google Watch 

        • Sleep profile

        • Heartbeat sensor

        • Health Metrics overtime and baseline.

      • Automated Retinal Disease Assesment (ARDA) - retina scanning. https://health.google/caregivers/arda/ & https://blog.google/intl/en-au/company-news/technology/ai-diabetic-retinopathy-screening/

      • Ultrasound: maternal (Philippines) and breast cancer (Taiwan)

      • Tuberculosis (TB)

      • Radiology mass contouring for cancer radiotherapy - Mayo Clinic

        • • Consider contouring the human spine and its relationship to the CNS/PNS.

      • Gemini interface or models??

    • Microsoft (MS) 

      • Open AI offerings??

      • https://dev.botframework.com/

      • Azure AI Health Bot /Azure for healthcare AI

      • LLaVA-Med - https://github.com/microsoft/LLaVA-Med  exploring various biomedical vision-language processing (VLP ) and vision question answering (VQA) research questions

      • Nuance Mix is an enterprise-grade software as a service (SaaS) platform that helps you create advanced conversational experiences for your customers.

      • DAX Copilot by Nuance part of MS? This AI‑powered ambient solution automatically documents patient encounters accurately and efficiently at the point of care with immediate note delivery. DAX Copilot embedded in Epic is accessed through Epic Haiku

      • MS also partnered with Epic, a health records software company,

    • Anthropic Model https://www.anthropic.com/claude

    • Meta's solution?

Software Architecture

• Client side browser: Next.js using .ts, styling with Tailwind CSS.

• Client Server Interface: https://vercel.com (serverless lambda functions?)

• Server Side: https://supabase.com (serverless lambda functions?, with PostgreSQL under the hood)

• https://www.bland.ai ?

• https://zapier.com/resources/guides/apis API integration

Potential Partners / Vendors:

• • health.google

    • https://www.googlecloudpresscorner.com/2023-08-29-HCA-Healthcare-Collaborates-with-Google-Cloud-to-Bring-Generative-AI-to-Hospitals

    • https://cloud.google.com/solutions/claims-acceleration-suite

  • Merative

    • Micromedex / DynaMedex

    • Merge / Imaging

    • Health Insights

    • Acquired from IBM

    • https://www.franciscopartners.com/investments?sector=healthcare&status=Current&region=north-america

    • UpDox