FaceNav Overview and Tutorials

More from User Help Files:

    - Speech Commands: Expanded vocabulary includes new commands for advanced mouse and keyboard actions, media control, and system navigation.

    - Customizable Gaming Keys: Nose gestures (up, down, left, right) can be mapped to any keyboard keys for gaming or productivity.

    - Facial Gestures as Triggers: Gestures like "smile" and "raise eyebrows" can be assigned to specific actions, providing non-verbal control.

    - Improved Visual Overlays: Configurable overlays for better feedback during tracking:

        NoseBox Overlay: Green box with a visual "nose dot" showing live tracking.

        Arrow Overlay: Highlights the active direction for keyboard-based nose tracking.

    - Low-Light Camera Mode: Enables an alert when the lighting conditions are suboptimal for tracking.

------------------------

** Modes of Operation **

------------------------

1. Nose Navigation with Speech

------------------------------

In this mode, your nose position controls mouse movements, with commands augmenting the experience for clicking, dragging, and keyboard inputs.

    - Green NoseBox:

        The NoseBox is the Area of Interest (AOI) displayed on the screen. It helps define where the mouse cursor will move.

        The size of the NoseBox can be adjusted via the "Zoom in" and "Zoom out" commands, allowing for either fine-tuned precision or broader movements.

    - Mouse Control:

        Joystick Mode: Relative movements of the nose within the NoseBox adjust the cursor position smoothly.

        Direct Nose Tracking: Absolute movements within the NoseBox directly map to the screen.

    - Key Commands:

        "Butterfly": Resets the resting position of the cursor to your nose's current position.

        "Click", "Right click", "Double click": Perform mouse clicks using speech.

    - Zoom Commands:

        "Zoom in": Enlarges the NoseBox, increasing precision.

        "Zoom out": Shrinks the NoseBox, increasing cursor responsiveness.

    - Dragging:

        "Hold left"/"Release left": Click-and-drag functionality for the left mouse button.

        "Hold right"/"Release right": Click-and-drag functionality for the right mouse button.

2. Nose Navigation with Keyboard Gestures

-----------------------------------------

This mode extends nose gestures to simulate keyboard actions, with two distinct behaviors:

    - Arrow Keys (Hold Down): Continuous arrow key presses while your nose remains outside the NoseBox boundaries. This is useful for gaming or smooth navigation.

    - Arrow Keys (Press Once): Triggers a single key press when your nose moves outside the NoseBox. Ideal for stepwise navigation or turn-based games.

Default mappings for gaming and productivity:

    - Nose Up: W or Up Arrow

    - Nose Down: S or Down Arrow

    - Nose Left: A or Left Arrow

    - Nose Right: D or Right Arrow

Mappings can be customized via the interface for specific workflows or games.

3. Facial Gestures

------------------

Non-speech-based interactions use facial expressions as input triggers. These can be assigned to any mouse or keyboard action.

    Available Gestures:

        Smile

        Raise Eyebrows

        Open Mouth

        Fish Face

For example:

    - A "smile" can simulate a left mouse click.

    - An "open mouth" can trigger a "space" key press.

4. Speech Commands

----------------

Mouse and Cursor Control:

    - "Butterfly": Reset the NoseBox center to your current nose position.

    - "Start"/"Stop": Enable or disable nose tracking.

Clicking:

    - "Click": Left mouse click.

    - "Right click": Right mouse click.

    - "Double click": Double left click.

Dragging:

    - "Hold left"/"Release left": Drag using the left mouse button.

    - "Hold right"/"Release right": Drag using the right mouse button.

Zoom:

    - "Zoom in": Reduce NoseBox size for increased precision.

    - "Zoom out": Enlarge NoseBox size for increased cursor responsiveness.

Keyboard Shortcuts

------------------

A comprehensive list of shortcuts is available, allowing speech commands to replace key presses:

Movement and Navigation:

    "Next", "Previous", "Up", "Down": Arrow key presses.

    "Page up", "Page down": Scroll the page.

Text Editing:

    "Copy", "Cut", "Paste".

    "Undo", "Redo", "Bold", "Italic", "Underline".

    "Find": Activates search functionality.

    "Select all".

Media Control:

    "Play"/"Pause".

    "Next track"/"Previous track".

    "Volume up"/"Volume down".

Window Management:

    "Switch Windows": Alt+Tab.

    "Minimize"/"Maximize": Adjust the active window.

    "Close": Alt+F4.

File Operations:

    "Save", "Print", "New Document".

    "Files": Opens File Explorer.

Overlay and Visual Feedback:

    - Green NoseBox: Displays the area of interest for nose navigation, with live feedback via a green box and a white "nose dot."

    - Triggered Color: The dot changes color when active.

    - Arrow Overlay: Highlights the active direction for keyboard-based nose tracking.

Camera Settings:

    - Default Camera: Select your input device for nose tracking.

    - Low Light Mode: Enables an alert when the lighting conditions are suboptimal for tracking.

Advanced Gaming Configuration

-----------------------------

Assign any keyboard action to nose gestures for gaming.

Defaults:

    Up: W

    Down: S

    Left: A

    Right: D

Easily remap keys for tailored gameplay.

How to Get Started:

    Position yourself in front of the camera and ensure your face is well-lit.

    Choose the tracking mode:

        - "Joystick" for relative mouse movements.

        - "Direct" for absolute nose-to-screen mapping.

    Adjust the NoseBox size using "Zoom in"/"Zoom out".

    Customize speech or gesture bindings to match your needs.

For more information visit www.motioninputgames.com or contact us at enquiries@motioninputgames.com

What hardware do I need to run this?

- A Windows 10/11 based PC or laptop, with a webcam! Ideally an Intel-based PC, 7th Gen and above, and 4GB RAM. An SSD is highly recommended. 

The more CPU cores the merrier! Running parallel ML and CV is highly compute intensive, even with our extensive optimisations. Doing simple mouse clicks in a web browser, should be much less intensive.

This is an AI PC application, and no data is stored, only settings information for use. 

What platforms does this run on?

- Windows 10 or 11. If you would like to help to test this with us, please contact us: https://forms.office.com/r/P3a3dsH8Gx or by email on our touchlesscomputing.org website.

How do I run the software?

- CLick on MotionInput Facial Navigation from the start menu. 

On some computers we have found that the Microsoft Visual C++ Redistributable is not installed. If no window appears, try installing this for X64 - https://docs.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170

Follow the instructions on the download link.

Errors:

It didn't run, what can I do?

- This is an early access version is to improve upon our final gold build.  

If you have an antivirus it may say warnings (or stop the app completely) from accessing the camera and microphone. The software requires those things to run, so you may have to alter settings in your antivirus and restart your machine.

Keep your Windows 10 and Windows 11 installations up to date, the oldest versions of Windows 10 have reported errors.

You are welcome to contact us with any other thoughts and requests on compatibility issues you have faced.

Non-technical points of interest:

This is very much in the realm of sci-fi! Can I do gestures like in <insert hollywood film here> ?

- Get in touch with us to send us a request!

What motivated you to build this?

- Covid-19 affected the world, and for a while before the vaccines, as well as the public getting sick, hospital staff were getting severely ill. To keep shared computers clean and germ free comes at a cost to various economies around the world. We saw a critical need to develop cheap/free software to help in healthcare, improve the way in which we work and so we examined many different methods for touchless computing. Along the journey, several major tech firms had made significant jumps in Machine Learning (ML) and Computer Vision. Some of the tech firms also had let go of past products which would have been useful if they were still in production, but the learning from them was still there. At the same time, we also realised that childhood obesity and general population health was deteriorating during lockdowns. So we developed several products to specifically look at how to get people moving more, with AI tuning of accuracy for various needs. We also saw how it can be designed and fine tuned for specific accessibility needs. This includes for cerebral palsy, MND, limb difference/amputees, muscular dystrophy and other musculoskeletal conditions, fingerspelling for the deaf and many more AI models. 

Videos of some of our other products:

www.tinyurl.com/mi3intel

www.tinyurl.com/mi34bbcnews

See our website for more information. 

Whats next?

- We have a lot of great products being developed and the tech firm companies on board are excited to see what our team will come up with next. Keep in touch with us, send us your touchless computing requests and especially if it can help people, we want to know and open collaborations.

Feedback form: 

https://forms.office.com/r/P3a3dsH8Gx or by email on our motioninputgames.com website.