VR-ML-Robot projectÂ
August 2023
To run the APK without the Boo: Put the VrHandy folder in your website Documents folder, update the ip and port in the DataMgr script SetIPPort()
To run the APK with the Boo and without the website: Tap the Live button, which will let your left hand control the simulated hand.Â
Sign Quiz: Learning different signsÂ
(using TensorFlow 2 and Barracuda at 72 hertz)
Cable-driven joints with servos in forearm
each finger has 4 DOF (can do Spock hand signal)
Live Long And Prosper
human-size with remote servos (cable)
(left above) hand tracking to simulated robot (range of motion)Â (Center above) Print times for an early version (Right above) AI stuff in PDF, zoom in to see details
Historical Documents: PDF's with links to videosÂ
download and view locally for best results (large files)Â
Here is the finalized, simplified code for the hand-tracking to servo pipeline. Also (below right) is a sample journal / daily progress report that I use to communicate. Â
Super Simple Code generated bones each frame
Notice the 4 Degrees Of Freedom, the knuckle is rotated 90°. (It can do the Spock hand signal: Live Long And Prosper)
Each finger (including the thumb) has 4 servos (knuckle servo rotated 90°)
20 servos for each hand, 3 servos (larger) for each wrist = 23 servos
Possible Use: Using hand tracking and a simulated hand, a Quest Unity app can train in sign language.
Possible Use: With a Quest 3 headset with pass-thru or Leap Motion hand tracker, a hand robot can have a conversation in sign language
Featuring Unity, Vuforia, UDP, remote Asset Bundles, and logging data to a website (php)
<<<< Extended Demo
Welcome 2019
Particles VR (stereo)
SHORT RESUME
2019 Jan: Unity dev, Unity plugins, C#, Unity Shaders, Unity Real time Audio, Physics, AR / VRÂ
Looking for an immediate position as a Unity developer
Your feedback is desired
Stretch Game
Reach around to score while stretching. Keeping fit without realizing it.
Pass-thru video
Tap to score
Control amount of connectors
Collision avoidance
WillyWonkaLift
Like an urban ski lift. Minimal visitor interaction. Mixed reality template for mobile IOS/Android built in Unity. The mobile gyro is used for orientation. The lift moves smoothly between waypoints.  This is a way to explore large three dimensional spaces.Â
Hello:
This is Amre Amer. I am looking for an immediate opening for
a Unity Developer position, with experience in AR/VR, Android,
UDP local networking, game development, prototyping and problem-solving.
My cell is 858 337 2218. I currently live in South San Francisco, CA, and
can relocate. My email is me@amre-amer.com.Â
I have good references from my previous work with a very interesting
startup that is having funding issues.
I look forward to speaking with you,
Amre
Realistic behavior is the goal.
July 2018 Augmented Reality with ArKit 1.5
Shaders
zone detection for game objects
works on GPU, frees up CPU
limited physics in shader
3d zones with layered shaders
computer vision avoidance
Simple 28x28 Image Classifier NN
Neural Network Classifier
Using 28x28 images, 784-node input layer, two 16-node hidden layers, 10-node output layer in Unity. Custom activation.  Â
Autonomous Driving Simulator Platform
AutoDrive
Exploring AutoDrive concepts in Unity C#. A three-day project to set up a learning environment for historical prediction and other concepts.
Studying Fibonacci and Neural Networks
Numbers at an 8 year old level
Trying out a fibonacci style neural layout, which seems to ignore pattern shift successfully.
I want to work with Unity 2018.Â
Conceptual thinking and prototyping skills to improve on existing ideas or create new ones. Â
Fast turn-around and good communication skills.
Lots of experience in fast Unity prototyping, optimizing, intellectual property, mesh control, simple physics, pass-thru video, billboarding, positioning, sensors and robotics, raspberry pi, 3d printing, drones, cabling, linux, c#, computer vision, machine learning.
Cool Achievements mid 2016 thru 2017
The last 18 months have been a fantastic experience at a "stealth mode startup", in Scotts Valley, CA. Here are some of my learning experiences:Â
Billboarding
Top Achievement 2017
Billboarding: More content with less CPU
To render multiple complex objects at 120 fps. By using several virtual cameras and billboards, filming one "heavy" 3D object, we can convincingly display billboarded copies each with their own AI, using the remainder of available CPU. Â
Intellectual Property
Top Achievement 2017
Intellectual Property: 2D / 3D Content Security
When intellectual property rights for 3D content need to be protected, keeping content on secure servers is crucial. Content is communicated with billboarded 2D video streams. For stereoscopic 3D, content can be shared using two billboards per viewer. Synchronized sensor data is sent back. This is a good business model for sharing proprietary content.
Mesh Control
Top Achievement 2017
Mesh Control for CPU Optimization
Mesh control is the ability to individually control thousands of quads for fish ai. One mesh instead of thousands of game objects is used. One or more images is displayed on thousands of individual quads in a single mesh with individual AI.
Simple Physics
Top Achievement 2017
Simple Physics
In addition to mesh-based quads and billboards, simple physics is used to create fish snakes or nettle tentacles with game object segments that follow each other. Simple physics limits the physics to what we need, using less CPU.
Unity
Unity 2017, Coding and iOS
I started with Unity in mid 2016, creating MindRaft (left image) as part of my interview process. Unity is great for fast prototyping, direct pass-thru video, Vuforia, HTC Vive, Leap, UDP, webcams and sensors. Â
Fish AI
Act Like a Fish
Fish AI brings game objects, mesh quads and procedural meshes to life, acting like fish, snakes, and nettles. Using look and target points, efficient CPU usage is achieved.
Hardware Support
Hardware Support
Hardware support includes cables, diagrams, charging and maintenance. HDMI, USB and power cables are used with a variety of connectors. Lightweight, tightly wrapped cabling is critical to a good presentation.
Positioning
Positioning
Accurate positioning is currently a challenge. HTC Vive, ZED, Leap, Vuforia and others are being explored. In-house experiments, like RGB positioning are useful for gaining knowledge and insight. Â
Communication
Communication with Team Members
Diagrams are used to communicate in 2D. OmniGraffle is a favorite of mine for more formal, worked-out ideas. Sketches is a fast way to go thru several ideas or to communicate rapidly. Speaking in person, one-on-one, is also necessary.Â
Sensor and Robotics
Sensors and Robotics
Raspberry Pi, Arduino, Parallax, Linux, Servos, LED's, PWM are used to transmit video streams and sensor data between servers, linux devices, and mobile devices. Infrared and proximity sensors and lasers are also used.
3D Printing
3D Printing
3D Printing is used to generate quick 3d models of a prototype concept, and functional finished products. Â
Vuforia and AR
Vuforia and Augmented Reality
Using markers to position virtual objects in the real world is a complement to 3D positioning systems. Stereoscopic 3D pass-thru video (stereo AR) is done with two webcams and proper virtual camera and screen game object placement.
Viewers
Viewer Design
Prototyping 2D and stereoscopic (3D) viewers is a big part of setting up a demo space. Â
Drones
Drones with Unity
Drones are used as sensor platforms to collect data and send that data back when possible. Unity or iOS use UDP to send commands to multiple drones simultaneously.Â
Game Of Life
Game of Life, (J. Conway) is a great exercise for an interview. Here is my experience with this amazing algorithm. This exercise includes both 2D and 3D with some open-ended challenges.
cvPong (Computer Vision 2D)
Updating a Pong-like exercise with computer vision (used in 2D) along with vector-estimating for paddle placement.
cvBots (Computer Vision 3D)
Following a green ball in 3D using stereoscopic camera vision, showing estimated distance with color.
cvFollow (mono cv)
Follow brightest green object, and turn when lost. Experimenting with blobs to identify and follow the largest blob (closest ball). In progress
cvArm (segments and joints)
Working with random numbers, joints and segments to control an arm. Simple code for remembering history for faster real-time reaction time.
Neural Network (home grown variant)
A N(N-1) network that values distance and signal ratios.