Research Statement

I was a member of the Language Acquisition and Robotics Group at the Beckman Institute, where I researched embodied cognition on a humanoid robotics platform from the Italian Institute of Technology.  Towards that objective, my Ph.D. work generalized methods of optimal control (state estimation and state feedback) on nonlinear systems using linearizing transformations in high dimensions over finite domains.  I have compiled the results of this work into a book that I intend to publish.  I have continued to expand on these ideas with a focus on computational scalability.  Future research projects that I intend to engage in will be very product-driven and will integrate state-of-the-art machine learning (GPU-accelerated MPC with ADMM and LSTM) into robotics applications.  I am particularly interested in mobile walking robotics, soft robotics, and prosthetics.

I have had a great deal of experience in robotics research. I designed automated part inspection systems for Lakeshore Vision and Robotics. I worked at NASA’s Goddard Space Flight Center in the modeling, control, and construction of a re-configurable tetrahedral rover. I contracted for Valve Software in the design of sensor fusion systems for head mounted VR tracking.  My most recent work was in autonomous driving and navigation at Petronics, a robotics startup at the Research Park in Champaign.  While there, I also worked on augmented reality applications, and I co-wrote and won a grant for over $1 million in SBIR funding.  My work at the startup has given me a great deal of system-level engineering experience and the ability to collaborate on large projects integrating algorithms and hardware.

There are currently two projects that I would like to work on.  The first is based on the World Model work of D. Ha and J. Schmidhuber, generalizing the VAE architecture to robotics applications.  The goal is to develop increasingly generalized forms of control for stochastic and nonlinear systems, with a variety of input and output data structures, e.g., images, point clouds, and mixed continuous and discrete signals.  I would like to seek out a partnership with NVIDIA who is keen to break into the robotics market with embedded deep learning (for potential funding and hardware).  I am currently engaged in a new startup with a focus on cloud-based machine learning services.  This work would be able to provide general cloud-based control solutions that would allow small light-weight robots to have access to scalable computing power at low cost.  I would love to run a graduate lab that deploys these solutions on novel robots.  The second line of research I would like to work on is finding product-driven applications for Generative Adversarial Networks, e.g., generating photo-realistic environments and learning game engines from real world data (recently feasibility demonstrated by NVIDIA's driving simulator and style-based generator).

Sententia (2018-2019)

I worked with Jeremy Adsitt to build a cloud document classification pipeline.
built image to text services with Tesseract and text and image classifiers using TensforFlow.

Petronics (2017-2018)

Robotics and control research at a startup based in Champaign, IL.
Visit the Kickstarter campaign and product website for details.

While at Petronics, I developed an AR control interface with integrated sensor fusion and map building, multi-session SLAM with a monocular camera and globally consistent occupancy, autonomous driving with surface-aware planning and dynamic obstacle avoidance, and motion stabilization for a mobile 360 camera.  I also co-wrote and won an SBIR grant for over $1 mil.  I primarily worked with Dr. Erik Johnson, Dr. David Jun, and Dario Aranguiz.


YouTube Video

Machine Learning on the iCub (2010-2017)

Visit the Language Acquisition and Robotics Group for regular updates.
For more videos, visit our YouTube channel CogRoboticsUIUC.


YouTube Video

Ball & Plate Balancing

Solving a Lego Maze

Fun Projects

Magnetic Levitation

Project Opiliones

VALVE Vortex Group 2012

I contracted for Valve over the summer of 2012 implementing adaptive tracking systems for AR and VR.  Our group consisted of Michael Abrash, Atman BinstockAaron Nicholls, Jason Jerald, Dean Dejong, Fabian Giesen, Ben Krasnow, Jeri Ellsworth, and Gordon Stoll. We were experimenting with many different head mounted displays.  My work focused on sensor fusion with noisy position measurements from fiducial markers and Sixense readings.  The HMDs had inertial sensors and gyros haphazardly taped on, and I combined these measurements to get high-accuracy low-latency estimates of head position and orientation.  I built sensor fusion tools for HMDs with an arbitrary arrangement of completely uncalibrated sensors.  Simply shaking the HMD produced all of the necessary parameters, and then the filter gave extremely smooth low latency tracking.  Some members of the group joined the Oculus team which is now owned by Facebook. Others left Valve to start Technical Illusions and work on CastAR.  The members that stayed partnered with HTC Vive

Gordon Stoll

NASA Lunar Planetary Science Academy 2009

In the summer of 2009, Cynthia Cheung approached me with a team lead opportunity for the LPSA at NASA's Goddard Space Flight Center. The members of my team were Aaron Silver, Cletis Nicklow and Grant Moore. Each team within the academy had its own project. Our project involved designing and implementing a laser system that could be used for both ranging and communication. We designed a system capable of meeting all the requirements, and implemented various portions of it on an FPGA platform. Aside from our primary project, there were many other exciting activities within the academy. We gave weekly reports and had many interesting guest speakers. We even spent a week at a meteor impact sight in Canada, collecting data and enjoying the scenery.

Aaron Silver, Cletis Nicklow, Grant Moore

NASA Tetrahedral Robotics Research (2006-2007)

YouTube Video

The tetrahedral system that we explored is classified as ART (Autonomous Reconfigurable Technology), which means it uses modular/reconfigurable components to construct complex robotic structures. The individual components that comprise this system are designed to be as simple as possible. TET robotics employ linear actuators in tetrahedral configurations. The coordinated extension and contraction of these linear actuators allows for a myriad of complex motions. In addition, the ability to replace broken parts easily and configure for different tasks makes such technology desirable for space robotics. NASA is presently exploring this technology for rovers, but has a large array of other applications in mind.

YouTube Video

The autonomy of these structures comes from the controller hierarchy. NASA wants to limit the amount of commands it needs to send to the rover, and the lag-time between transmission and reception is appreciable even at the speed of light. For this reason it is ideal to be able to send a basic set of commands like move to this location and perform test. To do this, 3 control levels are required. The first involves the command sent by NASA. Then the rover's central brain must translate this command into the necessary length configurations for the structure. This includes both walking and avoiding obstacles. These length command then proceed to each strut to be controlled independently by a PID or some type of decentralized adaptive control for precise movement. Coordination on the second control level is being explored in two main ways. The simplest approach involves constructing a library of basic movement commands (walking gaits) and executing them in a desired order. After a base library has been complete a neural based control is sought to learn new motions through interacting with the environment.

Summer 2006

NASA / ESMD Faculty Student Fellowship GSFC outside of D.C.
Dynamic modeling and control implementation for a 12TET

Using inertial tensors and motion constraints, we were able to construct dynamic kinematics models from the Euler-Lagrange equations constructed within SimMechanics. This software made the assembly of modular components quick and easy. It operates within MATLAB using a graphical block interface just like Simulink. In fact it can interact with Simulink and other packages including custom MATLAB code. If you are not familiar with Simulink, it is very similar to LabView, but in my opinion it offers considerably more modeling power. Using SimMechanic, a single strut was constructed with the desired frictional and motor behaviors, and then it was simply cut and paste into nodal configurations. Once the model was obtained Simulink and MATLAB code was used to implement the strut control and high level walking gait commands.

During that summer, We were employed on site at Goddard Space Flight Center in Maryland for modeling and control work. The other two members of the Hope College team were Aaron Silver and our mentor Dr. Miguel Abrahantes. We teamed up with another controls team from Georgia. All of our work was coordinated. We worked alongside NASA employees and other student teams exploring the mechanical and electrical engineering aspects of the project. Each week we reported progress and discussed new ideas. Our models became invaluable to understanding the dynamics of the 12TET and optimal mechanical designs such as minimizing the diameter and spread of the struts meeting at the node.

Miguel Abrahantes, Aaron Silver

12TET Kinematics model constructed in MATLAB w/ Simulink & SimMechanics:

YouTube Video

Below is a concept video from the prototype that was built in parallel to our model.

YouTube Video

Spring 2007

NASA/MSGC Hope College:
Exploration of potential strut controllers Application of Decentralized Adaptive Controller in 1TET Model
The adaptive Seraji controller was determined to be unstable over long executions and discrete inputs.
The struts responded well to PI control, which was ultimately chosen for application in 4TET.

Summer 2007

NASA/ESMD Hope College:
Design, Construction & Control of a 4TET Prototype

The 4TET followed the construction of a 1TET assembled as an Engineering Design project. This was the first closed loop controlled walking gait of an over-constrained tetrahedral robot ever! The summer team consisted of Dr. Miguel Abrahantes, myself, Aaron Silver and Dan Lithio. Collectively we developed the control scheme and hardware for the 4TET 's autonomous control. The control system took 5 weeks to implement and cost $1k in hardware.

Control: MATLAB, USB DAQ, IFI controller Board, Custom String POTs, ESCs and Relays

We used an onboard processor from IFI that read in 2 analog signals from a reference passed through the laptop. The analog signals were multiplexed to correspond to each strut. This let the processor know what length the struts should be. The struts reported their lengths via spring loaded string potentiometers I constructed from retractable key chains. The struts themselves were made from power car antennas. The on board control was a simple PI control where the weighted addition of error and integral error defined how much power to send to the motor. The non-responsive dead-zone of the motor was corrected for. The power was controlled with 7 PWM signals out of the processor. Alternate struts were controlled via relays because we never need to control the 3 floor struts. Each PWM went to a Victor 883 electronic speed controller which was capable of reversible polarity and was provided with a 12V power supply.

YouTube Video

Our implementation that summer required a wired tether, but Miguel implemented a wireless communication system the following summer.

YouTube Video

Closing Thoughts:

This project was of particular interest to me as it extended on the concept of emergence within a complex system. Consider a system composed of simple fundamental units with simple interactions between each unit. In the presence of an energy flow through an open system, entropy within the system can actually decrease as long as entropy external to the system is increasing. This phenomenon is capable of capturing all the order we see around us, be it man made or natural. What I find most fascinating about this is even the simplest systems are capable of producing chaos while the most complex systems are capable of producing awe inspiring order . Mathematics is the key. It provides a window into the most fascinating realms of reality thought to be forever unintelligible.

Senior Design Projects

Fall 2006

Construction of a Dual Prop Micro Air Vehicle that uses payload tilt for control
Performs stable hover via two counter rotating sets of blades to cancel torque
Leans in desired direction of motion by shifting the battery forward

Spring 2007

Construction of an affordable linear actuator with controlled 5:1 extension
Project completed ahead of schedule leaving time to construct and test a 1TET 

Lakeshore Vision & Robotics (2005-2007)

LVR was a small privately owned company. I met the owner, Dr. Greg Caskey, as a part time Physics professor teaching Modern Physics. This job called on a nice mix of my Physics and Engineering background. A lot of the work I did for LVR involved the construction of mathematical models to interpret scan data and determine part geometries. This involved post processing of scan data and nonlinear regression for model comparisons. In addition repeatability and fit confidence were tested with each system. I set up new hardware for data acquisition primarily taking advantage of encoders and camera triggers. Some controls work was also done to eliminate vibrations and provide smooth motion. Most of the programming was done alongside my partner Robert Van Ark, who was a 2003 Computer Science graduate from Grand Valley. We did most of the low level hardware programming in C++ and constructed the graphical user interfaces with VB6.

Summer 2005

  • Internship in Zeeland, MI
  • Construction of a color inspection system and a 3D laser profile system

Fall 2007

  • Returned to company at new location in Holland, MI
  • Implementation of an image processing system for real-time 3D scans
  • Construction of a high resolution system for spherical parts w/ TIR< 5µm