This work presents a low-power motion gesture recognition system-on-chip (SoC) for smart devices. The SoC incorporates a low-power image sensor and a memory-efficient outermost-edge-based gesture sensing DSP. The DSP utilizes a self-adaptive motion detector that automatically updates a motion-pixel threshold for accurately sensing hand movements. A convolution-based noise-tolerant feature extraction technique is also developed for preventing detection errors caused by random noises in the images from the low-power sensor. The feature extraction architecture is highly accelerated utilizing parallelisms and pipelining for achieving low-latency real-time gesture recognition. Measurements from a test chip fabricated in 65nm CMOS show that the SoC consumes 213.7 μW with only 3 μW dynamic power at 30fps. The SoC occupies only 0.54mm2, making it very well-suited for wearable devices and sensor nodes. The image sensor is fully operational down to 0.6 V while the DSP can be scaled down to 0.46 V. The average recognition accuracy of the system is 85% while the latency is 1.056ms.