Our assistive feeding device combines computer vision, robotics, and embedded systems to create an affordable solution for individuals with limited upper body mobility.
The system uses real-time mouth tracking to position food at the optimal location, giving users control over their feeding process. Built with accessibility in mind, the entire system costs under $600 and is fully open-source so other teams can replicate it.
The project was awarded the Olin College SAG Grant for prototyping support and the Olin College SEED Grant for continued development. We have since formed Munchkin LLC, licensed in Massachusetts, to support structured user testing at Briarwood Rehabilitation & Healthcare Center and future development.
V3 is our current development version, rebuilding the system with higher-torque servos, an integrated plate holder, and a modular tool-changer. User testing will hopefully be happening at Briarwood Rehabilitation & Healthcare Center in Fall 2026.
V3 introduces a fully automatic fork-and-spoon switcher. The mechanism locks and releases each utensil in under 2 seconds — no hands required, and the correct tool is always selected based on what's being served.
Upgraded to 30 kg·cm servos with 0.1° angular precision. Smoother motion, more torque, and far better positional accuracy than v1.
The plate holder is now built into the device body and folds flush when not in use — no separate tray to carry, and no setup required.
The arm now folds completely within the device body, bringing the whole system to a compact footprint for easy transport and storage.
An integrated touchscreen lets users select feeding mode, speed, and other settings directly on the device — no phone or laptop needed.
New external ports support switches, sip-and-puff controllers, and other assistive devices — enabling partial or full control depending on each user's capabilities.
Computer vision will identify what's on the plate, track portion sizes, and provide nutritional estimates in real-time — making Munchkin a smarter feeding companion.
We explored several design approaches before landing on our final solution. Our initial concepts included a stationary arm with rotating base, a gantry-style system, and a jointed arm similar to industrial robots.
4-DOF articulated arm, 4-DOF SCARA-style, 2-DOF pan-tilt mechanism
4-DOF articulated arm with base rotation, shoulder, elbow, and wrist joints
We chose the 4-DOF articulated configuration for maximum workspace flexibility and natural motion. The additional wrist joint (vs. 3-DOF) allows proper utensil orientation regardless of approach angle, critical for smooth feeding. While more complex than simpler designs, the improved dexterity justified the added servos and control complexity.
Interactive inverse kinematics using CCD algorithm • Move your mouse to control the arm
MediaPipe Face Mesh, OpenCV Haar Cascades, custom CNN model, depth camera only
MediaPipe Face Mesh with RealSense depth camera
MediaPipe provides 468 facial landmarks at 30fps with excellent accuracy on Raspberry Pi 4. Combines well with RealSense depth data for precise 3D positioning. Pre-trained models eliminated weeks of custom ML training, and open-source implementation made debugging straightforward. Haar Cascades were too inaccurate, and custom CNNs would require extensive datasets we didn't have.
See mouth tracking in action
Tip: Select "Always allow" for best experience
Raspberry Pi, Arduino + OpenCV on laptop, Jetson Nano, microcontroller with cloud processing
Raspberry Pi 4B (4GB RAM)
Raspberry Pi 4 offers the perfect balance of compute power and cost ($62 vs. $200+ for Jetson). Quad-core ARM processor handles MediaPipe at 30fps while managing servo control and safety monitoring. Built-in USB 3.0 for camera, GPIO for safety buttons, and strong community support made development smooth. Arduino lacked ML capability, and cloud processing introduced unacceptable latency for safety-critical feeding motions.
Test your reaction time vs Raspberry Pi 4
Click when the screen turns green!