← Back to Projects
Final Project · Olin College · Fall 2024 SAG Grant Recipient SEED Grant Recipient Munchkin LLC · Massachusetts

Robotic Assistive Feeding Device

Real-time face tracking · Automatic utensil changing · 4-DOF robotic arm · Olin College

0:00 / 0:00

Our assistive feeding device combines computer vision, robotics, and embedded systems to create an affordable solution for individuals with limited upper body mobility.

The system uses real-time mouth tracking to position food at the optimal location, giving users control over their feeding process. Built with accessibility in mind, the entire system costs under $600 and is fully open-source so other teams can replicate it.

The project was awarded the Olin College SAG Grant for prototyping support and the Olin College SEED Grant for continued development. We have since formed Munchkin LLC, licensed in Massachusetts, to support structured user testing at Briarwood Rehabilitation & Healthcare Center and future development.

Munchkin V3

V3 is our current development version, rebuilding the system with higher-torque servos, an integrated plate holder, and a modular tool-changer. User testing will hopefully be happening at Briarwood Rehabilitation & Healthcare Center in Fall 2026.

Feature 01

Automatic Tool Changer

V3 introduces a fully automatic fork-and-spoon switcher. The mechanism locks and releases each utensil in under 2 seconds — no hands required, and the correct tool is always selected based on what's being served.

<2stool swap
2utensil modes
Autoselection
High-precision servos
Feature 02

High-Precision Servos

Upgraded to 30 kg·cm servos with 0.1° angular precision. Smoother motion, more torque, and far better positional accuracy than v1.

30 kg·cm 0.1° precision
Integrated folding plate holder
Feature 03

Integrated Plate Holder

The plate holder is now built into the device body and folds flush when not in use — no separate tray to carry, and no setup required.

Folds flat No assembly
Folding robotic arm
Feature 04

Folding Robotic Arm

The arm now folds completely within the device body, bringing the whole system to a compact footprint for easy transport and storage.

Self-contained Travel-ready
Onboard touchscreen
Feature 05

Onboard Touchscreen

An integrated touchscreen lets users select feeding mode, speed, and other settings directly on the device — no phone or laptop needed.

On-device UI Mode selection
Adaptive control ports
Feature 06

Adaptive Control Ports

New external ports support switches, sip-and-puff controllers, and other assistive devices — enabling partial or full control depending on each user's capabilities.

Switch access Sip & puff Modular
Coming Soon Feature 07

AI Food Tracking

Computer vision will identify what's on the plate, track portion sizes, and provide nutritional estimates in real-time — making Munchkin a smarter feeding companion.

Computer vision Nutrition data AI-powered
V1 — Original Design

Design decisions and trade-offs

We explored several design approaches before landing on our final solution. Our initial concepts included a stationary arm with rotating base, a gantry-style system, and a jointed arm similar to industrial robots.

01

Robotic Arm Configuration

OPTIONS

4-DOF articulated arm, 4-DOF SCARA-style, 2-DOF pan-tilt mechanism

OUR CHOICE

4-DOF articulated arm with base rotation, shoulder, elbow, and wrist joints

RATIONALE

We chose the 4-DOF articulated configuration for maximum workspace flexibility and natural motion. The additional wrist joint (vs. 3-DOF) allows proper utensil orientation regardless of approach angle, critical for smooth feeding. While more complex than simpler designs, the improved dexterity justified the added servos and control complexity.

Base
Shoulder
Elbow
Error
0px

Interactive inverse kinematics using CCD algorithm • Move your mouse to control the arm

02

Mouth Tracking Approach

OPTIONS

MediaPipe Face Mesh, OpenCV Haar Cascades, custom CNN model, depth camera only

OUR CHOICE

MediaPipe Face Mesh with RealSense depth camera

RATIONALE

MediaPipe provides 468 facial landmarks at 30fps with excellent accuracy on Raspberry Pi 4. Combines well with RealSense depth data for precise 3D positioning. Pre-trained models eliminated weeks of custom ML training, and open-source implementation made debugging straightforward. Haar Cascades were too inaccurate, and custom CNNs would require extensive datasets we didn't have.

📹

See mouth tracking in action

Tip: Select "Always allow" for best experience

03

Computing Platform

OPTIONS

Raspberry Pi, Arduino + OpenCV on laptop, Jetson Nano, microcontroller with cloud processing

OUR CHOICE

Raspberry Pi 4B (4GB RAM)

RATIONALE

Raspberry Pi 4 offers the perfect balance of compute power and cost ($62 vs. $200+ for Jetson). Quad-core ARM processor handles MediaPipe at 30fps while managing servo control and safety monitoring. Built-in USB 3.0 for camera, GPIO for safety buttons, and strong community support made development smooth. Arduino lacked ML capability, and cloud processing introduced unacceptable latency for safety-critical feeding motions.

Test your reaction time vs Raspberry Pi 4

Click when the screen turns green!

System Architecture

Hover over components for detailed specifications INPUT Intel RealSense Depth Camera RGB-D Video Stream USB 3.0 E-Stop (Primary) Hard-Wired to GPIO E-Stop (Shoulder Mount) Pluggable - GPIO Power Supply 5V 5A Pi Power Power Supply 9V 3A Servo Power Power Switch Controls BusLinker Power (Servo Control) PROCESSING Raspberry Pi 4 Model B Main Processing Unit MediaPipe Face Detection Mouth Tracking Open/Close Detection Motion Control Inverse Kinematics Position Calculation Servo Commands Safety Monitor E-Stop Detection Emergency Pause Serial Control USB Communication Servo Protocol Cooling Fan Thermal Management BusLinker V2.2 Serial Bus Servo Control Converts USB → Serial Bus Protocol Controls 4 servos OUTPUT Servo ID 0 - Base Pan HiWonder LX-15D Rotational Axis - Joint 0 Servo ID 1 - Shoulder HiWonder LX-15D Vertical Axis - Joint 1 Servo ID 2 - Elbow HiWonder LX-15D Vertical Axis - Joint 2 Servo ID 3 - Wrist HiWonder LX-15D Vertical Axis - Joint 3 End Effector Hybrid Spoon & Fork Holder Quick-release mechanism 3D Printed PLA USB 3.0 GPIO 5V 9V USB → Serial Serial Bus

← Back to All Projects