↑ ↓ space · N notes
GenAI Kids · Fall 2026 Class 02 / 08
Camera workshop · Hand Tracker V2

Pixels →
Hand →
Robot

How Reachy Mini turns 2.7 million camera numbers into “follow my hand” — about 30 times every second.

BIG robot first · live demo · then we ride the pipeline, station by station

Reachy Mini desktop robot
Reachy Mini · camera → head + antennas
Just out · Sep 17, 2026 Best home robot right now

Figure Helix 2.5

Figure’s smartest computer brain yet — it watches the camera and moves the whole body. It walks into homes it never saw before.

🧹

TIDY

living rooms

🧺

TOWELS

fold + basket

🛏️

BEDS

make the bed

30 homes near San Francisco · the robot never practiced there
With big practice first: about 56% finished the whole job · from scratch: about 9%
KID LINE A giant computer brain that learns runs the WHOLE body.
Today we go small and open the hood: one camera picture, station by station.
Helix 2.5 · 3 jobs · ~24s · tap play
WATCH · tidy · towels · beds in brand-new homes
More: figure.ai/news/helix-2-5-zero-shot-30-home-generalization
Live demo · wave your hand

Wave → Reachy follows

📷

CAMERA

sees a picture

→
✋

HAND

finds your hand

→
🤖

HEAD + ANTENNAS

follow you

Watch #1 · the head follows your palm — left, right, up, down
Watch #2 · the antennas copy your pointer finger tilt · two hands = one antenna each
TRY Hide your hand · count to 1 · watch
SPACE One Mini · 50 cm away · stay seated

Big question: HOW does a picture become a head move? Let’s ride the pipeline.

The map · 6 stations

One picture’s trip through Reachy

Stations 1–2 · the camera’s job: turn the world into numbers
Stations 3–4 · two brains that learned from tens of thousands of hand photos
Stations 5–6 · math the app’s makers invented — distances, angles, small steps
Stations 1–2 · pixels

A picture is just numbers

hover or tap a dot · toy camera: 28 × 20 = 560 dots

One dot = 3 numbers

R
—
G
—
B
—

each number goes from 0 (none) to 255 (max)

Reachy’s real picture

1280 × 720 = 921,600 dots
× 3 numbers = 2,764,800 numbers
about 30 times every second

Mirror trick: the app flips the picture so your left is Reachy’s left. · No “hand” in here yet — just numbers.
Station 3 · find the hand

Brain #1 draws a box

? HAND · 93% sure zoom in only look in here scanning the whole picture…
scan → box → zoom · repeats every picture

The hand-finding brain

  • Practiced on about 6,000 real hand photos from around the world
  • Takes in the whole picture at once → draws a box + says how sure it is
  • The app only trusts it when it’s at least 50% sure

Speed trick 🏃

Once it finds you, it just follows the box picture to picture. It only re-searches the whole picture if it loses your hand.

Next station: a second brain looks only inside the box and marks 21 dots.
Station 4 · 21 special dots

How Brain #2 finds 21 dots

Looks only inside the box. Same 21 jobs every time: wrist + each finger’s 3 knuckles and tip. Nobody wrote a rule — it learned.

screen shows them slowly · the brain places all 21 together

How it learned · guess → check → nudge

  • Answer sheets · ~30,000 real photos, all 21 dots marked by people
  • Guess · brain sees a photo (answers hidden) → places 21 dots · first guesses land on the wall
  • Check + nudge · compare to the answer sheet → shift itself a tiny bit → next photo · millions of nudges
  • Finds the patterns itself · rounded tip + shadow · crease = knuckle · can even guess hidden joints
  • + 100,000 computer-drawn hands with exact 3D joints → that’s how it learned how close (z)

0 wrist · 1–4 thumb · 5–8 pointer · 9–12 middle · 13–16 ring · 17–20 pinky

Every dot = 3 numbers

x (how far right) · y (how far down) · z (how close)
21 dots × 3 = 63 numbers · it also says LEFT or RIGHT hand

2,764,800 → 63. The picture just got 40,000× smaller — and now it’s about a hand.
Station 5 · pick the clues

63 numbers in · 3 clues out

Picking the clues (grown-ups say feature extraction) = a recipe a person wrote: a few dots in → one clue out. Yellow math — not a learned brain.

clue 1 · #10 palm spot · fingertips fly, this knuckle stays
PICTURE numbers · 0 → 1 0101x → righty → down (0.30, 0.60)
where the brain says the palm is
ROBOT numbers · −1 → +1 (0,0) straight ahead −1+1−1+1mirror-flipped: + = your right (+0.40, +0.20)
middle = 0 · edge = ±1 · what the head wants

Recipe: robot-x = (0.5 − x) × 2  ·  robot-y = (y − 0.5) × 2  →  (0.3, 0.6) becomes (+0.4, +0.2). Middle of the picture = (0, 0) = look straight ahead.

0°−60°+60° base knuckle (dot #5) stays here #8 tip #5 base +0°
clue 2 · arrow #5 → #8 · up = 0° · right + · left −
SAME MOVEMENTSYMMETRIC (mirror) antennas: +0°
two hands = one antenna each · 5° per tick so they glide
2,764,800 → 63 → 3. palm-x · palm-y · finger angle. That’s all the motors ever hear about your hand.
Station 6 · motors

Nine motors · three moves

head move 1 of 6 · turn 9 motor 9 · turns the whole body · helps when the neck runs out of turn 1 2 3 4 5 6 6 neckmotors 7 8 antennamotors
six neck motors push + pull together · the head can move six ways
6

NECK

hold up the head

1

BODY

turns it · joins in on big turns

2

ANTENNAS

one motor each

Six ways the head can move

↔ turn ✓ ↕ nod ✓ ↻ tilt ⇕ up · down ⇔ left · right ⤢ forward · back

✓ = the two moves this app uses · no single motor owns a move — all six work together

Three clues → three moves

  • palm left · right → turn the head
  • palm up · down → nod the head
  • finger angle → the antennas · two hands → one each

no tilt · no slide · the app never asks for the body — but the robot turns it itself when the neck runs out (~65°)

Why it glides: the app never says “jump to the hand.” It says “move a little closer” — 50 times a second. Fifty tiny moves look like one smooth glide.
Recap · the trip with real numbers

Same map · now with the numbers

The app’s control panel

camera-feed dots: 🔴 palm · 🔵 finger base · 🟢 finger tip · stay ≥ 50 cm from the camera
📺 Camera feed
🎯 Tracking
📡 Antennas
✋ SideLEFT ▾
↔️ ModeSAME ▾
Bonus app · Dance Mirror · same pipeline, whole body

Dance → Reachy copies you (bigger!)

YOU · 33 body dots REACHY · mirror × 1.5 a mirror: your left = its right
arm up → that side’s antenna · tilt → tilt · bounce → bob · turn → whole body turns
A kid dancing while Reachy Mini copies the pose
Dance Mirror · built by GenAI for Kids · huggingface.co/spaces/yzhsjtu/dance_mirror
  • Same 6 stations — but Brain #1 + #2 now find your whole body: 33 dots
  • New clues: head turn · nod · tilt · body turn · arms up · bounce
  • “How extra?” slider multiplies every move ×1 → ×2.6 · step away 1.2 s → eases home
TRY Sim first, then real · 1 m back · arm up · tilt · turn · bounce
SIM ≠ REAL Sim camera stays still · real camera rides in the moving head
SPOT Which stations changed? Which stayed?
Dance Mirror · stations 3–4 · find the body

Two brains again — now for the whole body

33 dots pop in by group · face · arms · hands · hips · legs

Brain #1 · person finder

Looks at the whole picture → draws a box around the person (not just a hand)

Brain #2 · dot marker

Works inside the box → marks 33 dots · about 18 pictures a second

Who’s who · 33 dots

  • 0–10 face · nose, eyes, ears, mouth ← head clues
  • 11–16 arms · shoulders, elbows, wrists ← body + hand clues
  • 17–22 hands · pinky, index, thumb — a tiny hand tracker (3 dots, not 21)
  • 23–24 hips ← bounce clue
  • 25–32 legs · knees, ankles, heels, toes (not used today)
Every dot = x, y, z + how sure · the app also gets the same 33 dots in metres (“world dots”) — handy for “which shoulder is nearer”.
Dance Mirror · stations 5–6 · clues → moves

You move → one clue → Reachy copies

Six moves. Each one is the same recipe: a few dots in → one clue → one move out. Yellow math — a person chose every row.

You doThe clue · which dotsReachy does
🙂↔ Turn your headnose slides left / right of the shoulder middlehead turns · body turns too
🙂↕ Nodnose gets closer to the shouldershead nods
🙂↻ Tiltthe ear-to-ear line slantshead tilts
🧍↻ Turn your bodyone shoulder gets nearer the camerawhole body turns
🙌 Arm upwrist above the shoulder?that side’s antenna · mirror
⬆⬇ Bouncehips above / below their usual heightReachy bobs

9 dots do all of it · nose · 2 ears · 2 shoulders · 2 wrists · 2 hips

The ruler: every clue is measured in shoulder-widths — so a big kid far away and a small kid up close give the same numbers.
“How extra?” ×1 → ×2.6 makes every move bigger · limits keep the motors safe · “move a little closer” every tick → it glides, like Hand Tracker.