Computer vision project — live on AWS

A camera watches a bouldering wall, finds the holds, tracks the climber, and suggests the next move.

Three models trained from scratch — YOLOv8 hold detection, HRNet pose tracking, a linear-regression move scorer — running as real AWS Lambda functions below, not a mock.

Why upload photos instead of just a live feed? Finding an actual bouldering wall to test in front of isn't practical for most visitors here (myself included, writing this at a desk). Try it with the bundled example below, upload your own climbing photos, or use your webcam if you do have a wall handy.

One honest gap: the move-scoring model was still in early data collection when I paused this project — no hand-graded "good move" labels ever got collected, so it's trained on a heuristic stand-in (reward upward progress, prefer a comfortable reach) instead. Treat suggestions as directional, not gospel.

Source — hold detection·pose tracking·move scoring·this page

Try it

Every result below comes from the real deployed Lambda functions. First call after idle time may take 10–30s while they cold-start — everything after that is a couple seconds.

Example bouldering wall
1. The wall
Example climber on the wall
2. The climber
Enable your camera to begin, or upload photos below
1Camera
2Scan the wall

Point the camera at an empty section of wall, then scan.

3Climb

Step into frame for a live suggestion every couple of seconds.

How the pipeline works

01

Hold detection

A YOLOv8 model, trained on ~300 hand-labeled photos, finds every bolt-on hold (single class: handle). Each detected hold is cropped, K-means finds its dominant color, and a second K-means pass clusters holds into "tracks" — the color-coded routes climbers actually follow.

02

Pose tracking

HRNet estimates a 17-point COCO skeleton for the climber, cropped to a YOLOv6-detected person first so joints stay sharp even on a full-frame photo. The camera stays fixed on a tripod in the intended flow, so the wall only needs scanning once.

03

Hold matching

For each hand and foot, the nearest detected hold on the chosen track becomes that limb's current position. From there, every same-track hold within roughly one body-length becomes a candidate next move — hands search upward, feet search below the hips.

04

Move scoring

A regularized linear regression scores every candidate move and the highest-scored one gets suggested. Trained on a heuristic target — net upward progress, comfortable reach — since real hand-graded labels were never collected before the project paused.

Built on

AWS Lambda

Three container-image functions — detection, pose, scoring — each independently deployable.

Docker + ECR

Each model ships as its own image, sized to fit compiled deps like torch and OpenCV that exceed Lambda's zip limits.

API Gateway

HTTP APIs front each Lambda; this page calls them directly from the browser.

S3 + DynamoDB

Model weights and run history — hold detections, poses, and scored moves per session.