What Changed in the Room?
Evidence-based 3D change detection from two phone videos of the same room: it reports objects added, removed or moved, in 3D and in plain language, and keeps viewpoint differences, occlusion and unseen areas from being reported as changes.
Problem to result
Who had the problem, what I built, the hardest part, and what it achieved.
Spotting what changed in a room from two phone videos is easy to get wrong: different viewpoints, occlusion and areas only one video saw all look like changes, and transparent objects are invisible to depth sensing.
A tool that compares two walkthrough videos of the same room and reports what was added, removed or moved, in 3D and in plain language.
Transparent objects. Depth sees straight through a glass, so a removed glass left no trace. Two approaches failed; the one that worked compares matched before and after views by appearance and only accepts a change that scores well below nearby unchanged places seen through the same views.
All 8 recorded test pairs correct, including the removed drinking glass; a 2 m chair move measured at 1.93 m in a synthetic test; false changes on high-angle pairs cut from up to 7 to 0; 8 of 8 synthetic tests pass.
- Thresholds (0.72 similarity, 0.10 below controls, 0.05 label margin) were tuned on the same 8 recorded pairs reported here; there is no separate held-out set.
- No real recording of a moved object: the 1.93 m move comes from a synthetic test, so move measurement is unproven on real footage.
- Ground truth for the recorded pairs was established by eye from matched views, not by an independent measurement.
Results: table and top-down maps
The outcome for every recorded test pair, then one top-down map per pair.
Results by test pair
| Test | Lighting / viewpoint | True change | System output | Unchecked (before / after) |
|---|---|---|---|---|
| test4 | cool white | clear plastic jug added | jug added | 7% / 9% |
| test7 | cool white | drinking glass removed | drinking glass removed | 11% / 4% |
| test6 | cool white | none (control) | no changes | 12% / 6% |
| test3 | cool white, high angle | none | no changes | 41% / 13% |
| test5 | cool white, high angle | none | no changes | 59% / 11% |
| test8 | cool white, high angle | none | no changes | 32% / 13% |
| test1 | warm, very dark, high angle | no visible change | no changes | 43% / 10% |
| test9 | warm, high angle | no visible change | no changes | 32% / 8% |
- 8 of 8 outputs match the ground truth. There is no test2 recording, so the pairs are test1 and test3 to test9.
- Verification (DINOv2 similarity of matched crops): jug 0.47 vs controls 0.84; glass 0.71 vs controls 0.87. A change must score below 0.72 and at least 0.10 below its controls; static places score 0.76 to 0.89.
- Registration tolerance (2.5 × measured ICP error, clamped to 1–5 cm): 1.1–1.5 cm on well-aligned pairs, 3.4–4.0 cm on high-angle pairs.
- Unchecked: share of surfaces one video saw that the other didn't see clearly. Changes there are not reported.








test7 · top-down map · drinking glass removed
cool white light. Blue and orange: before and after camera paths; grey: unchanged; yellow: seen by only one video and left unchecked (11% of the before video, 4% of the after video); red: removed / old position, green: added / new position.
Example output files
The real results summary, report.md and changes.json files the system wrote for the recorded test pairs.
WHAT CHANGED IN THE ROOM — 8 recorded test pairs (thresholds tuned on these pairs) ────────────────────────────────────────────────────────────────────────────── test light / viewpoint ground truth output test4 cool white clear plastic jug added added jug test7 cool white drinking glass removed removed drinking glass test6 cool white no change (control) no changes test3 cool white, high angle no change no changes test5 cool white, high angle no change no changes test8 cool white, high angle no change no changes test1 warm, very dark, high angle no visible change no changes test9 warm, high angle no visible change no changes All 8 outputs match the ground truth (judged by eye from matched views). Three high-angle pairs gave false alarms before the control-place verification step. VERIFICATION (DINOv2 similarity of matched crops vs control places) ────────────────────────────────────────────────────────────────────────────── jug (test4, added) 0.47 controls 0.84 glass (test7, removed) 0.71 controls 0.87 static places, cool-white tests 0.76 to 0.89 (a change must score < 0.72 and >= 0.10 below controls) SYNTHETIC TESTS (tests/test_changes.py) ────────────────────────────────────────────────────────────────────────────── 8/8 pass: hidden and unseen objects 100% unknown, static box 0% changed, moved chair measured at 1.93 m (true 2 m) with no false rotation.
Project Walkthroughs
Click any video to play.
test7 · drinking glass removed
Reference (left) and test (right): the glass next to the black mug is outlined before and marked missing after. Found by the appearance channel, because the depth model sees through glass.
test4 · clear jug added
The new jug is outlined in the test video and a dashed box marks where it will appear in the reference. Its carved 3D box is 16 × 11 × 32 cm.
test6 · no change (control)
Same desk, nothing changed: the system reports no changes. Yellow tint marks surfaces only one video saw, which are left unchecked rather than reported.
Problems & Fixes
What went wrong while building it, and how each problem was solved.
No Python, ffmpeg or COLMAP on the machine
Cause: Fresh environment
Installed Python 3.11 and ffmpeg with winget; project venv; PyTorch 2.11 with CUDA 12.8
Source Code
GitHub repositories for this project.