# Robot-view episode audit

Checked November 19, 2026. The sample depicts autonomous operation on the original MotionOrigin concept rig. The same rig and capture layout also generated a sample-pod packing episode.

## Source reconstruction

- `historical_reference` · [Flash ALOHA](https://johnyao93.github.io/static/files/flash-aloha.pdf?v=2), Figure 6 and Appendix V. The authors used left wrist, right wrist and fixed overhead cameras. Figure 6 shows how wrist cameras naturally rotate with the arm and how braces, the gripper and nearby surfaces enter the frame. It also shows the operator mask used to reconcile collection and deployment views. Appendix V lists the ARX experiments at 50 Hz, capture at 640 × 360 and resizing to 320 × 180 for ACT. The HESSIAN tossing/catching setup used separate rates for its two arms. Exact sensor mounts, fields of view, optical distortions, and hardware clock delays were not recovered. The simulated mount transforms are original and illustrative.
- `historical_reference` · The Flash ALOHA operator directly guided backdrivable arms with scissor handles, while encoders recorded positions. The paper’s slower ALOHA 2 puppeteering baseline uses a separate setup. The autonomous example here intentionally contains no teaching operator. [Project page](https://johnyao93.github.io/flash-aloha/) · [Capture contract](https://github.com/JohnYao93/flash-aloha/blob/aec35bb/src/contracts/standard_capture.py)
- `historical_reference` · The repository’s generic [bimanual example](https://raw.githubusercontent.com/JohnYao93/flash-aloha/aec35bb/src/config/tasks/bimanual.yaml) specifies 100 Hz, three camera names and a 14-dimensional action representation. The MotionOrigin export matches the paper’s ARX capture rate and three camera roles. It uses the MotionOrigin rig’s pose schema.
- `observed_current` · The tomato animation clock comes from the first successful blemished case by ID in the first expanded holdout seed (11011, case 1). The reduced-model policy completes it in 7.033333 s. This case is selected to illustrate a successful run, and the entire holdout remains the source for aggregate performance. The renderer maps phase boundaries onto the approved articulated scene with monotone interpolation. It does not use the reduced model’s positional state as observed joint or camera motion.
- `proposal` · Pod packing uses a new fixture and a short, flat-bottomed sample pod, with an authored trajectory. Reusing the cameras demonstrates portability of the recording setup across these two scenes. A joint-limit, reach, contact and tracking check on a calibrated robot is required before treating the output as feasible action labels for hardware.

## What the dataset contains

The two episodes contain 732 timestamps and 2,196 original RGB images across three camera views. The ZIP contains per-episode metadata, RGB WebP files, `observations.jsonl` and `next_pose_targets.jsonl`. Observations store simulator-local link quaternions, world camera transforms in millimetres and viewport visibility probes. The next step targets are authored poses from the subsequent simulator tick. They are not physical encoder readings, force measurements, actuator commands, or calibrated finger placements inferred from RGB. The terminal observation has no subsequent target. The published preview upscales three 640 × 360 camera views onto a 3840 × 2160 SVG canvas.

The audit decodes every image, checks each index and 20 ms timestamp, and tests constancy of both wrist-to-camera transforms and the world-to-overhead camera relationship. It verifies that the object does not drift during release. To quantify the importance of synchronization, the audit pairs the same pose sequence with a deliberate 100 ms shift. The 95th-percentile displacement of the larger-moving wrist is 49.64 mm for tomato and 44.05 mm for packing. This is a geometric sensitivity check in the reconstruction, not a measured failure rate for real sensor skew.

Three of five unobstructed target probes count as a visible frame. For the tomato, all 353 timestamps show the fruit in at least one view. The right wrist covers all 83 placement and release samples; the left covers 12. Camera coverage means the object is visible by this probe rule. Blemish recognition or contact observability require separate validation.

## Offline export check

`observed_current` · Checked November 19, 2026. The SVGs store prerecorded camera frames inside one SVG group per capture timestamp. The browser selects the group by playback time. The archive stores the same observations as individual files, plus authoring metadata. To verify, read the frame groups `defs/g#f0`, `defs/g#f1` and onward, decode each group’s three WebP data URIs in left, right, overhead order, and compare the bytes with each stream’s `rgb_path` in the matching archive observation. Join `next_pose_targets.jsonl` by `observation_index` and compare the target timestamp, pose and closure values with the following sample. The terminal observation has no target.

| Episode | Timestamp groups | Byte-identical camera images | Checked next-pose joins |
| --- | ---: | ---: | ---: |
| Tomato sorting | 353 | 1,059 | 352 |
| Sample-pod packing | 379 | 1,137 | 378 |
| Total | 732 | 2,196 | 730 |

The rendering code samples an articulated scene along fixed trajectories. The reduced tomato experiment independently supplied one phase schedule. The frame comparison validates the offline packaging of its visual reconstruction and the pod example. Physical feasibility, contact accuracy, learning and transfer require their own tests.

## Reproduce the capture

With Python 3, NumPy and Pillow installed, plus Node, Playwright and Chrome:

```sh
python3 research/tomato-sorting/capture_timing.py
capture_dir=$(mktemp -d)
node scripts/render_robot_episodes.cjs "$capture_dir"
python3 scripts/bake_robot_episodes.py "$capture_dir"
node scripts/build_dexterity_study.cjs
```

For transfer, use the same target robot, camera calibration and action convention across tasks. Generate each new task as a single trajectory with the relevant object, fixture and contacts. Render the observation views from that trajectory and pair them with simulator targets. Test camera alignment and replay on a small real-robot calibration set before measuring held-out physical performance.
