Scene setup

A location for data collection needs only the handheld grippers and the anchor tags. The setup takes approximately 30 seconds.

How the tags workCopy Anchor Link

The camera on each gripper has two lenses:

• The observation lens looks at the fingers and the workspace. The policy sees only this image.
• The auxiliary lens looks back at the operator, the room and the anchor tags. The processing uses this image to find the position of the gripper.

The anchor tags are large AprilTags that do not move. They set the world frame and the metric scale of the data. The tags are behind the gripper, thus they are not in the observation image. A policy cannot learn to depend on them.

Each gripper also has three small tags: one on the base and one on each finger. These tags give the angles of the fingers. Before you record, make sure that the gripper has its tags (step 9 of the handheld gripper).

Print the anchor tagsCopy Anchor Link

Anchor tags (PDF)

1. Print each page at 100 % (Actual size). Do not use Fit to page.
2. Check the size of each tag (see below).
3. Attach each tag to a flat, rigid board. The tag must stay flat.

The PDF has two tags of the tagStandard41h12 family, with IDs 0 and 1. Each tag is 180 mm wide. Its inner square is the tag size that the processing uses: 100 mm.

Check the tag sizeCopy Anchor Link

The anchor tags set the metric scale of the data. If the printer changes the size of a tag, all the distances in the data change by the same amount.

1. Measure the inner square of a printed tag. It must be 100.0 mm.
2. Measure the whole tag. It must be 180 mm.
3. If a value is different, print the tag again at 100 % (Actual size). Do not use Fit to page.

You can use tags with other IDs. Each anchor tag must have a different ID. Do not use the IDs of the gripper tags, 2000 to 2005. The processing must know the IDs of the anchor tags. The first ID sets the world frame.

Place the anchor tagsCopy Anchor Link

1. Put each tag on a stand.
2. Put the stands behind the operator, one at each side, as in the drawing. The body of the operator must not hide the tags from the auxiliary lens.
3. Stand the tags upright, with the printed side toward the workspace. Do not lay a tag flat. At the edge of the view of the lens, the processing cannot find a flat tag.
4. Do not move the tags during a session.

One anchor tag is the minimum. Two tags are the standard setup. Each additional tag gives the processing more data.

More grippers can use the same location. Each gripper that sees an anchor tag, or the tags of a different gripper, is in the same world frame.

LightCopy Anchor Link

Keep the gripper tags out of hard shadows. If the edge of a shadow goes across a tag, the processing can fail to find the tag. Outdoors in sunlight, look for shadow edges before you record.

RecordCopy Anchor Link

1. Charge the cameras. One charge records for approximately 80 minutes.
2. Start the recording on the camera of each gripper.
3. Immediately, hold the grippers still for approximately 1 second, near the anchor tags.
4. Do the task.
5. Tap the grippers on the table several times. You can also tap the grippers against each other.
6. Immediately, stop the recordings.

The processing uses the hold to find the start of the task, and the taps to find the end. It uses the sound of the recordings to align the recordings of the two grippers.

More informationCopy Anchor Link

The paper Specialist: Yet Another Universal Manipulator Interface gives the details of the setup and the processing.