Use an AprilTag scanner

Why would you want to detect AprilTags? AprilTags, similar to QR codes, can encode a variety of information and are widely used in robotics and computer vision applications. Unlike QR codes, AprilTags are designed for scenarios where precise localization, identification, and orientation are critical. They are commonly used for tasks such as guiding autonomous vehicles, tracking objects, and enhancing augmented reality systems.

In this codelab, you'll learn how to detect and decode AprilTags using a Viam module. We'll leverage the apriltag and OpenCV Python libraries to process images from a camera and extract information encoded in AprilTags. By the end, you'll have a working solution that can identify AprilTags in real-time, enabling customizable actions for a variety of applications, from navigation to interactive robotics.

apriltag demo

What You'll Build

A camera that can detect and decode AprilTags

Prerequisites

A computer with MacOS, Windows, or Linux to flash your Raspberry Pi and configure the device's components using the Viam app
Hardware and supplies requirements
- 1 - Raspberry Pi 4
- 1 - microSD card to use with your Pi
- 1 - power supply for your Pi
- 1 - USB web camera

What You'll Need

All the hardware components listed in prerequisites.
Sign up for a free Viam account, and then sign in to the Viam app

What You'll Learn

How to use a webcam to scan and detect AprilTags
How to configure and test a device's components using Viam
How to use a vision service in Viam
How to use modules from the Viam registry

Watch the Video

See a demonstration and overview of the AprilTags scanner in this video.

The Raspberry Pi boots from a USB flash drive (or microSD card). You need to install Raspberry Pi OS on a USB flash drive that you will use with your Pi. For more details about alternative methods of setting up your Raspberry Pi, refer to the Viam docs.

Install Raspberry Pi OS

Connect the USB flash drive (or microSD card) to your computer.
Download the Raspberry Pi Imager and launch it.
Click CHOOSE DEVICE. Select your model of Pi, which is Raspberry Pi 4.
Click CHOOSE OS. Select Raspberry Pi OS (64-bit) from the menu.
Click CHOOSE STORAGE. From the list of devices, select the USB flash drive you intend to use in your Raspberry Pi.
Configure your Raspberry Pi for remote access. Click Next. When prompted to apply OS customization settings, select EDIT SETTINGS.
Check Set hostname and enter the name you would like to access the Pi by in that field, for example, test.
Select the checkbox for Set username and password and set a username (for example, your first name) that you will use to log into the Pi. If you skip this step, the default username will be pi (not recommended for security reasons). And specify a password.
Connect your Pi to Wi-Fi so that you can run viam-server wirelessly. Check Configure wireless LAN and enter your wireless network credentials. SSID (short for Service Set Identifier) is your Wi-Fi network name, and password is the network password. Change the section Wireless LAN country to where your router is currently being operated.
Select the SERVICES tab, check Enable SSH, and select Use password authentication.
Be sure that you remember the hostname and username you set, as you will need this when you SSH into your Pi.
Save your updates, and confirm YES to apply OS customization settings. Confirm YES to erase data on the USB flash drive. You may also be prompted by your operating system to enter an administrator password. After granting permissions to the Imager, it will begin writing and then verifying the Linux installation to the USB flash drive.
Remove the USB flash drive from your computer when the installation is complete.

Connect with SSH

Place the USB flash drive into your Raspberry Pi and boot the Pi by plugging it in to an outlet. A red LED will turn on to indicate that the Pi is connected to power.
Make sure you are using a 5V 3A USB-C power supply to power your Raspberry Pi 4. Using a power supply with inadequate amperage can lead to instability, throttling, or unexpected behavior. Additionally, while USB boot is enabled by default on newer Raspberry Pi 4 models, older versions may require a firmware update to enable it. Refer to the Raspberry Pi documentation for detailed setup instructions and compatibility.
Once the Pi is started, connect to it with SSH. From a command line terminal window, enter the following command. The text in <> should be replaced (including the < and > symbols themselves) with the user and hostname you configured when you set up your Pi.
```
ssh <USERNAME>@<HOSTNAME>.local
```
If you are prompted "Are you sure you want to continue connecting?", type "yes" and hit enter. Then, enter the password for your username. You should be greeted by a login message and a command prompt.
Update your Raspberry Pi to ensure all the latest packages are installed
```
sudo apt update
sudo apt upgrade
```

Configure your machine

In the Viam app under the LOCATIONS tab, create a machine by typing in a name and clicking Add machine.
Click View setup instructions.
To install viam-server on the Raspberry Pi device that you want to use to communicate with and control your webcam, select the Linux / Aarch64 platform for the Raspberry Pi, and leave your installation method as viam-agent.
Use the viam-agent to download and install viam-server on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi.
The setup page will indicate when the machine is successfully connected.

Add your USB webcam

Connect the USB webcam to the Raspberry Pi.
In the Viam app, find the CONFIGURE tab. It's time to configure your hardware.
Click the + icon in the left-hand menu and select Component.
Select camera, and find the webcam module. This adds the module for working with a USB webcam. Leave the default name camera-1 for now.
Notice adding this component adds the webcam hardware component called camera-1. The collapsible panel on the right corresponds to the part listed in the left sidebar. From the Attributes section of the panel, select a video_path.
Click Save in the top right to save and apply your configuration changes.
At the bottom of the camera-1 panel, expand the TEST section to ensure you have configured the camera properly.
If any problems occur, check under the LOGS tab to see what might be going wrong. Refer to the troubleshooting guide if needed.

Now that your hardware is working the way you want it, it's time to add a vision service to detect and decode an AprilTag.

In the Viam app, click the + icon in the left-hand menu and select Service, and then vision.
Search for a module called apriltag. Then click Add module, and Create a new vision service called vision-1.
Notice this creates two new items in the left sidebar. The first is your new vision service called vision-1, and the second is your new apriltag module.
In the vision-1 panel under the Depends on section, check the camera-1 resource. This configures the vision service to depend on data coming in from the webcam.
Save your changes in the top right and wait a few moments for the configuration changes to take effect.
At the bottom of the vision-1 panel, expand the TEST section to ensure you have configured the vision service properly. Point the webcam towards an AprilTag to see if the camera detects it.
If the camera detects an AprilTag, a bounding box will highlight the item in the video feed, decode the data, and display it on the right under Labels. In the example shown here, the data decoded was unique ID 234.
Troubleshooting:
- Check under the LOGS tab of the Viam app to see what might be going wrong.
- Make sure you are testing with a valid AprilTag. You can generate AprilTags here. Make sure to select tag family tag36h11 since the module we are using defaults to that tag family.

What you learned

How to use a webcam to scan and detect AprilTags
How to configure and test a device's components using Viam
How to use a vision service in Viam
How to use modules from the Viam registry

Building advanced features with Viam and AprilTags

At this point, you have configured and tested your machine and webcam to detect and decode AprilTags, but nothing else is happening automatically, you can create automatic processes that trigger actions when AprilTags are detected, similar to the steps in this QR code detection codelab.

Now that you have a camera that detects and decodes AprilTags, you can:

train and deploy a custom vision model to detect and classify other objects alongisde AprilTags
extend the apriltag module in the Viam registry to use different tag families, add custom functionality such as pose detection, or trigger specific actions based on detected tags
add more components and services to enhance the functionality of your machine, such as integrating advanced robotics controls or feedback systems

Real-world applications and projects for AprilTags

AprilTags are used in the physical world for tasks that require precise localization, identification, and orientation. They are used across various industries and projects, including:

Navigation and Localization: AprilTags serve as waypoints for autonomous navigation, enabling roving robots to traverse offices, homes, or outdoor spaces with high accuracy.
Access Control: Secure access systems can link specific AprilTags to trigger actions like unlocking doors, disabling alarms, or raising barriers.
Interactive Robotics: Robots can use AprilTags to identify objects or trigger specific actions, such as picking up or delivering items to designated areas.
Augmented Reality: Physical objects tagged with AprilTags can be recognized and interacted with in augmented reality applications.
Logistics and Warehousing: AprilTags track and identify packages or inventory in warehouses and distribution centers, streamlining operations.
Manufacturing: They guide assembly line operations, ensuring precision in automated workflows.

Notable real-world use cases

New York City Subway: AprilTags are being used to improve accessibility for visually impaired and non-English speakers by aiding navigation.
FIRST Tech Challenge: AprilTags provide precise location and orientation for robots during competitions, enabling autonomous navigation and task execution.

NYC subway Apriltags over stairwell NYC subway Apriltags by trains