Home Assistant is one of the most popular smart home platforms in the world with a vast ecosystem of integrations to monitor and automate the devices around our homes. The Viam community integration makes it possible to collect images and sensor data from those existing connected devices, train custom machine learning models, and run them locally as part of an automation workflow.

In this codelab, you'll learn how to install the integration through the Home Assistant Community Store (HACS) and use a vision service to detect people from a camera connected to Home Assistant.

Home Assistant notification of a person detected

What You'll Build

Prerequisites

What You'll Need

What You'll Learn

Watch the Video

See a demonstration and overview of the integration in this video.

Create your machine

  1. In the Viam app under the LOCATIONS tab, create a machine by typing in a name and clicking Add machine. add machine
  2. Click View setup instructions. setup instructions
  3. To install viam-server on the Raspberry Pi device that you want to use to communicate with and control your webcam, select the Linux / Aarch64 platform for the Raspberry Pi, and leave your installation method as viam-agent. select platform
  4. Use the viam-agent to download and install viam-server on your Raspberry Pi. Follow the instructions to run the command provided in the setup instructions from the SSH prompt of your Raspberry Pi. installation agent
  5. The setup page will indicate when the machine is successfully connected. successful toast

Add an ML model service

  1. In the Viam app, click the + icon in the left-hand menu and select Service, and then ML model. select ML model
  2. Search for a module called TFLite CPU. Then click Add module, and Create a new ML Model service called mlmodel-1. This provides the ability to run Tensorflow Lite, a.k.a LiteRT, models on the machine. add module
  3. Notice this creates two new items in the left sidebar. The first is your new ML Model service called mlmodel-1, and the second is the tflite_cpu module from the Registry.
  4. In the mlmodel-1 panel, click "Select model" in the Model section, search for "COCO" under the Registry tab, and select the "EfficientDet-COCO" model from the list. This is a general object detection model that is very efficient, as the name implies, for resource-constrained devices like a Raspberry Pi. add model
  5. Click Save in the top right to save and apply your configuration changes.

Add a vision service

  1. In the Viam app, click the + icon in the left-hand menu and select Service, and then vision. select vision
  2. Search for a module called mlmodel. Then click Add module, and Create a new Vision service called vision-1. add module
  3. Notice adding this service adds the vision service called vision-1. From the ML Model section of the panel, select mlmodel-1. select ML model for vision service
  4. Save your changes in the top right and wait a few moments for the configuration changes to take effect.

As a community integration, Viam is available through the Home Assistant Community Store (HACS). After getting started with HACS, you can add the Viam integration from the store.

  1. From the HACS dashboard in Home Assistant, search for Viam at the top, and select the top result. search Viam
  2. The integration documentation will be displayed. Click on the blue "Download" button in the bottom right to install the code onto your Home Assistant device. Viam integration docs
  3. Once the installation is complete, go to the "Settings" page and select "Devices & services" to view the integrations dashboard. Home Assistant integrations dashboard
  4. Click on "Add Integration", search for Viam, and select the top result. search for Viam integration
  5. In the configuration form, enter the API key ID and API key for the machine you set up earlier. You can find those values under the "Connect" tab in the Viam app, in the API keys section. machine API keysenter API keys for Viam machine
  6. Click "Submit" to create the connection between Home Assistant and the Viam app. Click "Finish" when the success message appears to confirm the setup is complete. Integration success message
  7. The Viam integration should now appear on your Integrations Dashboard. Viam integration in dashboard

The Viam integration provides a few services that can be used in Home Assistant automations. For this codelab, you'll use the object detection service to intelligently notify when people have been detected by existing cameras connected to your home.

  1. In the Home Assistant app, go to the "Settings" page and select "Automations & scenes" to view the Automations Dashboard. automations dashboard
  2. Click "Create Automation" and select "Create new automation" from the options in the modal. Create automation menu
  3. The "New automation" page will include sections for "When", "And if", "Then do" to configure for the workflow. new automation page
  4. Click "+ Add Trigger" to open the selection modal, then search for "Time pattern" and select the top result. time pattern trigger
  5. Enter "/30" in the "Seconds" field to trigger the automation every 30 seconds. You can configure different patterns if you wish to check more or less frequently. time pattern settings
  6. Click "+ Add Action" to open the selection modal, then search for "Detect object" and select the top result. detect objects actions
  7. In the "Viam machine" field, select the configured machine from earlier. Enter the name of the vision service, "vision-1", in the "Detector Name" field. Check the "Camera Entity" box and select the connected camera you would like to use for this automation. Then set the "Response variable" to "people_detections". detect objects configuration
  8. Click "+ Add Action" to open the selection modal, then search for "Condition" and select the top result. condition action
  9. Select "Template" for the "Condition type" and enter the following into the "Value template" field:
    { { people_detections.detections|count > 0 } }
    condition config
  10. Click "+ Add Action" to open the selection modal, then search for "Persist" and select the top result. notification action
  11. Click the action menu (three vertical dots) on the new "Persistent Notifications: ‘Create'" card, and select "Edit in YAML" to display the YAML editor for creating templated notifications. Replace the data: {} line with the following code:
    data:
  title: People Alert
  message: >
    Detected {{ people_detections.detections|join(", ", "name")|default("nothing",
    true) }}: ![captured image]({{ people_detections.img_src }})
    This will create a notification with a list of detected people and the associated image. notification config
  12. Click the "Save" button in the bottom-right corner to enter a name for the automation and complete the setup.

Congratulations, you now have automated people detection for your smart home! Building on this experience you can do the following to make it even smarter:

What You Learned

Additional Viam resources