🛠 Accélérer l'inférence IA avec un HAT NPU (Hailo)

Objectives

Port a neural network inference to an NPU accelerator connected to the Raspberry Pi 5. Note: the BCM2712 SoC does not contain an embedded NPU; hardware acceleration is provided by an AI HAT+ (Hailo-8L 13 TOPS or Hailo-8 26 TOPS) or an AI HAT+ 2 (Hailo-10H, 40 TOPS + 8 GB dedicated RAM), connected via PCIe through the FPC/HAT+ connector.

Hardware

• Raspberry Pi 5 (8 GB recommended) running 64-bit Raspberry Pi OS + 27 W power supply
• AI HAT+ or AI HAT+ 2 (Hailo) with heatsink/active cooling
• Active Cooler is essential: the HAT and SoC heat up during continuous inference
• Official camera (libcamera) for a complete detection pipeline

Complete code

# 1) Installer la pile Hailo (dépôt officiel Raspberry Pi)
sudo apt update && sudo apt install -y hailo-all
sudo hailoall status          # vérifie driver + firmware
ls /dev/hailo0                # présent => NPU visible

# 2) Détecteur YOLO prêt à l'emploi fourni avec la pile
hailo_detection_yolov8.py --input camera --show-fps

# 3) Suivi thermique pendant la charge (bridage à 80-85 °C)
watch -n1 vcgencmd measure_temp

#!/usr/bin/env python3
# infer_hailo.py — inférence minimale via HailoRT (API Python)
import hailort

# Charger un réseau compilé .hef (obtenu avec TAPPAs/compile_hailo)
dev = hailort.Device.create_hailo
dev.load_hef("yolov8s.hef")
print("NPU détecté, réseau chargé :", dev.get_arch)
# ... puis création d'inférence, envoi des trames et lecture des sorties

Expected verification

hailoall status returns driver and firmware OK, and /dev/hailo0 exists. The detection script displays bounding boxes on the camera stream with a FPS rate significantly higher than CPU inference. Temperature remains below 80 °C thanks to cooling.

Common errors

• Believing in an integrated NPU: the BCM2712 does not have one; without a Hailo HAT+, all inference runs on the four Cortex-A76 cores (or on the GPU, without a dedicated accelerator).
• Forgetting to enable the PCIe/HAT+ port: the HAT does not appear; configure the firmware for HAT+ mode and then reboot.
• Overheating: without an Active Cooler, the SoC enters the 80-85 °C throttling range and FPS drops.
• Firmware/driver version incompatibility after apt upgrade: rerun hailoall status and reinstall hailo-all.
• Network not compiled for the chip: a .hef compiled for Hailo-8 does not necessarily work on Hailo-8L; recompile with the correct TAPPAs compiler.
• Weak power supply: the HAT adds a power consumption spike; stay at 5 V/5 A.