🛠 Optimize local AI with OpenVINO and Intel Arc GPU

Objectifs

This tutorial aims to configure the development environment to leverage the combined computing power of the CPU and Intel Arc GPU via the OpenVINO toolkit. You will learn to convert a pre-trained neural network model for efficient execution on the xe graphics driver under Linux.

Matériel nécessaire

• LattePanda Mu Ultra (Core Ultra 7 256V tested)
• Mini Carrier Board
• Compliant USB-C power supply
• Installed NVMe SSD (min 128 GB)
• Stable Internet connection

Étapes

1. Install system dependencies required for the compiler and graphics libraries under Ubuntu 24.04 or 26.04.

sudo apt update && sudo apt install build-essential cmake git python3-pip libgl1-mesa-glx

1. Create a Python virtual environment to isolate OpenVINO dependencies.

python3 -m venv openvino_env
source openvino_env/bin/activate

1. Install the openvino-dev package which includes conversion tools and runtimes for CPU, GPU, and NPU.

pip install openvino-dev[tf,onnx,pytorch]

1. Verify that the Intel Arc GPU is correctly detected by OpenVINO. The xe driver must be active.

python3 -c "import openvino as ov; core = ov.Core(); print(core.available_devices)"

1. Download a simple neural network model (e.g., ResNet50) and convert it for Intel architecture.

ow convert_model --output_dir ./models --input_shape [1,3,224,224] --input_model resnet50.onnx

1. Run a test inference specifically targeting the GPU device to take advantage of the Lunar Lake architecture.

python3 -c "import openvino as ov; core = ov.Core(); model = core.read_model('./models/resnet50.xml'); compiled = core.compile_model(model, 'GPU'); print('GPU inference successful')"

Vérification

The inference command must return a success message without driver errors. Ensure GPU usage increases via intel_gpu_top during execution to confirm hardware acceleration.

Pour aller plus loin

Explore using the integrated NPU for heavy loads while keeping the GPU for display, or test INT8 quantization to reduce latency and energy consumption.