Déploiement de l'IA en périphérie pour les équipes embarquées et IoT

Ce qu'elle couvre

Ce programme de niveau praticien donne aux ingénieurs embarqués et IoT toutes les compétences nécessaires pour mettre en production des inférences IA sur du matériel contraint. Les participants travaillent en pratique avec ONNX, TensorFlow Lite, Core ML et des runtimes LLM edge comme Llama.cpp et llamafile, en couvrant la quantisation, l'élagage et les optimisations spécifiques au matériel. Les sessions abordent les contraintes réelles : budget batterie, limitation thermique, mémoire restreinte et mises à jour de modèles en OTA. Le format combine des modules conceptuels courts avec des exercices de laboratoire sur des appareils physiques ou émulés.

À l'issue, vous saurez

• Convert a trained PyTorch or TensorFlow model to ONNX, TFLite, and Core ML formats and validate parity across runtimes
• Apply INT8 post-training quantisation and measure accuracy-latency trade-offs on a target device
• Run a quantised LLM (Llama.cpp or llamafile) on an edge device and profile tokens-per-second against thermal and battery budgets
• Design and implement a power-aware inference pipeline that respects duty-cycle constraints on battery-powered hardware
• Build and execute an OTA model update workflow with rollback safety on a representative IoT device

Sujets abordés

• Model conversion and interoperability: ONNX, TensorFlow Lite, Core ML
• Quantisation (INT8, FP16) and structured pruning for edge targets
• Edge LLM runtimes: Llama.cpp, llamafile, MLC LLM
• Hardware accelerators: NPUs, DSPs, GPU microcontrollers (ARM Ethos, Apple Neural Engine)
• Battery budget analysis and power-aware inference scheduling
• Thermal management and throttling strategies
• OTA model updates and versioning on constrained devices
• Benchmarking latency, throughput, and memory footprint on real hardware

Modalité

Delivered as a 3-5 day intensive bootcamp, on-site or remote with hardware kits shipped to participants in advance. Approximately 60% hands-on lab time, 40% guided instruction. Participants receive a reference board (e.g. Raspberry Pi 5 or STM32 dev kit) or use their own target platform. Labs use Docker-based toolchains to minimise setup friction. Remote delivery uses shared cloud-hosted hardware via SSH where physical shipping is not feasible.

Ce qui fait que ça marche

• Start with a hardware-in-the-loop benchmark early in the project to set realistic constraints before model selection
• Adopt a model-card discipline that records accuracy, latency, power draw, and thermal behaviour for every candidate model
• Involve firmware and ML engineers in joint design reviews so power budgets are agreed before training begins
• Use automated regression tests that run the inference pipeline on the target device in CI/CD, catching regressions before release

Erreurs fréquentes

• Attempting to deploy full-precision FP32 models without quantisation, then discovering the device lacks the memory and compute budget at integration time
• Ignoring thermal throttling during sustained inference, leading to unpredictable latency spikes in production
• Treating model accuracy on desktop benchmarks as a proxy for on-device accuracy without re-validating after quantisation
• Skipping OTA update planning until late in the product lifecycle, resulting in fragile manual reflashing processes

Fournisseurs à considérer

• Edge Impulse (training & certification)www.edgeimpulse.com/experts →
• Coursera – AI on the Edge Specialization (Duke University)www.coursera.org/specializations/ai-on-the-edge →
• Arm Education – ML on Arm learning pathslearn.arm.com/learning-paths/cross-platform/ml-on-arm →
• DeepLearning.AI – AI on the Edge courseswww.deeplearning.ai/short-courses/ →

Sources

• TensorFlow Lite documentation – On-device ML →
• Llama.cpp – Local inference for large language models →