LattePanda Mu Ultra YOLO26 Edge-AI Inference Evaluation

Why Use YOLO for Edge-AI Evaluation?

YOLO (You Only Look Once) is a widely used family of real-time computer-vision models for applications such as monitoring, industrial inspection, robotics, and perception systems. Testing several YOLO26 model sizes across CPU, GPU, and NPU backends provides a useful view of the platform’s inference throughput and backend compatibility. It does not, by itself, define complete application performance: camera decoding, image resizing, post-processing, tracking, I/O, power, and model accuracy also matter.

The device under test is the LattePanda Mu Ultra, a compact x86 compute module based on Intel Core Ultra Series 2 processors. Tests were reported for the Core Ultra 7 256V and Core Ultra 5 226V configurations, using YOLO26 n/s/m/l detection models in FP32, FP16, and INT8 export variants on CPU, GPU, and NPU.

Test Environment and Reporting Limits

Important: The latency figures below are model-inference measurements under an unspecified input resolution. They must not be described as 4K end-to-end processing latency or camera-pipeline frame rate.

The driver version is consistent with Intel’s v1.35.0 release, whose verified configuration lists Ubuntu 24.04, OpenVINO 2026.2, and Lunar Lake support. However, the following details are required for a reproducible public benchmark:

• Exact processor SKU is used for every table; BIOS, firmware, kernel, GPU driver, Ultralytics version, and model file hash.
• Input resolution (imgsz), batch size, synchronous or asynchronous mode, number of warm-up runs, and whether preprocessing, post-processing, and data transfer are included.
• Power profile, cooling solution, ambient temperature, and sustained clock behavior.
• INT8 calibration dataset, calibration fraction, quantization command, and post-export accuracy (mAP50 and mAP50–95).
• Variation reported as standard deviation, percentile range, or coefficient of variation—not only 'FPS fluctuation <10'.

Results at a Glance

• The GPU delivered the highest throughput for the YOLO26n, YOLO26s, and YOLO26m models, reaching up to 420.1 FPS with YOLO26n FP16.
• The NPU demonstrated consistent acceleration across the tested model sizes, including successful YOLO26l FP32 and FP16 inference at approximately 64–65 FPS.
• INT8 quantization significantly improved CPU inference performance in several tested configurations, although performance gains varied across models and compute backends.
• In the current test environment, YOLO26l GPU inference was not completed and INT8 conversion was unsuccessful. Further software and configuration validation is required.
• The Core Ultra 7 256V and Core Ultra 5 226V both delivered strong AI inference performance. Actual performance differences varied by model and precision configuration.

Detailed 256V Results

YOLO26n

GPU FP16 delivered the highest performance at 420.1 FPS. In this test, INT8 did not provide additional performance gains on the GPU or NPU, indicating that the optimal precision may vary by compute backend.

YOLO26s

GPU INT8 delivered the highest throughput at 235.2 FPS, while CPU INT8 improved performance from 17.1 FPS to 47.2 FPS. Production deployment should balance these performance gains against application-specific accuracy requirements.

YOLO26m

GPU INT8 delivered the highest throughput at 148.3 FPS, while NPU INT8 achieved 87.7 FPS. The optimal compute backend should be selected based on the application’s performance and deployment requirements.

YOLO26l

The NPU successfully ran the YOLO26l FP32 and FP16 model variants at approximately 64–65 FPS, demonstrating its ability to handle larger vision models. GPU inference and INT8 conversion were not completed in the current test environment and require further software validation.

Choosing a Compute Backend

For maximum throughput, the GPU is the preferred choice for validated models. The NPU provides another acceleration option for sustained edge-AI workloads, while the CPU offers broad compatibility across different deployment scenarios. The optimal compute engine depends on the model, precision, software configuration, and application requirements.

Precision and Quantization

1.Smaller Model Footprint

Quantization significantly reduces the model footprint. For YOLO26n, the exported model size decreases from 9.6 MB in FP32 to 5.3 MB in FP16 and 3.2 MB in INT8, making deployment more storage- and memory-efficient.

2.Higher CPU Performance with INT8

INT8 quantization significantly improves CPU inference performance. YOLO26n increases from 56.0 to 125.4 FPS, while YOLO26m rises from 5.2 to 16.9 FPS. Performance gains on the GPU and NPU vary by model, highlighting the importance of selecting the optimal precision for each compute engine.

3.Balancing Performance and Accuracy

Quantization can improve deployment efficiency while potentially affecting model accuracy. Developers should validate the quantized model with application-specific datasets and select the precision that provides the best balance of speed, model size, and detection accuracy.

Implementation Notes

For optimal NPU performance in the tested software environment, export the OpenVINO model with non-end-to-end processing:

model.export(format="openvino", end2end=False)

When exporting FP32, FP16, and INT8 variants, store each model in a separate directory to prevent files from being overwritten.

Before running inference, verify that OpenVINO can detect the NPU:

python -c "from openvino import Core; print(Core().available_devices)"

The output should include NPU.

GPU execution of YOLO26l and INT8 conversion were not completed in the current test environment. Further software validation and log analysis are required.

Core Ultra 7 256V vs. Core Ultra 5 226V

The measured difference is workload-dependent. The data does not establish a consistent 256V lead, and the missing 226V YOLO26l result must not be interpreted as a failure. Intel specifies 64 GPU TOPS and 47 NPU TOPS for the 256V, versus 53 GPU TOPS and 40 NPU TOPS for the 226V; application performance still depends on software, power, and workload behavior.

Conclusion

• LattePanda Mu Ultra delivers strong YOLO26 inference performance across its CPU, GPU and NPU.
• The GPU achieved the highest throughput for YOLO26n, YOLO26s and YOLO26m, reaching up to 420.1 FPS.
• The NPU successfully accelerated every tested FP32 and FP16 model size, including YOLO26l.
• INT8 quantization significantly improved CPU performance in several test configurations.
• With flexible heterogeneous computing in a compact x86 compute module, Mu Ultra is well suited for edge vision and on-device AI applications.
• Actual performance may vary depending on the model, precision, software configuration and deployment environment.

Sources Used for Verification

• Ultralytics — YOLO26 model documentation
• Ultralytics — OpenVINO integration
• Ultralytics — model export documentation
• Ultralytics — YOLO26 end-to-end detection
• Intel — Linux NPU driver v1.35.0 release
• Intel — Core Ultra 5 226V specifications
• Intel — Core Ultra 7 256V product family specifications