Edge Computer Vision on Jetson with ONNX and TensorRT, Managed by AWS IoT

Key Takeaways

• Edge CV is an engineering problem, not just a model export.
• ONNX plus TensorRT is a common path to production performance.
• AWS IoT provides identity, telemetry, and update control for fleets.
• Hybrid patterns reduce bandwidth by uploading evidence only when needed.

Edge deployment is often the right choice when bandwidth is limited, latency must be very low, or data cannot leave a site. This post outlines a practical approach for deploying computer vision models on NVIDIA Jetson devices, packaging them with ONNX and TensorRT, and managing fleets via AWS IoT.

When edge makes sense

• Low latency is required, for example under 200 ms
• Privacy constraints limit cloud upload
• Network reliability is variable
• You need local actuation, for example alarms or PLC triggers

Model packaging workflow

1. Train and validate model
2. Export to ONNX
3. Convert and optimize with TensorRT
4. Benchmark on target hardware

References:

• ONNX: https://onnx.ai/
• NVIDIA TensorRT: https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html
• NVIDIA Jetson platform: https://developer.nvidia.com/embedded/jetson-modules

Device management with AWS IoT

AWS IoT can manage:

• Device identity and certificates
• Config delivery
• Telemetry and health metrics
• Over the air updates

References:

• AWS IoT Core: https://docs.aws.amazon.com/iot/latest/developerguide/what-is-aws-iot.html
• AWS IoT Greengrass: https://docs.aws.amazon.com/greengrass/v2/developerguide/what-is-gg.html

Inference architecture on device

Recommended components:

• A local inference service, containerized
• A watchdog and auto restart
• A ring buffer for recent frames
• Local event storage, synced when online

Hybrid architecture: edge plus cloud

A common pattern:

• Edge detects events and stores evidence locally
• Cloud receives event summaries
• Full clips upload only on triggered events

This reduces bandwidth and cost.

Storing metadata in MongoDB

MongoDB is useful for:

• Fleet inventory
• Model versions per device
• Event history and maintenance logs

References:

• MongoDB time series collections: https://www.mongodb.com/docs/manual/core/timeseries-collections/

zion computer vision services: https://lidvizion.ai/

FAQs

Q: Can we run PyTorch directly on Jetson? Yes, but TensorRT often improves latency and throughput for production.

Q: How do we update models safely? Use staged rollouts, track model version per device, and keep the previous version available for rollback.

Q: How do we monitor accuracy drift at the edge? Store small samples and event statistics, then periodically review and label a subset to measure performance.

Internal reference:

• Lid Vi

Q: What should we link to internally? A: Link to relevant solution pages like Computer Vision or Document Intelligence, and only link to published blog URLs on the main domain. Avoid staging links.