Jetson API Field Guide · Source archive

Local AI: Vision Pipeline

Evolving from a concentrated vision analysis to full on pipeline

Originally published 2026-07-20 on Advisory Hour.

Originally published as a Advisory Hour Substack post on 2026-07-20. This local copy preserves the text and public supporting media alongside the research record.

Original post

Local AI: Vision Pipeline image from the original Advisory Hour post.
Local AI: Vision Pipeline image from the original Advisory Hour post.

I’ve written quite a few times on how I’m using local vision for different projects. I find it extends token usage considerably, and at the same time I have a sense that the outputs off the coding harnesses are better. I believe the reason for this is the ruthless interrogation of the image. I’m not just getting a prompt, I’m doing a full on briefing on what the image contains.

An agent (or me, by using my web UI) gets back a very detailed briefing of the image. This is a complex image I’m doing an analysis on. What is it? In words?

Local AI: Vision Pipeline image from the original Advisory Hour post.
Local AI: Vision Pipeline image from the original Advisory Hour post.

I’m using several different vision systems not so much a factory orchestration, but in a concert. During the Summer into AI competition I bumped into OpenCV5 and also added on SAM2, a segmentation software that I’ll show examples of here in a moment. OpenCV5 is great, it’s like this great toolbox of vision capabilities. Why not wrap them in an API and then just tie it all into a pipeline that the vision system could work with? So, that’s what I did.

Local AI: Vision Pipeline image from the original Advisory Hour post.
Local AI: Vision Pipeline image from the original Advisory Hour post.

The important caveat here is that I’ve not yet put the full extent of the pipeline onto a new project. In my own interpretation of how things are going, I can see quite a few things that need adjusted. For example, it’s leaning a little too hard in the direction of citation and evidence flows to justify the pipeline and not quite enough on the prompting side.

The way I’m tackling this is with a control panel, which looks like the following image. This panel also gives some sense of what this suite of tools is capable of. It’s funny to me in a sad way: I can recall very expensive imaging solutions that did 0.001% of what this system is doing on my Surface Pro-and it’s largely all written by Codex.

Local AI: Vision Pipeline image from the original Advisory Hour post.
Local AI: Vision Pipeline image from the original Advisory Hour post.

The other vision API capabilities remain present. I bolted on a full vision pipeline because I just wanted to put the entire set of capabilities to work in one long set of exchanges. It’s incredible to me that one can prompt out a software factory that handles complex vision analysis and handoffs. It might be worthwhile to feed this article to your AI tool of choice and collect some ideas on how that might work.

Here’s another article on local vision. It might get you thinking about other possible use cases.

EricLocal vision tools: Eyeballing early build of Independence TownI’ve written before on the use of Local AI for vision tasks and why this is such a great unlock for your agentic loops. It can be explained simply with a casual algorithm example…Read more20 days ago · Eric

Subscribe now

Connected work