> which is present on the drone and use direct IMU input and ESC commands from the Jetson
Does the jetson board even have the appropriate UARTs on it to talk directly to the ESCs? Your typical hobby grade 5" class size ESC (either 4-in-1 or discrete) cannot talk to two different controllers at the same time. If it's already wired to the four UART outputs from a 30x30 size flight controller (such as something STM32H7 based running betaflight), the ESC cannot be in communication with any other device.
Right, that was my point about the physical layout not really being conducive to NN-driven stabilization, on my cursory read.
Digging in, you could probably bitbang DShot on Orin GPIOs. It would be really sketchy since it's presumably running Linux on out-of-order cores with giant cache, so there's nothing close to realtime going on, but it's so damn fast that the theoretical problem probably doesn't matter and I bet it would work fine.
Orin also has "safety island" for ASIL applications which has lockstep Cortex-R52s. This would be ideal to run an integrated flight controller RTOS but it's supposedly fused off on consumer boards, lame. There's also a Cortex-R5 called "SPE" that could probably run a flight control RTOS as well.
Regardless of all of this theory craft, I was either totally wrong or accidentally right in two ways, depending on how you want to look at it: according to a sibling post, the STM32 board actually runs the NN-backed stabilization loop rather than a PID one! I've seen this done in research before but never in a meaningfully successful way, so I'm even more impressed now - as far as I can tell, it's _two_ novel solves, an actually working PID-or-better NN-based stabilization system on STM32 _and_ inside-out vision based flight planning for racing on the Nvidia board.