The Versal AI edge SOMs are mildly overpriced. The boards are worth it, but in the embedded space Nvidia is offering the cheapest solutions so an FPGA based application will always need to justify the additional cost for slightly worse performance, by arguing that the application has latency requirements that a GPU cannot help with.

GPUs tend to perform worse when you have small batches and frequent kernel launches. This is especially annoying in cases where a simple kernel wide synchronization barrier could solve your problems, but CUDA expects you to not synchronize like that within the kernel, you're supposed to launch a sequence of kernels one after the other. That's not a good solution if a for loop over n iterations turns into n kernel calls.

It's not just that the boards are expensive; you'll also need a Vivado license to create any designs for it. That license is at least several thousand dollars for the Versal devices.

Need something like this $160 Zuboard entry to 5-figure Zynq market, https://news.avnet.com/press-releases/press-release-details/...

> the smallest, lowest power, and most cost-optimized member of the Zynq UltraScale+ family.. jump-start.. MPSoC-based end systems like miniaturized, compute-intensive edge applications in industrial and healthcare IoT systems, embedded vision cameras, AV-over-IP 4K and 8K-ready streaming, hand-held test equipment, consumer, medical applications and more.. board is ideal for design engineers, software engineers, system architects, hobbyists, makers and even students

F = Field P = Programmable G = Gate <---- important A = Array

You aren't "programming", you're "wiring gates together". In other words, you can build custom hardware to solve a problem without using a generic CPU (or GPU) to do it. FPGAs are implemented as a fabric of LUTs (Look-up Tables) which take 4- or 6- (or more) inputs and produce an output. That allows Boolean algebra functions to be processed. The tools you use (Vivado / ISE / YoSys / etc.) take a your intended design, written in a HDL (Hardware Design Language) such as Verilog or VHDL, and turn it into a configuration file which is injected into the FPGA, causing it to be configured to into the hardware you want (if you've done it right). FPGAs are a stepping stone between generic hardware such as a CPU or GPU and a custom ASIC. They win when you can express the problem in specialized hardware much better than writing code to do something on a CPU/GPU. Parallelization is the key to many FPGA designs. Also, you don't have to spend >$1MM on a mask set to go have an ASIC fabricated by TSMC, etc.

They are useful for AI, but it's a completely different beast than a GPU.

FYI, amd has some prototype alternative programming models for these NPU engines now, although they are certainly very immature: https://github.com/Xilinx/mlir-aie/tree/main/programming_gui...

the cores/arches themselves are referred to by a bagillion different names AIE1 AIE2 AIEML Phoenix Strix blah blah (and *DNA refers to the driver/runtime not the core/arch itself) but NPU exclusively refers to consumer edge SoC products.