For anyone looking to dive deeper and actually understand how FFmpeg and libav work under the hood, I highly recommend Leandro Moreira's tutorial [0]. For me, it's hands down the best and most comprehensive explanation out there.
[0] https://github.com/leandromoreira/ffmpeg-libav-tutorial