ffmpeg is awesome. However, there are some issues that all ffmpeg-based players share that I wish someone would tackle (either by fixing ffmpeg or making something new). Seeking performance and reliability (especially single frame seeking) is a big one. Robustness in the face of truncated or otherwise corrupted video files is another. Ffmpeg isn't terrible in these areas, and I realize that there are underlying issues that make these things very hard, but still I believe it is possible to do much better.
Yep. It's a lot of work creating a non-stuttering, fast-seeking player using ffmpeg/libav. For high resolution / high bitrate streams ffmpeg doesn't issue frames smoothely in itself (I-frames takes longer) so I had to create a whole caching structure and issue the frames from a different thread to get it to play smoothely in my application...
On the other hand there are so many edge cases around codecs and assumptions around what is and is not coming so I think that if a different API was created there would likely be a lot of cases where it would break? Something like the NDI API would be great, it is really quick to integrate.