I'm pretty ok with the way I have things now. It probably isn't worth buying a new video card just for playing back these files. Could the bulk of the work be offloaded to a faster GPU? I don't know, but I think Dan is right. I did have that option turned on in VLC, but my cheap old GPU probably can't do much of anything. It plays only 13 fps from an SSD, so it seems the CPU is the most likely bottleneck. It plays the Photo JPEG clips at 11 fps, with disk usage at a bit under 50%, and CPU around 22% (which may be the equivalent of 88%, as it's probably not multi-threaded, either). I found a new (to me) video player that I like better than VLC, so I've made that my default player.
Too bad, as H.264 is literally 1/10 the size, and looks and plays beautifully.