I have watched 3kliksphilip and LinusTechTips demo video upscaling AI stuff, and it always looks really really weird and gross. It seems really unnatural, and has the same kind of "nope that's wrong" as if you maxed the saturation on the photo. It feels more like a highschooler touching everything up in photoshop than correctly understanding what the objects in the video should look like.
Also, in their recent demo of a video upscaling technology, Nvidia clearly kneecapped the pre-upscale footage to make it seem like the upscaler was doing more than it actually was. If you take the source video they were using and convert it using settings they claimed they were using for a low bitrate version, you get a much clearer and cleaner video than they showed.
> Nvidia clearly kneecapped the pre-upscale footage
I'm not so sure. High bitrate content like game streaming does end up losing visual quality with lossy encoding. The average 1080p Twitch stream struggles to look as clean as Nvidia's 1080p source footage.
Given the circumstances, I think this is a great option. Most videos don't look right when encoded on YouTube or blown up at 2x resolution. A destructive AI pipeline is your best shot at reducing the destructive artifacting introduced at the encoding stage.
This news is still something to celebrate as it shows the way forward.
The information is in the video files and our brain can't just use it but math can.
I can imagine using/buying a upscale model optimized for an old tv show for example. Or using a model for my prev game streams.
There is temporal information and high res pictures from sets, styles or actors at the time of the recordings. All information which can be used.
If its not yet good enough, it will.
Lets see if this forces a global standard or opensource model plug and play or whatever.
And if you look at the other nvidia research like were the model learns the face and than interpolates all further movements also a really good use case.
Also, in their recent demo of a video upscaling technology, Nvidia clearly kneecapped the pre-upscale footage to make it seem like the upscaler was doing more than it actually was. If you take the source video they were using and convert it using settings they claimed they were using for a low bitrate version, you get a much clearer and cleaner video than they showed.