FAST: Free Adaptive Super-Resolution via Transfer for Compressed Videos

Applications

With its ability to be combined with existing algorithms, this invention has wide application for imaging or video industries that support real-time upsampling of low-resolution videos streamed from the cloud and onto high-resolution displays.

Problem Addressed

Existing super-resolution (SR) algorithms used to upsample low-resolution videos focus more on the visual quality of upsampled video frames rather than the real-time processing capability. The computation cost required to process upsampled video frames causes a bottleneck to running these algorithms in real-time. The inventors demonstrate an effective technique that promotes faster video super-resolution with negligible computation cost.

Technology

Free Adaptive Super-resolution via Transfer (FAST) is a patented technique that accelerates image-based transfer on compressed videos to enhance video resolution. FAST leverages the inter-frame similarity between adjacent frames in a video sequence with a typical frame rate of 30 fps (frames per second). For further acceleration, FAST uses non-overlapping block division embedded in the compressed video to avoid redundant computations in the transfer process. These key features of FAST promote efficient and accelerated super-resolution by exploiting information already embedded in compressed videos.

FAST accelerates state-of-the-art super-resolution algorithms (e.g. SR-CNN) by up to an order of magnitude with acceptable quality loss up to 0.2 dB and is an important step towards enabling real-time super-resolution on streamed videos for large displays.

Advantages

  • Novel technique that amortizes away the running time of single image based SR methods
  • Leverages information already embedded in compressed videos for fast video super-resolution