How Super Resolution Zoom Algorithm Brings the Night Sight feature and Super-Res zoom of the Pixel 3 devices To The Future.

'This approach, which includes no explicit de-mosaicing step, serves to both increase image resolution and boost signal to noise ratio,' write the Google researchers in the paper the video is based on. 'Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones.'

I have been following and working with “RAISR: Rapid and Accurate Image Super-Resolution” for a while, as far as 2016, as I have a keen interest on imaging science. Digital zoom is my target has never been satisfied with most of the current technologies. Machine learning based algorithms seem to do a better job in taking tough because a   lower resolution image and "reconstruct" missing details reliably, much differently from the typical digital zoom process. Which is a small crop of a single image is scaled up to produce a much larger image and traditionally, this is done by linear interpolation methods, this recreates information that is missing in the original image, and usually introduces artifacts and lacks texture and details. In contrast, most modern single-image zoom procedures use machine learning and produce better results like the RAISR.

This brings us to the Google Pixel 3;

Digital zoom using algorithms (rather than lenses) has long been the “ugly duckling” of mobile device cameras. As compared to the optical zoom capabilities of DSLR cameras, the quality of digitally zoomed images has not been competitive, and conventional wisdom is that the complex optics and mechanisms of larger cameras can't be replaced with much more compact mobile device cameras and clever algorithms. 

With the new Super Res Zoom feature on the Pixel 3, we are challenging that notion. 

The Super Res Zoom technology in Pixel 3 is different and better than any previous digital zoom technique based on upscaling a crop of a single image, because we merge many frames directly onto a higher resolution picture. This results in greatly improved detail that is roughly competitive with the 2x optical zoom lenses on many other smartphones. Super Res Zoom means that if you pinch-zoom before pressing the shutter, you’ll get a lot more details in your picture than if you crop afterwards.

 Google uses the Super Resolution in the Night Sight feature and Super-Res zoom of the Pixel 3 devices and one could learn more about the technology on this article from 2018 on Google AI blog.