I saw this on Reddit the other day and was a little blown away by the video.
Anyone who has ever tried to take a photo of a large group of people at a family reunion or office gathering knows the virtual impossibility of finding one image where everyone has their act together. As Vivek Bhardwaj explained at the BlackBerry World 2012 keynote general session, when the BlackBerry 10 Camera snaps a photo of multiple people, it can compensate for the occasional blink or lookaway or smile failure. It allows the photographer to select the face of any subject in the frame with a suboptimal expression and step back frame by frame to a point where the eyes, smile, and focus were in unison. It then blends the older face capture into the photo so that everyone in the shot will be happy with the result
I am at once floored by the utter simplicity of the solution and duly impressed by the amount of technological wizardry involved in making this happen. There is nothing new about any of the digital features being used here. Simple point-and-shoot cameras have had facial recognition and smile detection for some time. Pretty much any digital camera has some video capture capability. The compositing of the timeshifted face shot into the photo is the sort of thing that can be done by anyone with proficiency in photo editing. What is remarkable is the apparent seamless automation of all these elements that the BlackBerry 10 Camera claims to enable.
Of course, it’s one thing to show a demo in a video onstage. It’s another to have it work perfectly every time on a piece of BlackBerry hardware. The first question I have concerns the quality and number of prior frames available. I don’t know how many megapixels the still photos will be, but if the prior capture is just 1080p video, those are only 2MP frames. I would assume the prior frames are full quality, but it remains to be seen what that frame rate would be and how many frames will reside in the buffer. Hopefully enough to find suitable images of all the faces in the image. However powerful this aspect of the system may be, it is still a big job to capture and hold that many images in memory.
The other big task is the compositing. Just because the face looks better in an earlier frame does not insure that the head/neck/shoulders remained stationary. If there is some misalignment there, the face will not look right in the final image. I am a photo editing novice, so there may be an arsenal of software solutions to get around this. If the native BB 10 Camera software can crunch all those operations by itself, then maybe RIM really has hit a home run here.
This is a pretty ambitious claim that Blackberry is making, but if it works as advertised, then the daunting task of the large group photo may become much more manageable in the near future.