Reinventing the photograph (Image: Michael Glenwood)
LIKE a lot of professionals, isn’t crazy about modern cameras. Granted, the London-based photographer keeps an iPhone in her pocket for the occasional snapshot, but really she likes nothing more than to take out her well-worn Hasselblad and hear the satisfying pah-clunk of a mechanical shutter. “As an artist and a photographer, I do believe in embracing new mediums,” she says. “But, to be honest, I use old technology more than new.”
With cameras, though, what counts as old is relative. Pannack’s camera phone and Hasselblad may be separated by several decades of innovation, but they have more in common than most gadgets spanning this timescale. Both focus light into an image using a series of glass lenses, like every other camera on the market. Whether that image is then captured on film or on the latest smartphone’s digital sensor, it is the result of a technique invented in the 1830s.
Advertisement
Photography has been the medium of the modern age, recording everything from momentous events like Neil Armstrong on the moon and the , to Marilyn Monroe on a subway grating and that selfie you took on New Year’s Eve, dressed as an orangutan and with a banana squashed in your friend’s ear. But all that could be just the start, because the way we take photographs and what we do with them afterwards is changing dramatically. The job performed by finely ground slivers of glass is being eclipsed by the work of finely crafted algorithms. As they take the lead, the 180-year reign of the lens might be over (Gallery: Click here to see the evolution of the camera in pictures). It is a fundamental reimagining of photography and, like an ageing Polaroid, the camera as we know it is fading from existence.
Digital photography has entered an age of extremes. There are gigapixel cameras that capture ultra-high-res images, and so-called single-pixel cameras that construct an image by taking multiple snaps with just one relatively cheap, low-powered detector. But the real battle for the future of photography is happening in our pockets. Although smartphone manufacturers have pushed camera technology massively in recent years, they are finding it increasingly difficult to cram a stack of lenses into a wafer-thin body. When Apple released promo shots of the iPhone 6 last year, it airbrushed out the protruding lens – the only component that could not be sufficiently miniaturised. But while the lens has reached its limit, computational photography is just getting going.
Most smartphone cameras give us the ability to take a panorama, essentially a super-wide-angle image. As you pivot on the spot, an algorithm records several images and then stitches them together, making slight adjustments for exposure, parallax and so on, so that the seams are invisible. Another common phone feature is high dynamic range (HDR) imaging, which allows you to photograph a very high-contrast scene – a person standing with the sun directly behind them, for instance – and still see details of their face rather than just a silhouette.
The HDR function works by merging several images of the same scene: at least one image with a long exposure to capture detail in the shadows, and one image with a short exposure to preserve highlight detail. “Under the hood, a lot of these computational-imaging techniques are already taking place,” says computer scientist at the King Abdullah University of Science and Technology in Thuwal, Saudi Arabia.
On the moon: A great photo stands the test of time (Image: NASA)
Computional trickery will soon be covering up even bigger imperfections. One shortcoming of smartphone photos, for example, is blurry edges – the natural product of cheap, miniaturised optics. Traditional cameras do not suffer from this because they usually use larger lenses that contain many individual elements. That is obviously a no-no for smartphones, so Heidrich is developing an algorithm that can correct optical aberrations. Since a bad lens typically affects the primary colours differently, the algorithm looks for areas where the red, green and blue layers that make up the image are misaligned. It then adjusts the image until they line up, .
Blurring can also come from camera shake, which is all too common when holding a phone at arm’s length. But an algorithm developed by Kari Pulli of photography tech start-up in Palo Alto, California, can help keep things steady. It takes readings on the fly from a smartphone’s built-in gyroscope, and uses them to adjust for motion blur without sacrificing genuine movement in the scene.
Risk and reward
Heidrich’s and Pulli’s ideas are set to improve smartphone image quality, but they will not do much to make up for our amateurish blunders. After all, photography – in the spirit of the pioneering French photojournalist Henri Cartier-Bresson – is all about the “decisive moment”, that crucial split second when you press the shutter. Pannack agrees. More goes into great photos than just a good-quality image, she says. Creativity is wrapped up with risk and what Pannack calls “beautiful mistakes”. Get it right and you’ve captured your friend’s expression the moment the banana finds his ear; get it wrong and you have accidentally focused on the background.
This is where we could see the biggest shift in how we take photos. What if we could simply take a shot and only later decide what its composition and focus should be? Hard though it might be to imagine this, computational photography is already showing us how it could be done.
“What if we could simply take a shot and decide how to compose it afterwards?”
In 2011, the California-based company launched the first consumer “light-field” camera. It still focuses light through a main lens towards a sensor, as in a normal camera. But the Lytro has an extra array of micro-lenses which capture bundles of rays and focus them on to individual pixels. The result is a device that captures the direction, as well as the intensity, of incoming light – known as the light field.
Using this information, algorithms can work out how the light would have behaved if the main lens had focused it differently. That leads to the key benefit of a light-field camera: the ability to refocus photos after they are taken. If your friend is out of focus, you can tap where they are on the screen and, as if you’ve had an instant cataract removal, their face turns crystal clear. Even better, the algorithms can calculate how the scene would look if the light had entered the lens at a slightly different angle, giving the option of viewing the scene from more than one perspective. If you were photographing a football tackle, you might be able to adjust the image to show the point of contact between the players, for example.
Refocusing a picture long after you took it is within our grasp (Image: Lytro)
Similar tech is headed to smartphones. The PiCam replaces the single lens on a smartphone with an array of 16 smaller ones. These are a small distance apart, so can also record depth information. “It kind of mimics the human eyes,” says Kartik Venkataraman, CEO and co-founder of Pelican Imaging, also in California, which developed the technology. Next-gen selfies will come in 3D, ready to be printed.
This year PiCam is being released as a development kit to smartphone manufacturers and we can expect light-field photography to be widely available soon. But not everyone involved in computational photography believes Lytro or PiCam have got the right idea. Pulli points out that taking up sensor space with a lens array kills the resolution of the final shot, unless yet more algorithms are employed to merge several low-res images. “You pay with low resolution, and have to expend a lot of effort trying to get it back,” he says.
Even so, there are other ideas for how to exploit light fields to great effect. One of these, says , a computer scientist at the University of Toronto in Canada, is relighting – the ability to impose light that was not there to begin with on a scene. Whereas traditional photographers have to laboriously set up a separate flash unit away from their camera to achieve certain lighting conditions, light-field cameras could add a spotlight effect automatically. Their depth information equates to a 3D map of the scene, which makes it possible to work out exactly where the shadows would naturally fall.
Kutulakos is developing yet another trick: separating light that has bounced directly off the subject from light that has bounced indirectly, scattered by different surface angles and textures. This technique involves placing an electronic, slit-shaped mask over a camera’s internal flash unit, limiting the outward light to one plane; another mask shields the camera’s sensor. When the masks match up in the same plane, only the direct light reaches the sensor.
Cameras – and our eyes – normally perceive a mixture of direct and indirect light. But things look quite different if . A glass turns black. Yellow foam turns grey. A latex glove becomes almost invisible. “We are only just starting to realise what happens when you take pictures in this way,” says Kutulanos.
Something particularly interesting happens when you photograph faces in direct light: they look stark and haggard, a consequence of masking out all the light scattered indirectly from beneath the skin. Indirect light, on the other hand, reveals perfectly smooth features and bright white teeth. You wouldn’t want this photo as your profile pic – it has an overall red cast – but Kutulanos believes a boost of the indirect channel could give anyone an instant makeover. More generally, if you can distinguish between the direct and indirect natural light in a scene, you will be better equipped to manipulate images, for example using Photoshop-style edits.
For Kutulanos, this is just another illustration of the level of control that could see computational photography supplanting traditional techniques. “I have colleagues who have been announcing the death of the camera for a few years already,” he says.
“Some people have been announcing the death of the camera for years”
And that death is on the cards. If the lens is the essence of traditional photography, then the final nail in the coffin will be imaging technology that has no need of one. Such a device has been designed by neuroscientist-turned-engineer Patrick Gill at Rambus, a technology company based in California. It forgoes a lens for a single spiral-shaped aperture, which diffracts incoming light into numerous mini-spiral “pixels”, each as unique as a snowflake. Although these mini-spirals arrive at the sensor in a blurry mess, their individual shapes allow an algorithm to unpack them (see diagram above). It is then just a matter of recording their intensity – as a digital camera does with normal pixels – to recreate the original scene.
For now, lensless technology is primitive. Gill shows an image of the Mona Lisa captured on his device. Although the enigmatic smile is there, the painting appears grey and blotchy. Nonetheless, it speaks volumes for the power of computational photography that, to preserve a moment in time, the most recognisable part of a camera is not even necessary. “I shouldn’t even call them cameras,” says Gill. “If my Rambus colleagues were here right now they’d give me a slap on the wrist!”
Not much bigger than a breadcrumb, and incredibly cheap to manufacture, Gill’s lensless cameras – or “lensless smart sensors”, as he is supposed to call them – could grace the surface of almost anything, and make photography even more ubiquitous than it already is. That raises questions about the effect incessant photography is having on our perception of the past (see “Making memories“), as well as questions about the future of those who make a living from being in the right place at the right time. If cameras are everywhere, and capturing the decisive moment is simply a matter of trawling back through an endless photostream, what place is there for a pro?
Pannack stands by her trusty Hasselblad, despite having just returned from Romania to discover that security scanners had blanked 30 rolls of film. “It’s kind of like saying, if we reinvented the paintbrush, would all painters be out of a job?” she says. “I don’t think so, because they would still need an eye – they need to know what works and what doesn’t.”
But with new technologies making photography snappier than ever, perhaps the rest of us can work on cultivating that expert eye.
Competition: To be in with a chance to win a Lytro camera click here
Leader: “Who said the camera never lies?“
Making memories
You either can’t stand them or are one yourself: people who choose to witness everything through their smartphone cameras. Those outstretched arms and selfie sticks have become a common sight. But is the habit affecting our ability to remember?
In 2013, psychologist at Fairfield University in Connecticut decided to find out, by . One group took no photos, another took photos of everything, and a third group took photos only of specific details. She then tested what they recalled.
Those who took photos indiscriminately remembered less than those who browsed the exhibits without a camera, possibly because the frequent snappers relied on the cameras to record what they were seeing, says Henkel. But those who used their cameras to zoom in on the details of exhibits recalled just as much as those without cameras. “It focuses your attention,” says Henkel. “All research tells us that focusing your attention is going to lead to better memory.”
Henkel’s study suggests that constant use of cameras could be free from ill effects, as long as we are selective and pay attention to what we are taking photos of. But photography can skew memory in another way.
In a classic 2002 experiment, , now at the University of Warwick, UK, and her colleagues gave a group of adults fake photos of a balloon ride and told them it took place when they were children. , or at least certain details.
For Henkel, this shows the dangers of the kind of photo-manipulation that computational photography will make all the easier (see main story).
This article appeared in print under the headline “Photo finish”
