The key to the smartphone camera revolution: Fewer, bigger pixels

Apple has exactly the right idea when it comes to smartphone cameras.

When it introduced the iPhone 5s, Apple increased the sensor size by 15 percent but left the resolution at a relatively paltry 8 megapixels. As I explained in my previous post, 8 megapixels is still enough to make an excellent-quality 8″ x 12″ print…which is more than enough for most consumers, especially those taking photos with smartphones. And even a 1080p television or monitor can only display about 2 megapixels full screen.

A comparison of various sensor sizes. The dark orange square represents a 35mm "full frame" camera. The light orange square represents the 1.6X crop APS-C sensor found in Canon's entry-level camera line, and the light green square represents Four Thirds and Micro Four Thirds. Meanwhile, the smaller sensors are what you would find in point-and-shoot cameras and, at the smallest point (the dark blue), the cameras in smart phones. Image courtesy of http://www.macrumors.com/2014/07/17/iphone6-sony-13-megapixel-imx220-sensor/
A comparison of various sensor sizes. The dark orange square represents a 35mm “full frame” camera. The light orange square represents the 1.6X crop APS-C sensor found in Canon’s entry-level camera line, and the light green square represents Four Thirds and Micro Four Thirds. Meanwhile, the smaller sensors are what you would find in point-and-shoot cameras and, at the smallest point (the dark blue), the cameras in smart phones. Image courtesy of MacRumors.

The advantage of this approach is that the individual pixels are larger (1.5 microns), and that means more light and better image quality.

A missed opportunity

Unfortunately, most smartphone manufacturers are moving in the opposite direction by increasing megapixel counts. For example, the Sony Xperia Z1 has 20 megapixels. Although the Sony’s sensor is a bit larger (matching many point-and-shoot cameras at 1/2.3″), the individual pixels are just 1.1 microns…and that means less light per pixel and reduced image quality. It also means larger file sizes — which is a major challenge given the relatively limited storage space on smartphones and the increasing trend to move away from card-based storage in smartphones. Even if users back up their photos regularly onto computer hard drives or cloud services, even those have limited storage space.

Just who is making 12″ x 18″ “high-quality” prints from a smartphone camera anyway? And would any photo from a camera with such small pixels be worth printing at this size anyway?

For too long, digital camera manufacturers have leapfrogged each other on megapixel counts as a selling feature for consumers, and this has carried over into smartphones. As a result, the real technological improvements in camera sensors for image quality and high-ISO performance have been stymied by smaller and smaller pixels. But the people who really need higher megapixel counts are professional photographers, not consumers. Yet the professional Nikon D4S with a full-frame sensor only prints 16.2-megapixel photos…because that’s all they need. But because of the larger sensor, the D4S has a pixel size of 7.3 microns — massive compared to even the best smartphone.

So why not just add bigger and bigger sensors to smartphones? Why not a full-frame sensor?

Certainly moving from 1/3.2″ on the iPhone 5 to 1/3″ on the iPhone 5S was a reasonable improvement, and the relatively obscure Nokia 808 Pureview offered a whopping 1/1.2″ sensor. But larger sensors cost significantly more in terms of manufacturing and require larger and larger lenses. That means more bulk. Not exactly something you want to slide into your pants pocket or replace when your 2-year wireless contract ends. Without a major revolution in lens design (which is dependent on the laws of physics), we can only realistically go so big with smartphone sensors.

Polka Boy performs at the biergarten at the historic Rathskeller restaurant in downtown Indianapolis. Shot with LG Google Nexus 5 smartphone. f2.52; 1/20 shutter; ISO 1531.
Polka Boy performs at the biergarten at the historic Rathskeller restaurant in downtown Indianapolis. Shot with LG Google Nexus 5 smartphone. f2.52; 1/20 shutter; ISO 1531; pixel size 1.3 microns.

Fuzzy (literally) math

So imagine if Sony had opted to go for 8 megapixels instead of 20 on the Xperia Z1 with its 1/2.3″ image sensor.

  1. A 1/2.3″ sensor measures 6.16 mm wide x 4.62 mm high.
  2. An 8-megapixel image measures 3,456 pixels wide x 2,304 pixels high.
  3. Divide 6.16 mm by 3,456 pixels to get 0.0018 mm per pixel.
  4. One micron is equivalent to 1/1,000 of a millimeter, so multiply by 1,000 to get 1.8 microns per pixel. That’s significantly larger than the iPhone 5S, and that would mean significantly better image quality.

As good as that is, what if the Nokia 808 Pureview only had 8 megapixels of resolution instead of 41? Let’s do the math again.

  1. A 1/1.2″ sensor measures 10.67 mm wide x 8 mm high.
  2. An 8-megapixel image measures 3,456 pixels wide x 2,304 pixels high.
  3. Divide 10.67 mm by 3,456 pixels to get 0.0031 mm per pixel.
  4. Multiply by 1,000 to convert millimeters to microns, and you get 3.1 microns per pixel.

That’s substantially larger than the pixel size of the Sony RX100 (2.4 microns per pixel), which is an award-winning, high-end compact camera with 20.2 megapixels of resolution.

Other tradeoffs – cropping and digital zoom

So part of the appeal of the 1/1.2″ Nokia 808 Pureview and its smaller 1/1.5″ sensor successor the Nokia Lumia 1020, was the 41-megapixel resolution that enabled “reinvented zoom.”

What the cameras in these phones are really doing when they “zoom” is cropping the image to a more reasonable 5 megapixels without any loss in image quality. This is quite useful for smartphones, most of which don’t have optical zoom lenses. But when the pixels are small to begin with (1.4 microns on the 808 and 1.1 microns on the 1020), “lossless” doesn’t mean a whole lot in terms of image quality.

But, as someone who regularly crops photos, if the full photo drops to 8 megapixels, then my crops could take them down even smaller. It leaves less flexibility…but even if I crop all the way in to 2 megapixels, that’s still 1080p resolution…great for displaying on screen (or Facebook, Twitter, Instagram) or making 4″ x 6″ prints. The addition of a real optical zoom lens on a smartphone minimizes the need for digital zoom or cropping in the first place.

What does the future hold?

I don’t know if we’ll ever get to DSLR image quality or lens quality with our smartphones, but considering the rapid pace of advancements the industry has already made, I would also not rule it out. As it stands today, the best smartphones can go toe to toe with point-and-shoot cameras in terms of image quality, although the new generation of bridge cameras like the Olympus Stylus 1, Sony RX10, RX100 and Panasonic FZ1000 are raising the bar higher than ever for what a point-and-shoot camera can offer. There are only a handful of smartphone cameras that offer optical zoom at all (the Samsung Galaxy K Zoom is basically a point-and-shoot camera first and a smartphone second, so it’s a bit awkward as a phone), but massive efforts are underway to bring real zoom lenses to smartphones while still allowing them to fit comfortably in your pocket.

The smartphone and camera industries just need to focus their energies in the right areas…and Apple has certainly set the precedent for that. As advancing sensor technology continues to squeeze more and more out of every pixel, then we can start to talk about larger and larger print sizes with more and more megapixels.

4K video may change still photography for the better

If you’re a pro photographer, especially a pro photojournalist, sports or wildlife photographer, the camera bodies of choice today are the Nikon D4S (MSRP $6,499.95) and the Canon EOS-1D X (MSRP $6,799.00). Not for the faint of wallet, especially with pricey full-frame lenses to buy on top of that.

And even if you have the budget for cameras like these, they are also bulky and heavy. The D4S body alone weighs 1,180 grams (about 2.6 pounds), and the EOS-1DX body alone weighs in at a whopping 1,530 grams (just under 3.4 pounds).  Add on a battery grip, a big full-frame telephoto lens and a bag with a few additional full-frame lenses, and you’re looking at a serious backache.

So why do professionals subject themselves to all that expense, bulk and weight? Because these cameras are the best tools available for their needs. Here are a few reasons why:

A comparison of various sensor sizes. The dark orange square represents a 35mm "full frame" camera. The light orange square represents the 1.6X crop APS-C sensor found in Canon's entry-level camera line, and the light green square represents Four Thirds and Micro Four Thirds. Meanwhile, the smaller sensors are what you would find in point-and-shoot cameras and, at the smallest point (the dark blue), the cameras in smart phones. Image courtesy of http://www.macrumors.com/2014/07/17/iphone6-sony-13-megapixel-imx220-sensor/
A comparison of various sensor sizes. The dark orange square represents a 35mm “full frame” camera. The light orange square represents the 1.6X crop APS-C sensor found in Canon’s entry-level camera line, and the light green square represents Four Thirds and Micro Four Thirds. Meanwhile, the smaller sensors are what you would find in point-and-shoot cameras and, at the smallest point (the dark blue), the cameras in smart phones. Image courtesy of MacRumors.

 

  • Full-frame sensors. Full-frame image sensors (that is, sensors the same size as 35mm film) provide lots of advantages in terms of shallow depth of field effects and image quality at high sensitivity (ISO) settings. Quite simply, bigger sensors allow more light to come in. A full-frame sensor (depicted in orange in the image below) is 36mm x 24mm.
  • Fast burst rates. The D4S can shoot continuously at 11 frames per second, which is very fast, and the EOS-1D X can shoot even faster at 14 frames per second.
  • Fast, accurate autofocus. Fast burst rates are meaningless without accurate autofocus, especially for fast-moving subjects like soccer players or birds in flight. You will just end up with a whole lot of blurry images. So these DSLR cameras have phase detect autofocus sensors.
  • Optical viewfinders. Those who use these cameras in the field can’t afford to miss a moment to capture the perfect shot. That’s one reason they tend to prefer optical viewfinders, because “Live View” on an electronic viewfinder is not truly live — electronic viewfinders are slightly delayed because the camera has to process the image coming in through the sensor as well as any exposure adjustments before the image appears in the viewfinder. For most photography work, the delay is not significant enough to matter, but for these users, it might be the difference between capturing the shot and missing it. Also, most electronic viewfinders do not display the entire frame.
Homeless car
This was the aftermath of a tornado on September 27, 2002 in Mooresville, Indiana. The tornado blew away the garage that had been sheltering this Corvette. Shot on a Nikon D1.
  • Weather sealing. Field work for sports, wildlife and general photojournalism can happen in all sorts of conditions and elements. I experienced this myself back in September 2002 when I was working as a reporter for The Mooresville-Decatur Times, and a tornado struck the town. After the tornados had passed, I had to go out and photograph the storm damage, and it was still raining quite a bit in the aftermath. It was a good thing the Nikon D1 I was using was weather sealed.
  • Fast ergonomics. These cameras are loaded with physical buttons that allow the photographer to change settings on the fly without having to rely on menus. The deep grips also help keep the camera steady in the photographer’s hand. Ergonomics might not be such a big deal for amateurs, but professionals who take very large numbers of photos and spend a lot of time holding a camera need a camera that “feels” right and doesn’t require navigating a lot of menus to change settings

A bold, new(ish) idea

But is there a better way? I think so. Enter the Panasonic Lumix GH4.

It has a burst rate of 12 frames per second, has a deep grip and lots of external dials just like a professional DSLR, a new “Depth from Defocus” autofocus system that is fast and accurate for tracking moving subjects, weather sealing and more. And, by the way, the MSRP is $1,699.99, and it only weights 560 grams (about 1.2 pounds).

Your fancy 1080p HDTV only has about 2 megapixels of resolution

Not only that, but it does something significant that the flagships from Canon and Nikon can’t: it shoots 4K ultra high-definition video. But why is this important to still photographers?

Video is, fundamentally, a series of photographs…one in each frame. And, at a minimum, video is shot at 24 frames per second.

Most modern high-definition cameras shoot 1080p video with a resolution of 1920 x 1080 pixels. If you multiply 1920 by 1080, you get approximately 2 million, or about 2 megapixels. So each image in the frame would have a very low resolution in terms of photography and would only be usable for making very small prints: a 4″ x 6″ at 300 dots per inch. Larger prints are possible, but the quality becomes degraded.

But 4K video is shot at 3840 x 2160 pixels. Multiply 3840 by 2160, and you get more than 8 million, or more than 8 megapixels. That’s enough resolution for an excellent 8″ x 12″ print — larger than a full page in National Geographic or Sports Illustrated. Cinema 4K is shot at 4096 x 2160, but that’s a strange aspect ratio for still photos.

Unless the photographer needs an even larger print than 8″ x 12″ and cannot fudge on print resolution at all (an 8-megapixel photo could cover a 2-page spread in National Geographic at 200 dpi instead of 300, and that’s still considered “good” resolution), then the GH4 offers the ability to capture very printable action photos at 24 frames per second.

Drawbacks

So why isn’t everyone jumping on the GH4 bandwagon? There are a few drawbacks to the GH4, but they don’t seem to be enough to make up for the price and weight difference or the potential for 24 fps photography.

  • Sensor size. The GH4 uses a Four Thirds sensor, which is considerably smaller than full frame: 18mm x 13.5mm, depicted in lime green above. There are some compromises in terms of depth of field and image quality at very high ISOs, but the GH4 produces images that are quite usable for 4K video up to ISO 3200, and usable up to ISO 6400 at smaller sizes, including 1080p video. Sure, a full-frame sensor provides super-shallow depth of field effects, but that’s more important for portraiture than for sports or wildlife photography. You can still get somewhat shallow depth of field with Micro Four Thirds cameras. And, since the sensor is smaller, the lenses can be a lot smaller and lighter too. For situations when you really need full frame image quality and 4K video, you can buy a Sony A7S for $2,499.99 in addition to the GH4 and still spend less and bear less weight than you would on a D4S or EOS 1D X. (Unfortunately the Sony A7S is not weather sealed, and its burst rate for stills is only 5 fps, or else I would recommend it instead of the GH4.)
The man on the other side of the glass.
This boy had his face pressed up against the glass at the zoo, making an interesting reflection. With my 45mm f1.8 portrait lens, I was able to achieve shallow depth of field in very low light even with a relatively small Four Thirds sensor. Shot with Olympus E-PL5.
  • No optical viewfinder. The biggest reason why the Panasonic GH4 and the Sony A7S are so light is that they do not have the pentamirror mechanism found in traditional DSLR cameras. And that means no optical viewfinder — only an electronic “Live View” viewfinder. But electronic viewfinders are not what they used to be — the electronic viewfinders on these cameras have so little lag that it’s “nigh imperceptible.” Both electronic viewfinders also cover 100% of the frame, which is an improvement over EVFs of the recent past. And the advantage of Live View is that you can see the results of exposure adjustments on the fly before shooting. EVF latency will never be zero, but with the GH4 and cameras like it, it’s awfully close.
  • Limited lens choices. Panasonic has the basics down in terms of professional lenses with its 12-35mm (24-70mm equivalent) and 35-100mm (70-200mm equivalent) weather-sealed, stabilized f2.8 zoom lenses. There are also some tremendous prime lenses in Panasonic’s lineup to get most of the shallow depth of field effects that you can find in a full-frame camera…including a 15mm (30mm equivalent) f1.7, a 25mm (50mm equivalent) f1.4 that I personally own and love, and 42.5mm (85mm equivalent) f1.2 portrait lens. Plus, Olympus has some great lenses of its own in the Micro Four Thirds lineup, with more pro lenses on the way. Canon and Nikon still have the edge in terms of lens selection, but a professional can build a fairly complete Micro Four Thirds lens kit at much lower prices and with much less weight than with full-frame Canon or Nikon lenses. (And maybe, just maybe, the excellent Olympus OM-D E-M1 will get 4K video as well…if not, we know Olympus is getting into the 4K game soon.)

Will this be enough to pry the Nikon D4S or the Canon EOS-1D X out of a professional photojournalist’s arthritic hand? Or will the next-generation pro DSLRs just start shooting 4K video too? Will we start to see more 8K video (with each frame being a 32-megapixel photo) or even higher resolutions? Memory cards that can hold terabytes of ultra HD video? Will the DSLR video revolution reverse course and place camcorders in every photographer’s hand instead of still cameras in every videographer’s hand? Will photo and video editing software converge to help photographers sift through 24 photos for every single second they were covering an event to find that one perfect image for publication? Smart phones that take professional-quality photos? It’s certainly an exciting time for photography.