UltraPixels, Photography, and the HTC One: A Chat With HTC’s Symon Whitehorn
HTC really pushed smartphone camera image quality front-and-center with the launch of the One and its UltraPixel sensor. As such, it’s part of a growing trend we’ve noticed that has handset manufacturers placing less of a value on raw pixel count, and instead looking to how they can really, qualitatively improve the pictures their phones are producing.
Not everyone is so quick to adopt the same level of enthusiasm for this new direction, and we’ve had our own concerns about what dialing-back the resolution on the One’s UltraPixel camera will mean for its users. Even with such misgivings, we wanted to learn more about what HTC was doing, and maybe even gain some new-found understanding and appreciation for UltraPixel technology in the process.
To help explain things to us, we turned to HTC’s Symon Whitehorn, Director of Special Projects, who was kind enough to spend some time talking with us about UltraPixel, the One, and smartphone cameras in general. Here’s an edited account of our conversation with him:
Pocketnow: Is the choice for four megapixels [with the HTC One] a technical limitation, or did HTC feel that four megapixels is “enough?”
Symon Whitehorn: I think that it comes down to understanding the type of pictures people are taking, what are the constraints, what’s going to really impact the quality – as you’d define it – with sort of eighty percent plus of the pictures they take every day? No camera can cover all scenarios. You know, you do need to focus on (to use a pun) on what are sort of those “prime” images they’re trying to capture, but also what are they doing with that content?
2.1 megapixels would have given us a much bigger pixel size, even with the UltraPixel, but frankly despite people saying, “well that was good enough,” that really wouldn’t have been enough in some scenarios, especially for what you can’t do with 4.0, like some zooming. We also wanted to have no compromise on print – even though that’s a rarer and rarer scenario – we certainly didn’t want to fall short in any ways with photo work or print, up to 8 x 10. There’s no differentiation up to that point.
We knew that one of our prime dissatisfiers with most customers – even employing two cameras – is low-light performance. And that’s one of the things we really wanted to crack, so we spent a lot of time focusing on that element of the equation: pictures in bars, pictures in restaurants… everyone can get really nice pictures on a sunny day on the beach. Any camera can achieve that. So it would sort of go into that fringe of where we think – you know – frankly, I think we’ve made one of the best point and shoot cameras you can get – for a restaurant.
P: How much of what UltraPixel does is software, and how much is hardware?
SW: A lot of that is actually hardware, giving you that performance – it’s literally physics. More photons are coming in than otherwise. There’s not a lot of processing going on to enhance light other than managing the ISO, which we’re still working on. It’s not just the sensor; it’s also the optics, it’s also the aperture – stuff that we’re just committed to continue to improve upon.
It’s never one thing, though, it’s always the entire package, and I think that for a long time a lot of smartphones had suffered from things being put together individually from silos. “This sensor’s right.” “This lens is right.” And nobody actually worked out that the [transfer function] between them wasn’t compatible, or they didn’t know there was a compromise there.
SW: There’s always going to be some processing you do after the picture to smooth noise, but actually [with the UltraPixel sensor] we have less noise to start with. Say you had a much higher megapixel camera with a much smaller pixel size – in comparison, a bucket versus lots of little cups collecting rain – you start getting all this interpolation and noise happening between those elements, so there’s stuff you have to cancel out. We just start from a better place; we don’t have as much noise to process out of the image from the start.
We’re not anti-megapixels. When I go shoot with my 5D II it’s a 22-megapixel camera. Then, the pixels are like eight microns. That’s why you’re not going to make as much noise. It’s really about starting from the right place; the less processing you can do the better.
P: Are we going to see UltraPixel cameras on most HTC phones going forward, or will it be saved for the flagship, higher-end devices?
SW: I wish I could answer that; there was a very good debate about that just yesterday. For now it’s only available on the HTC One, but if it gets such great feedback – we’ll wait and see. Obviously, that’s something we’d love to do; I can’t confirm or deny that at this point.
P: Are there any limitations – besides just megapixel count – where you kind of wish the UltraPixel camera could do better, and might do better in future generations?
SW: Smartphone cameras really are pushing physics, frankly – you know, in these super-tiny forms – I’m actually quite amazed what the industry as a whole has achieved in the past few years. I think some of the biggest steps forward are going to have to come in optics, and in some of those areas – frankly, more conventional areas: optics, aperture. There are obviously other ways experimenting with different layouts of pixels.
So, there’s a lot of room to maneuver in, a lot of room to get better. Once again, we’re clearly committed to that, and we’re open to lots of different ideas. We’ve seen a lot of interesting work being done.
P: One thing that we think could be improved [with the HTC One] is the sharpness of the image. Sometimes you can get some extremely sharp photos, but other times it just looks a little – not blurry or fuzzy, by any means – but if you could manually set the sharpness, you’d want to kick it up a few notches.
SW: I just want to make something very clear: we’ve all been playing with pre-production units – my One is working pretty well, but I saw another around the office that wasn’t working very well – so some of these could be – and are being – improved and tweaked, but very often there are decisions being made in sharpening algorithms that frankly, when you look back on some of the past products, they relied on too much over-sharpening. I don’t like over-sharpening; I’d rather have a slightly more natural-looking image.
So, yeah, there are decisions being made in the algorithms for different situations. Obviously, when you’ve got a brighter picture you’ve got more light to play with, and have to digitally stop down. I think there are little things we can do. I’m not sure whether those inconsistencies are because yours is a pre-production unit, or because those are the algorithms. They’ll all be totally refined, obviously – with the final production unit, you’ll have that.
P: Something we’ve yet to see with smartphone cameras is an optical zoom. Do you foresee that happening within the next five years?
SW: Yes I do. I think the next bounding box to go into is going to be focal length. I think then the question comes around to behavior… [with a simple camera] my wife took the best pictures she ever shot. With a camera with infinitely variable zoom, then she didn’t get as many good pictures. So, it’s a trade-off.
We have to ask ourselves, is a variable like telephoto a good thing in one of these devices? I think we’ll be working on that, because it’s another natural bounding box to go and play with… I think there’s some great smartphone photography going on right now, I think precisely because of some of the limitations of it.
P: Can the quality of the One’s photos improve further with just a software update?
SW: I can’t promise that; I would hope so! You know, there’s a lot we could do. We really want to get the information back from people like yourself of what’s satisfying. We like to fine-tune the performance of our products based on the feedback we get.
Pictures are emotional, and the emotive response to them is what we’re going to be tuning for. We like to make sure we don’t over-sharpen and make things unnatural. It’s all about the balance that we’re trying to create.
P: If you could defy physics and invent the ultimate camera to put in a smartphone, what kind of specs would it have? What would be the ideal camera in your mind, to take the most amazing pictures?
SW: Then I’d be telling you what I’m working on next, which I can’t give away!
But no, ideally, I actually want to see a leap forward in optics. I would like to get sharper optics – better optics. The trade-off always is “how much physical volume are people going to tolerate from their device and could still live with it on a daily basis?” I would hate to see smartphone cameras – and especially our cameras – start to replicate the errors I saw in the point-and-shoot market. They just become bloated with features and functions that frankly are irrelevant and start to becomes these jack-of-all-trades instead of focusing on just taking the best pictures you can take.
For me, the perfect camera is going to be wrapped around that sort of idea of daily use. I think we’re on the right track for a smartphone camera… it would be a mistake to try to replicate or match what a conventional camera does.
P: Was one of the considerations with going with four megapixels that it allows Zoe to work, that it allows these highlight videos to be generated on the fly, because you’re not dealing with fifteen megabyte files?
SW: Absolutely, yeah. We can apply effects real easily, we can post-process them easily. As you play with it, it’s instantaneous. Part of that’s our dedicated [imaging chip], and a whole bunch of processing horsepower, as well, and it just gets exponentially better because we have smaller files to start with.
P: We talk a lot about dedicated camera buttons because it seems like they’re gone and they’re never coming back. Was a dedicated camera button ever considered for the One?
SW: It was considered and it’s something that’s certainly not off the table for us. There are factions for and factions against. This doesn’t signal a rejection of the shutter button for us. At all.
P: What does the future look like for HTC cameras?
SW: One of the things we haven’t touched on is we want to get a very rich dynamic range, and we want to just really elevate the photographic quality – even if you’re going to put some vignetting all over it [like Instagram effects] – working with the old guys at Kodak, they’d be looking at some of this stuff and say, “why are people doing this to their images? It took us years to get vignetting out!”
At the end of the day we want to get the best digital pic we can… the camera shouldn’t get in the way of you and the moment. I’m finding now that I’m taking out my HTC One and shooting with that more than I am with my so-called “real” camera, because I’m doing more with the content.
The idea is that you have this true-to-life starting point with the images you can get. So we’re going to continue to work on that; this is a commitment to photography that we have. It’s not just myself – we have a huge team here of people working on it, and we have a lot of experts on the outside helping us, as well. We look at trends in photography, we look at trends in imaging and video.
I’m personally very fascinated by what people do with our devices. I’m going to be really excited to see what people do with Zoe and highlight videos – I think they’re really interesting, as sort of a creative starting point.
Big pixels and low light [performance] are always going to be a criteria for us, just because we’ve seen what it can do for people, but that’s not our only point of focus. There are loads of things we’re developing.
P: Any final thoughts?
SW: I think where you’re going to see some of the most interesting work is going to be what you do with your content. I think that’s why a lot of the decisions we made earlier-on about UltraPixels and all that benefits – it’s a holistic approach to imaging, rather than looking at a silo. We don’t have a lot of divisions just looking a lenses, or looking at sensors, but we have a very holistic approach to developing our imaging solutions.
The HTC One X was a big step forward in imaging, and I think [the One] is a quantum leap on from that, and I think that’s just going to continue.
Symon Whitehorn, thanks so much for talking with us. We can’t wait to see what comes next from HTC and its camera technology!