A new camera phone champion hits the market (the Samsung Galaxy S9), and it’s time for another super-detailed comparison. All the photos here (bar one) are deliberately shown in low light. My aim here was to push the optics and image processing of the three smartphone cameras to the limit – every phone camera takes a good snap in bright sunlight, but they all struggle once light gets low. Or do they?

Why push phone cameras to the limit? Because if you can find one that takes good snaps in terrible light then you’ve got a margin for error in real life shots, i.e. you can take photos in bars and clubs and living rooms, and at night, with more confidence that things will work out well.

iPhone X, main camera, f/1.8

Galaxy S9, main camera f/1.5 (or f/2.4 if stopped down in bright light)

Pixel 2 camera, f/1.8 aperture

Notes:

• I’m not going to be testing the small aperture zoom lens of the iPhone X – this only works well in bright light anyway, plus there’s scope for a proper zoom test against the Samsung Galaxy S9+ when I get that in for review too. In the same vein, any testing of ‘portrait’ mode will need another feature too, since the standard Galaxy S9 doesn’t have this computational effect.
• All photos were taken handheld (though I do have steady hands!) and in full ‘auto’ mode (apart from controlling whether the flash fires), as the average user would do. Each of these phone cameras either have ‘pro’ modes or (in the iPhone’s case) alternative camera software that can do more with the camera hardware, but then comparing results is somewhat meaningless.
• All test photos were taken from the same spot in each case, so framing differences are due to the different field of view of the phone cameras concerned.
• In each case, where there was a choice of focus points, I made sure to tap in the centre, or on the intended subject, so that focussing should be identical in each case (though the Pixel 2 made a mess of one of the shots, as you’ll see).
• I look at 1:1 crops in each case, since only then do we see the actual image quality produced. In normal use you’d only see pixel detail on (e.g.) a 4K TV, but as soon as you start cropping a photo later then you’ll be glad of high quality pixels!
• Testing was done on the Pixel 2 front with the XL model, since that was to hand, but the cameras are identical, so for the purposes of this comparison the images are 100% appropriate to the smaller Pixel 2.
• Behind the scenes, the phones take several exposures and then combine them (out of the user’s control) for best results. With modern chipsets and OIS in hardware, all of this is hidden – the output is just the one JPG, which is what I’m looking at here. And please don’t talk about ‘RAW’ output in the comments below – such processing is absolutely only for pros who want to mess around with their photos later. Here I’m looking at what the typical buyer sees.
• All the test photos here can be downloaded for your interest and further analysis here on Dropbox.

The test scenes here are arranged in approximate order of light levels, i.e. you won’t start to see any real differences in photo detail and fidelity until we get halfway down the page!

Test 1: Hazy sun

Yes, yes, this is a bright light shot, but included here just for reference, to prove my point that phone images in good light are virtual indistinguishable. Here’s the overall scene, shot in hazy (and somewhat elusive) winter sun here in the UK:

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

Clear results all round, but the iPhone X’s image processing clearly has less edge enhancement applied to it – look at the greenery, look at the bricks in the wall – and so I’m giving this the nod by a point here.

Apple iPhone X: 10 pts; Samsung Galaxy S9: 9 pts; Google Pixel 2: 9 pts

Test 2: Gloomy plane

With winter gloom set in and light pretty awful, here’s one of my favourite test subjects – plenty of details to inspect at around 50m distance. Here’s the overall scene::

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

Similar results (and scores) to test 1 – all three photos are excellent, considering the lighting, but the iPhone X has the edge (pun intended) because of its lack of edge enhancement – details look more natural, more lifelike, while the S9 and Pixel manage to make too much of tiny details and texture. In my humble opinion.

Apple iPhone X: 10 pts; Samsung Galaxy S9: 9 pts; Google Pixel 2: 9 pts

Test 3: Bright details in the gloom

Same poor light conditions, last one before I head indoors – this time with the delicate and bright reflective clock face on the clock tower – another excellent test subject. Here’s the overall scene:

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

You’re going to start accusing me of bias at this point, because I’m going to give this one to the iPhone X too, by a single point – the Pixel 2’s shot is too artificial in terms of contrast, while the Galaxy S9 make the scene lighter than it was (which might be a good thing, in this case!) but it also doesn’t handle the fine number detail on the clock face as well.

Apple iPhone X: 10 pts; Samsung Galaxy S9: 9 pts; Google Pixel 2: 9 pts

Test 4: Low light church interior

Inside the church, light levels were low and I was shooting details at a good 30m, in this case focussing on the lectern area, with some writing to examine and with finely detailed railings. Here’s the overall scene, made to look lighter than it really was by the OIS:equipped phones:

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

For the fourth time in a row, the iPhone takes it, I’m afraid, with better control of noise and detail – you can just about read the (10 commandments) text on the stone slab, while the words fade into enhanced noise on the S9 and Pixel 2. Plus the gold eagle looks more natural and less artificial.

Apple iPhone X: 9 pts; Samsung Galaxy S9: 7 pts; Google Pixel 2: 8 pts

Test 5: Low living room light

Things are starting to get seriously tricky for the phone cameras now… With just an atmospheric LED light, a shot at about a metre of a painting in a living room. I’ve learned not to examine pixel details in the painting itself, though, since you then start to hit brush strokes and their own uncertainties! Here’s the overall scene:

And here are 1:1 crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

The Google Pixel 2’s HDR+ processing and intelligence is starting to pay off here, with far better dynamic range (thanks to combining loads of shorter shots), and only the LEDs blown out. It even gets the colours right. The iPhone X is in second place, some way behind in terms of handling the brightest parts of the frame, with the new S9 in last place and less dynamic range than I’d have expected.

Apple iPhone X: 7 pts; Samsung Galaxy S9: 6 pts; Google Pixel 2: 9 pts

Test 6: Darker detail

The same test scene as above, but this time I wanted to crop in on a darker frame corner, to look at handling of noise in really dark parts of the shot:

Here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

Noise levels are around the same in each case (remember that the wall is a solid colour), with perhaps the Galaxy S9 having a slight edge, as you might expect with its f/1.5 aperture. It also gets the colour of the wall just right, though exact coloration will be different than in daylight due to the unusual lighting, of course. A narrow S9 win….

Apple iPhone X: 8 pts; Samsung Galaxy S9: 9 pts; Google Pixel 2: 8 pts

Test 7: Extreme low light

A bookcase, shot about 0.5m in light so low than I had difficulty seeing the book titles with my own light-adjusted eyes. Here’s the overall scene, again made to look a lot lighter than it was because of the great camera specs and processing:

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

Pushing things this far has at last caused the iPhone X to drop behind. Where it’s showing clear artefacts down at the pixel level, the Google Pixel 2’s HDR+ software produces something that’s much cleaner and clearer. But it’s the Galaxy S9’s photo that is astonishingly clear – probably too artificially so – the combination of multi-shot and larger aperture show Samsung up well here.

Apple iPhone X: 6 pts; Samsung Galaxy S9: 9 pts; Google Pixel 2: 8 pts

Test 8: Party time

My standard ‘moving person, low light, party mock-up’ shot, with flash forced on. If a phone can handle this then it’s good for an evening out. Here’s the overall scene:

And here are 1:1 central crops from, in turn, the iPhone X, the Galaxy S9, and the Pixel 2:

None of these photos really pass muster compared to what a standalone camera with a real Xenon flash would have produced. Of the three shots here, the Pixel 2’s seems most useable, in terms of freezing motion (kind of), thanks to it being able to discard some of its low light multiple shots and align the rest, plus it auto-adjusts for the flash and thus makes the scene natural. while the S9 gets the scene light enough and freezes motion thanks to its large aperture, but it all looks very two dimensional and not very natural. The iPhone X’s shot is simply too blurry and noisy, even when viewed as-is and not cropped in.

Apple iPhone X: 5 pts; Samsung Galaxy S9: 7 pts; Google Pixel 2: 8 pts

Verdict

Adding up all the points (this is the fun bit!) gives:

• Google Pixel 2: 68 pts 
• iPhone X: 65 pts
• Samsung Galaxy S9: 65 pts

So a narrow victory overall for the Pixel 2, mainly thanks to cleverer software than any physics advantage. Though I have to give three huge caveats here. Firstly, the margin of victory is relatively small overall and a different set of test subjects might have elicited slightly different results. Secondly, and along the same lines, I wasn’t testing zooming here – the iPhone X has a second 2x zoom lens, of course, and this would skew things back towards it – as I hope to do in a future article against the Samsung Galaxy S9+, which also has the zoom lens.

Finally, I can’t emphasise enough that a large amount of imaging performance in modern phone cameras is down to software. Which can be changed. Which means that continued OS and application updates to all three contenders here will keep their output continuously improving through 2018.