Every year, our tech experts test a wide range of smartphones from brands like Apple, Google, Samsung and more so that you can better compare your options and find a phone that suits your needs.
Here's an insight into how we do our testing.
On this page:
Our expert testers
All the tests and procedures described below are carried out or overseen by professional testers in specialist smartphone testing laboratories in Europe. The test method is decided by representatives of consumer organisations from around the world who have many years of specialised expertise in smartphone testing.
How we choose what we test
The lab is constantly testing smartphones, but each batch can take some weeks to complete. This is because we only test products bought from retail outlets, so we know that what we're testing is representative of what you're likely to see on store shelves (and the phones haven't been 'tweaked' in any way for better performance).
We fast-track testing for models we think you might want to know about, like the latest Google, Samsung or Apple phones, rather than simply including them in our next scheduled batch test.
How we test smartphones
Testers rate the following aspects of each phone to help determine the best performers.
Camera quality (photo and video)
We assess photo (still) image quality in studio and outdoor environments, in high and low light conditions. Rear and selfie cameras are tested as well as the camera flash if available. Camera apps are set to fully automatic mode and default focal length at maximum resolution.
A technical assessment of the sharpness, contrast and colour balance is conducted, followed by a measurement of the shutter delay, picture sequence speed, autofocus speed and start-up time. Then, a visual appraisal of test photos is conducted using the following settings:
- Picture quality (daylight) – Still life (Studio) and Portrait (outdoor)
- Portrait (outdoor light simulation) for normal and Bokeh effect portraits
- Universal target test picture (daylight illumination and 4 x zoom)
- Wide-angle quality and suitability for camera and camcorder
- Picture quality in low light conditions without and with flash, if available
- Front camera 'selfie' picture quality – bright and low light
Video quality is assessed in a studio environment designed to simulate daylight in full HD, HD at 60FPS and 4K UHD if available. We also test image stabilisation using the rear and selfie cameras and sound quality during video recording.
Assessing colour accuracy and contrast during the smartphone camera test.
Battery life
A complete discharge of the battery from 100% charge based on a typical user day. This includes calls, navigation, camera use, web browsing and periods where the phone is in on but not in use. The test cycle is conducted using a robot arm that simulates a human finger, for test consistency across devices.
Each device is tested with two brightness settings, 300 nits and maximum. Display is set to dim after one minute of inactivity. Two notifications per hour (display switches on / vibration alarm) are sent to the device under test. Charge time is also recorded and assessed at 15 minutes and to total charge.
Display
We assess the size of the display as well as quality of the image with regard to sharpness, brightness and colour clarity. This includes three readability assessments – indoor in low light, outdoor in bright light and from the sides.
Durability
Assesses resistance to scratches, shock (drop test) and rain. Scratch test involves attempting to scratch the screen across five different pressure loads and followed by the protective glass around the camera.
To assess shock, the phones are turned on and placed in a tumble barrel with a stone surface and repeatedly dropped from 80cm, 100 times. The phones are checked for damage at 25 and 50 drops and a final function assessment occurs after 100 drops.
To test rain and water resistance, the phones are placed on a horizontal surface and 'rained on' for five minutes. Then, we check functionality immediately after and again at two and three days. Phones that claim to be waterproof are submerged to the stated depth based on IP rating/manufacturer claims. Then, functionality is checked immediately after and again at two and three days.
Replicating a typical user day with a robotic finger during the battery discharge test.
Sound quality
Sound score is based on the quality of sound from the integrated speakers and listening through high quality headphones. We also measure maximum, undistorted speakerphone volume from a distance of one metre.
Performance
A benchmark test of processor performance and speed. Tests are performed five to ten times successively to check if the devices get warm or even hot. The hottest part of the smartphone on the display or back side is measured with an infrared thermometer.
Features
A performance assessment of additional features that don't fit into other test categories, including voice assistant, IP certification (water/dust resistance) gyroscope, compass and more. This test also involves measuring signal strength and performance of a 4G network and the GPS accuracy.
Ease of use
General ease of use assessment covering day-to-day use such as navigation, making/receiving calls (acoustic test of speech quality), sending texts, using apps, web browsing and so on. The acoustic test is conducted using a dummy head with an artificial ear for consistency across devices.
Functions
General functions test including touch screen feel and responsiveness (e.g. copy/paste) and unlock methods. This includes ease of use of the camera app and transferring files to a computer.
Different testers also describe their ergonomic impressions of the touchscreen when swiping with their fingers, using multitouch gestures (zoom with 2 fingers), kinetic scrolling, swipe writing (if possible), copy and paste functions and, if possible, using the supplied pen. In case of buttons or keypads the lab experts evaluate these solutions as well.
Privacy and security
A substantial assessment of built-in data protection tools and whether the user is able to easily secure, access and delete personal information from the device, such as ease of wiping the drive during a factory reset. Data protection includes brute force, vulnerability and biometric assessments.
Conducting an acoustic test of call clarity in an anechoic chamber.
Test criteria explained
The CHOICE Expert Rating is made up of:
- Camera quality (photo and video) (25%)
- Battery life (15%)
- Display (15%)
- Durability (10%)
- Sound quality (10%)
- Performance (10%)
- Features (5%)
- Ease of use (5%)
- Functions (3%)
- Security (2%)
We're on your side
For more than 60 years, we've been making a difference for Australian consumers. In that time, we've never taken ads or sponsorship.
Instead we're funded by members who value expert reviews and independent product testing.
With no self-interest behind our advice, you don't just buy smarter, you get the answers that you need.
You know without hesitation what's safe for you and your family.
And you'll never be alone when something goes wrong or a business treats you unfairly.
Learn more about CHOICE membership today
Stock images: Getty, unless otherwise stated.
