Instant messaging apps have quickly evolved from simple niche SMS alternatives into full-featured social platforms and content sharing mediums. According to Statista, the number of mobile phone messaging app users worldwide is projected to grow from 1.6 billion in 2016 to 2.5 billion in 2021 and eMarketer estimated over three-quarters (76.3%) of the world’s smartphone users used an IM app in 2017.
Messaging apps form one of the most diverse app categories, with fierce competition in diverse verticals such as target demographic, geographical regions, and business-or-leisure, and users are blessed with bountiful options of popular IM apps such as WhatsApp, Facebook Messenger, Google Hangouts, Slack, Viber, and WeChat. Since instant messaging apps are one of the most used apps on the phones, their battery drain performance can directly affect the device’s battery life.
Today the common practice in the app industry for testing the battery drain of mobile apps is to run the app on a fully charged phone for an extended period of time, e.g., 10 hours, and monitor how much the battery level has dropped. Such manual testing is not only time-consuming and thus facing the growing mismatch with the ever-shortening app release cycle (e.g. weekly), but also difficult for testing important app features that require user interactions like messaging. In this blog, we discuss how it takes only minutes to set up automated battery drain testing for messaging apps using Mobile Enerlytics’ battery usage testing solution, called Eagle Tester, to perform automated battery drain testing of each and every app build, to effortlessly catch and fix energy issues before every app release.
In a nutshell, Eagle Tester performs automated battery drain testing via seamless integration with major CI platforms such as Jenkins, where battery drain testing is triggered by every code commit and release build. Its intuitive dashboard manages and displays historical battery drain trend of past releases.
Writing a battery drain test for your app takes only minutes
Assume you already use Jenkins or some other CI to perform app tests. To turn such tests into battery tests, simply insert startMeasure(String testName) and then stopMeasure(String testName) at the start and end of your test. Remember to use the same test name for the two calls!
Writing equivalent tests for competing apps is just as easy
To ensure consistency among battery tests of different apps, we create a base test class with as much as possible common functionalities implemented. Then for each new app we want to test we simply extend the parent class and make the minimal amount of changes necessary to run the test.
In the following example, the base class describes the testing sequence and the extended classes only need to implement the sendImage() function (because different apps handle attachments in slightly different ways). This test was very simple: sending 9 text messages, including 2 URLs and one containing emoticons, and one image attachment. All messaging apps then ran the exact same test sequence.
public abstract class MessageAppTest {
void sendTextMessage(String message) throws UiObjectNotFoundException {
messageBox.setText(message);
sendButton.click();
}
protected abstract void sendImage() throws UiObjectNotFoundException;
@Test
public void batteryTest() throws InterruptedException, UiObjectNotFoundException {
eagleTester.startMeasure("messageBatteryTest");
sendTextMessage("Text 1");
sendTextMessage("Text 2");
sendTextMessage("Text 3");
sendTextMessage("Text 4");
sendTextMessage("Text 5");
sendTextMessage("Here is our website: https://mobileenerlytics.com/");
sendTextMessage("Here is a video URL: https://www.youtube.com/watch?v=hNyMD8nFnoc");
sendTextMessage("Here are some emoticons: <3 ");
sendTextMessage("Text 10 (last)");
sendImage();
eagleTester.stopMeasure("messageBatteryTest");
}
@After
public void tearDown() throws InterruptedException {
eagleTester.finish();
}
}
Here is an implementation of a subclass test for Google Hangouts.
public class GoogleHangoutsTest extends MessageAppTest {
@Before
public void setupChat() throws UiObjectNotFoundException, InterruptedException {
super.setupApp("com.google.android.talk");
messageBox = mDevice.findObject(new UiSelector().
text("Write a message"));
sendButton = mDevice.findObject(new UiSelector().
resourceId("com.google.android.talk:id/floating_send_button"));
}
@Override
protected void sendImage() throws UiObjectNotFoundException {
mDevice.findObject(new UiSelector().
resourceId("com.google.android.talk:id/gallerypicker_indicator_icon")).click();
mDevice.findObject(new UiSelector().
resourceId("com.google.android.talk:id/gallery_item_image")).click();
mDevice.findObject(new UiSelector().
resourceId("com.google.android.talk:id/floating_send_button"));
}
}
Full test files are here.
Messaging App Comparison: Results
After writing the extended test classes for 7 of the most popular IM apps: Facebook Messenger, Hike messenger, Skype, Slack, Telegram, WhatsApp, and Google Hangout, to send a same picture, we run batched all the tests with Jenkins and went for coffee, while Jenkins drove the equivalent tests for the 7 IM apps under the same conditions (stable WiFi connection, background apps closed, the same image sent, etc). Back from coffee, all test results were ready in Eagle Tester’s dashboard display. The results of this 10-text, 1-minute long battery drain test for 7 IM apps are shown below.
Summary of Results:
- Mean: 1,425 uAh
- Median: 1,050 uAh (Google Hangouts)
- Max: 3,500 uAh (Skype)
- Min: 415 uAh (Telegram)
Takeaways
It might seem impossible that Skype could take 8.4x as much power as Telegram on the exact same action sequence, and even 3.5x as much as Google Hangouts seems ridiculous. Without going into the battery drain diagnosis capability of Eagle Tester (see the next blog), we already could make some easy inferences from just using the app.
One thing that jumped out: Skype uses many more un-necessary animations to do simple actions that other messaging apps do not. For example:
- A loading icon on every sent message
- Animated emoticons as opposed to still images
- Sliding animation for each new message appearing in the chat history
Conclusion
In this blog, we showed how easy it is to write battery drain tests using Eagle Tester; each test typically captures a potential battery drain scenario and is written once but will be automatically triggered by CI upon every app code commit or release build. We also showed how easy it is to write equivalent battery drain tests to benchmark similar/competing apps. Finally, a simple test showed 7 popular IM apps differ in battery drain by up to 8.4X in performing a common texting sequence!