After over a decade since Apple released the first iPhone, the billions of smartphones today are still living with the same lithium-ion battery whose capacity has barely improved in the past 15 years. Over the same time span, mobile apps have become far more complex and power-hungry than they used to be. As such, making sure app releases do not have battery glitches (unexpected excessive battery drain) has become a growing concern faced by app vendors, especially after app markets like Google Play have undertaken efforts to downrank apps that drain much battery.
The common practice in the app industry today for testing the battery drain of mobile apps is to run the app on a fully charged phone for an extended period of time, e.g., 10 hours, and watch how much the battery level has dropped. Such manual testing is not only time-consuming and thus slowing down the development cycle or CI workflow, but more importantly treating apps as black boxes without giving insights to the detected battery issues to help developers fix them.
In this blog, we use a popular open source app, Omni-Notes, to show how it takes only minutes to set up automated battery drain testing using Mobile Enerlytics’ battery usage testing solution, called Eagle Tester, to perform automated battery drain testing of each and every app build, to catch and fix energy issues before every app release.
Bug Injection
We set up a battery test for Omni-Notes in 10 minutes on February 28, 2018, and Jenkins triggered battery drain test for every code commit for the next 3 days in the Master branch, totaling 25 commits. We also include tests for the past 5 releases that dates back to November 2017.
We injected a simple “energy bug” – a dummy while loop shown below – into the last commit of the Master Branch in the performToggle() method of the Fab component of app. The method gets called every time the Fab button is clicked. Since the test concentrates on this activity, we injected the bug here.
Bug Detection
Using Eagle Tester to test app battery drain takes 3 easy steps: (1) Write battery drain tests, (2) Configure Eagle Tester in CI (we will use Jenkins as an example), which will drive the tests automatically upon each code commit or app build, (3) Examine the test output in Eagle Tester’s dashboard.
Step 1: Writing a battery drain test for your app
We have discussed in our previous blog how one can adapt an app test (e.g. for performance) to battery drain test by inserting only 2 lines of code (API calls to the Eagle Tester library.)
For the Omni-Notes app, we reused one of the Android Unit Tests written by the developers of the app. Since the test was short, we iterate it 50 times.
@Test
public void fbTest() throws IOException, InterruptedException {
fb = new FabLifecycleTest();
eagleTester.startMeasure("fabActionsTest");
Thread.sleep(5000);
for(int i=0;i<=50;i++){
fb.fabActionsTest(); // the actual test
Thread.sleep(1000);
}
eagleTester.stopMeasure("fabActionsTest");
}
Step 2: Configuring Jenkins to perform Battery Drain tests
Configuring Jenkins to run battery drain tests takes 6 easy steps:
(1) Download and install the Eagle Tester plugin in Jenkins.
(2) Navigate to the Eagle Tester section in Manage Jenkins>Configure System and configure the plugin as shown below. Enter the username and password registered with our website in the below section.
(3) Connect your Git repository to the Jenkins job.
(4) Enable the Eagle Tester plugin inside the job and populate the Package Name field with the package name of the app as shown in the below image.
(5) Issue the commands to run the test in the Execute Shell section of the job.
(6) Build the job.
Step 3: Examining test output
Eagle Tester provides a dashboard that displays all the test results in an intuitive manner.
Upon entering the dashboard, we see the Overview page which gives a quick glance of the battery drain of the Master branch compared to the last Release, the breakdowns of the difference in terms of battery drain tests, phone components, and app threads, and the status of past and ongoing battery drain tests. For the Omni-Notes app, we see the Master branch (most recent commit) incurs 20.2% higher battery drain compared to the last Release, suggesting there is a potential battery drain issue in the Master branch!
To confirm the battery drain hike, we click on the “Release Trend” page which shows the battery drain result across all recent app releases for the above test. We see that the battery drain hike indeed happens with the Master branch.
To drill down the cause of the battery drain hike, we navigate to the Test Analysis page by clicking on Test Analysis on the left panel. The default energy drain breakdown by phone components shows that the GPU energy drain appears stable, while the CPU energy drain has experienced an increase.
To isolate the battery drain hike due to individual components, we unselect the GPU component, by clicking the GPU icon. We see (shown below) that, the two most recent commits, 1c5cf7a and e296a9a, have 84% and 89% higher CPU energy than previous commits, suggesting the CPU increase is due to code changes first introduced in Commit 6ae5aaa. Git-Compare of Commit 1c5cf7a and 6ae5aaa precisely identifies the code change (shown earlier) that contributed to the energy hike.
Takeaway
In this blog, using an open-source app Omni-Notes, we showed the simple steps one needs to follow to set up Eagle Tester in CI to performed continuous, automated Battery Drain testing for each and every code commit and app building, to detect battery drain hikes and isolate the root cause in the app source code.