An Experimental Approach to Writing Unit Tests (using the Android Notepad Sample)

In our 20% time, a few of us at Google (but not on the Android team) have been working on the problem of how to write unit tests for Android.  After some experimentation, we've come up with the approach described below.  We've found it to work well for us; take a look and let us know your thoughts (you can leave comments here).  Note that this approach uses the Java JVM--not Android's Dalvik VM.  We explain the reasons for this decision below.

Of course, functional tests are important too, so while we were at it we wrote some doc on How to Write Functional Tests (with the Android Notepad Sample).

Why write unit tests?

Small, isolated unit tests can help you thoroughly test edge conditions in your code.  Writing small unit tests can also help you write your code so that dependencies are explicit and limited, which improves its readability and flexibility.

What are the problems?

There are several obstacles to writing small tests for Android:
  • The Dalvik VM has limited support for mocking.  For example, it is possible to run the EasyMock jar on Dalvik, but the class extension module doesn't run because it has dependencies on the JVM class loader.  This means you can only mock interfaces--and not classes--using EasyMock.  (The Android API provides "mock" versions of system classes, such as Activity and Context, but these are actually "stubs" rather than true mocks--they are not able to verify behavior.)
  • Some aspects of the Android API make it difficult to create mocks regardless of which VM is used, because many methods are declared final, making them impossible to mock through subclassing.  And much of Android's functionality is presented through static functions, which are also difficult to mock.
  • If you want to run on the JVM, you can link against the provided android.jar file, but the implementation of every method in that .jar file simply throws an exception.  Even if you were able to mock the classes in the .jar file, any of your application's classes which extend an Android class would still wind up calling methods in android.jar and throwing an exception.

What are the solutions?       

Until the problems above are resolved, we recommend the following workarounds.  These are meant to be adopted together.

1) Test on your development machine, using the JVM.  This gives you the benefit of using standard mock and test libraries with the JVM.  

It also provides the following additional benefits:
  • Faster startup time
  • Code coverage utilities
However, this approach will not find problems due to differences between the Dalvik VM and the JVM.  In practice (with programs of several thousand lines of code), we haven't discovered any significant differences between the two VMs, and we would consider any difference to be a bug in one or the other VM.

We don't recommend that you do performance testing with the JVM.

2) Use a mocking framework that supports mocking statics and finals, such as PowerMock, jmockit, or jeasytest.  (This document relies on PowerMock, because that's the framework we are familiar with).

An alternative is to create a thin layer of java interfaces around API classes which contain static or final items; you can then code against this layer and mock it out with EasyMock.  This is the approach we originally took, but it's a significant amount of work, and may also result in a size and speed penalty on a platform that is already severely constrained.

Further reading about mocking statics and mocking finals.

3) Use a version of android.jar which does not throw exceptions.  We've built one here (based on decompiling android.jar and manually removing the exceptions); we're looking into a more robust and automated way to create this file.  This is only necessary when you need to test code that is a subclass of an API.  

In part 2 of this tutorial (IN PROGRESS), we'll rework the example to use a delegate, and remove the need for a custom android.jar.


In Eclipse, create a new project specifically for the unit tests
  • File / New / Java Project
    Note that this is Java project, not an Android project.
  • File / Properties / Java Build Path / Projects, and add the project that you want to test.

Add the PowerMock test libraries
  • Download powermock libraries
    http://code.google.com/p/powermock/downloads/list:  download "powermock-1.2.5-with-dependencies.zip".  
  • Unzip the files.
  • Add all the unzipped jars to your project: 
    File / properties / java build path / libraries / Add External Jars...  Select all the jar files that you have unzipped.
    (You also need to add android.jar as an external jar.)
See this note if you're using ant.

Write a test

We'll write a test for method onCreateOptionsMenu()in the class NotepadV1:

    class NotepadV1 {

@Override

public boolean onCreateOptionsMenu(Menu menu) {

    boolean result = super.onCreateOptionsMenu(menu);

    menu.add(0, INSERT_ID, 0, R.string.menu_insert);

    return result;

}

...

    }


  • Create a new Junit test Case:  File / New / JUnit test case.  Select JUnit 4
    Put the test in the same package as the code you are testing.
  • Create a new class named NotepadV1Test.java.  
  • Write the test code.
If you've used a mocking library before, the test itself should look familiar.  Note that when using PowerMock, you often call the underlying mocking framework of your choice, in this case, EasyMock.

    @RunWith(PowerMockRunner.class)   // Enable PowerMock's replayAll() and verifyAll() methods.

    class NotepadV1UnitTest {

      @Test

public void testOnCreateOptionsMenu() {

    Menu menu = PowerMock.createMock(Menu.class);

    MenuItem menuItem = PowerMock.createMock(MenuItem.class);


    EasyMock.expect(menu.add(0, Notepadv1.INSERT_ID, 0, R.string.menu_insert))

                     .andReturn(menuItem);


    PowerMock.replayAll();

    Notepadv1 notepadv1 = new Notepadv1();

    assertTrue(notepadv1.onCreateOptionsMenu(menu));

    PowerMock.verifyAll();

}

    }

  • Run your test: Run > Run As JUnit Test.

About PowerMock

PowerMock extends the EasyMock, JMock, and Mockito frameworks (the download includes all of those libraries).  PowerMock adds some new functions, but it also replaces some existing functions.  For example, here we use PowerMock's createMock instead of EasyMock in order to get support for replayAll() and verifyAll().

Because of this overloading, we do not recommend the use of the * wildcard in import statements, so that it's clear which version of the function you are using.  (With the exception of import org.junit.Assert.*, since those methods are unambiguous.)

What's next?





Subpages (1): Testing onCreate
ċ
android.jar
(2100k)
Stephen Ng,
Jul 8, 2009, 9:12 AM
Comments