Testing

Every shipped Android feature survives three test environments: a plain JVM, a JVM with Android shadowed in, and a real or emulated device. Picking the right one for a given assertion is the discipline. This page covers the pyramid - unit / Robolectric / instrumentation - plus the doubles, DI, and screenshot tools that decide how much of the suite sits at the cheap layers. The layer-separation prereq is Architecture; Compose test rule and StateFlow-under-test patterns are touched on here and unpacked in Compose and State Management.

The Android test pyramid

The pyramid is three execution environments stacked by cost, with a "shift left" rule that pushes every test as far down as it can correctly live.

If a behavior can be proven at a lower layer without faking the assertion itself, prove it there. A ViewModel that derives UI state from a repository's Flow is testable at the JVM layer with a fake repo - no device, Activity, or Robolectric needed. Reserve the upper layers for what only they can prove: that an Espresso click() on the FAB navigates, that a Compose Modifier.clickable reacts.

A healthy suite is mostly bottom layer - thousands of fast JVM tests, hundreds of Robolectric, dozens of instrumentation. Inverted pyramids are the "40-minute suite that still misses bugs" smell.

Unit testing

Unit tests run on a regular JVM with src/test/java - the cheapest layer.

JUnit 4 vs JUnit 5 vs Kotest

JUnit 4 is the default - @RunWith(AndroidJUnit4::class) and the instrumentation runner are built around it. JUnit 5 (Jupiter) works in src/test via the de.mannodermaus.android-junit5 plugin, but instrumentation tests need JUnit 4 because androidx.test doesn't speak Jupiter. Most teams stay on JUnit 4 for uniformity.

Kotest is the Kotlin-first alternative - a behavioral DSL (describe / it, should), property-based testing, English-reading matchers. Strong fit for pure-domain modules.

class CartTotalSpec : DescribeSpec({
    it("never goes negative under coupons") {
        forAll<List<Item>, Coupon> { items, coupon ->
            cart(items).applyCoupon(coupon).total() >= 0
        }
    }
})

Truth, AssertK, and Strikt all beat raw assertEquals - they produce diff-style failure messages that point at the wrong field, not a wall of expected Cart(...) but got Cart(...).

Coroutines under test

Suspending code needs a deterministic clock and dispatcher. kotlinx-coroutines-test ships runTest, whose TestDispatcher skips delay instantly and exposes advanceTimeBy(ms) / advanceUntilIdle() for explicit time control.

@Test fun debouncesSearchInput() = runTest {
    val vm = SearchViewModel(fakeRepo, StandardTestDispatcher(testScheduler))
    vm.onQueryChanged("kot")
    advanceTimeBy(299); assertThat(fakeRepo.calls).isEmpty()  // before debounce window
    advanceTimeBy(1); assertThat(fakeRepo.calls).hasSize(1)   // window elapsed
}

The non-obvious requirement: injectable dispatchers. Code that hardcodes Dispatchers.IO can't be redirected to TestDispatcher - the coroutine runs on the real IO pool and runTest's skip-delay doesn't apply. Inject a CoroutineDispatcher; production binds the real one, tests bind the test scheduler.

Robolectric

Robolectric is the middle layer: a JVM-side reimplementation of the Android framework via "shadows" - drop-in replacements for Context, Activity, Resources, SharedPreferences, Handler. Tests annotated with @RunWith(AndroidJUnit4::class) and located in src/test/ run on the JVM but see an Android API surface.

@RunWith(AndroidJUnit4::class)
class PrefsRepositoryTest {
    private val context = ApplicationProvider.getApplicationContext<Context>()
 
    @Test fun writesAndReadsBack() {
        val repo = PrefsRepository(context)            // touches SharedPreferences
        repo.saveToken("abc"); assertThat(repo.token()).isEqualTo("abc")
    }
}

Sweet spot: code that needs Context or Resources but doesn't render or call hardware. SharedPreferences, resource lookups, simple Activity / Fragment lifecycle (Robolectric.buildActivity().create().resume()), Handler / Looper interactions.

Limits: anything depending on a real graphics pipeline, network stack, or hardware. WebView doesn't shadow (requires Chromium); neither do MediaCodec, Camera, GPS, or real sockets. A test hitting them no-ops, throws, or returns null and becomes a fiction. If correctness depends on a real Android version's behavior (a View.draw pixel, a WindowInsets measurement, a Binder round-trip), the test belongs on a device.

Instrumentation tests

Instrumentation tests run inside the app's own process on a real or emulated device. They live in src/androidTest/ and execute under androidx.test.runner.AndroidJUnitRunner - the runner that the Gradle connectedDebugAndroidTest task drives.

Android Test Orchestrator

By default, every test in an androidTest module shares one process. State leaks - a static singleton initialized by test A is still initialized for test B - cause order-dependent flakes. The Android Test Orchestrator runs one test per process: a fresh fork per @Test.

android {
    testOptions { execution = "ANDROIDX_TEST_ORCHESTRATOR" }
    defaultConfig.testInstrumentationRunnerArguments["clearPackageData"] = "true"
}

Forks aren't free, so runs get slower. The benefit is hermeticity: a pass means "in isolation," not "in this order." Past 50 instrumentation tests, the orchestrator pays for itself.

Espresso (Views)

Espresso is the View-system UI test framework. The fluent API matches a view, performs an action, asserts state, and synchronizes implicitly - onView returns only when the UI is idle.

@Test fun loginNavigatesHome() {
    onView(withId(R.id.email)).perform(typeText("a@b.c"))
    onView(withId(R.id.submit)).perform(click())
    onView(withId(R.id.home_title)).check(matches(isDisplayed()))
}

The default idle policy waits for the main Looper to drain and any registered IdlingResource to report idle. Work Espresso can't see (a Dispatchers.IO coroutine, a non-instrumented HTTP client) needs a custom IdlingResource or the test races. Most flakes trace here.

Compose test rule

Compose needs a different harness - there's no View tree to query with withId. createComposeRule() for pure Compose, or createAndroidComposeRule<MyActivity>() when hosted in an Activity, exposes setContent { ... } and a semantics tree.

@get:Rule val compose = createComposeRule()
 
@Test fun toggleFlipsState() {
    compose.setContent { MaterialTheme { ToggleRow(label = "wifi") } }
    compose.onNodeWithText("wifi").assertIsOff().performClick().assertIsOn()
}

Assertions speak the semantics layer (assertIsOn, assertIsDisplayed, assertContentDescriptionEquals), not pixel coordinates. The rule auto-synchronizes with recomposition; compose.waitForIdle() is the escape hatch for animations.

UI Automator is the third instrumentation tool, scoped to cross-app flows - the notification shade, settings, other apps. The only option when a test must leave the app under test and come back.

Test doubles

The Meszaros taxonomy applies cleanly. Stubs return canned values. Fakes are working implementations with shortcuts (an in-memory MutableMap as a database). Mocks verify interactions (verify(api).post(...)) - the only double that asserts a call was made.

For state-heavy collaborators - repositories, data sources, anything backed by Flow - prefer a fake. A fake backed by a real MutableStateFlow behaves like the real one: collectors see updates, the StateFlow deduplicates, combine works. A mocked one needs re-stubbing every Flow on every test, can't model "emit after a delay," and tests what the test set up rather than what the code does.

class FakeFeedRepository : FeedRepository {
    private val items = MutableStateFlow<List<Post>>(emptyList())
    override fun observe(): Flow<List<Post>> = items
    fun emit(posts: List<Post>) { items.value = posts }
}

Mocks earn their place for interaction tests (asserting analytics received track("login_success") is naturally verify) and third-party APIs with final methods where writing a fake means re-implementing a vendor SDK. MockK and Mockito-Kotlin both rewrite bytecode to mock final classes (Kotlin's default final otherwise blocks). Fake by default, mock when you must.

DI for testability

Tests force clean dependency wiring. A class that news its dependencies is untestable; one that takes them as parameters is trivially testable. Hilt formalizes that for Android.

@TestInstallIn(components = [SingletonComponent::class], replaces = [NetworkModule::class])
@Module object FakeNetworkModule {
    @Provides fun api(): FeedApi = FakeFeedApi()
}
 
@HiltAndroidTest
class FeedScreenTest {
    @get:Rule(order = 0) val hilt = HiltAndroidRule(this)
    @get:Rule(order = 1) val compose = createAndroidComposeRule<MainActivity>()
}

@TestInstallIn swaps a production module for a test module across the test process; @UninstallModules is the per-test escape hatch. order = 0 is essential - the Compose rule depends on Hilt being ready before the Activity launches.

Without DI, tests reach for mockkStatic, mockkObject, or PowerMock - all of which trade real wiring for class-loader trickery that fails subtly. Constructor injection plus a test-module swap is the path that scales.

Screenshot tests

Screenshot tests render UI and diff against a stored golden image. They catch the regression class unit tests can't: "padding broke," "dark-mode color wrong," "text overflowed on a small screen." Two architectures dominate.

Paparazzi renders Compose or Views on the JVM using LayoutLib (the renderer Android Studio's preview uses). Tests run at unit-test speed against a fixed device profile and produce PNGs that diff against committed goldens. Trade-off: LayoutLib is a re-implementation, so renderings can disagree with a device for fonts, shadows, and effects. Shot and Roborazzi run instrumentation tests that capture from a real device - slower, but pixels are authoritative.

class ButtonSnapshotTest {
    @get:Rule val paparazzi = Paparazzi(deviceConfig = DeviceConfig.PIXEL_5)
 
    @Test fun primary_button_three_states() {
        paparazzi.snapshot { Column { Button("idle"); Button("pressed"); Button("disabled") } }
    }
}

Prefer screenshots for static visuals with branching state (a button in three states, a card with/without image, a screen in light/dark). Avoid them for animated or interactive surfaces - the snapshot moment is arbitrary and goldens flake. Goldens live in version control; reviewers accept new ones for intentional redesigns.

Hermeticity

A test is hermetic when its outcome depends only on its inputs - not the wall clock, device locale, network, or order. Hermetic tests are repeatable, parallelizable, fast; non-hermetic ones are the source of every "flaky on CI, passes locally" report.

The non-hermetic surfaces on Android, and the wrappers that fix them:

• Wall clock - inject a Clock (java.time.Clock); bind Clock.fixed(...) in tests.
• Locale and timezone - in @Before, Locale.setDefault(Locale.US) and TimeZone.setDefault(TimeZone.getTimeZone("UTC")).
• Random - inject a seeded Random; tests use Random(0).
• Network - real HTTP fails on sealed CI. Use OkHttp's MockWebServer or fake the API client.

Anything that returns a different answer at 3 AM than 3 PM is a hermeticity hazard. Wrap it.

Test orchestration

Gradle has two test targets. ./gradlew test runs the JVM suite (Robolectric included). ./gradlew connectedDebugAndroidTest runs instrumentation against connected devices. CI fans out across an API-level x form-factor matrix.

Gradle Managed Devices turn that matrix from a CI script into a Gradle declaration. Declare a device, the build provisions the emulator, runs the suite, tears down.

testOptions.managedDevices.devices {
    create<ManagedVirtualDevice>("pixel6Api34") {
        device = "Pixel 6"; apiLevel = 34; systemImageSource = "aosp-atd"
    }
}

./gradlew pixel6Api34DebugAndroidTest runs the suite against that emulator. CI matrices that were 200 lines of YAML collapse to one Gradle block plus parallel task invocations.

Mature shape: JVM + Robolectric on every PR (sub-minute), instrumentation on a small device matrix per PR (5-10 min), full matrix nightly. ./gradlew check runs everything plus lint - the merge gate.

See also

• Architecture - layer separation is what makes the bottom of the pyramid possible.
• Compose - the Compose test rule's semantics assertions, setContent, and waitForIdle.
• State Management - StateFlow under test via Turbine (flow.test { awaitItem() }).
• Dependency Injection - @TestInstallIn, @UninstallModules, and HiltAndroidRule.
• Performance and Profiling - Macrobenchmark is the perf-side cousin; runner and orchestrator are shared.

Used in

• Crash Reporter SDK - the breadcrumb buffer is JVM-unit-testable with a fake Clock; the JNI-side native crash handler is the one corner that needs a real device.
• Photo Gallery App - the UploadWorker is testable with WorkManagerTestInitHelper's TestDriver, which advances the WorkManager scheduler manually.
• Messenger App - WebSocket reconnect logic is a JVM-unit-test target with a fake transport; MockWebServer covers the HTTP-side flows.
• Newsfeed App - the LazyVerticalGrid scroll path is a Compose-test-rule target; Paparazzi covers static card variants across light/dark and large-font configs.