System Design

Feature Flag SDK

mediumFreeAsked at
AmazonMicrosoftStripe
By Rotem Meidan ·

"Design a feature flag SDK for Android, like LaunchDarkly or Firebase Remote Config."

Analyzing the Problem

LaunchDarkly and Firebase Remote Config are feature flag SDKs: a remote on/off switch for already-shipped code, decoupling deploy from release so a feature can be turned on without a new build. The SDK evaluates typed flags (getBoolean, getString, getInt, getJson) from any thread, applies targeting rules against developer-supplied user attributes, computes sticky percentage rollouts deterministically, picks up changes within seconds of the console flipping them, honors a global kill switch, buffers exposure events for A/B analysis, and persists the last-known-good flag set across kill, reboot, and reinstall. Only the client SDK is in frame; the targeting console that decides those values is not.

A flag SDK is a value read from anywhere, at any time, that a server can change underneath you. Three terms name the mechanics that makes that safe:

  • SSE (Server-Sent Events) - a long-lived, one-way HTTP stream. The client opens an ordinary GET and the server holds it open, writing events down it as they happen. One direction only, which is all a push needs, and because it is plain HTTP the proxies and CDNs in the path pass it through untouched. Each event carries an id, so a client that reconnects names the last one it saw and gets only the gap replayed.
  • @Volatile and the snapshot swap - reading a field from many threads without a lock is only safe if the writer's store is guaranteed visible to every later read; otherwise a thread can sit on a stale cached copy indefinitely. Marking a reference @Volatile gives that guarantee. Swap a whole immutable object behind it rather than mutating in place, and every reader sees either the entire old flag set or the entire new one - never a half-applied mix, and never a lock on the read path.
  • Last-known-good cache - flags must answer before the network does, so the last set that worked is kept on disk and restored at launch. The SDK is never blocked on a round trip to be correct; it is only ever more or less fresh.

Why this shape? Because the read is the constraint. getBoolean() is called from a render tree, so it cannot block, cannot throw, and cannot wait for I/O - which forces every expensive thing (network, disk, parsing) off the read path and onto a background one, leaving the reader with nothing but a pointer dereference.

Phase 01

Requirements

Clarifying Questions

QuestionAnswer
Server-evaluated or client-evaluated targeting?Client-evaluated - the SDK runs targeting rules against user attributes on device
Refresh model - push, poll, or both?Both - server can push, client polls as fallback
Exposure tracking for A/B?Yes - record which variant each user saw
Kill switch in scope?Yes - a global switch that disables a flag regardless of targeting

Functional Requirements

  • Evaluate typed flags via getBoolean / getString / getInt / getJson from any thread
  • Apply targeting rules against developer-supplied user attributes
  • Compute sticky percentage rollouts deterministically from user ID + flag key
  • Pick up a flag change within seconds when the server can reach the device, and within minutes when it can't
  • Honor a global kill switch that disables a flag regardless of targeting
  • Buffer and upload exposure events for A/B analysis
  • Persist last-known-good flag set across kill, reboot, and reinstall on user data

Non-Functional Requirements

  • Evaluation costgetBoolean() <10 µs amortised; no I/O, no locks on the hot path
  • Fail-openAny internal exception returns the developer-supplied default; SDK never throws
  • Init costflags.initialize() <3 ms wall-clock; first eval succeeds before first frame
  • Push latency<5 s from console to client while the device is reachable
  • Fallback cadence15 min default; configurable; respects battery constraints
  • Binary sizeCore <300 KB shrunk; no kotlin-reflect
Phase 02

API Design

Public API

Flags.initialize(context, sdkKey = "client-abc123") {
    user("u_123") {                                       // immutable User snapshot
        attribute("plan", "premium")
        attribute("country", "DE")
        attribute("install_age_days", 42)
    }
    bundledDefaults(R.raw.flags_defaults_json)            // R.raw = Android build-time resource bundle; see dd1
    streaming(true)                                       // SSE on; falls back to polling
    pollInterval(15.minutes)
    onUpdate { changed ->                                 // optional callback for live UI refresh
        if ("checkout_v2" in changed) refreshCheckoutScreen()
    }
}
 
// Evaluation - typed, default-required, fail-open.
val newCheckout = Flags.getBoolean("checkout_v2", default = false)
val theme       = Flags.getString("ui_theme", default = "system")
val limit       = Flags.getInt("rate_limit_rpm", default = 60)
val config      = Flags.getJson<PricingConfig>("pricing_config", default = PricingConfig.SAFE)
 
Flags.identify(newUser)                                   // attribute change; re-evaluates locally
Flags.refresh()                                           // force a sync; fire-and-forget

The default argument is also the type witness and the fail-open value - if the SDK can't evaluate (no cache, exception, kill switch), the default is what flows. See the fail-open and bootstrap deep dives below.

Phase 03

High-Level Design

Architecture

The core axis is execution context, one lane per context: init at the top seeds the snapshot, the lock-free eval-time read sits under it, the background refresh lane below that updates it, and the server sits at the bottom. Read and refresh run concurrently around two pieces of shared state - the on-disk cache (the lifetime handoff that lets a cold start evaluate flags before any network call returns) and the in-memory @Volatile snapshot (the cross-thread handoff that lets any thread read without a lock). Both straddle the Eval/Refresh boundary rather than sitting inside either lane, because the contexts that write them are never the context that reads them. Arrows are data flow: flag values move from their source to the eval call site, which passes the user ID on to the bucketer.

The evaluation path reads a single @Volatile FlagSnapshot reference - no locks, no I/O. The refresh path pulls flag sets from the server (SSE push, poller backstop), merges them, and publishes atomically by swapping the reference and persisting to the cache. The cache sits between refresh and eval-time because it is the contract between execution contexts that don't share a process lifetime: refresh rewrites it on every flag change, and a cold-start eval reads it before the network has answered. Bundled defaults seed the snapshot directly at init, never via the cache; the cache restores the last-known-good set on cold start before any network call returns. Each evaluation also records to an ExposureBuffer that is drained by an UploadWorker to /v1/exposures:batch - the same drain pattern as the analytics pipeline, not shown here because it is a side-output, not part of the flag data flow.

Happy-Path - First Launch with Bundled Defaults

  1. Flags.initialize() registers the user snapshot, sets a @Volatile reference to a FlagSnapshot built from R.raw.flags_defaults_json, and returns within ~2 ms. The app is free to render.
  2. On the SDK's background thread, the cache loader reads the last-known-good flag set from DataStore (Jetpack's on-disk key-value store). If present and not expired, it replaces the snapshot reference atomically (AtomicReference.set).
  3. Flags.getBoolean("checkout_v2", false) reads the current snapshot, finds the flag rule, evaluates against the user attributes, applies sticky bucketing if the rule is a percentage rollout, and returns the variant. Total time: ~5 µs.
  4. The evaluation also calls exposureBuffer.record("checkout_v2", "variant_A", user.id); the buffer is drained by an UploadWorker like the analytics pipeline.
  5. The SSE client connects to /v1/flags/stream; the server sends a full eval-set (the server-delivered bundle of flag rules) on connect, then a delta whenever a flag changes.
  6. On delta, FlagSetMerger builds a new immutable FlagSnapshot from the previous + delta, persists it to DataStore, then publishes it via AtomicReference.set.
  7. The onUpdate callback fires with the set of changed keys so UI code can refresh affected screens.
  8. If SSE disconnects, the periodic WorkManager (deferred background scheduler) poller runs every 15 minutes and pulls the full eval-set as a backstop - no flag change is lost for more than the poll interval.
Phase 04

Deep Dives

Scenario

The user opens the app on a plane. No cached flags exist (first launch after reinstall on user data). Every getBoolean() call needs an answer before the first frame renders. Blocking on the network would lock the UI; returning random variants would break determinism.

Three-tier fallback from most-fresh to most-conservative.

Tier 1: server-fresh flag set. If the cache loader found a recent (TTL within 24 h) set in DataStore, the snapshot reference points at it. Used on warm cold starts where the user opened the app before.

Tier 2: stale cache. If the cache exists but is older than the TTL, the SDK still uses it - stale flags beat default-only behaviour for the seconds it takes to reach the server. The exposure event records stale_cache = true so analytics can exclude those from variant comparisons if desired.

Tier 3: bundled defaults. The release engineer ships flags_defaults.json in R.raw at build time. It's a snapshot of every flag and its safe default at the time of build. The SDK loads it from the APK with zero I/O latency (resources are mmap'd by Android). This is what a first-install user gets until the first sync completes.

fun initialize(ctx: Context, sdkKey: String, block: FlagsConfig.() -> Unit) {
    val config = FlagsConfig().apply(block)
    snapshot.set(FlagSnapshot.from(ctx.resources.openRawResource(config.bundledDefaults)))
    scope.launch {
        cacheStore.load()?.let { if (!it.isExpired) snapshot.set(it) }
        streamer.connect()
    }
}

The bundled defaults file is regenerated by a CI job after every release branch is cut, so the defaults stay close to current server state. Stale bundled defaults are not a correctness bug (the kill switch and fail-open semantics protect against them) but they shorten the cold-start divergence window.

Note
Bundled defaults are the contract - if a flag is missing from the bundled JSON, the SDK returns the developer's default argument and the dashboard surfaces the missing key. The developer's default must always be a safe fallback, not the desired new behaviour.
Scenario

getBoolean("checkout_v2", false) is called inside a Compose render tree. An NPE there crashes the whole screen. Any exception path inside the SDK is, transitively, an exception path inside every screen that reads a flag.

BadThrow on missing key

The principled approach: if the flag isn't defined, the developer made a mistake; throw FlagNotFoundException to surface it. Reality: a flag added in the server console but not yet in the client's cache (typical during a rollout) crashes every screen that reads it. The "correct" exception is a production outage.

GoodReturn default on missing key, log a warning

Missing flag → return default. Type mismatch → return default. Now the developer's default argument is essential: it's the fallback when anything goes wrong. The remaining gap is in internal exceptions: a corrupted cache entry, a NumberFormatException inside rule evaluation. Those still propagate.

GreatOuter try/catch wrapping the entire evaluation path

Wrap the whole get*() body in a try/catch that returns default on any throwable, with a debug-builds-only warning log. Internal logic is allowed to throw freely - the boundary catches it. The exception is captured and dispatched to a dedicated error queue that's uploaded with exposures, so the SDK team sees the failure rate without the host app ever crashing.

fun getBoolean(key: String, default: Boolean): Boolean = try {
    val rule = snapshot.get().rule(key) ?: return default     // missing key → default
    val variant = evaluator.evaluate(rule, currentUser)        // may throw on bad rule
    exposureBuffer.record(key, variant.id, currentUser.id)
    variant.asBoolean() ?: default                             // type mismatch → default
} catch (t: Throwable) {
    errorReporter.record(key, t)
    default
}

The catch is intentionally broad - Throwable, not Exception. A StackOverflowError inside rule evaluation must not bring down a render frame.

Note
Fail-open is non-negotiable - any SDK that can throw from a getBoolean() call is one bug away from a global app crash. The discipline is "internal code is allowed to throw; the public boundary catches everything".
Scenario

The user is in the 30% rollout for flag A and the 70% rollout for flag B. Across app restarts, they must see the SAME variant of A (sticky) AND a statistically independent assignment for B (no correlation between flags). Sequential RNG fails both requirements.

Hashing userId alone is the correlated-rollout antipattern. Every flag in a 30% rollout hits the same 30% of users - your experiments are confounded by construction. flagKey in the hash is the one-line fix, and it's the difference between A/B data you can trust and A/B data that tells you nothing.

BadRandom per evaluation

if (Random.nextDouble() &lt; 0.3) variant_A else variant_B. Each call gets a fresh roll - the user sees A on one screen and B on the next. Useless for product decisions; impossible to A/B test.

GoodHash the user ID alone

bucket = hash(userId) % 100. Sticky across restarts and across flags. But flag A's "in rollout" set and flag B's "in rollout" set are the same users at the same percentages - a user in the bottom 30% sees the new variant of every flag in a 30% rollout. Correlated rollouts pollute experiment data.

GreatHash userId + flagKey + salt with MurmurHash3

The bucket is MurmurHash3(userId || flagKey || salt) % 100. The flag key in the hash decorrelates rollouts: the user's bucket for flag A is independent of their bucket for flag B. The salt rotates per environment so dev/staging/prod don't share buckets - useful when tweaking rollout percentages.

object BucketingHasher {
    private const val SEED = 0xC6A4A793.toInt()
 
    fun bucket(userId: String, flagKey: String, salt: String): Int {
        val input = "$userId:$flagKey:$salt".toByteArray()
        val h = murmur3_32(input, SEED)                        // MurmurHash3 32-bit
        return (h.toLong() and 0xFFFFFFFFL).toInt() % 100      // 0..99
    }
}
 
fun applyRollout(rule: PercentageRollout, userId: String): Variant {
    val bucket = BucketingHasher.bucket(userId, rule.key, rule.salt)
    return if (bucket &lt; rule.percentage) rule.treatmentVariant else rule.controlVariant
}

The treatmentVariant is the new behavior being rolled out; the controlVariant is the unchanged baseline a user outside the rollout falls back to. MurmurHash3 is used because it has well-known uniformity properties at 32-bit output, no cryptographic overhead, and a 5-line implementation. SHA-256 would also work but costs an order of magnitude more per evaluation, and we have no security requirement here - just bucket uniformity.

Note
Salt rotation is the experiment-reset switch - when ops decides to re-shuffle the 30% rollout (e.g. to break a confound), they bump the salt and the user set changes entirely without touching the percentage.
Scenario

The ops team flips a flag in the console. Every connected client should respect the change within seconds. Mobile networks drop SSE connections on backgrounding, captive portals, cellular hand-off - so "every connected client" excludes the half that are mid-reconnect.

SSE with exponential reconnect

SSE is the cheapest thing that meets the <5 s kill-switch budget: a WebSocket would buy a back-channel the SDK never uses, and paying for a second protocol to negotiate and keep alive earns nothing here.

OkHttp's EventSource is a thin SSE client. The SDK connects on init, processes event: flag-update messages, and reconnects on error with exponential backoff + jitter. The server uses the Last-Event-ID header (echoed by the SDK on reconnect) to send any updates the client missed during the disconnect window.

val request = Request.Builder()
    .url(streamUrl)
    .header("Last-Event-ID", lastEventId.toString())          // server resumes from here
    .header("X-SDK-Version", sdkVersion)
    .build()
EventSources.createFactory(client).newEventSource(request, object : EventSourceListener() {
    override fun onEvent(es: EventSource, id: String?, type: String?, data: String) {
        lastEventId = id?.toLongOrNull() ?: lastEventId
        merger.applyDelta(FlagDelta.parse(data))
    }
    override fun onFailure(es: EventSource, t: Throwable?, response: Response?) {
        scheduler.scheduleReconnect(backoff.next())            // exponential + jitter
    }
})

Polling as the floor

Even with SSE, a WorkManager job polls /v1/flags/eval-set every 15 minutes. The poll is the floor on update latency when SSE is unreachable (corporate WiFi blocking long-lived connections, captive portal, app backgrounded with battery saver). On Doze/Standby (Android's idle battery-saving modes) the poll cadence drops to the OS-allowed minimum, which is fine - the user isn't seeing the app anyway.

App lifecycle

On ProcessLifecycleOwner (Jetpack's app-wide foreground/background signal) backgrounding, the SDK closes the SSE connection - it's not worth holding a persistent socket while the user isn't in the app, and the OS will tear it down anyway. On foreground, it reconnects immediately with the last event ID. A flag change that fired while backgrounded is delivered as soon as the user returns.

The resume turns on one detail: the client persists only the cursor, never the events. Step through a backgrounding where ops kills a flag while the socket is down, and watch the kill land on reconnect rather than minutes later via the poll.

Note
SSE is a latency optimisation, polling is the correctness floor - the design must work without SSE; SSE just makes propagation feel instant when it succeeds.
Scenario

The cached flag set is the answer the user gets on cold start before any network call returns. A corrupted cache, a partially-written cache, or a cache from a newer SDK version that the current client can't parse must not crash the SDK.

Atomic cache writes

The cache is a single JSON file stored via Jetpack DataStore. DataStore handles atomicity (write-to-temp + rename) and concurrent reads/writes through its serializer interface. A partial write on power loss is rolled back to the previous good state.

Schema versioning per cache record

Every cache entry carries a schemaVersion. On load, if the version is newer than the SDK knows, the SDK discards the cache and reverts to bundled defaults until a fresh sync. Older versions go through a migrator (v1 → v2 → v3). The migrator is in the SDK because the cache is on-device; the server's eval-set always speaks the version the SDK declares in its sync request.

Corruption tolerance

The cache load is wrapped in the same try/catch pattern as evaluation. A JsonParseException returns "no cache" rather than crashing init. The error is buffered like any other internal failure and ships in the next exposure batch.

suspend fun loadOrNull(): FlagSnapshot? = try {
    val raw = dataStore.data.first()                          // suspends; first emission
    when (raw.schemaVersion) {
        SCHEMA_V3 -> FlagSnapshot.fromV3(raw)
        SCHEMA_V2 -> migrateV2(raw)
        else      -> { errorReporter.record("cache_unknown_version"); null }
    }
} catch (t: Throwable) {
    errorReporter.record("cache_load_failed", t); null         // fall back to bundled defaults
}
Note
The cache is an optimisation, never a correctness dependency - if it's missing, broken, or schema-mismatched, the SDK falls back to bundled defaults without raising. The developer's default argument is the ultimate floor.

Tradeoffs

DecisionChosenAlternativeWhy
Evaluation locationClient-side from delivered rulesServer-evaluated per requestLatency at eval time is non-negotiable; offline tolerance is free
BootstrapBundled defaults + cache + freshDefault arg onlyFirst launch behaviour matters; bundled defaults bridge the gap
Failure modelFail-open with broad catchThrow on internal errorRender-tree calls can't throw; default arg is the floor
Bucketing hashMurmurHash3(userId+flagKey+salt)Hash userId onlyDecorrelates rollouts across flags
Refresh transportSSE + 15 min polling floorPolling onlySSE is the latency win; polling is the correctness floor
Kill orderingKill checked before targeting and overridesKill as a special targeting ruleOps needs one button that beats everything
Exposure dedupePer-session, in-memoryNo dedupeChatty flags would otherwise dominate upload volume
Cache schemaVersioned with migratorHard-code current schemaSDK versions skew across the install base; cache must tolerate

Done reading? Mark it so it sticks in your dashboard.

Next in System DesignAnalytics / Logging SDK

Discussion (0)

Sign in to join the discussion.
No comments yet. Start the discussion.