Experiment with user retention strategies using Firebase Predictions

You can use Firebase Predictions to identify users who are likely to disengage from your app, and apply a user retention strategy to those users. For example, if you have a mobile game, you could grant a gift of in-game currency to users that are predicted to disengage.

This guide shows you how you can implement this kind of prediction-based user retention strategy, and how you can create an Firebase A/B Testing experiment to determine the efficacy of this strategy compared to not granting any gifts.

Before you begin

Before you can start using predictions to determine your app's user retention policy, you must be using Google Analytics for Firebase in your app. In particular, you must:

  • Enable Analytics data sharing in the Firebase console.
  • Explicitly log any Analytics events relevant to your app that aren't automatically logged. By logging these events, you can improve the quality of future classifications.
  • Have sufficient event data volume for Firebase to make meaningful predictions. Typically, 10,000 monthly active users, 500 positive examples, and 500 negative examples provide Firebase Predictions with enough data.

Make your gift policy remotely configurable

First, set up your app to grant gifts to players depending on the value of a Remote Config parameter. Because you only want to grant this gift once, you should also track whether a player has already received the gift.

In the steps that follow, you will configure Predictions and A/B Testing to set the parameter remotely based on various criteria, but for now, your app won't grant any gifts.

For example:

  1. Import the Analytics and Remote Config SDKs:

    iOS (Swift)

    Add the SDKs to your Podfile:

    pod 'Firebase/Core'
    pod 'Firebase/RemoteConfig'
    

    Then, import them:

    import Firebase
    

    Android

    implementation 'com.google.firebase:firebase-core:16.0.6'
    implementation 'com.google.firebase:firebase-config:16.1.2'
    
  2. Initialize Remote Config and set the default value of the grant_retention_gift variable to false. Later, you will set up Predictions and A/B Testing to set this variable remotely, but by setting a default value in your app, you ensure that your app functions correctly before remotely-configured variables are retrieved.

    iOS (Swift)

    self.remoteConfig = RemoteConfig.remoteConfig()
    self.remoteConfig.setDefaults(["grant_retention_gift": false])
    
    // ...
    
    self.remoteConfig.fetch() { (status, error) -> Void in
        if status == .success {
          self.remoteConfig.activateFetched()
        }
    
        // Act on the retrieved parameters. For example, grant the retention gift to
        // players who haven't yet received one.
        let shouldGrantGift = self.remoteConfig["grant_retention_gift"].booleanValue
        if shouldGrantGift && !playerAlreadyReceivedGift() {
            grantGiftToPlayer()
        }
    }
    

    Java
    Android

    final FirebaseRemoteConfig config = FirebaseRemoteConfig.getInstance();
    
    Map<String, Object> remoteConfigDefaults = new HashMap<>();
    remoteConfigDefaults.put("grant_retention_gift", false);
    config.setDefaults(remoteConfigDefaults);
    
    // ...
    
    config.fetch(CACHE_EXPIRATION)
            .addOnCompleteListener(this, new OnCompleteListener<Void>() {
                @Override
                public void onComplete(@NonNull Task<Void> task) {
                    if (task.isSuccessful()) {
                        config.activateFetched();
                    }
    
                    // Act on the retrieved parameters. For example, grant the
                    // retention gift to players who haven't yet received one.
                    boolean shouldGrantGift = config.getBoolean("grant_retention_gift");
                    if (shouldGrantGift && !playerAlreadyReceivedGift()) {
                        grantGiftToPlayer();
                    }
                }
            });

    Kotlin
    Android

    val config = FirebaseRemoteConfig.getInstance()
    
    val remoteConfigDefaults = HashMap<String, Any>()
    remoteConfigDefaults["grant_retention_gift"] = false
    config.setDefaults(remoteConfigDefaults)
    
    // ...
    
    config.fetch(CACHE_EXPIRATION)
            .addOnCompleteListener(this) { task ->
                if (task.isSuccessful) {
                    config.activateFetched()
                }
    
                // Act on the retrieved parameters. For example, grant the
                // retention gift to players who haven't yet received one.
                val shouldGrantGift = config.getBoolean("grant_retention_gift")
                if (shouldGrantGift && !playerAlreadyReceivedGift()) {
                    grantGiftToPlayer()
                }
            }

At this point, you can deploy your game to the App Store or Play Store. For now, players will get the default behavior, which is to receive no retention gift, but because the gift policy is controlled by a remotely configurable variable, you can change and experiment with retention strategies using only the Firebase console, without having to push app updates to your users.

Start predicting user churn

Next, set up your Firebase project to start predicting when your users might disengage:

In the Firebase console, open the Predictions section. If you haven't already agreed to the Predictions terms of service, do so.

After you have agreed to the terms of service, Predictions will be enabled for your project. The Predictions section of the Firebase console allows you to define custom predictions; however, to predict churn, you can make use of the built-in prediction, which uses an aggregation of Analytics events to predict whether a user is about to stop using your app. These predictions will become available after you enable Predictions and have logged a few weeks' worth of Analytics events.

Create a user retention strategy experiment

Next, create an A/B Testing experiment that tests the effectiveness of the gift on user retention:

To create the experiment:

  1. In the Firebase console, open the A/B Testing section.

  2. Create a new experiment:

    1. Click Create experiment > Remote Config.

    2. Choose your app from the list and specify how many of your users you want to include in the experiment. You can also choose to exclude certain user categories, such as high spenders, from the experiment.

    3. Choose Retention (2-3 day) from the list of goal metrics, and select any additional metrics you want to track, such as user engagement, app removals, and ad clicks.

    4. Define two variants:

      • No retention gift (control group)
      • Grant gift to churners

      For the Control group, create a grant_retention_gift parameter and set it to (no change). Users assigned to the control group will get the default behavior, which is to not receive a gift.

      For the Grant gift to churners variant, set the grant_retention_gift parameter to true and set the user subset to Predict: churn (medium risk tolerance). Users assigned to this variant get a gift if they are predicted to churn.

Start the experiment and let it run for several days or more, until A/B Testing declares a leader. If the experiment cannot determine a leader, you might need to expand the experiment to more users.

Roll out the winning variant to all users

After A/B Testing has collected enough information to declare a leader—in this case, the variant that maximized retention—you can decide whether to roll out the winning variant (or another variant) to all of your users.

In the A/B Testing section of the Firebase console, open the details view of the completed experiment. From this view, you can see how each variant performed according to your goal metric and any secondary metrics you selected. With this information, you can decide whether to roll out the leading variant or another variant.

To roll out a variant to all users, click more_vert > Roll out the leader on the experiment's details page.

Send feedback about...

Need help? Visit our support page.