Experiment with user retention strategies using Firebase Predictions

You can use Firebase Predictions to identify users who are likely to disengage from your app, and apply a user retention strategy to those users. For example, if you have a mobile game, you could grant a gift of in-game currency to users that are predicted to disengage.

This guide shows you how you can implement this kind of prediction-based user retention strategy, and how you can create an A/B Testing experiment to determine the efficacy of this strategy compared to not granting any gifts and granting gifts to any user who achieves some goal (such as reaching level 2).

Before you begin

Before you can start using predictions to determine your app's user retention policy, you must be using Google Analytics for Firebase in your app. In particular, you must:

  • Enable Analytics data sharing in the Firebase console.
  • Explicitly log any Analytics events relevant to your app that aren't automatically logged. By logging these events, you can improve the quality of future classifications.
  • Have sufficient event data volume for Firebase to make meaningful predictions. Typically, 10,000 monthly active users, 500 positive examples, and 500 negative examples provide Firebase Predictions with enough data.

Start predicting user churn

The first step is to set up your Firebase project to start predicting when your users might disengage.

  1. In the Firebase console, open the Predictions section. If you haven't already agreed to the Predictions terms of service, do so.

    After you have agreed to the terms of service, Predictions will be enabled for your project. The Predictions section of the Firebase console allows you to define custom predictions; however, for churn prediction, you can make use of the built-in prediction, which is based on an aggregation of Analytics events. This predictions will become available around 24 hours after you enable Predictions and have logged some baseline events.

  2. Add a Remote Config parameter that corresponds to the churn prediction:

    1. In the Firebase console, click Remote Config.
    2. Add a new parameter named (for example) will_churn. Create a new condition for this parameter that applies when churn is predicted. Then, set the value for the new condition to true and set the default value to false.

Create a user retention strategy experiment

Next, create an A/B testing experiment that tests three user retention strategies:

  • Never grant gifts (control group)
  • Grant gifts when players achieve some goal, such as reaching level 2
  • Grant gifts when players are predicted to churn

To create the experiment:

  1. In the Firebase console, open the Remote Config section.
  2. Click the A/B testing button.
  3. Create a new experiment:

    1. Choose your app from the list and specify how many of your users you want to include in the experiment. You can also choose to exclude certain user categories, such as high spenders, from the experiment.

    2. Define three variants: one for each user retention strategy.

      Then, create an gift_policy parameter and assign the value gift_never to the control group variant, gift_achievement to the second variant, and gift_likelychurn to the prediction-based variant. Your app uses this parameter to determine whether and when to give a gift to a particular user.

    3. Choose Retention (1 day) from the list of goal metrics, and select any additional metrics you want to track, such as purchase revenue, longer-term retention, and daily engagement.

      Finally, in the Advanced options section, select the gift_policy_set Analytics event, which you explicitly log in your app. Note that this event will not appear in the list until it has been logged once.

Implement your user retention strategies in your app

Now that you have set up your predictions and experiments in the Firebase console, implement the user retention strategies from each of your experiment variants.

  1. Import the Analytics and Remote Config SDKs:

    iOS (Swift)

    Add the SDKs to your Podfile:

    pod 'Firebase/Core'
    pod 'Firebase/RemoteConfig'
    

    Then, import them:

    import Firebase
    

    Android

    compile 'com.google.firebase:firebase-core:11.6.2'
    compile 'com.google.firebase:firebase-config:11.6.2'
    
  2. Retrieve the gift policy for the current user using Remote Config.

    iOS (Swift)

    self.remoteConfig = RemoteConfig.remoteConfig()
    self.remoteConfig.setDefaults(["gift_policy": "gift_never"])
    
    // ...
    
    self.remoteConfig.fetch() { (status, error) -> Void in
        if status == .success {
          self.remoteConfig.activateFetched()
        }
    
        // Act on the retrieved parameters
    
        // Execute the gift policy retrieved with Remote Config
        self.executeGiftPolicy()
    
        // ...
    }
    

    Android

    mFirebaseRemoteConfig = FirebaseRemoteConfig.getInstance();
    
    Map remoteConfigDefaults = new HashMap<String, Object>();
    remoteConfigDefaults.put("gift_policy", "gift_never");
    mFirebaseRemoteConfig.setDefaults(remoteConfigDefaults);
    
    // ...
    
    mFirebaseRemoteConfig.fetch(cacheExpiration)
        .addOnCompleteListener(this, new OnCompleteListener<Void>() {
            @Override
            public void onComplete(@NonNull Task<Void> task) {
                if (task.isSuccessful()) {
                    mFirebaseRemoteConfig.activateFetched();
                }
    
                // Act on the retrieved parameters
    
                // Execute the gift policy retrieved with Remote Config
                executeGiftPolicy();
    
                // ...
            }
        });
    
  3. Finally, execute your app's gift policy based on the parameters you retrieved. Also, at this time, log the gift_policy_set event to indicate that user can be considered to be participating in the experiment.

    iOS (Swift)

    func executeGiftPolicy() {
        let giftPolicy = self.remoteConfig["gift_policy"].stringValue
        let willChurn = self.remoteConfig["will_churn"].booleanValue
    
        if giftPolicy == "gift_achievement" {
            grantGiftOnLevel2()
        } else if (giftPolicy == "gift_likelychurn") && willChurn {
            grantGiftNow()
        }
    
        Analytics.logEvent("gift_policy_set", parameters: nil)
    }
    

    Android

    private void executeGiftPolicy() {
        String giftPolicy = mFirebaseRemoteConfig.getString("gift_policy");
        boolean will_churn = mFirebaseRemoteConfig.getBoolean("will_churn");
    
        if (giftPolicy.equals("gift_achievement") {
          grantGiftOnLevel2();
        } else if (giftPolicy.equals("gift_likelychurn") && will_churn) {
          grantGiftNow();
        }
    
        FirebaseAnalytics.getInstance(this).logEvent("gift_policy_set", null);
    }
    

After a couple of days, you will start to see some results and understand if giving a gift to users impacts retention and you can make a decision based on this data. When you are satisfied with the results, you can roll out the winning behavior to 100% of your users by stopping the experiment and setting the gift_policy parameter to a fixed value.

Send feedback about...

Need help? Visit our support page.