Experiment with promotion strategies using Firebase Predictions

With Firebase Predictions, you can optimize your in-app promotions for each of your users based on the likelihood of the user making an in-app purchase. For example, you can promote your more expensive "premium" bundle to users likely to spend, and promote your less expensive basic bundle to other users.

Then, using Firebase A/B Testing, you can conduct an experiment to see how this prediction-based strategy affects in-app purchase revenue compared to simply always promoting the basic bundle or always offering the premium bundle.

Before you begin

Before you can start using predictions to determine your app's promotion strategy, you must be using Google Analytics for Firebase in your app. In particular, you must:

  • Enable Analytics data sharing in the Firebase console.
  • Explicitly log spending-related events that are not automatically collected, such as ecommerce_purchase. (Firebase automatically logs the in_app_purchase event for in-app purchases processed by the App Store and Play Store.) You should also generally log any other events that are relevant to your app, to maximize the data available to make classifications.
  • Have sufficient event data volume for Firebase to make meaningful predictions. Typically, 10,000 monthly active users, 500 positive examples, and 500 negative examples provide Firebase Predictions with enough data.

Make your promotions remotely configurable

First, set up your app to promote either your basic or premium bundle depending on the value of a Remote Config parameter. In the steps that follow, you will configure Predictions and A/B Testing to set the parameter remotely based on various criteria, but for now, your app will always promote the basic bundle.

For example:

  1. Import the Analytics and Remote Config SDKs:

    iOS (Swift)

    Add the SDKs to your Podfile:

    pod 'Firebase/Core'
    pod 'Firebase/RemoteConfig'
    

    Then, import them:

    import Firebase
    

    Android

    implementation 'com.google.firebase:firebase-core:16.0.4'
    implementation 'com.google.firebase:firebase-config:16.1.0'
    
  2. Initialize Remote Config and set the default value of the promoted_bundle variable to basic. Later, you will set up Predictions and A/B Testing to set this variable remotely, but by setting a default value in your app, you ensure that your app functions correctly before remotely-configured variables are retrieved.

    iOS (Swift)

    self.remoteConfig = RemoteConfig.remoteConfig()
    self.remoteConfig.setDefaults(["promoted_bundle": "basic"])
    

    Android

    final FirebaseRemoteConfig config = FirebaseRemoteConfig.getInstance();
    
    Map remoteConfigDefaults = new HashMap<String, Object>();
    remoteConfigDefaults.put("promoted_bundle", "basic");
    config.setDefaults(remoteConfigDefaults);
    
  3. Promote the bundle specified by the value of promoted_bundle. Until you set up Predictions, this value will always be basic.

    iOS (Swift)

    self.remoteConfig.fetch() { (status, error) -> Void in
        if status == .success {
          self.remoteConfig.activateFetched()
        }
    
        // Act on the retrieved parameters
    
        // Set the bundle to promote based on parameters retrieved with Remote
        // Config. This depends entirely on your app, but, for example, you might
        // retrieve and use image assets based on the specified bundle name.
        self.promotedBundle = self.remoteConfig.getString("promoted_bundle")
    
        // ...
    }
    

    Android

    config.fetch(cacheExpiration)
            .addOnCompleteListener(this, new OnCompleteListener<Void>() {
                @Override
                public void onComplete(@NonNull Task<Void> task) {
                    if (task.isSuccessful()) {
                        config.activateFetched();
                    }
    
                    // Act on the retrieved parameters
    
                    // Set the bundle to promote based on parameters retrieved with
                    // Remote Config. This depends entirely on your app, but for
                    // example, you might retrieve and use image assets based on the
                    // specified bundle name.
                    mPromotedBundle = config.getString("promoted_bundle");
                    // ...
                }
            });
    

At this point, you can deploy your app to the App Store or Play Store. Users will get the default promotions behavior (always promote the basic bundle), but because promotions are controlled by a remotely configurable variable, you can change and experiment with promotion strategies using only the Firebase console, without having to push app updates to your users.

Start predicting user spending

Next, set up your Firebase project to start predicting your users' spending behavior.

In the Firebase console, open the Predictions section. If you haven't already agreed to the Predictions terms of service, do so.

After you have agreed to the terms of service, Predictions will be enabled for your project. The Predictions section of the Firebase console allows you to define custom predictions; however, for spending predictions, you can make use of the built-in spend and not_spend predictions, which use an aggregation of Analytics events to predict whether a user will make an in-app purchase or an ecommerce purchase (subscription purchases are not currently considered in this prediction). These predictions will become available after you enable Predictions and have logged a few weeks' worth of spending events.

Create a promotions strategy experiment

Now create an A/B Testing experiment that tests the effect of the prediction-based promotions strategy on revenue compared to always promoting the basic bundle and always promoting the premium bundle.

It is beneficial for an experiment to have three variants—a control group, a group that unconditionally sets a parameter, and a prediction-based group—so the experiment can determine whether the prediction-based approach has a greater benefit than from simply applying the treatment to everyone. However, an experiment with three variants requires more participants for enough data to be collected. If your app doesn't have enough users to support running an experiment with three variants, it can still be okay to have an experiment with two variants, omitting the experiment where the treatment is applied to everyone.

To create the experiment:

  1. In the Firebase console, open the A/B Testing section.

  2. Create a new experiment:

    1. Click Create experiment > Remote Config.

    2. Choose your app from the list and specify how many of your users you want to include in the experiment. You can also choose to exclude certain user categories, such as high spenders, from the experiment.

    3. Choose Purchase revenue from the list of goal metrics, and select any additional metrics you want to track, such as user engagement, retention, app removals, and ad clicks.

    4. Define three variants:

      • Always promote the basic bundle (control group)
      • Always promote the premium bundle
      • Promote the basic or premium bundle depending on the spending prediction

      For the Control group, create a promoted_bundle parameter and set it to (no change). Users assigned to the control group will get the default behavior, which is to always see the basic promotion.

      For the Always promote premium variant, set promoted_bundle parameter to premium. Users assigned to this variant always see the premium promotion.

      For the Promote premium to likely spenders variant, set promoted_bundle parameter to premium and set the user subset to Predict: spend (medium risk tolerance). Users assigned to this variant see the premium promotion if they are predicted to spend.

Start the experiment and let it run for several days or more, until A/B Testing declares a leader. If the experiment cannot determine a leader, you might need to expand the experiment to more users.

Roll out the winning variant to all users

After A/B Testing has collected enough information to declare a leader—in this case, the variant that maximized purchase revenue—you can decide whether to roll out the winning variant (or another variant) to all of your users.

In the A/B Testing section of the Firebase console, open the details view of the completed experiment. From this view, you can see how each variant performed according to your goal metric and any secondary metrics you selected. With this information, you can decide whether to roll out the leading variant or another variant.

To roll out a variant to all users, click more_vert > Roll out the leader on the experiment's details page.

Оставить отзыв о...

Текущей странице
Нужна помощь? Обратитесь в службу поддержки.