Prediction Risk Tolerance

When predicting user behavior, there is always a degree of uncertainty that requires a trade-off: you must decide whether to include fewer users in a predicted group for higher overall accuracy, or to include more users for lower overall accuracy.

You tell Predictions how much uncertainty you are willing to tolerate by choosing a risk tolerance level for a prediction. Depending on the prediction and the number of available Analytics events, you might have one or more of the following risk tolerance levels available for a prediction:

  • High Risk Tolerance: This risk tolerance level targets more users than other risk tolerance levels with the lowest level of accuracy. This risk tolerance level is sometimes available when other risk tolerance levels are not.
  • Medium Risk Tolerance: This risk tolerance level targets a moderate number of users with a moderate accuracy level.
  • Low Risk Tolerance: This risk tolerance level targets the minimum number of users with the highest available accuracy level.

When the risk tolerance slider is set to one of these risk tolerance levels for a prediction, the tile for that prediction in the Firebase console shows the following metrics:

  • Percentage of your app's user base who can be targeted with that prediction at that risk tolerance level.
  • Total number of your app's users who can be targeted with that prediction at that risk tolerance level.

Example of Risk Tolerance

Suppose you have an app with 20,000 users, and you want to predict which users will stop using your app (or churn) in the next few days so that you can do something to encourage them to keep using your app.

In the figures below, each face represents 1,000 of your users, with those groups who are satisfied and who won't churn in green, and those who are dissatisfied and who will churn in red.

With high risk tolerance, Predictions might create a group like the one in the figure below, which includes 10,000 of the 13,000 users who are dissatisfied. However, the group also includes 4,000 users who are satisfied, and who you might not want to target in your re-engagement strategy. Because 10,000 of the group's 14,000 users were correctly predicted to be dissatisfied, the targeting accuracy of the group is about 71.4%.

sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied
sentiment_very_satisfied sentiment_very_satisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied
sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied
sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied

On the other hand, the figure below shows what a group created with low risk tolerance might look like. This group contains fewer false matches—only 1,000 users—but also includes 4,000 fewer dissatisfied users than the previous group. In this group, 6,000 of the group's 7,000 users were correctly predicted to be dissatisfied, so the targeting accuracy of the group is about 85.7%.

sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied sentiment_very_satisfied
sentiment_very_satisfied sentiment_very_satisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied
sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied
sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied sentiment_very_dissatisfied

Send feedback about...

Need help? Visit our support page.