Announcing Cloud Firestore (beta): Try the new, scalable, flexible database from Firebase and Google Cloud Platform. Learn more about Cloud Firestore.

Scale your Data

The best way to optimize performance and scale your data in Firebase Realtime Database is to split your data across multiple Realtime Database instances, also known as database sharding. Sharding gives you the flexibility to scale beyond the limits that apply to individual database instances, in addition to load balancing and performance optimization.

When to shard your data

You might want to shard your data across multiple databases if you're using Realtime Database and fit into any of the following scenarios:

  • You want to scale beyond the limit of 100,000 simultaneous connections, 1,000 write operations/second, or any of the other limits for a single database instance.
  • You have multiple, discrete data sets and want to optimize performance (for example, a chat app that serves separate, independent groups of users).
  • You want to balance load across multiple databases to improve uptime and reduce the risk of overloading a single database instance.

How to shard your data

To shard your data, follow these steps (described in greater detail below):

  1. Map your data to multiple databases according to your app's specific needs.
  2. Create multiple Firebase projects, each with its own Realtime Database.
  3. Configure your app so it connects to the Realtime Database instance necessary for each data set.
  4. (Optional) To unify your user base, set up custom authentication and sync users across multiple Firebase projects.

Map your data

When you're mapping your data to multiple databases, try to satisfy the following conditions:

  • Each query only runs against a single database instance. Realtime Database doesn't support queries across database instances.
  • No sharing or duplication of data across database instances (or minimal sharing or duplication).
  • Each app instance only connects to one database at any given moment.

For example, if you build a chat application that serves multiple organizations, you can create a database instance for each organization and store all the chat data in unique database instances. In this case, organization A and organization B don't share data, there isn't any duplicate data in your databases, and you only perform queries against a single database instance. Additionally, users in each organization only connect to their organization's database when they use the chat app.

You can then create several database instances in advance and use the organization's ID to map a team to its database instance. For example, organization A maps to Realtime Database A.

You can also store a map of organization IDs to Firebase project IDs and programmatically look up which database instance corresponds to the connecting client. However, this mapping strategy requires more overhead than the simpler ID-mapping strategy outlined above.

The way you map data for your app depends on your particular use case, but the conditions outlined above can help you define what works for your data.

Create multiple Firebase projects

  1. Create a Firebase project for each Realtime Database instance you need.
  2. Add your iOS, Android, or web apps to each Firebase project, and save the config file for later use. Note the following caveats to the standard process:

    • You don't need the SHA signing certificate for your Android apps.
    • If you access Realtime Database from your server, make sure to also create service accounts for each Firebase project.

Create as many projects as you need. If your project is on the Blaze pricing plan, your total Realtime Database charges are calculated based on your use of each database, so sharding doesn't increase your costs.

Connect your app to multiple database instances

  1. Configure your app for multiple projects.
  2. Store your Firebase project config files:

    • Client-side: Store Firebase config values (app ID, API key, database URL) of all instances in your app and use them to connect to the corresponding instance. This offers better performance, but is less flexible if you need to change your mapping strategy in the future.
    • Server-side: Store your Firebase project configs in your server or using Cloud Functions, and send them back to your client app when necessary. This has more computation and network overhead, but is more flexible if you change your mapping strategy.
  3. Optimize the connections on each database: If each client needs to connect to multiple databases during a session, you can reduce the number of simultaneous connections to each database instance by connecting to each database instance for only as long as is necessary.

For example, if an organization has different teams, and each team has its own database but users can chat across teams, connect user A to database 10 while user A is chatting with team 10, then disconnect from database 10 and connect to database 20 if user A chats with team 20.

Sync users across databases

Use custom tokens to authenticate users and give them access to all the database instances in your app. Each custom token is tied to a user ID and can authenticate users across Firebase projects within your app.

Get more advice

If you need more help sharding your data across multiple database instances, reach out to the Firebase experts on our Slack channel or on Stack Overflow.