Join us for Firebase Summit on November 10, 2021. Tune in to learn how Firebase can help you accelerate app development, release with confidence, and scale with ease. Register

Reduce index costs with map fields

This page describes how to use a map field to manage index settings for a group of subfields.

As a best practice, you should remove unused indexes to reduce storage costs and improve write performance. By default, Cloud Firestore builds a single-field index for each field in a document. You can control single-field indexing by defining index exemptions but with a maximum of 200 single-field index exemptions per database. It's possible to reach this limit before disabling all your unused single-field indexes.

You can avoid reaching the exemption limit by grouping document fields with the same index requirements under a map field. You can then apply an index exemption to the map field, and the same exemption applies to the map's subfields.

Solution: Use map fields to help manage indexes

Imagine an app that depends on a collection of game_event documents. Consider the following two data models:

Top-level document fields

Node.js
db.collection('game_events').doc().set({
   timestamp: Firestore.FieldValue.serverTimestamp(),
   user_id: 'huDIl8H88kFAFAdcHayf',
   team_id: '6Q5BhBESeTPk8LT0O59I',
   event_type: 'rare_item_drop',
   display_text: 'You found a rare item!',
});

Map field and subfields

In this data model, all the document fields become subfields of the details field:

Node.js
db.collection('game_events').doc().set({
  details: {
    timestamp: Firestore.FieldValue.serverTimestamp(),
    user_id: 'huDIl8H88kFAFAdcHayf',
    team_id: '6Q5BhBESeTPk8LT0O59I',
    event_type: 'rare_item_drop',
    display_text: 'You found a rare item!',
  }
});

Assume this app always queries game_event documents based on user_id and timestamp or team_id and timestamp. For example:

Node.js
let query_user_events = db.collection('game_events')
                          .where('details.user_id', '==', 'huDIl8H88kFAFAdcHayf')
                          .orderBy('details.timestamp');

let query_team_events = db.collection('game_events')
                          .where('details.team_id', '==', '6Q5BhBESeTPk8LT0O59I')
                          .orderBy('details.timestamp');

Notice the following about this app:

  • The app depends on the composite indexes for details.user_id, timestamp and details.team_id, timestamp.
  • The app does not use the single-field indexes for timestamp, user_id, team_id, event_type, or display_text.

Based on these index requirements, it's a good idea to disable the single-field indexes for timestamp, user_id, team_id, event_type, or display_text. Now, compare the exemptions required for the two data models.

Disabling indexes for top-level fields

To disable the single-field indexes in a top-level fields data model, you must define an exemption for each field. Your exemption count increases by five, and if you add a new field to your data model, you must define another exemption to disable its single-field index.

Disabling indexes for subfields

To disable the single-field indexes for a map and subfields data model, you can define a single exemption for the map field. An exemption on a map field applies the same indexing settings to the map's subfields. If you add a new subfield to the details field, the exemption automatically disables the new subfield's single-field index.

For example, using the Firebase CLI, add this index exemption to your firestore.indexes.json file to disable the single-field indexes for the game_events collection:

{
    "collectionGroup": "game_events",
    "fieldPath": "details",
    "indexes": []
},

If you later require a single-field index for one of the subfields, you can override the map field's index setting with an exemption. An exemption on a subfield overrides that subfield's inherited index settings. For example:

{
    "collectionGroup": "game_events",
    "fieldPath": "details.event_type",
    "indexes": [
      {
        "order": "ASCENDING",
        "queryScope": "COLLECTION"
      },
    ]
},

When to use this approach

In the example above, the map and subfields approach reduced the number of exemptions from five to one. Imagine, however, a similar document data model with two-hundred fields. This approach reduces the number of exemptions from 200 to 1.

You should consider using a map field and subfields approach when your document data model contains multiple fields with unused single-field indexes. You should especially consider this approach for documents with many fields.