> [!NOTE]
> **Note:** For additional Pipelines setup instructions and examples for client server libraries, refer to the [Cloud Pipelines documentation](https://cloud.google.com/firestore/docs/pipeline/overview).

## Background

Pipeline operations provide a new query interface for
Cloud Firestore that supports advanced query functionality and complex
expressions. It introduces many new functions, including `min(...)`, `max(...)`,
`substring(...)`, `regex_match(...)`, and `array_contains_all(...)`, and stages
to be able to perform complex transformations.

## Getting Started

To install and initialize client SDKs, refer to the instructions in the
following guides:

- [Get started with mobile and web SDKs](https://firebase.google.com/docs/firestore/enterprise/quickstart.md.txt#initialize).
- [Get started with server client libraries](https://firebase.google.com/docs/firestore/enterprise/quickstart-server.md.txt#initialize).

## Syntax

The following sections give an overview of the syntax for
Pipeline operations.

### Concepts

One notable difference with Pipeline operations is the
introduction of explicit "stage" ordering. This makes it possible to express
more complex queries. However, it is a notable deviation from the existing query
interface using Core operations where the ordering of stages was
implied. Consider the following Pipeline operations example:

### Web

```javascript
const pipeline = db.pipeline()
  // Step 1: Start a query with collection scope
  .collection("cities")
  // Step 2: Filter the collection
  .where(field("population").greaterThan(100000))
  // Step 3: Sort the remaining documents
  .sort(field("name").ascending())
  // Step 4: Return the top 10. Note applying the limit earlier in the
  // pipeline would have unintentional results.
  .limit(10);
```

##### Swift

```swift
let pipeline = db.pipeline()
  // Step 1: Start a query with collection scope
  .collection("cities")
  // Step 2: Filter the collection
  .where(Field("population").greaterThan(100000))
  // Step 3: Sort the remaining documents
  .sort([Field("name").ascending()])
  // Step 4: Return the top 10. Note applying the limit earlier in the pipeline would have
  // unintentional results.
  .limit(10)
```

### Kotlin

```kotlin
val pipeline = db.pipeline()
    // Step 1: Start a query with collection scope
    .collection("cities")
    // Step 2: Filter the collection
    .where(field("population").greaterThan(100000))
    // Step 3: Sort the remaining documents
    .sort(field("name").ascending())
    // Step 4: Return the top 10. Note applying the limit earlier in the pipeline would have
    // unintentional results.
    .limit(10)
```

### Java

```java
Pipeline pipeline = db.pipeline()
    // Step 1: Start a query with collection scope
    .collection("cities")
    // Step 2: Filter the collection
    .where(field("population").greaterThan(100000))
    // Step 3: Sort the remaining documents
    .sort(field("name").ascending())
    // Step 4: Return the top 10. Note applying the limit earlier in the pipeline would have
    // unintentional results.
    .limit(10);
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

pipeline = (
    client.pipeline()
    .collection("cities")
    .where(Field.of("population").greater_than(100_000))
    .sort(Field.of("name").ascending())
    .limit(10)
)
```

##### Go

```go
pipeline := client.Pipeline().
	Collection("cities").
	Where(firestore.FieldOf("population").GreaterThan(100000)).
	Sort(firestore.Orders(firestore.Ascending(firestore.FieldOf("name")))).
	Limit(10)
```

### Initialization

Pipeline operations have a very familiar syntax coming
from existing Cloud Firestore queries. To get started, you initialize a
query by writing the following:

### Web

```javascript
const { getFirestore } = require("firebase/firestore");
const { execute } = require("firebase/firestore/pipelines");
const database = getFirestore(app, "enterprise");
const pipeline = database.pipeline();
```

##### Swift

```swift
let firestore = Firestore.firestore(database: "enterprise")
let pipeline = firestore.pipeline()
```

### Kotlin

```kotlin
val firestore = Firebase.firestore("enterprise")
val pipeline = firestore.pipeline()
```

### Java

```java
FirebaseFirestore firestore = FirebaseFirestore.getInstance("enterprise");
PipelineSource pipeline = firestore.pipeline();
```

##### Python

```python
firestore_client = firestore.client(default_app, "your-new-enterprise-database")
pipeline = firestore_client.pipeline()
```

##### Go

```go
client, err := firestore.NewClientWithDatabase(ctx, projectID, databaseID)
if err != nil {
	fmt.Fprintf(w, "firestore.NewClientWithDatabase failed: %v", err)
	return err
}
defer client.Close()
pipeline := client.Pipeline().Collection("books")
```

### Structure

There are a few terms that are important to understand when creating
Pipeline operations: stages, expressions, and functions and
subquery wrappers.

![Example demonstrating stages and expressions in a query](https://docs.cloud.google.com//firestore/native/docs/images/pipeline-queries/pipeline_query_stage_expression.jpg)

**Stages:** A pipeline may consist of one or more stages. Logically, these
represent the series of steps (or stages) taken to execute the query.

> [!NOTE]
> **Note:** In practice, stages may be executed out of order to improve performance. However, this does not modify the intent or correctness of the query.

**Expressions:** Stages will often accept an expression allowing you to express
more complex queries. Expression may be simple and consist of a single function
like `eq("a", 1)`. You can also express more complex expressions by nesting
expressions like `and(eq("a", 1), eq("b", 2)).`

**Subquery Wrappers:** Functions like `array()` and `scalar()` allow you to
embed a nested pipeline as an expression within a stage.

### Fields / Constants / Variables

Pipeline operations support complex expressions. As such, it is
important to differentiate whether a value represents a **field** , a
**constant** , or a **variable**.

While **fields** refer to data within documents, and **constants** allow
specifying any value as an argument to an expression, **variables** allow
defining \& using temporary values that are scoped to the query execution rather
than the documents being processed. The following provides an overview of these
concepts, see the [`let(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/let.md.txt) stage for more information on
how to read \& write **variables** during query execution.

|   | Fields | Constants | Variables |
|---|---|---|---|
| **Purpose** | access or store fields into documents | specify a fixed value | use temporary values during pipeline execution |
| **SDK Usage** | `field("name")` | `constant("val")` | `variable("name")` |
| **Scope** | local to current document | global | global to pipeline and sub-pipelines |
| **Undefined Reference** | evaluates to `absent` | N/A | generates runtime error |

**Examples:**

### Web

```javascript
const pipeline = db.pipeline()
  .collection("cities")
  .where(field("name").equal(constant("Toronto")));
```

##### Swift

```swift
let pipeline = db.pipeline()
  .collection("cities")
  .where(Field("name").equal(Constant("Toronto")))
```

### Kotlin

```kotlin
val pipeline = db.pipeline()
    .collection("cities")
    .where(field("name").equal(constant("Toronto")))
```

### Java

```java
Pipeline pipeline = db.pipeline()
    .collection("cities")
    .where(field("name").equal(constant("Toronto")));
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field, Constant

pipeline = (
    client.pipeline()
    .collection("cities")
    .where(Field.of("name").equal(Constant.of("Toronto")))
)
```

##### Go

```go
pipeline := client.Pipeline().Collection("cities").
	Where(firestore.FieldOf("name").Equal(firestore.ConstantOf("Toronto")))
```

## Stages

### Input Stages

The input stage represents the first stage of a query. It defines the initial
set of documents you are querying over. For Pipeline operations,
this largely is similar to existing queries, where most queries start with
either a [`collection(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/input/collection.md.txt) or
[`collection_group(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/input/collection-group.md.txt) stage. Two new input
stages are [`database()`](https://firebase.google.com/docs/firestore/pipelines/stages/input/database.md.txt) and
[`documents(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/input/documents.md.txt) where
[`database()`](https://firebase.google.com/docs/firestore/pipelines/stages/input/database.md.txt) allows returning **all**
documents in the database, while [`documents(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/input/documents.md.txt)
acts identical to a batch read.

### Web

```javascript
let results;

// Return all restaurants in San Francisco
results = await execute(db.pipeline().collection("cities/sf/restaurants"));

// Return all restaurants
results = await execute(db.pipeline().collectionGroup("restaurants"));

// Return all documents across all collections in the database (the entire database)
results = await execute(db.pipeline().database());

// Batch read of 3 documents
results = await execute(db.pipeline().documents([
  doc(db, "cities", "SF"),
  doc(db, "cities", "DC"),
  doc(db, "cities", "NY")
]));
```

##### Swift

```swift
var results: Pipeline.Snapshot

// Return all restaurants in San Francisco
results = try await db.pipeline().collection("cities/sf/restaurants").execute()

// Return all restaurants
results = try await db.pipeline().collectionGroup("restaurants").execute()

// Return all documents across all collections in the database (the entire database)
results = try await db.pipeline().database().execute()

// Batch read of 3 documents
results = try await db.pipeline().documents([
  db.collection("cities").document("SF"),
  db.collection("cities").document("DC"),
  db.collection("cities").document("NY")
]).execute()
```

### Kotlin

```kotlin
var results: Task<Pipeline.Snapshot>

// Return all restaurants in San Francisco
results = db.pipeline().collection("cities/sf/restaurants").execute()

// Return all restaurants
results = db.pipeline().collectionGroup("restaurants").execute()

// Return all documents across all collections in the database (the entire database)
results = db.pipeline().database().execute()

// Batch read of 3 documents
results = db.pipeline().documents(
    db.collection("cities").document("SF"),
    db.collection("cities").document("DC"),
    db.collection("cities").document("NY")
).execute()
```

### Java

```java
Task<Pipeline.Snapshot> results;

// Return all restaurants in San Francisco
results = db.pipeline().collection("cities/sf/restaurants").execute();

// Return all restaurants
results = db.pipeline().collectionGroup("restaurants").execute();

// Return all documents across all collections in the database (the entire database)
results = db.pipeline().database().execute();

// Batch read of 3 documents
results = db.pipeline().documents(
    db.collection("cities").document("SF"),
    db.collection("cities").document("DC"),
    db.collection("cities").document("NY")
).execute();
```

##### Python

```python
# Return all restaurants in San Francisco
results = client.pipeline().collection("cities/sf/restaurants").execute()

# Return all restaurants
results = client.pipeline().collection_group("restaurants").execute()

# Return all documents across all collections in the database (the entire database)
results = client.pipeline().database().execute()

# Batch read of 3 documents
results = (
    client.pipeline()
    .documents(
        client.collection("cities").document("SF"),
        client.collection("cities").document("DC"),
        client.collection("cities").document("NY"),
    )
    .execute()
)
```

##### Go

```go
// Return all restaurants in San Francisco
results1, err := client.Pipeline().Collection("cities/sf/restaurants").Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}

// Return all restaurants
results2, err := client.Pipeline().CollectionGroup("restaurants").Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}

// Return all documents across all collections in the database (the entire database)
results3, err := client.Pipeline().Database().Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}

// Batch read of 3 documents
results4, err := client.Pipeline().
	Documents([]*firestore.DocumentRef{
		client.Collection("cities").Doc("SF"),
		client.Collection("cities").Doc("DC"),
		client.Collection("cities").Doc("NY"),
	}).
	Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}
```

As with all other stages, the order of results from these input stages is not
stable. A [`sort(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/sort.md.txt) operator should always be added if
a specific ordering is required.

### Where

The [`where(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/where.md.txt) stage acts as a standard filter
operation over documents generated from the previous stage and mostly mirrors
the existing "where" syntax for existing queries. Any document for where a given
expression evaluates to a non-`true` value is filtered out from the returned
documents.

Multiple [`where(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/where.md.txt) statements can be chained
together, and act as an `and(...)` expression. For example the following two
queries are logically equivalent and can be used interchangeably.

### Web

```javascript
let results;

results = await execute(db.pipeline().collection("books")
  .where(field("rating").equal(5))
  .where(field("published").lessThan(1900))
);

results = await execute(db.pipeline().collection("books")
  .where(and(field("rating").equal(5), field("published").lessThan(1900)))
);
```

##### Swift

```swift
var results: Pipeline.Snapshot

results = try await db.pipeline().collection("books")
  .where(Field("rating").equal(5))
  .where(Field("published").lessThan(1900))
  .execute()

results = try await db.pipeline().collection("books")
  .where(Field("rating").equal(5) && Field("published").lessThan(1900))
  .execute()
```

### Kotlin

```kotlin
var results: Task<Pipeline.Snapshot>

results = db.pipeline().collection("books")
    .where(field("rating").equal(5))
    .where(field("published").lessThan(1900))
    .execute()

results = db.pipeline().collection("books")
    .where(Expression.and(field("rating").equal(5),
      field("published").lessThan(1900)))
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results;

results = db.pipeline().collection("books")
    .where(field("rating").equal(5))
    .where(field("published").lessThan(1900))
    .execute();

results = db.pipeline().collection("books")
    .where(Expression.and(
        field("rating").equal(5),
        field("published").lessThan(1900)
    ))
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import And, Field

results = (
    client.pipeline()
    .collection("books")
    .where(Field.of("rating").equal(5))
    .where(Field.of("published").less_than(1900))
    .execute()
)

results = (
    client.pipeline()
    .collection("books")
    .where(And(Field.of("rating").equal(5), Field.of("published").less_than(1900)))
    .execute()
)
```

##### Go

```go
results1, err := client.Pipeline().
	Collection("books").
	Where(firestore.FieldOf("rating").Equal(5)).
	Where(firestore.FieldOf("published").LessThan(1900)).
	Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}

results2, err := client.Pipeline().
	Collection("books").
	Where(firestore.And(
		firestore.FieldOf("rating").Equal(5),
		firestore.FieldOf("published").LessThan(1900),
	)).
	Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}
```

### Select / Add \& Remove Fields

The [`select(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/select.md.txt),
[`add_fields(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/add-fields.md.txt), \&
[`remove_fields(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/remove-fields.md.txt) all allow you to modify
the fields being returned from a previous stage. These three are generally
referred to as projection-style stages.

The [`select(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/select.md.txt) and
[`add_fields(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/add-fields.md.txt) allow you to specify the
result of an expression to a user-provided field name. The
[`select(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/select.md.txt) will only return the documents with
specified field names while [`add_fields(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/add-fields.md.txt)
extends the schema of the previous stage (potentially overwriting values with
identical field names).

The [`remove_fields(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/remove-fields.md.txt) allows specifying a
set of fields to remove from the previous stage. Specifying field names that
don't exist is a no-op.

See the [Restrict the Fields to Return](https://firebase.google.com/docs/firestore/pipelines/get-started-with-pipelines.md.txt#restrict_the_fields_to_return) section
below but in general using such a stage to restrict the result to only the
fields needed in the client is helpful in reducing the cost and latency for most
queries.

### Aggregate / Distinct

The [`aggregate(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/aggregate.md.txt) stage lets you perform a series
of aggregations over the input documents. By default, all documents are
aggregated together, but an optional `grouping` argument can be provided,
allowing the input documents aggregated into different buckets.

### Web

```javascript
const results = await execute(db.pipeline()
  .collection("books")
  .aggregate(
    field("rating").average().as("avg_rating")
  )
  .distinct(field("genre"))
);
```

##### Swift

```swift
let results = try await db.pipeline()
  .collection("books")
  .aggregate([
    Field("rating").average().as("avg_rating")
  ], groups: [
    Field("genre")
  ])
  .execute()
```

### Kotlin

```kotlin
val results = db.pipeline()
    .collection("books")
    .aggregate(
        AggregateStage
            .withAccumulators(AggregateFunction.average("rating").alias("avg_rating"))
            .withGroups(field("genre"))
    )
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results = db.pipeline()
    .collection("books")
    .aggregate(AggregateStage
        .withAccumulators(
            AggregateFunction.average("rating").alias("avg_rating"))
        .withGroups(field("genre")))
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

results = (
    client.pipeline()
    .collection("books")
    .aggregate(
        Field.of("rating").average().as_("avg_rating"), groups=[Field.of("genre")]
    )
    .execute()
)
```

##### Go

```go
snapshot := client.Pipeline().
	Collection("books").
	Aggregate(
		firestore.Accumulators(firestore.Average("rating").As("avg_rating")),
		firestore.WithAggregateGroups("genre"),
	).
	Execute(ctx)
```

When `groupings` is not specified, this stage will produce only a single
document, otherwise a document will be generated for each unique combination of
`groupings` values.

The [`distinct(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/distinct.md.txt) stage is a simplified
aggregation operator which allows generating just the unique `groupings`
without any accumulators. It behaves identically to that of
[`aggregate(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/aggregate.md.txt) in all other regards. The
following example shows:

### Web

```javascript
const results = await execute(db.pipeline()
  .collection("books")
  .distinct(
    field("author").toUpper().as("author"),
    field("genre")
  )
);
```

##### Swift

```swift
let results = try await db.pipeline()
  .collection("books")
  .distinct([
    Field("author").toUpper().as("author"),
    Field("genre")
  ])
  .execute()
```

### Kotlin

```kotlin
val results = db.pipeline()
    .collection("books")
    .distinct(
        field("author").toUpper().alias("author"),
        field("genre")
    )
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results = db.pipeline()
    .collection("books")
    .distinct(
        field("author").toUpper().alias("author"),
        field("genre")
    )
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

results = (
    client.pipeline()
    .collection("books")
    .distinct(Field.of("author").to_upper().as_("author"), "genre")
    .execute()
)
```

##### Go

```go
snapshot := client.Pipeline().
	Collection("books").
	Distinct(firestore.Fields(
		firestore.ToUpper(firestore.FieldOf("author")).As("author"),
		firestore.FieldOf("genre"),
	)).
	Execute(ctx)
```

## Functions

Functions are a building block for creating expressions and complex queries.
For a complete list of functions with examples, refer to the
[Functions reference](https://firebase.google.com/docs/firestore/pipelines/functions/all_functions.md.txt).
As a quick reminder, consider the structure of a typical query:

![Example demonstrating stages and functions in a query](https://docs.cloud.google.com/firestore/native/docs/images/pipeline-queries/pipeline_query_stage_expression.jpg)

Many stages accept expressions which contain one or more functions. The most
common function usage will be found in the
[`where(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/where.md.txt) and
[`select(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/select.md.txt) stages. There are two main types of
functions that you should be familiar with:

### Web

```javascript
let results;

// Type 1: Scalar (for use in non-aggregation stages)
// Example: Return the min store price for each book.
results = await execute(db.pipeline().collection("books")
  .select(field("current").logicalMinimum(field("updated")).as("price_min"))
);

// Type 2: Aggregation (for use in aggregate stages)
// Example: Return the min price of all books.
results = await execute(db.pipeline().collection("books")
  .aggregate(field("price").minimum().as("min_price"))
);
```

##### Swift

```swift
var results: Pipeline.Snapshot

// Type 1: Scalar (for use in non-aggregation stages)
// Example: Return the min store price for each book.
results = try await db.pipeline().collection("books")
  .select([
    Field("current").logicalMinimum(["updated"]).as("price_min")
  ])
  .execute()

// Type 2: Aggregation (for use in aggregate stages)
// Example: Return the min price of all books.
results = try await db.pipeline().collection("books")
  .aggregate([Field("price").minimum().as("min_price")])
  .execute()
```

### Kotlin

```kotlin
var results: Task<Pipeline.Snapshot>

// Type 1: Scalar (for use in non-aggregation stages)
// Example: Return the min store price for each book.
results = db.pipeline().collection("books")
    .select(
        field("current").logicalMinimum("updated").alias("price_min")
    )
    .execute()

// Type 2: Aggregation (for use in aggregate stages)
// Example: Return the min price of all books.
results = db.pipeline().collection("books")
    .aggregate(AggregateFunction.minimum("price").alias("min_price"))
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results;

// Type 1: Scalar (for use in non-aggregation stages)
// Example: Return the min store price for each book.
results = db.pipeline().collection("books")
    .select(
        field("current").logicalMinimum("updated").alias("price_min")
    )
    .execute();

// Type 2: Aggregation (for use in aggregate stages)
// Example: Return the min price of all books.
results = db.pipeline().collection("books")
    .aggregate(AggregateFunction.minimum("price").alias("min_price"))
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

# Type 1: Scalar (for use in non-aggregation stages)
# Example: Return the min store price for each book.
results = (
    client.pipeline()
    .collection("books")
    .select(
        Field.of("current").logical_minimum(Field.of("updated")).as_("price_min")
    )
    .execute()
)

# Type 2: Aggregation (for use in aggregate stages)
# Example: Return the min price of all books.
results = (
    client.pipeline()
    .collection("books")
    .aggregate(Field.of("price").minimum().as_("min_price"))
    .execute()
)
```

##### Go

```go
// Type 1: Scalar (for use in non-aggregation stages)
// Example: Return the min store price for each book.
results1, err := client.Pipeline().
	Collection("books").
	Select(firestore.Fields(
		firestore.LogicalMinimum(firestore.FieldOf("current"), firestore.FieldOf("updated")).As("price_min"),
	)).
	Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}

// Type 2: Aggregation (for use in aggregate stages)
// Example: Return the min price of all books.
results2, err := client.Pipeline().
	Collection("books").
	Aggregate(firestore.Accumulators(
		firestore.Minimum("price").As("min_price"),
	)).
	Execute(ctx).Results().GetAll()
if err != nil {
	fmt.Fprintf(w, "GetAll failed: %v", err)
	return err
}
```

## Limits

For the most part Enterprise edition doesn't impose limits on the
shape of the query. In other words, you're not limited to a small number of
values in an `IN` or `OR` query. Instead, there are two primary limits you
should be aware of:

- **Deadline:** 60 seconds (identical to Standard edition).
- **Memory Usage:** 128 MiB limit on the amount of materialized data during query execution.

## Errors

You may encounter failed queries for a number of reasons. Here is a link to [common errors](https://firebase.google.com/docs/firestore/enterprise/understand-error-codes.md.txt) and the associated action you can take:

|---|---|
| **Error Code** | **Action** |
| `DEADLINE_EXCEEDED` | The query you are executing exceeds a 60 second deadline and requires additional optimization. See the performance section for tips. If you are unable to root cause the problem, reach out to the team. |
| `RESOURCE_EXHAUSTED` | The query you are executing exceeds the memory limits and requires additional optimization. See the performance section for tips. If you are unable to root cause the problem, reach out to the team. |
| `INTERNAL` | [Contact](https://firebase.google.com/support/troubleshooter/databases.md.txt) the team for support. |

## Performance

Enterprise edition databases do not require that an index is always present.
This means that a query can exhibit higher latency compared to existing queries
which would have just failed immediately with a `FAILED_PRECONDITION` missing
index error. To improve the performance of Pipeline operations,
there are a couple of steps you can take.

### Create Indexes

#### Index Used

Query explain lets you identify if your query is being served by an index or
falling back to a less efficient operation like a table scan. If your query is
not being fully served from an index, you can create an index by following the
instructions.

#### Creating Indexes

You can follow the existing [index management documentation](https://firebase.google.com/docs/firestore/query-data/indexing.md.txt) to create indexes. Before creating an index, familiarize yourself with [general best practices](https://firebase.google.com/docs/firestore/query-data/index-overview.md.txt) with indexes in Cloud Firestore. To ensure your query can leverage indexes, follow the [best practices](https://firebase.google.com/docs/firestore/query-data/index-overview.md.txt#index_properties) to create indexes with fields in the following order:

1. All fields that will be used in equality filters (in any order)
2. All fields that will be sorted on (in the same order)
3. Fields that will be used in range or inequality filters in decreasing order of query constraint selectivity

For example, for the following query,

### Web

```javascript
const results = await execute(db.pipeline()
  .collection("books")
  .where(field("published").lessThan(1900))
  .where(field("genre").equal("Science Fiction"))
  .where(field("rating").greaterThan(4.3))
  .sort(field("published").descending())
);
```

##### Swift

```swift
let results = try await db.pipeline()
  .collection("books")
  .where(Field("published").lessThan(1900))
  .where(Field("genre").equal("Science Fiction"))
  .where(Field("rating").greaterThan(4.3))
  .sort([Field("published").descending()])
  .execute()
```

### Kotlin

```kotlin
val results = db.pipeline()
    .collection("books")
    .where(field("published").lessThan(1900))
    .where(field("genre").equal("Science Fiction"))
    .where(field("rating").greaterThan(4.3))
    .sort(field("published").descending())
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results = db.pipeline()
    .collection("books")
    .where(field("published").lessThan(1900))
    .where(field("genre").equal("Science Fiction"))
    .where(field("rating").greaterThan(4.3))
    .sort(field("published").descending())
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

results = (
    client.pipeline()
    .collection("books")
    .where(Field.of("published").less_than(1900))
    .where(Field.of("genre").equal("Science Fiction"))
    .where(Field.of("rating").greater_than(4.3))
    .sort(Field.of("published").descending())
    .execute()
)
```

##### Go

```go
snapshot := client.Pipeline().
	Collection("books").
	Where(firestore.FieldOf("published").LessThan(1900)).
	Where(firestore.FieldOf("genre").Equal("Science Fiction")).
	Where(firestore.FieldOf("rating").GreaterThan(4.3)).
	Sort(firestore.Orders(firestore.Descending(firestore.FieldOf("published")))).
	Execute(ctx)
```

The recommended index is a collection scope index on `books` for `(genre [...], published DESC, avg_rating DESC).`

#### Index density

Cloud Firestore supports sparse and non-sparse indexes. For more
information, see
[Index density](https://firebase.google.com/docs/firestore/pipelines/index-overview#index_density).

#### Covered Queries + Secondary Indexes

Cloud Firestore can skip fetching the full document and just return
results from the index if all fields being returned are present in a secondary
index. This normally leads to a significant latency (and cost) improvement.
Using the sample query below:

### Web

```javascript
const results = await execute(db.pipeline()
  .collection("books")
  .where(field("category").like("%fantasy%"))
  .where(field("title").exists())
  .where(field("author").exists())
  .select(field("title"), field("author"))
);
```

##### Swift

```swift
let results = try await db.pipeline()
  .collection("books")
  .where(Field("category").like("%fantasy%"))
  .where(Field("title").exists())
  .where(Field("author").exists())
  .select([Field("title"), Field("author")])
  .execute()
```

### Kotlin

```kotlin
val results = db.pipeline()
    .collection("books")
    .where(field("category").like("%fantasy%"))
    .where(field("title").exists())
    .where(field("author").exists())
    .select(field("title"), field("author"))
    .execute()
```

### Java

```java
Task<Pipeline.Snapshot> results = db.pipeline()
    .collection("books")
    .where(field("category").like("%fantasy%"))
    .where(field("title").exists())
    .where(field("author").exists())
    .select(field("title"), field("author"))
    .execute();
```

##### Python

```python
from google.cloud.firestore_v1.pipeline_expressions import Field

results = (
    client.pipeline()
    .collection("books")
    .where(Field.of("category").like("%fantasy%"))
    .where(Field.of("title").exists())
    .where(Field.of("author").exists())
    .select("title", "author")
    .execute()
)
```

##### Go

```go
snapshot := client.Pipeline().
	Collection("books").
	Where(firestore.FieldOf("category").Like("%fantasy%")).
	Where(firestore.FieldOf("title").FieldExists()).
	Where(firestore.FieldOf("author").FieldExists()).
	Select(firestore.Fields("title", "author")).
	Execute(ctx)
```

If the database already has a collection scope index on `books` for
`(category [...], title [...], author [...])` then it can avoid fetching
anything from the main documents themselves. In this case the order in the
index does not matter, `[...]` is used to signify that.

### Restrict the Fields to Return

By default, a Cloud Firestore query returns all fields in a document,
analogous to a `SELECT *` in relational systems. If however your application
only needs a subset of the fields, the [`select(...)`](https://firebase.google.com/docs/firestore/pipelines/stages/transformation/select.md.txt)
or `restrict(...)` stages can be used to push this filtering server-side. This
will decrease both the response size (decreasing the network egress cost) as
well as improving latency.

## Troubleshooting Tools

### Query Explain

Query Explain lets you bring visibility in execution metrics and details about
indexes used.

### Metrics

Pipeline operations if fully integrated with existing [Cloud Firestore metrics](https://firebase.google.com/monitoring/api/metrics_gcp_d_h#gcp-firestore).

## Known Issues / Limitations

### Specialized Indexes

Pipeline operations don't yet support existing `array-contains` \& `vector` [index types](https://firebase.google.com/docs/firestore/query-data/index-overview.md.txt#index_modes). Instead of just rejecting such queries, Cloud Firestore will attempt to use other existing `ascending` \& `descending` indexes. It is expected that with such `array_contains` or `find_nearest` expressions will be slower than their existing equivalents due to this.

### Realtime and Offline Support

Pipeline operations don't have realtime and offline capabilities.

## What's next

- Begin exploring the [Functions and Stages reference documentation](https://firebase.google.com/docs/firestore/pipelines/functions/aggregate-functions.md.txt).
- Learn how to [Perform joins with subqueries](https://firebase.google.com/docs/firestore/pipelines/perform-joins-with-sub-pipelines.md.txt).