Pipeline

addFields

fun addFields(field: Selectable, vararg additionalFields: Selectable): Pipeline

Adds new fields to outputs from previous stages.

This stage allows you to compute values on-the-fly based on existing data from previous stages or constants. You can use this to create new fields or overwrite existing ones.

The added fields are defined using Selectables, which can be:

• Field: References an existing document field.

• AliasedExpression: Represents the result of a expression with an assigned alias name using Expr.alias

Example:

firestore.pipeline().collection("books")
  .addFields(
    field("rating").as("bookRating"), // Rename 'rating' to 'bookRating'
    add(5, field("quantity")).as("totalCost")  // Calculate 'totalCost'
  );

aggregate

fun aggregate(aggregateStage: AggregateStage): Pipeline

Performs optionally grouped aggregation operations on the documents from previous stages.

This stage allows you to calculate aggregate values over a set of documents, optionally grouped by one or more fields or functions. You can specify:

• Grouping Fields or Expressions: One or more fields or functions to group the documents by. For each distinct combination of values in these fields, a separate group is created. If no grouping fields are provided, a single group containing all documents is used. Not specifying groups is the same as putting the entire inputs into one group.

• AggregateFunctions: One or more accumulation operations to perform within each group. These are defined using AliasedAggregate expressions, which are typically created by calling AggregateFunction.alias on AggregateFunction instances. Each aggregation calculates a value (e.g., sum, average, count) based on the documents within its group.

Example:

// Calculate the average rating for each genre.
firestore.pipeline().collection("books")
  .aggregate(
    Aggregate
      .withAccumulators(average("rating").as("avg_rating"))
      .withGroups("genre"));

aggregate

fun aggregate(
    accumulator: AliasedAggregate,
    vararg additionalAccumulators: AliasedAggregate
): Pipeline

Performs aggregation operations on the documents from previous stages.

This stage allows you to calculate aggregate values over a set of documents. You define the aggregations to perform using AliasedAggregate expressions which are typically results of calling AggregateFunction.alias on AggregateFunction instances.

Example:

// Calculate the average rating and the total number of books
firestore.pipeline().collection("books")
    .aggregate(
        field("rating").average().as("averageRating"),
        countAll().as("totalBooks")
    );

aggregate

fun aggregate(aggregateStage: AggregateStage, options: AggregateOptions): Pipeline

Performs optionally grouped aggregation operations on the documents from previous stages.

This stage allows you to calculate aggregate values over a set of documents, optionally grouped by one or more fields or functions. You can specify:

• Grouping Fields or Expressions: One or more fields or functions to group the documents by. For each distinct combination of values in these fields, a separate group is created. If no grouping fields are provided, a single group containing all documents is used. Not specifying groups is the same as putting the entire inputs into one group.

• AggregateFunctions: One or more accumulation operations to perform within each group. These are defined using AliasedAggregate expressions, which are typically created by calling AggregateFunction.alias on AggregateFunction instances. Each aggregation calculates a value (e.g., sum, average, count) based on the documents within its group.

distinct

fun distinct(group: Selectable, vararg additionalGroups: Any): Pipeline

Returns a set of distinct values from the inputs to this stage.

This stage runs through the results from previous stages to include only results with unique combinations of Expr values Field, FunctionExpression, etc).

The parameters to this stage are defined using Selectable expressions or strings:

• String: Name of an existing field

• Field: References an existing document field.

• AliasedExpression: Represents the result of a function with an assigned alias name using Expr.alias

Example:

// Get a list of unique author names in uppercase and genre combinations.
firestore.pipeline().collection("books")
    .distinct(toUppercase(field("author")).as("authorName"), field("genre"))
    .select("authorName");

distinct

fun distinct(groupField: String, vararg additionalGroups: Any): Pipeline

Returns a set of distinct values from the inputs to this stage.

This stage runs through the results from previous stages to include only results with unique combinations of Expr values (Field, FunctionExpression, etc).

The parameters to this stage are defined using Selectable expressions or strings:

• String: Name of an existing field

• Field: References an existing document field.

• AliasedExpression: Represents the result of a function with an assigned alias name using Expr.alias

Example:

// Get a list of unique genres.
firestore.pipeline().collection("books")
    .distinct("genre");

execute

fun execute(): Task<Pipeline.Snapshot>

Executes this pipeline and returns the results as a Task of Snapshot.

execute

fun execute(options: Pipeline.ExecuteOptions): Task<Pipeline.Snapshot>

Executes this pipeline and returns the results as a Task of Snapshot.

findNearest

fun findNearest(
    vectorField: Field,
    vectorValue: DoubleArray,
    distanceMeasure: FindNearestStage.DistanceMeasure
): Pipeline

Performs a vector similarity search, ordering the result set by most similar to least similar, and returning the full set in the order of similarity.

findNearest

fun findNearest(
    vectorField: String,
    vectorValue: DoubleArray,
    distanceMeasure: FindNearestStage.DistanceMeasure
): Pipeline

Performs a vector similarity search, ordering the result set by most similar to least similar, and returning the full set in the order of similarity.

findNearest

fun findNearest(
    vectorField: String,
    vectorValue: Expression,
    distanceMeasure: FindNearestStage.DistanceMeasure,
    options: FindNearestOptions
): Pipeline

Performs a vector similarity search, ordering the result set by most similar to least similar, and returning the first N documents (specified by FindNearestOptions.withLimit) in the result set.

Example:

// Find books with similar "topicVectors" to the given targetVector
firestore.pipeline().collection("books")
    .findNearest("topicVectors", targetVector, FindNearest.DistanceMeasure.COSINE,
       new FindNearestOptions()
         .withLimit(10)
         .withDistanceField("distance"));

limit

fun limit(limit: Int): Pipeline

Limits the maximum number of documents returned by previous stages to limit.

This stage is particularly useful when you want to retrieve a controlled subset of data from a potentially large result set. It's often used for:

• Pagination: In combination with offset to retrieve specific pages of results.

• Limiting Data Retrieval: To prevent excessive data transfer and improve performance, especially when dealing with large collections.

Example:

// Limit the results to the top 10 highest-rated books
firestore.pipeline().collection("books")
    .sort(field("rating").descending())
    .limit(10);

offset

fun offset(offset: Int): Pipeline

Skips the first offset number of documents from the results of previous stages.

This stage is useful for implementing pagination in your pipelines, allowing you to retrieve results in chunks. It is typically used in conjunction with limit to control the size of each page.

Example:

// Retrieve the second page of 20 results
firestore.pipeline().collection("books")
    .sort(field("published").descending())
    .offset(20)  // Skip the first 20 results
    .limit(20);   // Take the next 20 results

rawStage

fun rawStage(rawStage: RawStage): Pipeline

Adds a raw stage to the pipeline by specifying the stage name as an argument. This does not offer any type safety on the stage params and requires the caller to know the order (and optionally names) of parameters accepted by the stage.

This method provides a way to call stages that are supported by the Firestore backend but that are not implemented in the SDK version being used.

removeFields

fun removeFields(field: Field, vararg additionalFields: Field): Pipeline

Remove fields from outputs of previous stages.

Example:

firestore.pipeline().collection("books")
  .removeFields(
    field("rating"), field("cost")
  );

removeFields

fun removeFields(field: String, vararg additionalFields: String): Pipeline

Remove fields from outputs of previous stages.

Example:

firestore.pipeline().collection("books")
  .removeFields(
    "rating", "cost"
  );

replaceWith

fun replaceWith(field: String): Pipeline

Fully overwrites all fields in a document with those coming from a nested map.

This stage allows you to emit a map value as a document. Each key of the map becomes a field on the document that contains the corresponding value.

Example:

// Input.
// {
//  "name": "John Doe Jr.",
//  "parents": {
//    "father": "John Doe Sr.",
//    "mother": "Jane Doe"
// }

// Emit parents as document.
firestore.pipeline().collection("people").replaceWith("parents");

// Output
// {
//  "father": "John Doe Sr.",
//  "mother": "Jane Doe"
// }

replaceWith

fun replaceWith(mapValue: Expression): Pipeline

Fully overwrites all fields in a document with those coming from a nested map.

This stage allows you to emit a map value as a document. Each key of the map becomes a field on the document that contains the corresponding value.

Example:

// Input.
// {
//  "name": "John Doe Jr.",
//  "parents": {
//    "father": "John Doe Sr.",
//    "mother": "Jane Doe"
// }

// Emit parents as document.
firestore.pipeline().collection("people").replaceWith(field("parents"));

// Output
// {
//  "father": "John Doe Sr.",
//  "mother": "Jane Doe"
// }

sample

fun sample(documents: Int): Pipeline

Performs a pseudo-random sampling of the input documents.

The documents parameter represents the target number of documents to produce and must be a non-negative integer value. If the previous stage produces less than size documents, the entire previous results are returned. If the previous stage produces more than size, this outputs a sample of exactly size entries where any sample is equally likely.

Example:

// Sample 10 books, if available.
firestore.pipeline().collection("books")
    .sample(10);

sample

fun sample(sample: SampleStage): Pipeline

Performs a pseudo-random sampling of the input documents.

Examples:

// Sample 10 books, if available.
firestore.pipeline().collection("books")
    .sample(Sample.withDocLimit(10));

// Sample 50% of books.
firestore.pipeline().collection("books")
    .sample(Sample.withPercentage(0.5));

select

fun select(fieldName: String, vararg additionalSelections: Any): Pipeline

Selects or creates a set of fields from the outputs of previous stages.

The selected fields are defined using Selectable expressions, which can be:

• String: Name of an existing field

• Field: Reference to an existing field.

• AliasedExpression: Represents the result of a expression with an assigned alias name using Expr.alias

If no selections are provided, the output of this stage is empty. Use Pipeline.addFields instead if only additions are desired.

Example:

firestore.collection("books")
  .select("name", "address");

// The above is a shorthand of this:
firestore.pipeline().collection("books")
   .select(field("name"), field("address"));

select

fun select(selection: Selectable, vararg additionalSelections: Any): Pipeline

Selects or creates a set of fields from the outputs of previous stages.

The selected fields are defined using Selectable expressions or strings, which can be:

• String: Name of an existing field

• Field: Reference to an existing field.

• AliasedExpression: Represents the result of a expression with an assigned alias name using Expr.alias

If no selections are provided, the output of this stage is empty. Use Pipeline.addFields instead if only additions are desired.

Example:

firestore.pipeline().collection("books")
  .select(
    field("name"),
    field("address").toUppercase().as("upperAddress"),
  );

sort

fun sort(order: Ordering, vararg additionalOrders: Ordering): Pipeline

Sorts the documents from previous stages based on one or more Ordering criteria.

This stage allows you to order the results of your pipeline. You can specify multiple Ordering instances to sort by multiple fields in ascending or descending order. If documents have the same value for a field used for sorting, the next specified ordering will be used. If all orderings result in equal comparison, the documents are considered equal and the order is unspecified.

Example:

// Sort books by rating in descending order, and then by title in ascending order for books with the same rating
firestore.pipeline().collection("books")
    .sort(
        Ordering.of("rating").descending(),
        Ordering.of("title")  // Ascending order is the default
    );

union

fun union(other: Pipeline): Pipeline

Performs union of all documents from two pipelines, including duplicates.

This stage will pass through documents from previous stage, and also pass through documents from previous stage of the other Pipeline given in parameter. The order of documents emitted from this stage is undefined.

Example:

// Emit documents from books collection and magazines collection.
firestore.pipeline().collection("books")
    .union(firestore.pipeline().collection("magazines"));

unnest

fun unnest(arrayWithAlias: Selectable): Pipeline

Takes a specified array from the input documents and outputs a document for each element with the element stored in a field with name specified by the alias.

For each document emitted by the prior stage, this stage will emit zero or more augmented documents. The input array is found in parameter arrayWithAlias, which can be an Expr with an alias specified via Expr.alias, or a Field that can also have alias specified. For each element of the input array, an augmented document will be produced. The element of input array will be stored in a field with name specified by the alias of the arrayWithAlias parameter. If the arrayWithAlias is a Field with no alias, then the original array field will be replaced with the individual element.

unnest

fun unnest(unnestStage: UnnestStage): Pipeline

Takes a specified array from the input documents and outputs a document for each element with the element stored in a field with name specified by the alias.

For each document emitted by the prior stage, this stage will emit zero or more augmented documents. The input array specified in the unnestStage parameter will for each element of the input array produce an augmented document. The element of the input array will be stored in a field with a name specified by the unnestStage parameter.

Optionally, an index field can also be added to emitted documents. See UnnestStage for further information.

unnest

fun unnest(arrayField: String, alias: String): Pipeline

Takes a specified array from the input documents and outputs a document for each element with the element stored in a field with name specified by the alias.

For each document emitted by the prior stage, this stage will emit zero or more augmented documents. The input array found in the previous stage document field specified by the arrayField parameter, will for each element of the input array produce an augmented document. The element of the input array will be stored in a field with name specified by alias parameter on the augmented document.

Example:

// Input:
// { "title": "The Hitchhiker's Guide to the Galaxy", "tags": [ "comedy", "space", "adventure" ], ... }

// Emit a book document for each tag of the book.
firestore.pipeline().collection("books")
    .unnest("tags", "tag");

// Output:
// { "title": "The Hitchhiker's Guide to the Galaxy", "tag": "comedy", ... }
// { "title": "The Hitchhiker's Guide to the Galaxy", "tag": "space", ... }
// { "title": "The Hitchhiker's Guide to the Galaxy", "tag": "adventure", ... }

unnest

fun unnest(arrayWithAlias: Selectable, options: UnnestOptions): Pipeline

Takes a specified array from the input documents and outputs a document for each element with the element stored in a field with name specified by the alias.

For each document emitted by the prior stage, this stage will emit zero or more augmented documents. The input array is found in parameter arrayWithAlias, which can be an Expr with an alias specified via Expr.alias, or a Field that can also have alias specified. For each element of the input array, an augmented document will be produced. The element of input array will be stored in a field with name specified by the alias of the arrayWithAlias parameter. If the arrayWithAlias is a Field with no alias, then the original array field will be replaced with the individual element.

Example:

// Input:
// { "title": "The Hitchhiker's Guide to the Galaxy", "tags": [ "comedy", "space", "adventure" ], ... }

// Emit a book document for each tag of the book.
firestore.pipeline().collection("books")
    .unnest("tags", "tag", new UnnestOptions().withIndexField("tagIndex"));

// Output:
// { "title": "The Hitchhiker's Guide to the Galaxy", "tagIndex": 0, "tag": "comedy", ... }
// { "title": "The Hitchhiker's Guide to the Galaxy", "tagIndex": 1, "tag": "space", ... }
// { "title": "The Hitchhiker's Guide to the Galaxy", "tagIndex": 2, "tag": "adventure", ... }

where

fun where(condition: BooleanExpression): Pipeline

Filters the documents from previous stages to only include those matching the specified BooleanExpression.

This stage allows you to apply conditions to the data, similar to a "WHERE" clause in SQL.

You can filter documents based on their field values, using implementations of BooleanExpression, typically including but not limited to:

• field comparators: Expr.equal, Expr.lessThan, Expr.greaterThan, etc.

• logical operators: Expr.and, Expr.or, Expr.not, etc.

• advanced functions: Expr.regexMatch, Expr.arrayContains, etc.

Example:

firestore.pipeline().collection("books")
  .where(
    and(
        gt("rating", 4.0),   // Filter for ratings greater than 4.0
        field("genre").equal("Science Fiction") // Equivalent to equal("genre", "Science Fiction")
    )
  );