Google 致力于为黑人社区推动种族平等。查看具体举措

将集合导出至 BigQuery

Firebase
Made by Firebase

将指定 Cloud Firestore 集合中的实时增量更新发送至 BigQuery。

此扩展程序的运作方式

Use this extension to export the documents in a Cloud Firestore collection to BigQuery. Exports are realtime and incremental, so the data in BigQuery is a mirror of your content in Cloud Firestore.

The extension creates and updates a dataset containing the following two BigQuery resources:

  • A table of raw data that stores a full change history of the documents within your collection. This table includes a number of metadata fields so that BigQuery can display the current state of your data. The principle metadata fields are timestamp, document_name, and the operation for the document change.
  • A view which represents the current state of the data within your collection. It also shows a log of the latest operation for each document (CREATE, UPDATE, or IMPORT).

If you create, update, delete, or import a document in the specified collection, this extension sends that update to BigQuery. You can then run queries on this mirrored dataset.

Note that this extension only listens for document changes in the collection, but not changes in any subcollection. You can, though, install additional instances of this extension to specifically listen to a subcollection or other collections in your database. Or if you have the same subcollection across documents in a given collection, you can use {wildcard} notation to listen to all those subcollections (for example: chats/{chatid}/posts).

Additional setup

Before installing this extension, you'll need to:

Backfill your BigQuery dataset

This extension only sends the content of documents that have been changed -- it does not export your full dataset of existing documents into BigQuery. So, to backfill your BigQuery dataset with all the documents in your collection, you can run the import script provided by this extension.

Important: Run the import script over the entire collection after installing this extension, otherwise all writes to your database during the import might be lost.

Generate schema views

After your data is in BigQuery, you can run the schema-views script (provided by this extension) to create views that make it easier to query relevant data. You only need to provide a JSON schema file that describes your data structure, and the schema-views script will create the views.

Billing

To install an extension, your project must be on the Blaze (pay as you go) plan

  • You will be charged a small amount (typically around $0.01/month) for the Firebase resources required by this extension (even if it is not used).
  • This extension uses other Firebase and Google Cloud Platform services, which have associated charges if you exceed the service’s free tier:
  • BigQuery (this extension writes to BigQuery with streaming inserts)
  • Cloud Firestore
  • Cloud Functions (Node.js 10+ runtime. See FAQs)

如何安装此扩展程序

使用 Firebase 控制台

要安装和管理扩展程序,您可以使用 Firebase 控制台。

使用控制台安装

使用 Firebase CLI

要安装和管理扩展程序,您还可以使用 Firebase CLI:

步骤 1:运行以下 npm 命令来安装 CLI 或将 CLI 更新到最新版本。

npm install -g firebase-tools
不能运行?请参阅 Firebase CLI 参考文档或者更改您的 npm 权限

步骤 2:运行以下命令来安装此扩展程序

firebase ext:install firestore-bigquery-export --project=projectId_or_alias
作者
许可
Apache-2.0
版本
0.1.10