A step-by-step tutorial to use Biq Blue as a service for Google BigQuery
Welcome to the step-by-step guide on how to set up Biq Blue "as a service" for your Google BigQuery account. Follow the five steps below in this exact order: each step prepares what the next one needs.
In short — 5 steps, about 10 minutes, metadata only:
- Create a service account named
biq-bluein your main project and download its JSON key. - Create the
biqbluedataset in the same region as your business tables and give the service account BigQuery Data Editor on this dataset only. - Grant the metadata-only roles at organization level: BigQuery Metadata Viewer, BigQuery Resource Viewer, BigQuery Job User.
- Upload the service account on go.biq.blue.
- Deploy your instance.
Biq Blue never reads the content of your tables: it only uses INFORMATION_SCHEMA metadata.
How it works
Biq Blue aggregates information from INFORMATION_SCHEMA views across your projects into tables within your BigQuery account.
This process enables Biq Blue to analyze your usage and make suggestions for improvement.
Biq Blue cannot access the contents of your tables, as it only requires metadata through specific organization-level IAM roles.
These roles are restricted to gathering job, table, storage and other meta information, ensuring Biq Blue never interacts with or views your actual data.
Additionally, Biq Blue requires the creation of a specific dataset where it can read and write its own tables, but these permissions are limited only to this dataset.
Prerequisites
- A Google Cloud main project that uses BigQuery (the service account and the
biqbluedataset live there). - The region of your business tables (a region such as
europe-west1, or a multi-region such asEU/US). - Permission to grant IAM roles at organization level (or at project level if you only have one project, see the FAQ).
Every step below shows the Google Cloud console way, plus the equivalent gcloud / bq commands if you prefer the CLI (or if an AI assistant is doing the setup for you).
Replace MAIN_PROJECT, REGION and ORG_ID with your own values.
Step 1: Create a service account
- In your main project, navigate to Google Cloud's service accounts console
- Click on + CREATE SERVICE ACCOUNT to create a new service account
biq-blue, no need to add roles now, neither grant users access to this service account
- Click on the new service account, go to the "Keys" tab, and "Add Key" in JSON
- Download the JSON
Same step with the gcloud CLI
gcloud iam service-accounts create biq-blue --project=MAIN_PROJECT --display-name="biq-blue"
gcloud iam service-accounts keys create biq-blue.json \
--project=MAIN_PROJECT \
--iam-account=biq-blue@MAIN_PROJECT.iam.gserviceaccount.com
# check: the service account exists
gcloud iam service-accounts describe biq-blue@MAIN_PROJECT.iam.gserviceaccount.com --project=MAIN_PROJECT
Step 2: Create the biqblue dataset
- In your main project, click on ⋮ > Create dataset and name it
biqblue, in the same region you use for your business tables - In the
biqbluedataset, click on + SHARING > Permissions > Add Principal and give the service account the following role: - BigQuery Data Editor (
roles/bigquery.dataEditor)
❤️ Biq Blue can now create its tables in this new dataset, and this dataset only. This is why the dataset must exist before you upload the service account: the service account has no permission to create datasets, and the upload step tests this dataset.
Same step with the bq CLI
# REGION = the location of your business datasets, e.g. EU, US or europe-west1
# (bq show --format=prettyjson MAIN_PROJECT:YOUR_BUSINESS_DATASET | grep location)
bq --project_id=MAIN_PROJECT mk --dataset --location=REGION MAIN_PROJECT:biqblue
bq --project_id=MAIN_PROJECT query --use_legacy_sql=false --location=REGION \
'GRANT `roles/bigquery.dataEditor` ON SCHEMA `MAIN_PROJECT.biqblue` TO "serviceAccount:biq-blue@MAIN_PROJECT.iam.gserviceaccount.com"'
# check: the dataset is in REGION and its access list contains the service account
bq show --format=prettyjson MAIN_PROJECT:biqblue
Step 3: Assign the required roles to your new service account at organization level
- Go to the IAM root section and select your organization (the root folder)
- Click on + GRANT ACCESS and give the
biq-blueservice account the following roles: - BigQuery Metadata Viewer (
roles/bigquery.metadataViewer) - BigQuery Resource Viewer (
roles/bigquery.resourceViewer) - BigQuery Job User (
roles/bigquery.jobUser)
❤️ These roles ensure that Biq Blue has the necessary authorizations to analyze INFORMATION_SCHEMA views, while preventing access to your table data. Privacy first!
Same step with the gcloud CLI
# ORG_ID = your organization id (gcloud organizations list)
for role in roles/bigquery.metadataViewer roles/bigquery.resourceViewer roles/bigquery.jobUser; do
gcloud organizations add-iam-policy-binding ORG_ID \
--member="serviceAccount:biq-blue@MAIN_PROJECT.iam.gserviceaccount.com" \
--role="$role"
done
# check: the three roles are listed
gcloud organizations get-iam-policy ORG_ID \
--flatten="bindings[].members" \
--filter="bindings.members:biq-blue@MAIN_PROJECT.iam.gserviceaccount.com" \
--format="value(bindings.role)"
No organization? Grant the same three roles on the project (gcloud projects add-iam-policy-binding MAIN_PROJECT ...) and tick Single project at the next step.
Step 4: Upload 🚀
Go back to the setup and upload your service account JSON key here.
Steps 1, 2 and 3 must be complete before this upload: Biq Blue immediately tests the permissions and the biqblue dataset with the uploaded service account.
Setting up from an AI coding assistant (Biq Blue MCP server)
If you use the Biq Blue MCP server (Claude Code, Cursor, OpenCode...), the upload and the next step are done with the biqblue_setup_instance tool, in this order:
biqblue_setup_instance step="service_account" data={ service_account_json: "<content of biq-blue.json>", single_project: false }
biqblue_setup_instance step="test_permissions"
biqblue_setup_instance step="configure" data={ dataset: "biqblue", project: "MAIN_PROJECT", region: "REGION" }
The configure step only tests the dataset created at step 2, it does not create it.
Final step: Deploy 💪
Follow the final instructions and click "deploy". Access your server and start reducing Google BigQuery costs right now.
(From the MCP server: biqblue_setup_instance step="deploy".)
The first data synchronization can take several tens of minutes if you have a lot of projects in Google BigQuery, but you can start using Biq Blue as soon as the 1st project is synchronized.
Frequently asked questions
- Can Biq Blue read the content of my BigQuery tables?
- No. Biq Blue only reads metadata through the
INFORMATION_SCHEMAviews (jobs, tables, storage, reservations). The roles it needs — BigQuery Metadata Viewer, Resource Viewer and Job User — do not give access to table contents. The only dataset it can write to is thebiqbluedataset you create for it. - Why must the
biqbluedataset exist before I upload the service account? - The service account only gets write access to this one dataset (BigQuery Data Editor on the dataset), so it cannot create datasets itself. The setup tests the dataset right after the upload: create it first, in the same region as your business tables, then upload.
- Which region should I choose for the
biqbluedataset? - The region (or multi-region such as
EUorUS) where your business tables live.INFORMATION_SCHEMAviews are regional, so Biq Blue analyzes the region of its dataset. - What if I do not have a Google Cloud organization?
- Grant the same three roles on the project instead of the organization and tick Single project when you upload the service account. Biq Blue then analyzes that project only.
- How long does the first synchronization take?
- Up to several tens of minutes when you have many projects, but you can start using Biq Blue as soon as the first project is synchronized.
- How much does it cost?
- The first month is free. See the pricing.
What's next
- Set up a BigQuery autoscaling reservation in 10 minutes
- Delete unused storage on BigQuery
- All articles
If you need a hand, just send us an email.