How to easily monitor Bigquery table usage?
BigQuery is a powerful data warehouse that can store and manage petabytes of data. However, it is important to monitor how your BigQuery tables are being used in order to ensure that they are being used efficiently and cost-effectively.
There are a few different ways to monitor BigQuery table usage. One way is to use the BigQuery INFORMATION_SCHEMA views, the TABLES and JOBS views especially. They are related by the referenced_tables fields in the JOBS view, but it's a time consumming job.
List tables and used fields
The easy way is to use Biq Blue. You can filter the list of tables by the queries that use them. So listing tables used by one specific user is just one click.
For each table, Biq Blue displays a list of the combinations of fields with which the table is used and alerts you if these fields do not match a partition or cluster. With a simple link, you can access the queries, for each combination.
Conclusion
Now you can list all your tables, examine how they are used by all your queries, and consider improvements to the table structure (partition and cluster) to better match the query usage and improve cost and speed.
⭐ Additional tips