GCP BigQuery

GCP BigQuery is a serverless and cost-effective enterprise data warehouse that works across clouds and scales with your data.

Cisco Cloud Observability supports monitoring the following GCP BigQuery entities:

BigQuery: Represents and displays metrics for the entire BigQuery service in a project.
BigQuery Dataset: A top-level container that is used to organize and control access to your tables and views.
BigQuery Data Transfer: A service that automates data movement into BigQuery on a scheduled, managed basis.

You must configure cloud connections to monitor this entity. See Configure Google Cloud Platform Connection.

Cisco Cloud Observability displays GCP entities on the Observe page. Metrics are displayed for specific entity instances in the list and detail views.

This document contains references to third-party documentation. Splunk AppDynamics does not own any rights and assumes no responsibility for the accuracy or completeness of such third-party documentation.

Detail View

To display the detail view of a GCP BigQuery entity:

Navigate to the Observe page.
Under Database & Storage, click GCP BigQuery.
The list view is now displayed.
From the list, click an instance Name to display the detail view.
The detail view displays metrics, key performance indicators, and properties (attributes) related to the instance you selected.

Navigate to the Observe page.
Under Database & Storage, click GCP BigQuery.
The list view is now displayed.
In the left-hand Relationships map, click GCP BigQuery Datasets.
The list view is now displayed.
From the list, click an instance Name to display the detail view.
The detail view displays metrics, key performance indicators, and properties (attributes) related to the instance you selected.

Navigate to the Observe page.
Under Database & Storage, click GCP BigQuery.
The list view is now displayed.
In the left-hand Relationships map, click GCP BigQueries Data Transfers.
The list view is now displayed.
From the list, click an instance Name to display the detail view.
The detail view displays metrics, key performance indicators, and properties (attributes) related to the instance you selected.

Metrics and Key Performance Indicators

Cisco Cloud Observability displays the following metrics and key performance indicators (KPIs) for GCP BigQuery entities. See Google Cloud metrics.

Display Name	Source Metric Name	Description
Jobs (Count)	`job/num_in_flight`	In-flight jobs.
BI Engine Query Fallbacks (Count)	`query/biengine_fallback_count`	The reasons that queries failed BI Engine execution.
Query Execution Times (s)	`query/execution_times`	Distribution of execution times for queries that completed successfully within the last sampling interval.
Query (Count)	`query/count`	In-flight queries.
Query Statement Scanned Bytes (By)	`query/statement_scanned_bytes`	Scanned bytes broken down by statement type.
Slots Allocated	`slots/allocated`	Number of BigQuery slots currently allocated for project.
Slots Total Allocated For Reservation	`slots/total_allocated_for_reservation`	Number of BigQuery slots currently allocated across projects in the reservation.
Storage Insertall Inserted Rows (Count)	`storage/insertall_inserted_rows`	The number of rows uploaded by the project using the InsertAll streaming API.
Storage Bytes (By)	`storage/insertall_inserted_bytes`	The number of bytes uploaded by the project using the InsertAll streaming API.

Display Name	Source Metric Name	Description
Storage Tables (Count)	`storage/table_count`	Number of tables.
Storage Stored Bytes (By)	`storage/stored_bytes`	Number of bytes stored. The top 100 tables in a dataset are labeled, while the rest are grouped with no table label.
Storage Uploaded Rows (Count)	`storage/uploaded_row_count`	Uploaded rows.
Storage Uploaded Bytes (By)	`storage/uploaded_bytes`	Uploaded bytes.

Display Name	Source Metric Name	Description
Transfer Config Last Execution Job Rows (Count)	`transfer_config/last_execution_job_rows`	Number of returned rows from last executed job of transfer configuration.
Transfer Config Run Duration (s)	`transfer_config/run_duration_seconds`	Distribution of the execution time (in seconds) of each transfer run per transfer configuration.
Transfer Config Runs (Count)	`transfer_config/active_runs`	Number of transfer runs that are running or pending of the transfer configuration.
Transfer Config Runs (Count)	`transfer_config/completed_runs`	Number of completed DTS runs per transfer configuration.

Properties (Attributes)

Cisco Cloud Observability displays the following properties for GCP BigQuery entities.

Display Name	Source Property Name	Description
ID	-	The ID of the BigQuery entity, generated by Splunk AppDynamics.
Name	-	The name of the BigQuery entity, taken from the ID.
Project ID	-	The ID of the GCP project.
Region	-	The `global` string.

Display Name	Source Property Name	Description
ID	`selfLink`	The ID of the dataset.
Name	`friendlyName` `selfLink`	The name of the dataset.
Project ID	-	The ID of the GCP project.
Region	`location`	The geographical region the resource is running.
Default Table Expiration	`defaultTableExpirationMs`	The default lifetime of all tables in the dataset in milliseconds.
Default Partition Expiration	`defaultPartitionExpirationMs`	The default partition expiration in milliseconds.
Creation Time	`creationTime`	The time this dataset was created, in milliseconds since the epoch.
Last Modified Time	`lastModifiedTime`	The date this dataset was last modified, in milliseconds since the epoch.
Is Case Insensitive	`isCaseInsensitive`	`TRUE` if the dataset and its table names are case-insensitive; otherwise, the value is `FALSE`.
Default Collation	`defaultCollation`	Defines the default collation specification of future tables created in the dataset.
Default Rounding Mode	`defaultRoundingMode`	Defines the default rounding mode specification of new tables created within this dataset.
Max Time Travel	`maxTimeTravelHours`	Defines the time travel window in hours.
Storage Billing Model	`storageBillingModel`	Updates `storageBillingModel` for the dataset.

Display Name	Source Property Name	Description
ID	`name`	The resource ID of the transfer configuration.
Name	`displayName` `name`	The user-specified display name for the data transfer.
Project ID	-	The ID of the GCP project.
Region	`-`	The geographical region the resource is running, trimmed from the GCP `name` field.
Data Source ID	`dataSourceId`	The data source ID. This cannot be changed once the data transfer is created.
Schedule	`schedule`	The data transfer schedule.
Disable Auto Scheduling	`disableAutoScheduling`	If set to `true`, automatic scheduling of data transfer runs for this configuration will be disabled.
Start Time	`startTime`	Specifies the time to start scheduling transfer runs.
End Time	`endTime`	Specifies the time to stop scheduling transfer runs.
Data Refresh Window Days	`dataRefreshWindowDays`	The number of days to look back to automatically refresh the data. For example, if `data_refresh_window_days = 10`, then every day BigQuery reingests data for `[today-10, today-1]`, rather than ingesting data for just `[today-1]`.
Disabled	`disabled`	Specifies if this configuration is disabled.
Update Time	`updateTime`	The data transfer modification time.
Next Run Time	`nextRunTime`	The next time the data transfer will run.
State	`state`	The state of the most recently updated transfer run.
Dataset Region	`datasetRegion`	The region in which the BigQuery dataset is located.

Retention and Purge Time-To-Live (TTL)

For all cloud and infrastructure entities, the retention TTL is 180 minutes (3 hours) and the purge TTL is 525,600 minutes (365 days).

Third party names, logos, marks, and general references used in these materials are the property of their respective owners or their affiliates in the United States and/or other countries. Inclusion of such references are for informational purposes only and are not intended to promote or otherwise suggest a relationship between Splunk AppDynamics and the third party.