Real-Time Cost Anomaly Detection Now Available for Google BigQuery On-Demand & Reservations
We’re excited to roll out a major upgrade to how DoiT helps you stay in control of your Google Cloud spend: real-time anomaly detection for Google BigQuery, including both on-demand and reservation workloads. Once enabled, you’ll receive alerts about unexpected BigQuery cost spikes in under an hour.
What’s new
Until now, anomaly insights for BigQuery relied on next-day billing-file ingestion. That meant if a bad query ran at 2 PM today, you wouldn’t know until tomorrow.
With our new real-time detection pipeline for BigQuery on-demand, we continuously ingest and analyze live BigQuery usage metadata, flag unusual usage patterns, and send you Slack or email alerts in less than an hour-not the next day.
Why it matters
Real-time detection for BigQuery means that you can:
- Catch runaway queries in minutes: Prevent accidental or inefficient queries from racking up costs before anyone notices.
- Protect against operational mistakes: Get alerted when abnormal query activity starts impacting spend.
- Strengthen your security posture: Real-time cost changes can signal unauthorized data access or compromised systems.
As DoiT’s BigQuery expert Sayle Matthews can attest, the risk of unchecked queries racking up significant costs in a short period of time is very high:
“One of the largest issues is seeing how much their bill is at any given moment and being able to alert them when a ‘runaway query’ hits. We have seen some examples where customers have single queries that cost $2,000 USD and run in less than a minute, and of course, these were run multiple times in quick succession. These mistakes lead to massive bills that aren't caught for days or weeks later.”
Watch this video for a more detailed breakdown of one such example of a runaway query that could have cost a customer thousands in unnecessary charges.
What you need to do
This feature is available for all DoiT customers with a paid Enhanced, Premium, or Enterprise subscription and a connected Google Cloud account. To enable it, take the following steps:
- Locate the service account of interest on the Google Cloud access & features page.
- Select the kebab menu (⋮) next to the project connection, and then select Edit.
- Select the Real-time Anomalies – BigQuery checkbox to add the feature.
- Select Generate gcloud commands.
- Follow the instructions displayed in the side panel to update your custom role.
- Select Done to enable the feature
Next steps
For more information about real-time anomaly detection, consult our Help documentation or raise a support ticket.