20 Jul Overview of Google Big Query
Today, businesses are rapidly migrating to the cloud. In fact, 94% of enterprises already use a cloud service, and by 2025, the data stored in cloud data centres will exceed 100 Zettabytes.
With the amount of data we use growing by the year, there’s an increasing need for platforms that help us organise this data to uncover useful insights.
However, it can be difficult to choose the best solution among the proliferation of data platforms.
In our continuing series into modern data platforms, we now look at Google’s Enterprise Data Warehouse solution, Big Query and how it could benefit your business.
What is Google Big Query?
Big Query is a fully managed data warehouse which provides a range of functions that leverage the benefits of Google’s mature cloud offerings:
- A scalable platform separating compute from storage to provide greater flexibility and choice
- A serverless architecture with zero infrastructure management
- Ability to encrypt all data (and options for encryption at rest)
- Next generation storage formats for optimised analytic workloads (to handle large scale datasets)
- A variety of user interfaces to access the data services including console interface, command line interface, client libraries and API’s.
- Like many leading platforms, Google provides the user options to pay as you consume (based on queries executed (by the second) and storage used), and/or to pay lower prices by reserving future capacity.
What are some of the key features ?
BigQuery’s greatest strengths are its data analytics features, which enable users to perform a broad variety of analysis:
- Adhoc analysis using standard SQL
- Geospatial analysis for analysis and visualisations of location data
- Machine Learning models can be built using standard SQL queries and using dozens of prebuilt standard AI algorithms
- In memory Business Intelligence Engine enabling end users to build rich, interactive dashboards and reports
The fact that Big Query uses ANSI SQL reduces the amount of time to become productive on BigQuery by using familiar and industry standard data programming languages.
Emerging features include BigQuery Omni which enables multicloud analytics across other clouds such as AWS and Azure. This is a very clever strategy by Google to support multi-cloud strategies, of which over 90% of organisations have now adopted. It is also moving into the data mesh space with Google Dataplex, which became generally available in late 2021.
How do I skill up on Big Query ?
Google offers accessible, free, hands on and online training to help practitioners learn how it works, how to configure it and how to get the most of it’s broad set of features. The Certification process is designed for all levels of skills and experience, starting with Cloud Foundational Certifications (typically no technical pre-requisites), Associate Certification and Professional Certifications.
To specialise in Data and Analytics, there are 3 paths – data analyst, data engineering and database engineer. The Data Analysis path includes 4 courses with over 40 hands on labs to teach you how to use BigQuery in a lab type environment. A perfect way to build your skills in Google.
How do the external analysts rate Big Query ?
The cloud data warehouse market is crowded with excellent vendors and technologies, so how does Big Query rate in this market? A few common observations include:
- BigQuery is extremely reliable over time when using large scale datasets (eg. Trillions of rows) with extremely low rates of downtime and/or outages
- Google have enabled free use of up to 1TB for Query and 10GB for storage per month, so its possible to get onto the platform and try it out
- The ability to build machine learning models using SQL is considered a massive advantage to many practitioners and opens up the platform to a broader set of skilled practitioners and traditional skillsets.
- BigQuery has the ability to easily and seamlessly connect to multiple data sources including AWS and Microsoft
- BigQuery captures meta data efficiently and quickly
Big Query is an impressive born in the cloud enterprise data warehouse with a very strong analytics capability embedded into the platform. Given Google’s heritage in the cloud, the BigQuery platform provides support for extremely large datasets, and has an extremely competitive commercial offering.
As more aspects of our life are becoming digitised, the importance of data is continuing to grow. Adopting a modern data platform for your business will help you remain competitive and adapt to the changing demands of the modern business landscape.
Learn More:
Cloud data platforms – our services
Migration to the cloud: the basics
7 steps to successful data migration – how to migrate to the cloud
Our Modern Data Platforms series