Google Cloud Platform Newsletter

Check Archive for older issues

News

Google Cloud Next ‘20: OnAir—delivering infrastructure for all your apps - A roll-up of new and recent infrastructure announcements at Google Cloud Next ‘20: OnAir.

Infrastructure for all: What happened at Next OnAir this week - In our third week at Next OnAir, we showed how Google cloud compute, networks, storage and global footprint support your applications.

Google Cloud Armor: Introducing 3 key features to protect your websites and applications - Simplifying the way you can use Cloud Armor to help protect your websites and applications.

In hybrid and multi-cloud environments, the network really matters - Enterprise Strategy Group details top networking considerations for hybrid and multi-cloud deployments.

Security, privacy, and compliance resources for Healthcare and Life Sciences customers - We have several recently published solution guides, whitepapers, and other assets to help Healthcare & Life Sciences organizations manage compliance.

New Private Service Connect simplifies secure access to services - The new Private Service Connect provides an easy, service-centric way to connect to Google Cloud services and protect network traffic.

Announcing the Grace Hopper subsea cable, linking the U.S., U.K. and Spain - The Grace Hopper private subsea cable ready to connect the UK and Spain to the US.

Google breaks AI performance records in MLPerf with world's fastest training supercomputer - Google set performance records in six out of the eight MLPerf benchmarks at the latest MLPerf benchmark contest.

Articles, Tutorials

Infrastructure, Networking, Security, Kubernetes

Sometimes Change is Bad: Immutable Infrastructure - Principles of immutable infrastructure.

Connecting an AWS and GCP VPC using an IPSec VPN Tunnel with BGP - The article explains how to set IPSec VPN Tunnel between AWS and GCP.

Azure Confidential Computing vs Google Cloud Confidential Computing - Deep dive into a comparison of Azure and GCP Confidential computing.

Stop downloading Google Cloud service account keys! - An alternative way to use Service Account keys instead of downloading them.

Network Latency and throughput— Measuring those metrics for Cloud Resources - An overview of different ways to measure latency and throughput for servers.

App Development, Serverless, Databases, DevOps

Preventing lateral movement in Google Compute Engine - Best practices, including concrete “dos and don’ts,” that can help you prevent security misconfigurations on Google Compute Engine.

Cloud Run adds support for gradual rollouts and rollbacks - Learn how to gradually roll out and roll back new changes to applications running on Cloud Run.

3 paths for disaster recovery for SAP systems on Google Cloud - An effective DR plan focuses on the technology systems supporting critical business functions; it involves a set of policies and procedures for recovering and/or continuing vital technology infrastructure and systems following any kind of disaster.

Top Five Reasons to use Crashlytics - A refresher on the essential tools that Crashlytics provides to help you debug crashes and get the most out of your crash reports.

How to Set Up a Deployment Pipeline on GCP with Cloud Build and Cloud Functions - Automatically deploying Cloud Function instances when changes get pushed to your Git repositories.

Cloud Functions For Firebase Boilerplate - A Github repository with boilerplate code.

Shiny on Google Cloud Run - Scale-to-Zero R Web Apps - Deploying R Shiny apps to Cloud Run.

Controlling Your Phone From the Cloud - Using Cloud IoT to communicate with an iPhone.

Spanner’s SQL Story - A description of evolution of SQL support for Cloud Spanner.

BigTable — Almost All You Need to Know - An overview of Cloud Bigtable. covering internal design, performance, tips for schema design etc.

Big Data, Analytics, ML&AI

Kafka to BigQuery using Dataflow - In this article, two different methods to connect Kafka to BigQuery using Dataflow are evaluated.

Computing Session for Event Data Using Google BigQuery - Explanation and examples on how to calculate sessions in BigQuery.

Get started with BigQuery and dbt, the easy way - Find here the quickest way to get started with dbt and BigQuery using only free offerings from Google Cloud.

Scheduled serverless dbt + BigQuery service - Using Cloud Run to execute periodically dbt commands.

How Servian helped REA Group repatriate 500TB of BigQuery data - Description of migrating BigQuery data from one region to another.

BigQuery Cost and Performance Optimization - Demonstrating on BigQuery public dataset on how to reduce query price.

BigQuery Omni: Distributed Query Engine Comes To Google Cloud - BigQuery’s distributed query engine extends support for multi-cloud data lakes.

Part 2: Building a Simple ETL Pipeline with Python and Google Cloud Functions — MySQL to BigQuery - Extracting data from a MySQL database and loading into Google BigQuery using Google Cloud Functions.

Document AI in Google Cloud Platform - Web system for text extraction from scanned documents based on Document AI.

Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib - Building a complete search engine on top of Kubernetes with AI personalized results.

Predicting Air Pollution with Prophet on GCP - Using EPA’s public data for time series forecasting.

Using GCP’s AI Platform to Predict Customer Churn - Developing a classification model to address customer churn.

Visualizing Pitcher Clusters: A Next OnAir Digital Experience - Analyzing baseball pitchers.

Various

GCP Data Engineering exam preparation, tips, and my journey from failing to passing experience - A story about failing and then succeeding in the certification exam.

Google Associate Cloud Engineer - Taking the online Associate Cloud Engineer exam.

Coursera’s Data Engineering with GCP Professional Certificate: better content or better marketing? - An honest review from a recently certified data student.

How to Pass The GCP Associate Cloud Engineer Exam - A complete guide for those who want to take this exam.

Slides, Videos, Audio

GCP Podcast - #229 Lucidworks with Radu Miclaus.

Kubernetes Podcast - #114 Scheduling, with David Oppenheimer.

Next OnAir sessions to get the most out of Google Cloud infrastructure - A curated list of breakout sessions from Google Cloud Next ‘20 OnAir infrastructure week.

Releases

Anthos - Anthos 1.3.3 is now available.

BigQuery - Updated version of Magnitude Simba ODBC driver includes performance improvements and bug fixes. INFORMATION_SCHEMA views for streaming metadata are now in alpha.

Compute Engine - N2D machine types are now available in asia-east1 in all three zones. When creating patch jobs, you can now choose whether to deploy zones concurrently or one at a time. N2 machines are now available in Sao Paulo southamerica-southeast1 in all three zones. You can access m2-megamem memory-optimized machine types in all zones that already have m2-ultramem memory-optimized machine types. Improved validation checks will be introduced on API calls to compute.googleapis.com starting on August 3, 2020 to increase reliability and REST API compliance of the Compute Engine platform for all users.

Cloud Dataflow - Dataflow now supports Dataflow Shuffle, Streaming Engine, FlexRS, and the following regional endpoints in GA: northamerica-northeast1 (Montréal) asia-southeast1 (Singapore) australia-southeast1 (Sydney).

Dataproc - Enabled Kerberos automatic-configuration feature. New sub-minor versions of Dataproc images: 1.3.65-debian10, 1.3.65-ubuntu18, 1.4.36-debian10, 1.4.36-ubuntu18, 1.5.11-debian10, 1.5.11-ubuntu18, 2.0.0-RC7-debian10, and 2.0.0-RC7-ubuntu18. 1.3+ images (includes Preview image): HADOOP-16984: Added support to read history files only from the done directory. Images 1.3 - 1.5: HIVE-20600: Fixed Hive Metastore connection leak. Images 1.5 - 2.0 preview: Upgraded the Cloud Storage connector to version 2.1.4 (see the GitHub change notes). Fixed an issue where optional components that depend on HDFS failed on single node clusters. Fixed an issue that caused workflows to be stuck in the RUNNING state when managed clusters (created by the workflow) were deleted while the workflow was running.

Dialogflow - GA (general availability) launch of mega agents. Beta launch of the Facebook Workplace integration. Beta launch of Dialogflow Messenger.

Cloud Functions - Cloud Functions is now available in the following regions: asia-south1 (Mumbai) asia-southeast2 (Jakarta) asia-northeast3 (Seoul) See Cloud Functions Locations for details.

GKE on Prem - Anthos GKE on-prem 1.3.3-gke.0 is now available. Fixes: Fixed CVE-2020-8559 described in Security bulletins.

IAM - We are delaying the upcoming changes for deleted members that are bound to a role.

Cloud Logging - The Logs field explorer panel is now generally available (GA).

Cloud Memorystore - Support for VPC Service Controls on Memorystore for Redis is now Generally Available.

Anthos Migrate - On July 28, 2020 we released Migrate for Anthos 1.4. For instructions on upgrading from version 1.3, see Upgrading Migrate for Anthos to 1.4. Added support for Anthos GKE on-prem clusters running on VMware. The Google Cloud Console provides a web-based, graphical user interface that you can use to manage your Google Cloud Console(GCP) projects and resources. You can use Migrate for Anthos to migrate Windows VMs to workloads on GKE. Migrate for Anthos now includes Custom Resource Definitions (CRDs) that enable you to easily create and manage migrations using an API automation solution or code. Added the new --json-key sa.json option to the migctl source create ce command to create a migration for Compute Engine, where sa.json specifies a service account. To edit the migration plan, you must now use the migctl migration get my-migration command to download the plan. Added the node-selectors and tolerations options to the migctl setup install installation command that lets you install Migrate for Anthos on a specific set of nodes or node pools in a cluster. The migctl migration cleanup command has been removed and is no longer necessary. In previous releases, you used a command in the form: migctl source create ce my-ce-src --project my-project --zone zone to create a migration for Compute Engine. The migctl migration logs command has been removed. By default Migrate for Anthos installs to and performs migrations in the v2k-system namespace. GKE on-prem preview: If a source was created with migctl source create using the wrong credentials, you could not delete the migration with migctl migration delete. 160309992: Editing a migration plan from the GUI console might fail if it was also edited using migctl. 161135630: Attempting multiple migrations of the same remote VM (from VMware, AWS or Azure) simultaneously, might result in a stuck migration process. 161214397: For Anthos on-prem, in case of a missing service-account to upload container images to the Container Registry, the migration might get stuck. 161110816: migctl migration create with a source that doesn't exist fails with a non-informative error message: request was denied. 161104564: Creating a Linux migration with wrong os-type specification causes the migration process to get stuck until deleted. 160858543, 160836394, 160844377, 154430477, 154403665, 153241390,153239696, 152408818, 151516642, 132002453: Unstable network in Migrate for Compute Engine infrastructure, or a GKE node restart, might cause migration to get stuck. 161787358: In some cases, upgrading from version v1.3 to v1.4 might fail with Failed to convert source message. 153811691, 153439420: Migrate for Anthos support for older Java does not handle OpenJDK 7 and 8 CPU resource calculations. 152974631: Using GKE nodes with CPU and Memory configurations below the recommended values might cause migrations to get stuck. 157890913, 160082702, 161125635, 159693579:A migration might continue to indicate that it is running, while an issue encountered prevents further processing.

Cloud Run - You can now tag Cloud Run revisions. Cloud Run is now available in asia-southeast1 (Singapore).

Security Command Center - Security Command Center v1beta1 API will be disabled on Jan. The SeverityLevel finding source property for all Security Health Analytics findings will be removed and replaced with a field named Severity, which retains the same values. The nodePools finding source property will be removed from the OVER_PRIVILEGED_SCOPES findings and replaced with a source property named VulnerableNodePools. The finding category of 2SV_NOT_ENFORCED is being renamed MFA_NOT_ENFORCED. The ExceptionInstructions source property will be removed from all Security Health Analytics findings. The ProjectId source property from all Security Health Analytics findings will be removed. The AssetSettings finding source property from PUBLIC_SQL_INSTANCE, SQL_PUBLIC_IP, SSL_NOT_ENFORCED, AUTO_BACKUP_DISABLED, SQL_NO_ROOT_PASSWORD, SQL_WEAK_ROOT_PASSWORD finding types will be removed, as it contains data duplicated from the asset entity. The Allowed finding source property from OPEN_FIREWALL findings will be replaced with changed a new field named ExternallyAccessibleProtocolsAndPorts, which will contain a subset of the values from the Allowed property. The SourceRanges finding source property from findings in OPEN_FIREWALL findings will be replaced with a new ExternalSourceRanges, which will contain a subset of the values from the SourceRanges property. As of Jan. As of Sept. The OffendingIamRoles source property in extensions of IAM Scanner Configurations will use structured data instead of a JSON-formatted string. The QualifiedLogMetricNames source property in specific Monitoring findings from Security Health Analytics will use a list instead of a character-separated string value. The AlertPolicyFailureReasons source property in specific Monitoring findings from Security Health Analytics will use a list instead of a character-separated string value. The CompatibleFeatures source property in WEAK_SSL_POLICY findings will use a list instead of a character-separated string value.

Cloud Spanner - The Cloud Spanner emulator is now generally available, enabling you to develop and test applications locally.

Cloud Storage Transfer - Transfers from Microsoft Azure Blob Storage are now generally available.

VPC Service Controls - General availability for the following integration: Memorystore for Redis.

Network Intelligence Center - Network Topology no longer supports infrastructure segments.

Anthos GKE deployed on-prem - Anthos GKE on-prem 1.3.3-gke.0 is now available. Fixes: Fixed CVE-2020-8559 described in Security bulletins.

Dialogflow Enterprise - GA (general availability) launch of mega agents. Beta launch of the Facebook Workplace integration. Beta launch of Dialogflow Messenger.

Anthos GKE on-prem - Anthos GKE on-prem 1.3.3-gke.0 is now available. Fixes: Fixed CVE-2020-8559 described in Security bulletins.