Workflows patterns and best practices - Part 1

For the last couple of years, we’ve been using Workflows, Google Cloud’s service orchestrator, to bring order to our serverless microservices architectures. As we used and gained more experience with Workflows and service orchestration, we shared what he had learned in conference talks, blog posts, samples, and tutorials. Along the way, some common patterns and best practices emerged. To help you take better advantage of Workflows and service orchestration on Google Cloud, we’ve summarized these proven patterns and best practices in a three-part series of blog posts. Read More ↗︎

Executing commands (gcloud, kubectl) from Workflows

In a previous post, I showed how to manage the lifecycle of a virtual machine using Workflows and the Compute Engine connector. This works well when there’s a connector for the resource you’re trying to manage. When there is no connector, you can try to use the API of the resource from Workflows, if there’s one. Alternatively, you can also use my favorite command line tool to manage the resource: gcloud. Read More →

Workflows that pause and wait for human approvals from Google Sheets

I’ve been writing a series of posts to showcase Google Workspace and Google Cloud Workflows integration. In my first post, I showed an IT automation use case in which a Google Sheets spreadsheet triggers a workflow to create virtual machines in Google Cloud. In the second post, I showed how to feed a Google Sheets spreadsheet with data from BigQuery using a workflow. In this third and final post of the series, I show how to design a workflow that pauses and waits for human approvals from Google Sheets. Read More →

Writing to Google Sheets from Workflows

In my previous post, I showed how to trigger a workflow in Google Cloud from a Google Sheets spreadsheet using Apps Script. In this post, I show how to do the reverse: write to Google Sheets from a workflow in Google Cloud. Use case Imagine you have some dataset in BigQuery. Periodically, you want to query and extract a subset of the dataset and save it to a Google Sheets spreadsheet. Read More →

Multi-environment service orchestrations

In a previous post, I showed how to use a GitOps approach to manage the deployment lifecycle of a service orchestration. This approach makes it easy to deploy changes to a workflow in a staging environment, run tests against it, and gradually roll out these changes to the production environment. While GitOps helps to manage the deployment lifecycle, it’s not enough. Sometimes, you need to make changes to the workflow before deploying to different environments. Read More ↗︎

GitOps your service orchestrations

GitOps takes DevOps best practices used for application development (such as version control and CI/CD) and applies them to infrastructure automation. In GitOps, the Git repository serves as the source of truth and the CD pipeline is responsible for building, testing, and deploying the application code and the underlying infrastructure. Nowadays, an application is not just code running on infrastructure that you own and operate. It is usually a set of first-party and third-party microservices working together in an event-driven architecture or with a central service orchestrator such as Workflows. Read More ↗︎

Triggering Workflows from Google Sheets

Is it possible to integrate Google Workspace tools such as Calendar, Sheets, and Forms with Workflows? For example, can you trigger a workflow from a Google Form or a Sheet? Turns out, this is not only possible but also easier than you might think. Let me show you how with a sample use case. Use case Imagine you are an administrator in charge of allocating virtual machines (VM) in your cloud infrastructure to users. Read More →

Creating Workflows that pause and wait for events

In Workflows, it’s easy to chain various services together into an automated workflow. For some use cases, you might need to pause workflow execution and wait for some input. This input could be a human approval or an external service calling back with data needed to complete the workflow. ​​With Workflows callbacks, a workflow can create an HTTP endpoint and pause execution until it receives an HTTP callback to that endpoint. Read More →

Taking screenshots of web pages with Cloud Run jobs, Workflows, and Eventarc

At Google Cloud I/O, we announced the public preview of Cloud Run jobs. Unlike Cloud Run services that run continuously to respond to web requests or events, Cloud Run jobs run code that performs some work and quits when the work is done. Cloud Run jobs are a good fit for administrative tasks such as database migration, scheduled work like nightly reports, or doing batch data transformation. In this post, I show you a fully serverless, event-driven application to take screenshots of web pages, powered by Cloud Run jobs, Workflows, and Eventarc. Read More ↗︎

Worklows state management with Firestore

In Workflows, sometimes, you need to store some state, a key/value pair, in a step in one execution and later read that state in another step in another execution. There’s no intrinsic key/value store in Workflows. However, you can use Firestore as a key/value store and that’s what I want to show you here. If you want to skip to see some samples, check out workflow.yaml. If you want to learn more about it, keep reading. Read More →