Eventarc

I talked about CloudEvents in the context of event-driven architectures before. In this post, let’s explore CloudEvents in more depth.

CloudEvents: Why? What?

CloudEvents is a popular specification for describing event data in a common way with the goal of increasing interoperability between different event systems.

Google Cloud’s Eventarc, open-source Knative, Azure’s Event Grid, and many more projects rely on CloudEvent specification to define their event formats.

CloudEvents, at its core, defines a set of metadata, called “context attributes”. This metadata is the minimal set of information needed to route an event to the proper component. Data that is not intended for routing is placed within the data (or base64 encoded data_base64) field of CloudEvent.

A few weeks ago, we announced third-party event sources for Eventarc, with the first cohort of third-party providers by our ecosystem partners. This blog post describes how to listen for events from one of those third-party providers, Datadog, and route them to a service in Google Cloud via Eventarc.

Datadog is a monitoring platform for cloud applications. It brings together end-to-end traces, metrics, and logs to make your applications and infrastructure observable. In Datadog, you can create monitors that track metrics, events, logs, integration availability, and network endpoints, and you can set alert thresholds on those monitors. Once these thresholds are reached, Datadog notifies your teams or services via email, Slack, and now a Google Cloud service via Eventarc.

In Workflows, it’s easy to chain various services together into an automated workflow. For some use cases, you might need to pause workflow execution and wait for some input. This input could be a human approval or an external service calling back with data needed to complete the workflow.

​​With Workflows callbacks, a workflow can create an HTTP endpoint and pause execution until it receives an HTTP callback to that endpoint. This is very useful for creating human-in-the-middle type workflows. In a previous blog post, Guillaume Laforge showed how to build an automated translation workflow with human validation using callbacks.

At Google Cloud I/O, we announced the public preview of Cloud Run jobs. Unlike Cloud Run services that run continuously to respond to web requests or events, Cloud Run jobs run code that performs some work and quits when the work is done. Cloud Run jobs are a good fit for administrative tasks such as database migration, scheduled work like nightly reports, or doing batch data transformation.

In this post, I show you a fully serverless, event-driven application to take screenshots of web pages, powered by Cloud Run jobs, Workflows, and Eventarc.

We’re happy to announce that you can now create Eventarc triggers to directly target Workflows destinations. Available as a preview feature, it simplifies event-driven orchestrations by enabling you to route Eventarc events to Workflows without having an intermediary service.

Integrating Eventarc and Workflows In a previous post, we talked about how to integrate Eventarc and Workflows. Since there was no direct integration between the two services, we had to deploy an intermediary Cloud Run service to receive events from Eventarc and then use the Workflows API to kick off a Workflows execution. Let’s take a look at a concrete example on how this worked.

Terraform is increasingly the preferred tool for building, changing, and versioning infrastructure in Google Cloud and across clouds. In an earlier post, I showed how to create Eventarc triggers using Google Cloud Console or via the command line with gcloud. In this post, I show how to create the same triggers with the google_eventarc_trigger Terraform resource.

See eventarc-samples/terraform on GitHub for the prerequisites and main.tf for full Terraform configuration.

Define a Cloud Run service as an event sink Before you can create a trigger, you need to create a Cloud Run service as an event sink for the trigger. You can use Terraform’s google_cloud_run_service resource to define a Cloud Run service:

You can now apply a path pattern when filtering in Eventarc. This is especially useful when you need to filter on resource names beyond exact match. Path pattern syntax allows you to define a regex-like expression that matches events as broadly as you like.

Let’s take a look at a concrete example.

Without path patterns

Let’s say you want to listen for new file creations in a Cloud Storage bucket with an AuditLog trigger.

December was a busy month for the Eventarc team, who landed a number of features at the end of the year. Let’s take a closer look at some of these new capabilities.

Cloud Storage trigger is GA Back in September, we announced the public preview of Cloud Storage triggers as the preferred way of routing Cloud Storage events to Cloud Run targets. They are now generally available. For more details, see the documentation on how to create a Cloud Storage trigger and check out my previous blog post on Cloud Storage triggers.

With event-driven architectures, it’s quite common to read events from a source in one region or project and route them to a destination in another region or another project. Let’s take a look at how you can implement cross-region and cross-project event routing in Google Cloud.

Cross-region event routing is straightforward in Google Cloud, whether you’re using Pub/Sub directly or Eventarc. Pub/Sub routes messages globally. When applications hosted in any region publish messages to a topic, subscribers from any region can pull from that topic. Eventarc enables you to route events across regions by creating a trigger in the region of the event’s source and specifying a destination in a different region. For more details, take a look at my previous blog post on Eventarc locations.

New locations in Eventarc Back in August, we announced more Eventarc locations (17 new regions, as well as 6 new dual-region and multi-region locations to be precise). This takes the total number of locations in Eventarc to more than 30. You can see the full list in the Eventarc locations page or by running gcloud eventarc locations list .

What does location mean in Eventarc? An Eventarc location usually refers to the single region that the Eventarc trigger gets created in. However, depending on the trigger type, the location can be more than a single region: