# Creating Workflows that pause and wait for events

Author: [Mete Atamel](https://atamel.dev/) | Published: 2022-08-05 | Topics: Workflows, Eventarc, Firestore, Serverless, Google Cloud Platform, How To | Canonical: https://atamel.dev/posts/2022/08-05_workflows_that_pause_for_events/


In [Workflows](https://cloud.google.com/workflows), it’s easy to chain various
services together into an automated workflow. For some use cases, you might need
to pause workflow execution and wait for some input. This input could be a human
approval or an external service calling back with data needed to complete the
workflow.

​​With Workflows
[callbacks](https://cloud.google.com/workflows/docs/creating-callback-endpoints),
a workflow can create an HTTP endpoint and pause execution until it receives an
HTTP callback to that endpoint. This is very useful for creating
human-in-the-middle type workflows. In a previous [blog
post](https://cloud.google.com/blog/topics/developers-practitioners/introducing-workflows-callbacks),
[Guillaume Laforge](https://twitter.com/glaforge) showed how to build an
automated translation workflow with human validation using callbacks.

Callbacks are great, but someone (or some service) has to know the callback
endpoint and make an HTTP call to that endpoint. And, many services send or
generate events, instead of calling HTTP endpoints. **Wouldn’t it be nice if you
could pause a workflow execution and resume when a specific event is received?**
For example, you could use this capability to create workflows that pause and
wait for a new message from a Pub/Sub topic or a new file to be created in a
Cloud Storage bucket.

## Event callbacks

While Workflows does not provide event callbacks out of the box, it’s possible
to have a workflow execution wait for an event by using callbacks, Firestore,
and Eventarc.

The idea is to use Firestore to store callback details from your original
workflow, use Eventarc to listen for events from various event sources, and use
a second event-triggered workflow to receive events from Eventarc and call back
to callback endpoints in your original workflow.

## Architecture

Here is the setup in more detail:

![Architecture diagram](/img/2022/workflows-that-pause-architecture.png)

1. A `callback-event-sample` workflow
   ([yaml](https://github.com/GoogleCloudPlatform/workflows-demos/blob/master/callback-event/callback-event-sample.yaml))
   creates a callback for an event source that it is interested in waiting
   events from.
1. It stores the callback information for the event source in a document in Firestore.
1. It carries on with its workflow and at some point, starts waiting for an event.
1. In the meantime, `callback-event-listener`
   ([yaml](https://github.com/GoogleCloudPlatform/workflows-demos/blob/master/callback-event/callback-event-listener.yaml))
   is ready to be triggered by events from a Pub/Sub topic and a Cloud Storage
   bucket with Eventarc.
1. At some point, Eventarc receives an event and triggers the event listener.
1. Event listener finds the document for the event source in Firestore.
1. Event listener calls back all the callback URLs registered with that event
   source. `callback-event-sample` workflow receives the event via its callback
   endpoint and stops waiting.
1. It deletes the callback URL from Firestore and continues with its execution.

Detailed set up instructions along with YAML workflow definitions are on
[GitHub](https://github.com/GoogleCloudPlatform/workflows-demos/tree/master/callback-event).

## Test

To test event callbacks, first execute the sample workflow:

```sh
gcloud workflows run callback-event-sample
```

Once it starts, the workflow will pause and wait, which you can confirm by its
running state on Google Cloud Console:

![Workflows execution waiting](/img/2022/workflows-execution-waiting.png)

To test Pub/Sub callbacks, send a Pub/Sub message:

```sh
TOPIC=topic-callback
gcloud pubsub topics publish $TOPIC --message="Hello World"
```

In the workflow logs, you should see that the workflow started and stopped
waiting for the event:

```sh
Started waiting for an event from source topic-callback
Stopped waiting for an event from source topic-callback
```

To test Cloud Storage events, upload a new file to Cloud Storage:

```sh
BUCKET=$PROJECT_ID-bucket-callback
echo "Hello World" > random.txt
gsutil cp random.txt gs://$BUCKET/random.txt
```

You should see that the workflow started and stopped waiting for the event:

```sh
Started waiting for an event from source $PROJECT_ID-bucket-callback
Stopped waiting for an event from source $PROJECT_ID-bucket-callback
```

At this point, the workflow should stop executing and you should also see the
received Pub/Sub and Cloud Storage events in the output:

![Workflows output](/img/2022/workflows-output.png)

---

This blog post covered how to create workflows that wait for Pub/Sub and Cloud
Storage events by using callbacks, Firestore, and Eventarc. You can extend the
sample to listen for any [events that Eventarc
supports](https://cloud.google.com/eventarc/docs/reference/supported-events).
Feel free to reach out to me on Twitter
[@meteatamel](https://twitter.com/meteatamel) for any questions
or feedback.