Serverless

Building microservices on Google Kubernetes Engine (GKE) provides you with maximum flexibility to build your applications, while still benefiting from the scale and toolset that Google Cloud has to offer. But with great flexibility comes great responsibility. Orchestrating microservices can be difficult, requiring non-trivial implementation, customization, and maintenance of messaging systems.

Cloud Run for Anthos now includes an events feature that allows you to easily build event-driven systems on Google Cloud. Now in beta, Cloud Run for Anthos’ event feature assumes responsibility for the implementation and management of eventing infrastructure, so you don’t have to.

Challenges in connecting services

When I think about my recent projects, I probably spent half of my time coding new services and the other half in connecting services. Service A calling Service B, or Service C calling an external service and using the result to feed into another Service D.

Connecting services is one of those things that ‘should be easy’ but in reality, it takes a lot of time and effort. You need to figure out a common connection format for services to use, make the connection, parse the results, and pass the results on. I’m not even mentioning error handling, retries and all those production readiness type features that you ultimately need to do.

My colleague Felipe Hoffa recently published a blog post titled Get started with BigQuery and dbt, the easy way. More specifically, he showed how to install dbt in Google Cloud Shell, configure it and manually run it to create a temporary dataset in BigQuery. This is great for testing dbt + BigQuery but how do you run this kind of setup in production?

dbt documentation states that Running dbt in production simply means setting up a system to run a dbt job on a schedule, rather than running dbt commands manually from the command line.

I finally got around to updating my Knative Tutorial from Knative v0.14.0 to the latest Knative v0.16.0 release. Since I skipped v0.15.0, I’m not sure which changes are due to v0.15.0 vs. v0.16.0. Regardless, there have been some notable changes that I want to outline in this blog post. This is not meant to be an exhaustive list. Feel free to let me know in the comments if there are other notable changes that I should be aware of.

.NET for Google Cloud Functions (Alpha)

I spoke at many .NET conferences over the last 3-4 years and one of the top requests I always received was: When will .NET be supported on Cloud Functions?

Unfortunately, I didn’t have a good answer for a while. That all changed last month with the following tweet from Jon Skeet from our C# team:

I'm thrilled that .NET support is coming to Google Cloud Functions, along with the .NET Functions Framework. Sign up for the public alpha at https://t.co/nASIACFCrg

Read More →

In part 3 of the blog series, I talked about how we transformed our Windows-only .NET Framework app to a containerized multi-platform .NET Core app.

This removed our dependency on Windows and enabled us to deploy to Linux-based platforms such as App Engine (Flex). On the other hand, the app still ran on VMs, it was billed per second even if nobody used it, deployments were slow and most importantly, it was a single monolith that was deployed and scaled as a single unit.

It’s hard to keep with Knative releases with a release every 6 weeks. I finally managed to update my Knative Tutorial for the latest Knative v0.12.0. In this blog post, I want to outline some of the differences I’ve observed.

Knative Serving

Knative Serving has been pretty stable in the recent releases and Knative Serving v0.12.0 is no exception. I didn’t need to update my tutorial specifically for this release.

Knative Eventing

Knative Eventing v0.12.0 changed the default yaml for Knative Eventing bundles. Now, they are under eventing.yaml (previously, it was release.yaml) and this is the yaml you need to point to install eventing. This makes sense as it’s more consistent with Knative Serving and its serving.yaml.

The default Knative Installation instructions for Google Kubernete Engine (GKE) is problematic (see bug 2266). In this post, I want to outline what the problem is, tell you what I do, and also provide you the scripts that work for me until a proper solution is implemented either in gcloud or Knative.

The problem

The default Knative Installation instructions tell you to create a GKE cluster as follows:

gcloud beta container clusters create $CLUSTER_NAME \
 --addons=HorizontalPodAutoscaling,HttpLoadBalancing,Istio \
 --machine-type=n1-standard-4 \
 --cluster-version=latest --zone=$CLUSTER_ZONE \
 --enable-stackdriver-kubernetes --enable-ip-alias \
 --enable-autoscaling --min-nodes=1 --max-nodes=10 \
 --enable-autorepair \
 --scopes cloud-platform

Notice the Istio add-on. This command creates a Kubernetes cluster with Istio already installed. This is good because Istio is a dependency of Knative but keep reading.

In my previous post, I talked about Knative v0.9.0 and some of the eventing changes in the latest release. I’ve been playing with Knative v0.9.0 since then to read Google Cloud Pub/Sub messages using PullSubscription and I ran into a rather fundamental issue that baffled me for a while. I’d like to outline the problem and the solution here, just in case it’s useful to others.

Knative Services as eventing sinks

In my PullSubscription, I could define Kubernetes Services as event sinks as follows:

Knative has been evolving pretty quickly. There’s a new release roughly every 6 weeks with significant changes in each release. Knative v0.7.0 was all about changes in Knative Serving (my post). Knative v0.8.0 was about deprecation of Knative Build in favor of Tekton Pipelines (my other post).

Knative Serving v0.9.0 and Eventing v0.9.0 have been released a little over a week ago. In Serving, there’s a v1 API and a number of improvements on autoscaling and cold starts. In Eventing, the way events are read changed quite a bit. I want to outline some of these changes here.