Gemini

Introduction

It’s no secret that LLMs sometimes lie and they do so in a very confident kind of way. This might be OK for some applications but it can be a real problem if your application requires high levels of accuracy.

I remember when the first LLMs emerged back in early 2023. I tried some of the early models and it felt like they were hallucinating half of the time. More recently, it started feeling like LLMs are getting better at giving more factual answers. But it’s just a feeling and you can’t base application decisions (or any decision?) on feelings, can you?

Introduction

Once in a while, you come across a feature that is so easy to use and so useful that you don’t know how you lived without it before. For me, Vertex AI’s Grounding with Google Search is one of those features.

In this blog post, I explain why you need grounding with large language models (LLMs) and how Vertex AI’s Grounding with Google Search can help with minimal effort on your part.

Introduction

Back in February, Google announced Gemini 1.5 Pro with its impressive 1 million token context window.

Larger context size means that Gemini 1.5 Pro can process vast amounts of information in one go — 1 hour of video, 11 hours of audio, 30,000 lines of code or over 700,000 words and the good news is that there’s good language support.

In this blog post, I will point out some samples utilizing Gemini 1.5 Pro in Google Cloud’s Vertex AI in different use cases and languages (Python, Node.js, Java, C#, Go).

A user recently reported an intermittent error with C# and Gemini 1.5 model on Vertex AI’s streaming API. In this blog post, I want to outline what the error is, what causes it, and how to avoid it with the hopes of saving some frustration for someone out there.

Error

The user reported using Google.Cloud.AIPlatform.V1 library with version 2.27.0 to use Gemini 1.5 via Vertex AI’s streaming API and running into an intermittent System.IO.IOException.

This week, I’m speaking at 3 meetups on Gemini Code Assist. My talk has a little introduction to GenAI and Gemini, followed by a series of hands-on demos that showcase different features of Gemini Code Assist.

In the demos, I setup Gemini Code Assist in Cloud Code IDE plugin in Visual Studio Code. Then, I show how to design and create an application, explain, run, generate, test, transform code, and finish with understanding logs with the help of Gemini.

Introduction

In my recent blog post, I’ve been exploring Vertex AI’s Gemini REST API and mainly talked about the streamGenerateContent method which is a streaming API.

Recently, a new method appeared in Vertex AI docs: generateContent which is the non-streaming (unary) version of the API.

In this short blog post, I take a closer look at the new non-streaming generateContent API and explain why it makes sense to use as a simpler API when the latency is not super critical.

Introduction

In my previous Using Vertex AI Gemini REST API post, I showed how to use the Gemini REST API from languages without SDK support yet such as C# and Rust.

There’s actually another way to use Gemini from languages without SDK support: GAPIC libraries. In this post, I show you how to use Vertex AI Gemini from GAPIC libraries, using C# as an example.

What is GAPIC?

At this point, you might be wondering: What’s GAPIC? GAPIC stands for Google API CodeGen. In Google Cloud, all services have auto-generated libraries from Google’s service proto files. Since these libraries are auto-generated, they’re not the easiest and most intuitive way of calling a service. Because of that, some services also have hand-written SDKs/libraries on top of GAPIC libraries.

Introduction

Back in December, Google announced Gemini, its most capable and general model so far available from Google AI Studio and Google Cloud Vertex AI.

The Try the Vertex AI Gemini API documentation page shows instructions on how to use the Gemini API from Python, Node.js, Java, and Go.

That’s great but what about other languages?

Even though there are no official SDKs/libraries for other languages yet, you can use the Gemini REST API to access the same functionality with a little bit more work on your part.