GenAI on Atamel.Dev

Codelab - Gemini for Developers

atamel@gmail.com (Mete Atamel) — Fri, 13 Feb 2026 00:00:00 +0000

Gemini for Developers

The Gemini ecosystem has evolved into a comprehensive suite of models, tools, and APIs. Whether you are “vibe-coding” a web app or deploying enterprise-grade agents, navigating the options can be overwhelming.

I am happy to announce a new Gemini for Developers codelab. This codelab is designed to teach you everything you need to know about the Gemini ecosystem, from the different model flavors to tools powered by Gemini to integration using the Google Gen AI SDK and the new Interactions API.

Gemini Interactions API - One interface for models and agents

atamel@gmail.com (Mete Atamel) — Tue, 03 Feb 2026 00:00:00 +0000

Interactions API overview

GenAI is rapidly moving from simple “prompt-and-response” patterns to complex, agentic workflows. To support this shift, Google recently introduced the Interactions API, a new unified foundation designed specifically for building with both models and agents.

In this post, I’ll introduce the core concepts of the Interactions API and walk through some of the samples available in my genai-samples repository.

What is the Interactions API?

Traditionally, developers had to use the Gemini API to talk to the models and another framework like Agent Development Kit (ADK) to create and manage agents. Currently in beta, the Interactions API simplifies this by providing a single interface for:

Introducing Google Cloud VertexAI Extensions for .NET

atamel@gmail.com (Mete Atamel) — Fri, 30 Jan 2026 00:00:00 +0000

Hero image

In October 2024, Microsoft announced the Microsoft.Extensions.AI.Abstractions and Microsoft.Extensions.AI libraries for .NET. These libraries provide the .NET ecosystem with essential abstractions for integrating AI services into .NET applications from various providers such as Open AI, Azure, Google.

Today, we’re happy to announce the Google.Cloud.VertexAI.Extensions library. This is the Vertex AI implementation of Microsoft.Extensions.AI. It enables .NET developers to integrate Google Gemini models on Vertex AI via the Microsoft.Extensions.AI abstractions.

Parallel agents in Antigravity

atamel@gmail.com (Mete Atamel) — Mon, 19 Jan 2026 00:00:00 +0000

Hero image

Google Antigravity transforms your regular IDE into an agentic development platform. In my previous blog posts, I showed some of the unique features of Antigravity compared to a regular IDE. In today’s blog post, I’ll talk about what makes Antigravity truly unique: Its ability to spin up and manage multiple agents.

Antigravity modes

Antigravity has two modes: Editor and Agent Manager.

Editor is your familiar IDE with an agent on the side to help you with tasks. You can read more about it in my previous blog post Google Antigravity Editor - Tips & Tricks and learn how to provide feedback to the agent in Provide Feedback to Google Antigravity and how to customize it in Customize Google Antigravity with rules and workflows.

Provide Feedback to Google Antigravity

atamel@gmail.com (Mete Atamel) — Wed, 10 Dec 2025 00:00:00 +0000

Google Antigravity Provide Feedback

At the heart of Google Antigravity is its ability to effortlessly gather your feedback at every stage of the experience. In this blog post, I will show you all the different ways you can provide feedback to Antigravity.

As the agent works on a task, it creates different artifacts along the way:

An implementation plan and a task list (before coding)
Code diffs (as it generates code)
A walkthrough to verify the results (after coding)

These artifacts are a way for Antigravity to communicate its plans and progress. More importantly, they’re also a way for you to provide feedback to the agent in Google docs style comments. This is very useful to effectively steer the agent in the direction you want.

Google Antigravity Editor - Tips & Tricks

atamel@gmail.com (Mete Atamel) — Mon, 01 Dec 2025 00:00:00 +0000

Google Antigravity Tips and Tricks

Google Antigravity is an agentic development platform where you have your familiar code editor along with a powerful agent on the side. In today’s post, I want to show you some tips and tricks for the code editor.

Setup and Extensions

In a typical setup, you’d have the editor, the terminal, and the agent visible:

Antigravity IDE

Customize Google Antigravity with rules and workflows

atamel@gmail.com (Mete Atamel) — Tue, 25 Nov 2025 00:00:00 +0000

Google Antigravity

Google Antigravity was announced last week as the next generation agentic IDE. I’m very impressed with it so far. It already helped me to upgrade my blog to the latest Hugo (that I’ve been putting off for a long time). It even recognized that some of the shortcodes (eg. Twitter) from the old version changed in the new version and automatically updated my blog posts with the new version of the shortcodes. Nice!

RAG just got much easier with File Search Tool in Gemini API

atamel@gmail.com (Mete Atamel) — Thu, 13 Nov 2025 00:00:00 +0000

File Search Tool

The Gemini team at Google recently announced the File Search Tool, a fully managed RAG system built directly into the Gemini API as a simple, integrated, and scalable way to ground Gemini. I gave it a try and I’m impressed how easy it is to use to ground Gemini with your own data.

In this blog post, I’ll introduce the File Search Tool and show you a concrete example.

Quick Guide to ADK Callbacks

atamel@gmail.com (Mete Atamel) — Mon, 03 Nov 2025 00:00:00 +0000

I’ve been exploring the Agent Developer Kit (ADK) and its powerful callbacks feature. In this blog post, I want to outline what callbacks are and provide a sample agent with all the callbacks implemented for a quick reference and testing.

At its core, an agent framework like ADK gives you a sequence of steps:

receive input → invoke model → invoke tools → return output

In real-world systems, we often need to hook into these steps for logging, guarding, caching, altering prompts or results, or dynamically changing behaviour based on session state. That’s exactly where callbacks come in. Think of callbacks as “checkpoints” in the agent’s lifecycle. The ADK framework automatically calls your functions at these key stages, giving you a chance to intervene.

Vibe coding an AI Trivia Quest app with Google AI Studio

atamel@gmail.com (Mete Atamel) — Mon, 27 Oct 2025 00:00:00 +0000

A few days ago, I saw Logan Kilpatrick’s Tweet about the new AI-first vibe coding experience in AI Studio:

Introducing the new AI first vibe coding experience in @GoogleAIStudio! Built to take you from prompt to production with Gemini, and optimized for AI app creation. Start building AI apps for free : )

More updates and features to come! pic.twitter.com/HpI7Dsl8Bj
— Logan Kilpatrick (@OfficialLoganK) October 21, 2025

Last time I tried AI Studio for vibe coding, it was mostly a single-page web application with all the code in a single file (which wasn’t ideal).

Introducing Google Gen AI .NET SDK

atamel@gmail.com (Mete Atamel) — Thu, 23 Oct 2025 00:00:00 +0000

Introducing Google Gen AI .NET SDK

Last year, we announced the Google Gen AI SDK as the new unified library for Gemini on Google AI (via the Gemini Developer API) and Vertex AI (via the Vertex AI API). At the time, it was only a Python SDK. Since then, the team has been busy adding support for Go, Node.js, and Java but my favorite language, C#, was missing until now.

Search Flights with Gemini Computer Use model

atamel@gmail.com (Mete Atamel) — Mon, 20 Oct 2025 00:00:00 +0000

Earlier this month, the Gemini 2.5 Computer Use model was announced. This model is specialized in interacting with graphical user interfaces (UI). This is useful in scenarios where a structured API does not exist for the model to interact with (via function calling). Instead, you can use the Computer Use model to directly interact with user interfaces such as filling and submitting forms.

It’s important to note that the model does not interact with the UI directly. As input, the model receives the user request, a screenshot of the environment, and a history of recent actions. As output, it generates a function call representing a UI action such as clicking or typing (see the full list of supported UI actions). It’s the client-side code’s responsibility to execute the received action and the process continues in a loop:

Secure your LLM apps with Google Cloud Model Armor

atamel@gmail.com (Mete Atamel) — Mon, 11 Aug 2025 00:00:00 +0000

Model armor

It’s crucial to secure inputs and outputs to and from your Large Language Model (LLM). Failure to do so can result in prompt injections, jailbreaking, sensitive information exposure, and more (as detailed in OWASP Top 10 for Large Language Model Applications).

I previously talked about LLM Guard and Vertex AI and showed how to use LLM Guard to secure LLMs. Google Cloud has its own service to secure LLMs: Model Armor. In this post, we’ll explore Model Armor and see how it can help to safeguard your LLM applications.

Gen AI Evaluation Service - Multimodal Metrics

atamel@gmail.com (Mete Atamel) — Tue, 05 Aug 2025 00:00:00 +0000

Multimodal metrics

This is the sixth and final post in my Vertex AI Gen AI Evaluation Service blog post series. In the previous posts, we covered computation-based, model-based, tool-use, and agent metrics. These metrics measure different aspects of an LLM response in different ways but one thing they all had in common: they are all for text-based outputs.

LLMs nowadays also produce multimodal (images, videos) outputs. How do you evaluate multimodal outputs? That’s the topic of this blog post.

Gen AI Evaluation Service - Agent Metrics

atamel@gmail.com (Mete Atamel) — Fri, 01 Aug 2025 00:00:00 +0000

Agent metrics

In my previous Gen AI Evaluation Service - Tool-Use Metrics post, we talked about LLMs calling external tools and how you can use tool-use metrics to evaluate how good those tool calls are. In today’s fifth post of my Vertex AI Gen AI Evaluation Service blog post series, we will talk about a related topic: agents and agent metrics.

What are agents?

There are many definitions of agents but an agent is essentially a piece of software that acts autonomously to achieve specific goals. They use LLMs to perform tasks, utilize external tools, coordinate with other agents, and ultimately produce a response to the user.

Gen AI Evaluation Service - Tool-Use Metrics

atamel@gmail.com (Mete Atamel) — Mon, 28 Jul 2025 00:00:00 +0000

Tool-use metrics

I’m continuing my Vertex AI Gen AI Evaluation Service blog post series. In today’s fourth post of the series, I will talk about tool-use metrics.

What is tool use?

Tool use, also known as function calling, provides the LLM with definitions of external tools (for example, a get_current_weather function). When processing a prompt, the model determines if a tool is needed and, if so, outputs structured data specifying the tool to call and its parameters (for example, get_current_weather(location='London')).

Gen AI Evaluation Service - Model-Based Metrics

atamel@gmail.com (Mete Atamel) — Tue, 08 Jul 2025 00:00:00 +0000

Model-based metrics

In the Gen AI Evaluation Service - An Overview post, I introduced Vertex AI’s Gen AI evaluation service and talked about the various classes of metrics it supports. In the Gen AI Evaluation Service - Computation-Based Metrics post, we delved into computation-based metrics, what they provide, and discussed their limitations. In today’s third post of the series, we’ll dive into model-based metrics.

The idea of model-based metrics is to use a judge model to evaluate the output of a candidate model. Using an LLM as a judge allows more flexible and rich evaluations that the computational/statistical metrics fail to do.

Gen AI Evaluation Service - Computation-Based Metrics

atamel@gmail.com (Mete Atamel) — Wed, 02 Jul 2025 00:00:00 +0000

In my Gen AI Evaluation Service - An Overview post, I introduced Vertex AI’s Gen AI evaluation service and talked about the various classes of metrics it supports. In today’s post, I want to dive into computation-based metrics, what they provide, and discuss their limitations.

Computation-based metrics are metrics that can be calculated using a mathematical formula. They’re deterministic – the same input produces the same score, unlike model-based metrics where you might get slightly different scores for the same input.

Gen AI Evaluation Service - An Overview

atamel@gmail.com (Mete Atamel) — Mon, 30 Jun 2025 00:00:00 +0000

Generating content with Large Language Models (LLMs) is easy. Determining whether the generated content is good is hard. That’s why evaluating LLM outputs with metrics is crucial. Previously, I talked about DeepEval and Promptfoo as some of the tools you can use for LLM evaluation. I also talked about RAG triad metrics specifically for Retrieval Augmented Generation (RAG) evaluation for LLMs.

In the next few posts, I want to talk about a Google Cloud specific evaluation service: the Gen AI evaluation service in Vertex AI. The Gen AI evaluation service in Vertex AI lets you evaluate any generative model or application against a set of criteria or your own custom criteria.

Evaluating RAG pipelines with the RAG triad

atamel@gmail.com (Mete Atamel) — Wed, 14 May 2025 00:00:00 +0000

Retrieval-Augmented Generation (RAG) emerged as a dominant framework for feeding Large Language Models (LLMs) the context beyond the scope of their training data and enabling LLMs to respond with more grounded answers and fewer hallucinations based on that context.

However, designing an effective RAG pipeline can be challenging. You need to answer questions such as:

How should you parse and chunk text documents for vector embedding? What chunk size and overlay size should you use?
What vector embedding model should you use?
What retrieval method should I use to fetch the relevant context? How many documents should you retrieve by default? Does the retriever 1.actually manage to retrieve the applicable documents?
Does the generator actually generate content that is in line with the retrieved context? What parameters (model, prompt template, temperature) work better?

The only way to objectively answer these questions is to measure how well the RAG pipeline works, but what exactly do you measure, and how do you measure it? This is the topic I’ll cover here.

DeepEval adds native support for Gemini as an LLM Judge

atamel@gmail.com (Mete Atamel) — Tue, 29 Apr 2025 00:00:00 +0000

DeepEval and Gemini

In my previous post on DeepEval and Vertex AI, I introduced DeepEval, an open-source evaluation framework for LLMs. I also demonstrated how to use Gemini (on Vertex AI) as an LLM Judge in DeepEval, replacing the default OpenAI judge to evaluate outputs from other LLMs. At that time, the Gemini integration with DeepEval wasn’t ideal and I had to implement my own integration.

Thanks to the excellent work by Roy Arsan in PR #1493, DeepEval now includes native Gemini integration. Since it’s built on the new unified Google GenAI SDK, DeepEval supports Gemini models running both on Vertex AI and Google AI. Nice!

Much simplified function calling in Gemini 2.X models

atamel@gmail.com (Mete Atamel) — Tue, 08 Apr 2025 00:00:00 +0000

Last year, in my Deep dive into function calling in Gemini post, I talked about how to do function calling in Gemini. More specifically, I showed how to call two functions (location_to_lat_long and lat_long_to_weather) to get the weather information for a location from Gemini. It wasn’t difficult but it involved a lot of steps for 2 simple function calls.

I’m pleased to see that the latest Gemini 2.X models and the unified Google Gen AI SDK (that I talked about in my Gemini on Vertex AI and Google AI now unified with the new Google Gen AI SDK) made function calling much simpler.

RAG with a PDF using LlamaIndex and SimpleVectorStore on Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 24 Mar 2025 00:00:00 +0000

LlamaIndex and Vertex AI

Previously, I showed how to do RAG with a PDF using LangChain and Annoy Vector Store and RAG with a PDF using LangChain and Firestore Vector Store. Both used a PDF as the RAG backend and used LangChain as the LLM framework to orchestrate RAG ingestion and retrieval.

LlamaIndex is another popular LLM framework. I wondered how to set up the same PDF based RAG pipeline with LlamaIndex and Vertex AI but I didn’t find a good sample. I put together a sample and in this short post, I show how to set up the same PDF based RAG pipeline with LlamaIndex.

Ensuring AI Code Quality with SonarQube + Gemini Code Assist

atamel@gmail.com (Mete Atamel) — Tue, 04 Mar 2025 00:00:00 +0000

In my previous Code Quality in the Age of AI-Assisted Development blog post, I talked about how generative AI is changing the way we code and its potential impact on code quality. I recommended using static code analysis tools to monitor AI-generated code, ensuring its security and quality.

In this blog post, I will explore one such static code analysis tool, SonarQube, and see how it improves the quality of AI-generated code.

Code Quality in the Age of AI-Assisted Development

atamel@gmail.com (Mete Atamel) — Tue, 28 Jan 2025 00:00:00 +0000

As developers transition from manual coding to AI-assisted coding, an increasing share of code is now being generated by AI. This shift has significantly boosted productivity and efficiency, but it raises an important question: how does AI-assisted development impact code quality? How can we ensure that AI-generated code maintains high quality, adheres to good style, and follows best practices? This question has been on my mind recently, and it is the topic of this blog post.

Improve the RAG pipeline with RAG triad metrics

atamel@gmail.com (Mete Atamel) — Tue, 21 Jan 2025 00:00:00 +0000

In my previous RAG Evaluation - A Step-by-Step Guide with DeepEval post, I showed how to evaluate a RAG pipeline with the RAG triad metrics using DeepEval and Vertex AI. As a recap, these were the results:

RAG triad with DeepEval

Answer relevancy and faithfulness metrics had perfect 1.0 scores whereas contextual relevancy was low at 0.29 because we retrieved a lot of irrelevant context:

The score is 0.29 because while the context mentions relevant information such as "The Cymbal Starlight 2024 has a cargo
capacity of 13.5 cubic feet", much of the retrieved context is irrelevant. For example, several statements discuss
towing capacity like "Your Cymbal Starlight 2024 is not equipped to tow a trailer", or describe how to access/load cargo
like "To access the cargo area, open the trunk lid using the trunk release lever located in the driver's footwell"
instead of focusing on the requested cargo capacity.

Can we improve this? Let’s take a look.

RAG Evaluation - A Step-by-Step Guide with DeepEval

atamel@gmail.com (Mete Atamel) — Tue, 14 Jan 2025 00:00:00 +0000

In my previous Evaluating RAG pipelines post, I introduced two approaches to evaluating RAG pipelines. In this post, I will show you how to implement these two approaches in detail. The implementation will naturally depend on the framework you use. In my case, I’ll be using DeepEval, an open-source evaluation framework.

Approach 1: Evaluating Retrieval and Generator separately

As a recap, in this approach, you evaluate the retriever and generator of the RAG pipeline separately with their own separate metrics. This approach allows to pinpoint issues at the retriever and the generator level:

Evaluating RAG pipelines

atamel@gmail.com (Mete Atamel) — Thu, 09 Jan 2025 00:00:00 +0000

Retrieval-Augmented Generation (RAG) emerged as a dominant framework to feed LLMs the context beyond the scope of its training data and enable LLMs to respond with more grounded answers with less hallucinations based on that context.

However, designing an effective RAG pipeline can be challenging. You need to answer certain questions such as:

How should you parse and chunk text documents for embedding? What chunk and overlay size to use?
What embedding model is best for your use case?
What retrieval method works most effectively? How many documents should you retrieve by default? Does the retriever actually manage to retrieve the relevant documents?
Does the generator actually generate content in line with the relevant context? What parameters (e.g. model, prompt template, temperature) work better?

The only way to objectively answer these questions is to measure how well the RAG pipeline works but what exactly do you measure? This is the topic of this blog post.

Gemini on Vertex AI and Google AI now unified with the new Google Gen AI SDK

atamel@gmail.com (Mete Atamel) — Tue, 17 Dec 2024 00:00:00 +0000

If you’ve been working with Gemini, you’ve likely encountered the two separate client libraries for Gemini: the Gemini library for Google AI vs. Vertex AI in Google Cloud. Even though the two libraries are quite similar, there are slight differences that make the two libraries non-interchangeable.

I usually start my experiments in Google AI and when it is time to switch to Vertex AI on Google Cloud, I couldn’t simply copy and paste my code. I had to go through updating my Google AI libraries to Vertex AI libraries. It wasn’t difficult but it was quite annoying.

Control LLM output with LangChain's structured and Pydantic output parsers

atamel@gmail.com (Mete Atamel) — Mon, 09 Dec 2024 00:00:00 +0000

In my previous Control LLM output with response type and schema post, I talked about how you can define a JSON response schema and Vertex AI makes sure the output of the Large Language Model (LLM) conforms to that schema.

In this post, I show how you can implement a similar response schema using LangChain’s structured output parser with any model. You can further get the output parsed and populated into Python classes automatically with the Pydantic output parser. This helps you to really narrow down and structure LLM outputs.

Tracing with Langtrace and Gemini

atamel@gmail.com (Mete Atamel) — Wed, 27 Nov 2024 00:00:00 +0000

Large Language Models (LLMs) feel like a totally new technology with totally new problems. It’s true to some extent but at the same time, they also have the same old problems that we had to tackle in traditional technology.

For example, how do you figure out which LLM calls are taking too long or have failed? At the bare minimum, you need logging but ideally, you use a full observability platform like OpenTelemetry with logging, tracing, metrics and more. You need the good old software engineering practices, such as observability, applied to new technologies like LLMs.

Batch prediction in Gemini

atamel@gmail.com (Mete Atamel) — Mon, 18 Nov 2024 00:00:00 +0000

LLMs are great in generating content on demand but if left unchecked, you can be left with a large bill at the end of the day. In my Control LLM costs with context caching post, I talked about how to limit costs by using context caching. Batch generation is another technique you can use to save time at a discounted price.

What’s batch generation?

Batch generation in Gemini allows you to send multiple generative AI requests in batches rather than one by one and get responses asynchronously either in a Cloud Storage bucket or a BigQuery table. This not only simplifies processing of large datasets, but it also saves time and money, as batch requests are processed in paralllel and discounted 50% from standard requests.

LLM Guard and Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 11 Nov 2024 00:00:00 +0000

LLM Guard and Vertex AI

I’ve been focusing on evaluation frameworks lately because I believe that the hardest problem while using LLMs is to make sure they behave properly. Are you getting the right outputs grounded with your data? Are outputs free of harmful or PII data? When you make a change to your RAG pipeline or to your prompts, are outputs getting better or worse? How do you know? You don’t know unless you measure. What do you measure and how? These are the sort of questions you need to answer and that’s when evaluation frameworks come into the picture.

Promptfoo and Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 04 Nov 2024 00:00:00 +0000

Promptfoo and Vertex AI

In my previous DeepEval and Vertex AI blog post, I talked about how crucial it is to have an evaluation framework in place when working with Large Language Models (LLMs) and introduced DeepEval as one of such evaluation frameworks.

Recently, I came across another LLM evaluation and security framework called Promptfoo. In this post, I will introduce Promptfoo, show what it provides for evaluations and security testing, and how it can be used with Vertex AI.

Firestore for Image Embeddings

atamel@gmail.com (Mete Atamel) — Tue, 29 Oct 2024 00:00:00 +0000

Firestore and LangChain

In my previous Firestore for Text Embedding and Similarity Search post, I talked about how Firestore and LangChain can help you to store text embeddings and do similarity searches against them. With multimodal embedding models, you can generate embeddings not only for text but for images and video as well. In this post, I will show you how to store image embeddings in Firestore and later use them for similarity search.

Firestore for Text Embedding and Similarity Search

atamel@gmail.com (Mete Atamel) — Wed, 09 Oct 2024 00:00:00 +0000

Firestore and LangChain

In my previous Persisting LLM chat history to Firestore post, I showed how to persist chat messages in Firestore for more meaningful and context-aware conversations. Another common requirement in LLM applications is to ground responses in data for more relevant answers. For that, you need embeddings. In this post, I want to talk specifically about text embeddings and how Firestore and LangChain can help you to store text embeddings and do similarity searches against them.

Persisting LLM chat history to Firestore

atamel@gmail.com (Mete Atamel) — Tue, 01 Oct 2024 00:00:00 +0000

Firestore and LangChain

Firestore has long been my go-to NoSQL backend for my serverless apps. Recently, it’s becoming my go-to backend for my LLM powered apps too. In this series of posts, I want to show you how Firestore can help for your LLM apps.

In the first post of the series, I want to talk about LLM powered chat applications. I know, not all LLM apps have to be chat based apps but a lot of them are because LLMs are simply very good at chat based communication.

Semantic Kernel and Gemini

atamel@gmail.com (Mete Atamel) — Mon, 19 Aug 2024 00:00:00 +0000

Semantic Kernel and VertexAI

Introduction

When you’re building a Large Language Model (LLMs) application, you typically start with the SDK of the LLM you’re trying to talk to. However, at some point, it might make sense to start using a higher level framework. This is especially true if you rely on multiple LLMs from different vendors. Instead of learning and using SDKs from multiple vendors, you can learn a higher level framework and use that to orchestrate your calls to multiple LLMs. These frameworks also have useful abstractions beyond simple LLM calls that accelerate LLM application development.

DeepEval and Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 12 Aug 2024 00:00:00 +0000

DeepEval and VertexAI

Introduction

When you’re working with Large Language Models (LLMs), it’s crucial to have an evaluation framework in place. Only by constantly evaluating and testing your LLM outputs, you can tell if the changes you’re making to prompts or the output you’re getting back from the LLM are actually good.

In this blog post, we’ll look into one of those evaluation frameworks called DeepEval, an open-source evaluation framework for LLMs. It allows to “unit test” LLM outputs in a similar way to Pytest. We’ll also see how DeepEval can be configured to work with Vertex AI.

Deep dive into function calling in Gemini

atamel@gmail.com (Mete Atamel) — Tue, 06 Aug 2024 00:00:00 +0000

Introduction

In this blog post, we’ll deep dive into function calling in Gemini. More specifically, you’ll see how to handle multiple and parallel function call requests from generate_content and chat interfaces and take a look at the new auto function calling feature through a sample weather application.

What is function calling?

Function Calling is useful to augment LLMs with more up-to-date data via external API calls.

You can define custom functions and provide these to an LLM. While processing a prompt, the LLM can choose to delegate tasks to the functions that you identify. The model does not call the functions directly but rather makes function call requests with parameters to your application. In turn, your application code responds to function call requests by calling external APIs and providing the responses back to the model, allowing the LLM to complete its response to the prompt.

Control LLM costs with context caching

atamel@gmail.com (Mete Atamel) — Fri, 19 Jul 2024 00:00:00 +0000

Context caching

Introduction

Some large language models (LLMs), such as Gemini 1.5 Flash or Gemini 1.5 Pro, have a very large context window. This is very useful if you want to analyze a big chunk of data, such as a whole book or a long video. On the other hand, it can get quite expensive if you keep sending the same large data in your prompts. Context caching can help.

Control LLM output with response type and schema

atamel@gmail.com (Mete Atamel) — Mon, 15 Jul 2024 00:00:00 +0000

Introduction

Large language models (LLMs) are great at generating content but the output format you get back can be a hit or miss sometimes.

For example, you ask for a JSON output in certain format and you might get free-form text or a JSON wrapped in markdown string or a proper JSON but with some required fields missing. If your application requires a strict format, this can be a real problem.

RAG API powered by LlamaIndex on Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 08 Jul 2024 00:00:00 +0000

LlamaIndex and Vertex AI

Introduction

Recently, I talked about why grounding LLMs is important and how to ground LLMs with public data using Google Search (Vertex AI’s Grounding with Google Search: how to use it and why) and with private data using Vertex AI Search (Grounding LLMs with your own data using Vertex AI Search).

In today’s post, I want to talk about another more flexible and customizable way of grounding your LLMs with private data: the RAG API powered by LlamaIndex on Vertex AI.

Grounding LLMs with your own data using Vertex AI Search

atamel@gmail.com (Mete Atamel) — Mon, 01 Jul 2024 00:00:00 +0000

Introduction

In my previous Vertex AI’s Grounding with Google Search: how to use it and why post, I explained why you need grounding with large language models (LLMs) and how Vertex AI’s grounding with Google Search can help to ground LLMs with public up-to-date data.

That’s great but you sometimes need to ground LLMs with your own private data. How can you do that? There are many ways but Vertex AI Search is the easiest way and that’s what I want to talk about today with a simple use case.

Give your LLM a quick lie detector test

atamel@gmail.com (Mete Atamel) — Thu, 06 Jun 2024 00:00:00 +0000

Lie Detector LLM

Introduction

It’s no secret that LLMs sometimes lie and they do so in a very confident kind of way. This might be OK for some applications but it can be a real problem if your application requires high levels of accuracy.

I remember when the first LLMs emerged back in early 2023. I tried some of the early models and it felt like they were hallucinating half of the time. More recently, it started feeling like LLMs are getting better at giving more factual answers. But it’s just a feeling and you can’t base application decisions (or any decision?) on feelings, can you?

Vertex AI's Grounding with Google Search - how to use it and why

atamel@gmail.com (Mete Atamel) — Wed, 29 May 2024 00:00:00 +0000

Introduction

Once in a while, you come across a feature that is so easy to use and so useful that you don’t know how you lived without it before. For me, Vertex AI’s Grounding with Google Search is one of those features.

In this blog post, I explain why you need grounding with large language models (LLMs) and how Vertex AI’s Grounding with Google Search can help with minimal effort on your part.

A tour of Gemini 1.5 Pro samples

atamel@gmail.com (Mete Atamel) — Tue, 07 May 2024 00:00:00 +0000

Introduction

Back in February, Google announced Gemini 1.5 Pro with its impressive 1 million token context window.

Gemini 1.5 Pro

Larger context size means that Gemini 1.5 Pro can process vast amounts of information in one go — 1 hour of video, 11 hours of audio, 30,000 lines of code or over 700,000 words and the good news is that there’s good language support.

In this blog post, I will point out some samples utilizing Gemini 1.5 Pro in Google Cloud’s Vertex AI in different use cases and languages (Python, Node.js, Java, C#, Go).

C# and Vertex AI Gemini streaming API bug and workaround

atamel@gmail.com (Mete Atamel) — Wed, 01 May 2024 00:00:00 +0000

A user recently reported an intermittent error with C# and Gemini 1.5 model on Vertex AI’s streaming API. In this blog post, I want to outline what the error is, what causes it, and how to avoid it with the hopes of saving some frustration for someone out there.

Error

The user reported using Google.Cloud.AIPlatform.V1 library with version 2.27.0 to use Gemini 1.5 via Vertex AI’s streaming API and running into an intermittent System.IO.IOException.

A Tour of Gemini Code Assist - Slides and Demos

atamel@gmail.com (Mete Atamel) — Wed, 24 Apr 2024 00:00:00 +0000

This week, I’m speaking at 3 meetups on Gemini Code Assist. My talk has a little introduction to GenAI and Gemini, followed by a series of hands-on demos that showcase different features of Gemini Code Assist.

In the demos, I setup Gemini Code Assist in Cloud Code IDE plugin in Visual Studio Code. Then, I show how to design and create an application, explain, run, generate, test, transform code, and finish with understanding logs with the help of Gemini.

Vertex AI Gemini generateContent (non-streaming) API

atamel@gmail.com (Mete Atamel) — Mon, 26 Feb 2024 00:00:00 +0000

Introduction

In my recent blog post, I’ve been exploring Vertex AI’s Gemini REST API and mainly talked about the streamGenerateContent method which is a streaming API.

Recently, a new method appeared in Vertex AI docs: generateContent which is the non-streaming (unary) version of the API.

In this short blog post, I take a closer look at the new non-streaming generateContent API and explain why it makes sense to use as a simpler API when the latency is not super critical.

Using Vertex AI Gemini from GAPIC libraries (C#)

atamel@gmail.com (Mete Atamel) — Wed, 14 Feb 2024 00:00:00 +0000

Gemini

Introduction

In my previous Using Vertex AI Gemini REST API post, I showed how to use the Gemini REST API from languages without SDK support yet such as C# and Rust.

There’s actually another way to use Gemini from languages without SDK support: GAPIC libraries. In this post, I show you how to use Vertex AI Gemini from GAPIC libraries, using C# as an example.

What is GAPIC?

At this point, you might be wondering: What’s GAPIC? GAPIC stands for Google API CodeGen. In Google Cloud, all services have auto-generated libraries from Google’s service proto files. Since these libraries are auto-generated, they’re not the easiest and most intuitive way of calling a service. Because of that, some services also have hand-written SDKs/libraries on top of GAPIC libraries.

Using Vertex AI Gemini REST API (C# and Rust)

atamel@gmail.com (Mete Atamel) — Mon, 05 Feb 2024 00:00:00 +0000

Introduction

Back in December, Google announced Gemini, its most capable and general model so far available from Google AI Studio and Google Cloud Vertex AI.

Gemini

The Try the Vertex AI Gemini API documentation page shows instructions on how to use the Gemini API from Python, Node.js, Java, and Go.

alt_text

That’s great but what about other languages?

Even though there are no official SDKs/libraries for other languages yet, you can use the Gemini REST API to access the same functionality with a little bit more work on your part.

Test and change an existing web app with Duet AI

atamel@gmail.com (Mete Atamel) — Mon, 29 Jan 2024 00:00:00 +0000

Duet AI

In the Create and deploy a new web app to Cloud Run with Duet AI post, I created a simple web application and deployed to Cloud Run using Duet AI’s help. Duet AI has been great to get a new and simple app up and running. But does it help for existing apps? Let’s figure it out.

In this blog post, I take an existing web app, explore it, test it, add a unit test, add new functionality, and add more unit tests all with the help of Duet AI. Again, I captured some lessons learned along the way to get the most out of Duet AI.

Create and deploy a new web app to Cloud Run with Duet AI

atamel@gmail.com (Mete Atamel) — Tue, 23 Jan 2024 00:00:00 +0000

Duet AI

I’ve been playing with Duet AI, Google’s AI-powered collaborator, recently to see how useful it can be for my development workflow. I’m pleasantly surprised how helpful Duet AI can be when provided with specific questions with the right context.

In this blog post, I document my journey of creating and deploying a new web application to Cloud Run with Duet AI’s help. I also capture some lessons learned along the way to get the most out of Duet AI.

C# library and samples for GenAI in Vertex AI

atamel@gmail.com (Mete Atamel) — Mon, 11 Dec 2023 00:00:00 +0000

In my previous post, I talked about multi-language libraries and samples for GenAI. In this post, I want to zoom into some C# specific information for GenAI in Vertex AI.

C# GenAI samples for Vertex AI

If you want to skip this blog post and just jump into code, there’s a collection of C# GenAI samples for Vertex AI.

C# GenAI samples for Vertex AI

Multi-language libraries and samples for GenAI in Vertex AI

atamel@gmail.com (Mete Atamel) — Tue, 28 Nov 2023 00:00:00 +0000

You might think that you need to know Python to be able to use GenAI with VertexAI. While Python is the dominant language in GenAI (and Vertex AI is no exception in that regard), you can actually use GenAI in Vertex AI from other languages such as Java, C#, Node.js, Go, and more.

Let’s take a look at the details.

Vertex AI SDK for Python

The official SDK for Vertex AI is Vertex AI SDK for Python and as expected, it’s in Python. You can initialize Vertex AI SDK with some parameters and utilize GenAI models with a few lines of code:

Generative AI Short Courses by DeepLearning.AI

atamel@gmail.com (Mete Atamel) — Tue, 04 Jul 2023 00:00:00 +0000

Introduction

In my previous couple posts (post1, post2), I shared my detailed notes on Generative AI Learning Path in Google Cloud’s Skills Boost. It’s a great collection of courses to get started in GenAI, especially on the theory underpinning GenAI.

Since then, I discovered another great resource to learn more about GenAI: Learn Generative AI Short Courses by DeepLearning.AI from Andrew Ng.

In this post, I summarize what each course teaches you to help you decide which course to take. I highly recommend taking all 4 courses. They’re full of useful information and short enough that even if you’re not fully interested in the topic, you can still get a good idea about it in a short amount of time.

Generative AI Learning Path Notes – Part 2

atamel@gmail.com (Mete Atamel) — Thu, 15 Jun 2023 00:00:00 +0000

If you’re looking to upskill in Generative AI, there’s a Generative AI Learning Path in Google Cloud Skills Boost. It currently consists of 10 courses and provides a good foundation on the theory behind Generative AI.

As I went through these courses myself, I took notes, as I learn best when I write things down. In part 1 of the blog series, I shared my notes for courses 1 to 6. In this part 2 of the blog series, I continue sharing my notes for courses 7 to 10.

Generative AI Learning Path Notes – Part 1

atamel@gmail.com (Mete Atamel) — Tue, 06 Jun 2023 00:00:00 +0000

If you’re looking to upskill in Generative AI (GenAI), there’s a Generative AI Learning Path in Google Cloud Skills Boost. It currently consists of 10 courses and provides a good foundation on the theory behind Generative AI and what tools and services Google provides in GenAI. The best part is that it’s completely free!

GenAI Learning Path

As I went through these courses myself, I took notes, as I learn best when I write things down. In this part 1 of the blog series, I want to share my notes for courses 1 to 6, in case you want to quickly read summaries of these courses.