# Grounding LLMs with your own data using Vertex AI Search

Author: [Mete Atamel](https://atamel.dev/) | Published: 2024-07-01 | Updated: 2024-06-25 | Topics: GenAI, VertexAI, Gemini, Google Cloud Platform | Canonical: https://atamel.dev/posts/2024/07-01_grounding_with_own_data_vertexai_search/


## Introduction

In my previous [Vertex AI's Grounding with Google Search: how to use it and why](https://cloud.google.com/blog/products/ai-machine-learning/using-vertex-ai-grounding-with-google-search) post, I explained why you need grounding with large language models (LLMs) and how Vertex AI’s grounding with Google Search can help to ground LLMs with public up-to-date data.

That’s great but you sometimes need to ground LLMs with your own private data. How can you do that? There are many ways but Vertex AI Search is the easiest way and that’s what I want to talk about today with a simple use case.

## Cymbal Starlight 2024

For the use case, imagine you own the 2024 model of a fictitious vehicle called Cymbal Starlight. It has a user’s manual in PDF format ([cymbal-starlight-2024.pdf](https://github.com/meteatamel/genai-beyond-basics/blob/main/samples/grounding/vertexai-search/cymbal-starlight-2024.pdf)) and you want to ask LLM questions about this vehicle from the manual. 

Since LLMs are not trained with this user manual, they’ll likely not be able to answer any questions about the vehicle but let’s see.

## Without Grounding

First, let's ask a question to the LLM about the vehicle without any grounding. You can do this in the [Vertex AI section of Google Cloud Console](https://console.cloud.google.com/vertex-ai/generative/language/create/text) against any model.

```log
Prompt: What is the cargo capacity of Cymbal Starlight?

Response: Unfortunately, I do not have access to real-time information,
including specific details about ships like "Cymbal Starlight" 
...
```

![Without grounding](https://raw.githubusercontent.com/meteatamel/genai-beyond-basics/main/samples/grounding/vertexai-search/images/without-grounding.png)

As you can see, the model thinks we're talking about a ship and cannot really answer the question.

There's also a
[main.py](https://github.com/meteatamel/genai-beyond-basics/blob/main/samples/grounding/vertexai-search/main.py)
sample code that you can run:

```shell
python main.py --project_id your-project-id
```

You get a similar response:

```log
Prompt: What is the cargo capacity of Cymbal Starlight?
Response text: I do not have access to real-time information,
including specific details about ships like the "Cymbal Starlight."
```

This is not surprising. Let’s now ground the LLM with this private PDF.

## Create datastore with PDF

To use a PDF for grounding, you need to upload the PDF to a Cloud Storage bucket
and set up a datastore to import from that bucket.

Create a Cloud Storage bucket with uniform bucket-level access and upload the PDF file:

```shell
gsutil mb -b on gs://cymbal-starlight-bucket
gsutil cp cymbal-starlight-2024.pdf gs://cymbal-starlight-bucket
```

Go to the [Agent Builder](https://console.cloud.google.com/gen-app-builder) section of the Google Cloud Console. Click on `Data Stores` on the left and `Create Data Store` button. Select `Cloud Storage`.

Point to the PDF file in the bucket and continue:

![PDF in bucket](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/pdf-in-bucket.png)

Give your datastore a name and click `Create`:

![Cymbal Datastore](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/cymbal-datastore.png)

You need to wait until the import is complete which can take up to 10-15 minutes:

![Cymbal datastore import](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/cymbal-datastore-import.png)

## Create a search app

Before you can use grounding, we need to create a search app and point to the datastore you just created.

Go to [Agent Builder](https://console.cloud.google.com/gen-app-builder) section of the Google Cloud Console and click on `Create App` button:

1. Select `Search` app type.
2. For Content, select `Generic`.
3. Make sure `Enterprise edition features` is enabled
4. Give your app a name and enter company info.

![Cymbal startlight app](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/cymbal-starlight-app.png)

In the next page, choose the data store you created earlier and click `Create`:

![Create app datastore](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/create-app-datastore.png)

## Setup grounding with Vertex AI Search

**Caution**: *Make sure the datastore import is complete before you continue. If you get an
error message: "Cannot use enterprise edition features" or something similar,
you might need to wait a little before trying again.*

Let's set up grounding with Vertex AI Search now.

Go back to [Vertex AI section of Google Cloud Console](https://console.cloud.google.com/vertex-ai/generative/language/create/text) and in the `Advanced` section, select `Enable grounding`:

![Enable grounding](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/enable-grounding.png)

Customize grounding and point to the datastore you just created:

![Customise grounding](https://github.com/meteatamel/genai-beyond-basics/raw/main/samples/grounding/vertexai-search/images/customise-grounding.png)

This is the format of the datastore string:

`projects/{}/locations/{}/collections/{}/dataStores/{}`

In my case, it's as follows:

`projects/genai-atamel/locations/global/collections/default_collection/dataStores/cymbal-datastore_1718710076267`

## With grounding

Finally, we're ready to ask questions about the vehicle with grounding enabled.

Let's start with the previous question:

```log
Prompt: What is the cargo capacity of Cymbal Starlight?
Response: The Cymbal Starlight 2024 has a cargo capacity of 13.5 cubic feet.
```

Let's ask another question:

```log
Prompt: What's the emergency roadside assistance phone number?
Response: 1-800-555-1212
```

Also run [main.py](https://github.com/meteatamel/genai-beyond-basics/blob/main/samples/grounding/vertexai-search/main.py) Python sample with grounding:

```shell
python main.py --project_id your-project-id \
    --data_store_path projects/your-project-id/locations/global/collections/default_collection/dataStores/your-datastore-id
```

You get a similar response:

```shell
Prompt: What is the cargo capacity of Cymbal Starlight?
Response text: The Cymbal Starlight 2024 has a cargo capacity of 13.5 cubic feet.
```

It works!

## Conclusion

In this blog post, I explored Vertex AI’s grounding with Vertex AI Search. It’s the easiest way of grounding LLMs with your own private data. If you want to learn more, here are some more resources:

* [Get started with generic search docs](https://cloud.google.com/generative-ai-app-builder/docs/try-enterprise-search)
* [Grounding decision flowchart](https://cloud.google.com/docs/ai-ml/generative-ai#grounding)
* [Getting Started with Grounding with Gemini in Vertex AI notebook](https://github.com/GoogleCloudPlatform/generative-ai/blob/main/gemini/grounding/intro-grounding-gemini.ipynb)
* [Grounding for Gemini with Vertex AI Search and DIY RAG talk](https://youtu.be/v4s5eU2tfd4)
* [Fix My Car sample app](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini/sample-apps/fixmycar)

