# RAG with a PDF using LlamaIndex and SimpleVectorStore on Vertex AI

Author: [Mete Atamel](https://atamel.dev/) | Published: 2025-03-24 | Topics: GenAI, GoogleAI, VertexAI, Gemini, Google Cloud Platform | Canonical: https://atamel.dev/posts/2025/03-24_rag_llamaindex_vertexai/


![LlamaIndex and Vertex AI](/img/2024/llamaindex_vertexai.png)

Previously, I showed how to do [RAG with a PDF using LangChain and Annoy Vector
Store](https://github.com/meteatamel/genai-beyond-basics/tree/main/samples/grounding/rag-pdf-annoy) and [RAG with a PDF
using LangChain and Firestore Vector
Store](https://github.com/meteatamel/genai-beyond-basics/tree/main/samples/grounding/rag-pdf-firestore). Both used a PDF
as the RAG backend and used LangChain as the LLM framework to orchestrate RAG ingestion and retrieval.

[LlamaIndex](https://www.llamaindex.ai/) is another popular LLM framework. I wondered how to set up the same PDF based
RAG pipeline with LlamaIndex and Vertex AI but I didn’t find a good sample. I put together a sample and in this short
post, I show how to set up the same PDF based RAG pipeline with LlamaIndex.

## Ingestion

Before you can use a PDF in a RAG pipeline, you need to first load the PDF into documents with `SimpleDirectoryReader`.
Each page in the PDF becomes a separate document:

```python
print("Read PDF into documents")
documents = SimpleDirectoryReader(
    input_files=["./cymbal-starlight-2024.pdf"]
).load_data()
```

Next, you need to split the documents into smaller chunks, create and store vector embeddings for each chunk.

First, you need an embedding model:

```python
print("Initialize embedding model")
credentials, project_id = google.auth.default()
embed_model = VertexTextEmbedding(
    credentials=credentials,
    model_name= "text-embedding-005",
)
```

You also need a `SentenceSplitter` to split documents into smaller chunks:

```python
text_splitter = SentenceSplitter(chunk_size=1024, chunk_overlap=20)
```

Finally, use `VectorStoreIndex.from_documents` to split the documents into smaller chunks and turn them into embeddings:

```python
print("Index documents")
index = VectorStoreIndex.from_documents(
    documents, embed_model=embed_model, transformations=[text_splitter]
)
```

## Querying

Now, we’re ready to query.

Create a query engine from the index and with a Vertex AI model:

```python
print("Initialize the query engine with the model")
llm = Vertex(model="gemini-2.0-flash-001")
query_engine = index.as_query_engine(llm=llm)
```

Start asking questions to the query engine:

```python
question = "What is the cargo capacity of Cymbal Starlight?"
response = query_engine.query(question)
print(f"Question: {question}")
print(f"Response: {str(response)}")
```

Run the full sample in [main.py](https://github.com/meteatamel/genai-beyond-basics/blob/main/samples/grounding/llamaindex-vertexai/main.py):

```sh
Read PDF into documents
Initialize embedding model
Index documents
Initialize the query engine with the model
Question: What is the cargo capacity of Cymbal Starlight?
Response: The cargo capacity of the Cymbal Starlight 2024 is 13.5 cubic feet.
```

As you can see, you get responses based on the PDF.

## Storage

Once the index is built, you can storage it to disk as follows:

```python
print("Store index locally")
index.storage_context.persist(persist_dir="index")
```

Next time, you can simply load the index:

```python
print("Reload index locally")
storage_context = StorageContext.from_defaults(persist_dir="index")
index = load_index_from_storage(storage_context, embed_model=embed_model)
```

## Conclusion

In this short blog post, I showed a sample on how to setup a PDF based RAG pipeline with LlamaIndex and Vertex AI.
For more information, you can check out these resources:

* [Build a RAG pipeline](https://docs.llamaindex.ai/en/stable/understanding/rag/)
* [LlamaIndex Vertex AI](https://docs.llamaindex.ai/en/stable/examples/llm/vertex/)