Multi-language libraries and samples for GenAI in Vertex AI

You might think that you need to know Python to be able to use GenAI with VertexAI. While Python is the dominant language in GenAI (and Vertex AI is no exception in that regard), you can actually use GenAI in Vertex AI from other languages such as Java, C#, Node.js, Go, and more.

Let’s take a look at the details.

Vertex AI SDK for Python

The official SDK for Vertex AI is Vertex AI SDK for Python and as expected, it’s in Python. You can initialize Vertex AI SDK with some parameters and utilize GenAI models with a few lines of code:

    vertexai.init(project=project_id, location=location)
    parameters = {
        "temperature": temperature,
        "max_output_tokens": 256,
        "top_p": 0.8,
        "top_k": 40
    }

    model = TextGenerationModel.from_pretrained("text-bison")
    response = model.predict(
        "Give me ten interview questions for the role of program manager.",
        **parameters,
    )

That’s great for Python but how can you do the same for other languages?

REST API for Vertex AI

It’s worth nothing that Vertex AI has a REST API. For example, you can utilize the same GenAI model via REST API as follows:

curl \
  -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  https://${REGION}-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/publishers/google/models/text-bison:predict -d \
  $'{
    "instances": [
      { "prompt": "Give me ten interview questions for the role of program manager."}
    ],
    "parameters": {
      "temperature": 0.2,
      "maxOutputTokens": 256,
      "topK": 40,
      "topP": 0.8
    }
  }'

You can also do the same REST API call in code from any language. The only complication is how you authenticate with Vertex AI and that depends on the language you’re using.

For example, here’s a C# console app and Cloud Functions utilizing the REST API and using application-default authentication.

Client libraries for Vertex AI

The REST API is a good start but there are also auto-generated client libraries for Vertex AI in multiple languages:

These client libraries take care of authentication and give you some strongly typed classes to work with. However, they are auto-generated and not as idiomatic to use as the Python SDK. They also use protobuf and you need to know how to construct the right protobuf which can be unintuitive.

For example, here’s the same sample, predict-text-prompt.js, using the Node.js client library:

  const {PredictionServiceClient} = aiplatform.v1;
  const {helpers} = aiplatform;
  const clientOptions = {
    apiEndpoint: 'us-central1-aiplatform.googleapis.com',
  };

  const publisher = 'google';
  const model = 'text-bison';

  const predictionServiceClient = new PredictionServiceClient(clientOptions);

  async function callPredict() {
    const endpoint = `projects/${project}/locations/${location}/publishers/${publisher}/models/${model}`;

    const prompt = {
      prompt:
        'Give me ten interview questions for the role of program manager.',
    };
    const instanceValue = helpers.toValue(prompt);
    const instances = [instanceValue];

    const parameter = {
      temperature: 0.2,
      maxOutputTokens: 256,
      topP: 0.95,
      topK: 40,
    };
    const parameters = helpers.toValue(parameter);

    const request = {
      endpoint,
      instances,
      parameters,
    };

    const response = await predictionServiceClient.predict(request);
    console.log(response);
  }

Definitely more verbose than the Vertex AI Python SDK but it works.

Multi-language GenAI samples for Vertex AI

Last but not least, there’s a collection of GenAI samples for Vertex AI in multiple languages: Python, Node.js, Java, Go, C#, and Ruby.

GenAI samples for Vertex AI

These samples show different GenAI use cases such as text classification, extraction, summarization, sentiment analysis and more. They are work in progress with some languages having more samples than others but they’re good starting points with GenAI on Vertex AI in different languages.

In this blog post, I summarized the state of Vertex AI SDKs and libraries and showed you a collection of samples for different languages.

SDK and client libraries:

Samples:

GenAI samples for Vertex AI

Feel free to reach out to me on Twitter @meteatamel for any feedback and questions.

GenAI  VertexAI  Google Cloud Platform 

See also