Google

Feature	Available
Tools	No
Multimodal	No

Chat UI can connect to the google Vertex API endpoints (List of supported models).

To enable:

Select or create a Google Cloud project.
Enable billing for your project.
Enable the Vertex AI API.
Set up authentication with a service account so you can access the API from your local workstation.

The service account credentials file can be imported as an environmental variable:

GOOGLE_APPLICATION_CREDENTIALS = clientid.json

Make sure your docker container has access to the file and the variable is correctly set. Afterwards Google Vertex endpoints can be configured as following:

MODELS=`[
  {
    "name": "gemini-1.5-pro",
    "displayName": "Vertex Gemini Pro 1.5",
    "endpoints" : [{
      "type": "vertex",
      "project": "abc-xyz",
      "location": "europe-west3",
      "extraBody": {
         "model_version": "gemini-1.5-pro-002",
      },
      // Optional
      "safetyThreshold": "BLOCK_MEDIUM_AND_ABOVE",
      "apiEndpoint": "", // alternative api endpoint url,
      "tools": [{
        "googleSearchRetrieval": {
          "disableAttribution": true
        }
      }]
    }]
  }
]`

GenAI

Or use the Gemini API API provider from:

Make sure that you have an API key from Google Cloud Platform. To get an API key, follow the instructions here.

You can either specify them directly in your .env.local using the GOOGLE_GENAI_API_KEY variables, or you can set them directly in the endpoint config.

You can find the list of models available here, and experimental models available here.

MODELS=`[
  {
    "name": "gemini-1.5-flash",
    "displayName": "Gemini Flash 1.5",
    "multimodal": true,
    "endpoints": [
      {
        "type": "genai",

        // Optional
        "apiKey": "abc...xyz"
        "safetyThreshold": "BLOCK_MEDIUM_AND_ABOVE",
      }
    ]
  },
  {
    "name": "gemini-1.5-pro",
    "displayName": "Gemini Pro 1.5",
    "multimodal": false,
    "endpoints": [
      {
        "type": "genai",

        // Optional
        "apiKey": "abc...xyz"
      }
    ]
  }
]`

< > Update on GitHub