Using Gemini with PuppyGraph AI

Version requirement

Gemini support is available in PuppyGraph 1.14.0 and later.

PuppyGraph AI can use Google Gemini models. Gemini is not a separate option in PuppyGraph. Instead, you point PuppyGraph at Google's OpenAI-compatible Gemini API endpoint and select the OpenAI request structure:

Setting	Value for Gemini
`AI_API_STRUCTURE`	`openai_style`
`AI_BASE_URL`	`https://generativelanguage.googleapis.com/v1beta/openai/`
`AI_MODELS`	Gemini model IDs, for example `gemini-3.8-flash`
`AI_API_KEY`	A Gemini API key from Google AI Studio

Gemini uses openai_style because Google's Gemini API offers an endpoint that follows the OpenAI Chat Completions structure. The API key and model IDs come from Google, and PuppyGraph sends its requests to Google's endpoint, so you do not need an OpenAI account or an OpenAI API key.

The rest of this page walks through these settings and lists common errors.

Prerequisites

A running PuppyGraph 1.14.0 or later deployment that you can configure with environment variables. The examples below use Docker Compose.
A Google account with access to Google AI Studio.

PuppyGraph AI sends catalog metadata, sampled rows, and generated queries to the model API. Review the Gemini API terms for how Google handles that data on the tier you use before connecting production data.

Get a Gemini API key

Open Google AI Studio and sign in.
Create an API key, or copy an existing one. AI Studio creates the key in a Google Cloud project.
Store the key somewhere safe, such as a secret manager or an environment variable on the host. Do not commit it to source control.

Free-tier keys have low rate limits and do not include every model. If PuppyGraph AI will build graph schemas over many tables, serve several users, or use Pro models, enable billing on the key's project. See Gemini API rate limits.

Configure PuppyGraph

Set the AI_* variables on the PuppyGraph container. In Docker Compose:

services:
  puppygraph:
    image: puppygraph/puppygraph:latest
    environment:
      PUPPYGRAPH_USERNAME: puppygraph
      PUPPYGRAPH_PASSWORD: puppygraph123
      AI_ENABLED: "true"
      AI_API_STRUCTURE: openai_style
      AI_BASE_URL: https://generativelanguage.googleapis.com/v1beta/openai/
      AI_MODELS: gemini-3.8-flash,gemini-3.1-pro-preview,gemini-3.5-flash-lite
      AI_API_KEY: ${GEMINI_API_KEY:?Set GEMINI_API_KEY for PuppyGraph AI}
    ports:
      - "8081:8081"
      - "8182:8182"
      - "7687:7687"

Then export the key and start the container:

export GEMINI_API_KEY=<your-gemini-api-key>
docker compose up -d

A few details matter here:

Set AI_API_STRUCTURE whenever you set AI_BASE_URL. PuppyGraph does not guess the structure from the URL, and rejects a base URL without one.
Use the base URL, not a full endpoint. PuppyGraph appends /chat/completions to it, so requests go to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. The trailing slash is optional.
Always set AI_MODELS. Its default value lists Anthropic models, which Google's endpoint does not serve.

Choose models

AI_MODELS is a comma-separated list. Every entry appears in the model selector on the AI page, and the first entry is the default. PuppyGraph AI relies on streaming and tool calling, so pick Gemini text models that support both. For example:

Model ID	When to use it
`gemini-3.8-flash`	A good default for schema building and graph questions.
`gemini-3.1-pro-preview`	Harder reasoning over large or unfamiliar schemas. Needs billing enabled, because the free tier has no quota for it. Preview models can change or be retired on short notice.
`gemini-3.5-flash-lite`	Lower cost and latency for simple questions.

Google adds and retires models regularly. Check the Gemini models list for current IDs, and use the exact ID string Google publishes.

Thinking and Gemini-specific options

Gemini 3 models always reason before they answer, and Google does not let you turn that off. PuppyGraph does not send OpenAI's reasoning_effort or Gemini's thinking_config, so each model uses its default thinking level. PuppyGraph also does not use Gemini-only options that Google's endpoint accepts through extra_body, such as cached_content. This describes the current setup, and later releases may add more control over these options.

Google describes its OpenAI compatibility layer as beta. See Gemini API OpenAI compatibility for the parameters it supports.

Let each user bring their own key

Instead of sharing one deployment key, you can let each user call Gemini with their own Gemini API key, so that each user's requests count against the quota and billing of their own Google Cloud project. To do that, leave out AI_API_KEY. Each user then enters their own Gemini key, either when the AI page asks for one or in the PuppyGraph AI section of the Preferences tab on the Settings page. If you do set AI_API_KEY, a user's own key overrides it for that user.

Users can change only the key. The API structure, base URL, and model list stay under the deployment's control. PuppyGraph binds each saved key to the API structure and base URL in effect when it was saved, so if you switch an existing deployment from Anthropic to Gemini, users who saved an Anthropic key are asked for a Gemini key the next time they open the AI page.

Set CLUSTER_SECURITYKEY before users save keys

PuppyGraph encrypts saved personal keys with a key derived from CLUSTER_SECURITYKEY, and the default CLUSTER_SECURITYKEY is public. Set your own value before users save keys. For details, see the note on personal keys in Configuration and provider errors.

Verify the connection

Open the PuppyGraph Web UI at http://localhost:8081 and log in.
Open AI from the left navigation.
Check that the model selector lists the Gemini models from AI_MODELS.
Send a short message, such as:
```
List the catalogs you can see.
```

A reply that names your catalogs (or says that none are connected) confirms that PuppyGraph reached Gemini and that tool calling works. From here, follow Build a graph schema or the full Building a Graph with PuppyGraph AI walkthrough with the Gemini configuration above in place of the Anthropic one.

Troubleshooting

Errors fall into two groups, and they show up in different places.

Configuration errors stop PuppyGraph before it contacts Google. The AI page shows them as Invalid AI API configuration: followed by the reason. Fix the setting and restart PuppyGraph.

Reason after `Invalid AI API configuration:`	Cause and fix
`AI_API_STRUCTURE is required when AI_BASE_URL is set`	`AI_BASE_URL` is set without `AI_API_STRUCTURE`. Add `AI_API_STRUCTURE: openai_style`.
`AI_API_STRUCTURE must be anthropic_style or openai_style`	`AI_API_STRUCTURE` is set to a provider name, such as `gemini`, instead of a request structure. Set it to `openai_style`.
`AI_BASE_URL: enter the API base URL, not a complete model endpoint`	`AI_BASE_URL` ends in `/chat/completions`. Remove that suffix.

Errors returned by Google appear in the chat as AI service error: OpenAI-style API error (status <code>), followed by Google's message. PuppyGraph replaces your API key with [redacted] if the message repeats it. For status 429 and 503, PuppyGraph first retries the request up to three times over about 7 seconds, or after the wait Google asks for in Retry-After if that is 30 seconds or less, so these errors appear only when the retries did not succeed.

Error in the chat	Cause and fix
Starts with `Anthropic API error` instead of `OpenAI-style API error`	`AI_API_STRUCTURE` is `anthropic_style`. Gemini needs `openai_style`.
Status 400 or 401 saying the API key is not valid	The key is not a Gemini API key (for example, an Anthropic or OpenAI key), or it was deleted in AI Studio. Create a new key in Google AI Studio.
Status 403	The Gemini API is not enabled for the key's project, or the key has API restrictions that exclude it. Check the key in AI Studio or the Google Cloud console.
Status 404 saying the model is not found	The model ID in `AI_MODELS` is misspelled, retired, or not available to your project. Compare it with the Gemini models list.
Status 429 with `RESOURCE_EXHAUSTED`	You hit a Gemini rate limit or quota, and it did not clear during PuppyGraph's retries. Wait and send the message again, choose a model with higher limits, or enable billing on the project. If the message shows `limit: 0` for the model, the free tier has no quota for that model, so it needs billing.
Status 503 with `UNAVAILABLE` and "high demand"	Google is overloaded for that model, and it did not recover during PuppyGraph's retries. Send the message again after a short wait, or switch to another model in the selector.
Status 400 with `Function call is missing a thought_signature in functionCall parts`	Your PuppyGraph version is earlier than 1.14.0, so it does not send Gemini thought signatures back with tool calls, which Gemini 3 models require. Upgrade to PuppyGraph 1.14.0 or later.
Status 400 mentioning the maximum number of output tokens	`AI_MAX_TOKENS` (default `64000`) exceeds the model's output limit. Lower it to the limit listed for the model.

If the AI page keeps asking for a key after you switched to Gemini, the users' saved keys belong to the previous API and are not sent to Google. Set AI_API_KEY, or have each user enter a Gemini key. Saved keys are also encrypted with a key derived from CLUSTER_SECURITYKEY, so users are asked again for their key after CLUSTER_SECURITYKEY changes.