Harnessing the Power of Gemini with Google AI Studio and LiteLLM

Google AI Studio provides a robust platform for developing AI applications, and when combined with LiteLLM, you unlock enhanced flexibility and control. This article dives deep into how to effectively use Gemini models through Google AI Studio with the help of LiteLLM.

What is Google AI Studio?

Google AI Studio is a comprehensive AI development platform designed for building and deploying generative AI applications. It offers a managed environment, making it easier to experiment with and utilize Google's powerful AI models like Gemini.

Key Features of Google AI Studio

• Managed Environment: Simplifies AI development.
• Generative AI Focus: Optimized for creating generative AI applications.
• Model Access: Direct access to Google's advanced models, including the Gemini family.

Integrating Gemini with LiteLLM

LiteLLM acts as a unified interface for various LLMs, including those from Google AI Studio. This integration allows you to seamlessly switch between different models and providers without altering your code.

Benefits of Using LiteLLM with Gemini

• Provider Agnostic: Easily switch between Gemini and other LLMs.
• Simplified API Calls: Consistent interface for different models.
• Enhanced Flexibility: Customize requests and manage configurations efficiently.

Setting Up Your Environment

Before you can start using Gemini with LiteLLM, you need to set up your environment and obtain the necessary API keys.

Prerequisites

1. Install LiteLLM: Use pip to install the LiteLLM Python package.

   pip install litellm
   

2. Obtain a Gemini API Key:

   • Go to the Google AI Studio.
   • Create a project and generate an API key.

3. Set Environment Variable: Set the GEMINI_API_KEY environment variable.

   import os
   os.environ["GEMINI_API_KEY"] = "your-api-key"
   

Making Your First API Call

Once your environment is set up, you can make your first API call to the Gemini model using LiteLLM.

Sample Code

from litellm import completion
import os

os.environ['GEMINI_API_KEY'] = "YOUR_GEMINI_API_KEY"  # Replace with your actual API key

response = completion(
    model="gemini/gemini-pro",
    messages=[{"role": "user", "content": "Write code for saying hi from LiteLLM"}]
)

print(response)

This code snippet demonstrates how to send a simple request to the gemini-pro model and print the response. Make sure to replace "YOUR_GEMINI_API_KEY" with your actual Gemini API key.

Key Parameters and Customizations

LiteLLM allows you to customize your requests to Gemini using various parameters. Here are some of the key OpenAI parameters supported by LiteLLM for Gemini:

• temperature: Controls the randomness of the output.
• top_p: Controls the cumulative probability of token selection.
• max_tokens: Limits the maximum number of tokens in the response.
• stream: Enables streaming responses.
• tools: Enables tool calling.
• tool_choice: Dictates which tool to use for a given call.
• response_format: Handle the type of response returned.
• n: Number of responses

Passing Gemini-Specific Parameters

You can also pass Gemini-specific parameters through LiteLLM. This allows you to leverage unique features offered by Gemini models.

from litellm import completion
import os

os.environ['GEMINI_API_KEY'] = "YOUR_GEMINI_API_KEY"

response = completion(
    model="gemini/gemini-1.5-pro",
    messages=[{"role": "user", "content": "List 5 popular cookie recipes."}],
    topK=1  # Gemini-specific parameter
)

print(response)

Validating Response Schema

When working with structured data, validating the response schema is crucial. LiteLLM supports sending response_schema parameter for Gemini-1.5-Pro on Google AI Studio.

Example with JSON Schema Validation

from litellm import completion, JSONSchemaValidationError
import os
import json

os.environ['GEMINI_API_KEY'] = "YOUR_GEMINI_API_KEY"

messages = [{ "role": "user", "content": "List 5 popular cookie recipes." }]
response_schema = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "recipe_name": { "type": "string" },
        },
        "required": ["recipe_name"],
    },
}

try:
    response = completion(
        model="gemini/gemini-1.5-pro",
        messages=messages,
        response_format={"type": "json_object", "response_schema": response_schema, "enforce_validation": True}
    )
    print(json.loads(response.choices[0].message.content))
except JSONSchemaValidationError as e:
    print(f"Raw Response: {e.raw_response}")
    raise e

This example demonstrates how to define a JSON schema and validate the response against it, ensuring data integrity.

Working with Images

Gemini-Pro-Vision model supports processing images. LiteLLM allows you to pass images via URLs or local file paths.

Sample Usage with Image URL

import os
import litellm
from dotenv import load_dotenv

load_dotenv()
os.environ["GEMINI_API_KEY"] = os.getenv('GEMINI_API_KEY')

prompt = 'Describe the image in a few sentences.'
image_url = 'https://storage.googleapis.com/github-repo/img/gemini/intro/landmark3.jpg'

messages = [
    {
        "role": "user",
        "content": [
            { "type": "text", "text": prompt },
            { "type": "image_url", "image_url": {"url": image_url} }
        ]
    }
]

response = litellm.completion(
    model="gemini/gemini-pro-vision",
    messages=messages,
)

content = response.get('choices', [{}])[0].get('message', {}).get('content')
print(content)

This code shows how to send an image URL to the gemini-pro-vision model and retrieve a description of the image.

Handling Inline Data

LiteLLM also supports sending inline data, such as audio streams, as base64 encoded strings, adhering to OpenAI's format.

Example with Audio Data

import litellm
from pathlib import Path
import base64
import os

os.environ["GEMINI_API_KEY"] = "YOUR_GEMINI_API_KEY"
litellm.set_verbose = True

audio_bytes = Path("speech_vertex.mp3").read_bytes()
encoded_data = base64.b64encode(audio_bytes).decode("utf-8")

model = "gemini/gemini-1.5-flash"
response = litellm.completion(
    model=model,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Please summarize the audio."},
                {
                    "type": "image_url",
                    "image_url": f"data:audio/mp3;base64,{encoded_data}",
                },
            ],
        }
    ],
)

print(response)

This example demonstrates how to encode an audio file to base64 and send it as inline data to the Gemini model.

Conclusion

Integrating Gemini with Google AI Studio via LiteLLM opens up a world of possibilities for AI development. By following this guide, you can effectively set up your environment, make API calls, customize requests, and handle various data types. This powerful combination allows you to leverage the advanced capabilities of Gemini in a flexible and efficient manner. Remember to consult the LiteLLM documentation and Google AI Studio documentation for more detailed information and advanced configurations.