gemini-docs/latest/content · Jun 26, 14:03 UTC
pages/embeddings.txt
TXT30.7 KB236 lines
route: /gemini-api/docs/embeddings
title: Embeddings
description:
The Gemini API offers embedding models to generate embeddings for text, images,
video, and other content. These resulting embeddings can then be used for tasks
such as semantic search, classification, and clustering, providing more accurate,
context-aware results than keyword-based approaches.
The latest model, gemini-embedding-2, is the first multimodal
embedding model in the Gemini API. It maps text, images,
video, audio, and documents into a unified embedding space, enabling cross-modal
search, classification, and clustering across over 100 languages. See the
multimodal embeddings section to learn more. For text-only
use cases, gemini-embedding-001 remains available.
Building Retrieval Augmented Generation (RAG) systems is a common use case for
AI products. Embeddings play a key role in significantly enhancing model outputs
with improved factual accuracy, coherence, and contextual richness. If you prefer
to use a managed RAG solution, we built the File Search
tool which makes doing RAG easier to manage and more cost effective.
Generating embeddings
Use the embedContent method to generate text embeddings:
Python
from google import genai
client = genai.Client()
result = client.models.embed_content(
model="gemini-embedding-2",
contents="What is the meaning of life?"
)
print(result.embeddings)
JavaScript
import { GoogleGenAI } from "@google/genai";
async function main() {
const ai = new GoogleGenAI({});
const response = await ai.models.embedContent({
model: 'gemini-embedding-2',
contents: 'What is the meaning of life?',
});
console.log(response.embeddings);
}
main();
Go
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"google.golang.org/genai"
)
func main() {
ctx := context.Background()
client, err := genai.NewClient(ctx, nil)
if err != nil {
log.Fatal(err)
}
contents := []*genai.Content{
genai.NewContentFromText("What is the meaning of life?", genai.RoleUser),
}
result, err := client.Models.EmbedContent(ctx,
"gemini-embedding-2",
contents,
nil,
)
if err != nil {
log.Fatal(err)
}
embeddings, err := json.MarshalIndent(result.Embeddings, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Println(string(embeddings))
}
REST
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-2:embedContent" \
-H "Content-Type: application/json" \
-H "x-goog-api-key: ${GEMINI_API_KEY}" \
-d '{
"model": "models/gemini-embedding-2",
"content": {
"parts": [{
"text": "What is the meaning of life?"
}]
}
}'
Note: While gemini-embedding-001 lets you generate individual embeddings for a
list of strings, Gemini Embedding 2 produces a single aggregated embedding for
multiple inputs. You can find more information in Embedding aggregation,
or you can use the Batch API if
you want to generate multiple embeddings at once.
Specify task type to improve performance
You can use embeddings for a wide range of tasks from classification to document
search. Specifying the right task type helps optimize the embeddings for the
intended relationships, maximizing accuracy and efficiency.
Task types with Embeddings 2
For text-only tasks with gemini-embedding-2, we strongly recommend you
add the task instruction in your prompt. This can be done by formatting the
query and the document with the correct task prefix.
The following tables show examples of how to format queries and documents for
symmetric and asymmetric use cases using the gemini-embedding-2 model.
Retrieval use cases (Asymmetric format)
In asymmetric use cases, add the task prefix to the query and apply
the document structure for the content you want to embed and retrieve.
Use case
Query structure
Document structure
Search query
task: search result | query: {content}
title: {title} | text: {content}
If there is no title, use title: none.
Question answering
task: question answering | query: {content}
title: {title} | text: {content}
Fact checking
task: fact checking | query: {content}
title: {title} | text: {content}
Code retrieval
task: code retrieval | query: {content}
title: {title} | text: {content}
Example usage
Python
# Generate embedding for a task's query. Use your correct task here:
def prepare_query(query):
# return f"task: question answering | query: {query}"
# return f"task: fact checking | query: {query}"
# return f"task: code retrieval | query: {query}"
return f"task: search result | query: {query}"
# Generate embedding for document of an asymmetric retrieval task:
def prepare_document(content, title=None):
if title is None:
title = "none"
return f"title: {title} | text: {content}"
Single-input use cases (Symmetric format)
In symmetric use cases, for the same task, use the same formatting
for the query and the document.
Use case
Input structure
Classification
task: classification | query: {content}
Clustering
task: clustering | query: {content}
Semantic similarity
task: sentence similarity | query: {content}
Do not use this for search or retrieval. It is intended for semantic textual similarity.
Example usage
Python
# Generate embedding for query & document of your task.
def prepare_query_and_document(content):
# return f'task: clustering | query: {content}'
# return f'task: sentence similarity | query: {content}'
return f'task: classification | query: {content}'
It is important that the task is used consistently. E.g., if documents are
embedded with f'task: classification | query: {content}', the query should
also be embedded following this task format.
Task types with Embeddings 1
For gemini-embedding-001, you can specify the task_type in the embedContent
method. For a complete list of supported task types, see the Supported task types
table.
Note: You cannot use the task_type field for the gemini-embedding-2 model. Instead, include the task as an instruction in your prompt, as detailed in the above section.
The following example shows how you can use SEMANTIC_SIMILARITY to check how
similar in meaning strings of texts are.
Python
from google import genai
from google.genai import types
import pandas as pd
from sklearn.metrics.pairwise import cosine_similarity
client = genai.Client()
texts = [
"What is the meaning of life?",
"What is the purpose of existence?",
"How do I bake a cake?",
]
result = client.models.embed_content(
model="gemini-embedding-001",
contents=texts,
config=types.EmbedContentConfig(task_type="SEMANTIC_SIMILARITY")
)
# Create a 3x3 table to show the similarity matrix
df = pd.DataFrame(
cosine_similarity([e.values for e in result.embeddings]),
index=texts,
columns=texts,
)
print(df)
JavaScript
import { GoogleGenAI } from "@google/genai";
// npm i compute-cosine-similarity
import * as cosineSimilarity from "compute-cosine-similarity";
async function main() {
const ai = new GoogleGenAI({});
const texts = [
"What is the meaning of life?",
"What is the purpose of existence?",
"How do I bake a cake?",
];
const response = await ai.models.embedContent({
model: 'gemini-embedding-001',
contents: texts,
config: { taskType: 'SEMANTIC_SIMILARITY' },
});
const embeddings = response.embeddings.map(e => e.values);
for (let i = 0; i < texts.length; i++) {
for (let j = i + 1; j < texts.length; j++) {
const text1 = texts[i];
const text2 = texts[j];
const similarity = cosineSimilarity(embeddings[i], embeddings[j]);
console.log(`Similarity between '${text1}' and '${text2}': ${similarity.toFixed(4)}`);
}
main();
Go
package main
import (
"context"
"fmt"
"log"
"math"
"google.golang.org/genai"
)
// cosineSimilarity calculates the similarity between two vectors.
func cosineSimilarity(a, b []float32) (float64, error) {
if len(a) != len(b) {
return 0, fmt.Errorf("vectors must have the same length")
}
var dotProduct, aMagnitude, bMagnitude float64
for i := 0; i < len(a); i++ {
dotProduct += float64(a[i] * b[i])
aMagnitude += float64(a[i] * a[i])
bMagnitude += float64(b[i] * b[i])
}
if aMagnitude == 0 || bMagnitude == 0 {
return 0, nil
}
return dotProduct / (math.Sqrt(aMagnitude) * math.Sqrt(bMagnitude)), nil
}
func main() {
ctx := c
…