04-06-01 — Azure AI Search + Azure OpenAI: On Your Data

O que é "On Your Data"

Azure OpenAI On Your Data é uma funcionalidade que permite conectar diretamente o Azure OpenAI ao Azure AI Search (ou Blob Storage / CosmosDB) passando uma propriedade extra na chamada da Chat Completions API. O serviço faz o retrieval automaticamente antes de gerar a resposta, incluindo citações com número da fonte.

Setup passo a passo

1. Criar o índice no Azure AI Search

from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndex,
    SearchField,
    SearchFieldDataType,
    SimpleField,
    SearchableField,
    VectorSearch,
    HnswAlgorithmConfiguration,
    VectorSearchProfile,
)
from azure.core.credentials import AzureKeyCredential

index_client = SearchIndexClient(
    endpoint="https://meu-search.search.windows.net",
    credential=AzureKeyCredential("SUA_CHAVE_ADMIN"),
)

fields = [
    SimpleField(name="id", type=SearchFieldDataType.String, key=True),
    SearchableField(name="content", type=SearchFieldDataType.String, analyzer_name="pt.lucene"),
    SearchableField(name="title", type=SearchFieldDataType.String),
    SimpleField(name="source", type=SearchFieldDataType.String, filterable=True),
    SearchField(
        name="content_vector",
        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
        searchable=True,
        vector_search_dimensions=1536,
        vector_search_profile_name="myHnswProfile",
    ),
]

vector_search = VectorSearch(
    algorithms=[HnswAlgorithmConfiguration(name="myHnsw")],
    profiles=[VectorSearchProfile(name="myHnswProfile", algorithm_configuration_name="myHnsw")],
)

index = SearchIndex(name="rag-index", fields=fields, vector_search=vector_search)
index_client.create_or_update_index(index)
print("Índice criado com sucesso.")

2. Indexar documentos com embeddings

from azure.search.documents import SearchClient
from openai import AzureOpenAI

openai_client = AzureOpenAI(
    azure_endpoint="https://meu-openai.openai.azure.com",
    api_key="SUA_CHAVE_OPENAI",
    api_version="2024-08-01-preview",
)

search_client = SearchClient(
    endpoint="https://meu-search.search.windows.net",
    index_name="rag-index",
    credential=AzureKeyCredential("SUA_CHAVE_ADMIN"),
)

documentos = [
    {"id": "1", "title": "Política de RH", "content": "Funcionários têm direito a 30 dias de férias anuais...", "source": "rh-politica.pdf"},
    {"id": "2", "title": "Manual de Vendas", "content": "O processo de vendas tem 5 etapas: prospecção, qualificação...", "source": "manual-vendas.pdf"},
]

docs_para_indexar = []
for doc in documentos:
    response = openai_client.embeddings.create(
        input=doc["content"],
        model="text-embedding-3-small",
    )
    doc["content_vector"] = response.data[0].embedding
    docs_para_indexar.append(doc)

result = search_client.upload_documents(docs_para_indexar)
print(f"Indexados: {len(result)} documentos")

3. Usar On Your Data — Python

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://meu-openai.openai.azure.com",
    api_key="SUA_CHAVE_OPENAI",
    api_version="2024-08-01-preview",
)

# Propriedade extra_body ativa o On Your Data
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Você é um assistente corporativo. Responda somente com base nos documentos fornecidos."},
        {"role": "user", "content": "Quantos dias de férias um funcionário tem direito?"},
    ],
    extra_body={
        "data_sources": [
            {
                "type": "azure_search",
                "parameters": {
                    "endpoint": "https://meu-search.search.windows.net",
                    "index_name": "rag-index",
                    "authentication": {
                        "type": "api_key",
                        "key": "SUA_CHAVE_SEARCH",
                    },
                    "query_type": "vector_semantic_hybrid",  # hybrid search ativo
                    "semantic_configuration": "default",
                    "top_n_documents": 5,
                    "fields_mapping": {
                        "content_fields": ["content"],
                        "title_field": "title",
                        "url_field": "source",
                        "vector_fields": ["content_vector"],
                    },
                    "embedding_dependency": {
                        "type": "deployment_name",
                        "deployment_name": "text-embedding-3-small",
                    },
                },
            }
        ]
    },
)

answer = response.choices[0].message.content
citations = response.choices[0].message.context.get("citations", [])

print(f"Resposta: {answer}")
for i, citation in enumerate(citations):
    print(f"[{i+1}] {citation['title']} — {citation['url']}")

4. Usando On Your Data — C#

using Azure;
using Azure.AI.OpenAI;
using Azure.AI.OpenAI.Chat;
using OpenAI.Chat;

// Configurar cliente
AzureOpenAIClient azureClient = new(
    new Uri("https://meu-openai.openai.azure.com"),
    new AzureKeyCredential("SUA_CHAVE_OPENAI"));

ChatClient chatClient = azureClient.GetChatClient("gpt-4o");

// Configurar On Your Data com Azure AI Search
AzureSearchChatDataSource dataSource = new()
{
    Endpoint = new Uri("https://meu-search.search.windows.net"),
    IndexName = "rag-index",
    Authentication = DataSourceAuthentication.FromApiKey("SUA_CHAVE_SEARCH"),
    QueryType = DataSourceQueryType.VectorSemanticHybrid,
    SemanticConfiguration = "default",
    TopNDocuments = 5,
    FieldMappings = new DataSourceFieldMappings
    {
        ContentFieldNames = { "content" },
        TitleFieldName = "title",
        UrlFieldName = "source",
        VectorFieldNames = { "content_vector" },
    },
    VectorizationSource = DataSourceVectorizer.FromDeploymentName("text-embedding-3-small"),
};

ChatCompletionOptions options = new();
options.AddDataSource(dataSource);

ChatCompletion completion = await chatClient.CompleteChatAsync(
    [new UserChatMessage("Quantos dias de férias um funcionário tem?")],
    options);

Console.WriteLine($"Resposta: {completion.Content[0].Text}");

// Acessar citações
AzureChatMessageContext? context = completion.GetAzureMessageContext();
if (context?.Citations is not null)
{
    foreach (AzureChatCitation citation in context.Citations)
    {
        Console.WriteLine($"Fonte: {citation.Title} — {citation.Url}");
    }
}

On Your Data vs RAG Custom — Quando usar cada um