Document grounding in the generative AI hub

If you run on SAP BTP, you don't have to build the whole pipeline yourself. The generative AI hub in SAP AI Core offers document grounding as a managed service: it fetches documents, chunks them, creates embeddings, stores them in a managed SAP HANA vector store, and plugs retrieval into the orchestration service as the grounding module. This lesson walks through the moving parts as SAP documents them, using Contoso's program assistant as the example.

Three APIs and one module

The endpoints all sit under $AI_API_URL/v2/lm/document-grounding/: pipelines for the Pipelines API, vector/collections and related paths for the Vector API, and retrieval/search for the Retrieval API. Every call carries your SAP AI Core token and an AI-Resource-Group header.

Option 1: a pipeline from a repository

SAP recommends the Pipelines API when documents already live in a repository. As of this review, the documented repositories are Microsoft SharePoint, AWS S3, SFTP, SAP Build Work Zone, SAP Document Management service, ServiceNow (knowledge base articles in HTML) and Google Drive. Supported file formats include PDF, HTML, TXT, DOCX, PPT and common image formats (JPEG, PNG, TIFF), with JSON listed for SharePoint, S3 and Google Drive. Content refreshes daily, and each pipeline handles at most 8,000 documents. Check the Pipelines API page before you design, because SAP adds repositories and changes limits.

Two details catch people. First, includePaths only takes folder paths inside the site's default document library, and you can't combine it with a metadata server in the same request. Second, the pipeline indexes everything it can read. SAP's documentation warns that document grounding has no mechanism to detect or filter personal or privileged data, so you must keep such content out of the indexed folders. Lesson 6 returns to this.

Option 2: your own chunks through the Vector API

When documents don't sit in a supported repository, or you need control over chunking, you create a collection (with an embeddingConfig naming the embedding model and optional metadata), then post documents made of your own chunks, each with optional metadata. You don't need a generic secret for this path. SAP publishes capacity limits, for example up to 100 documents per ingestion request and up to 100 chunk results per search, so batch large loads.

Wiring the grounding module

In an orchestration configuration, the grounding module has the type document_grounding_service. Its filters name which repositories to search (a list of repository IDs, or "*" for all), the data_repository_type (vector for your own content), and a search_config such as max_chunk_count, which defaults to 10. Its placeholders map an input variable, such as groundingRequest, to an output variable, such as groundingOutput, which your prompt template then uses as {{?groundingOutput}}. You can also ask for metadata, such as a document's webUrl, with metadata_params, so the model can cite where each passage came from.

For region availability and pricing, SAP points to SAP Note 3437766. Pipeline creation is rate-limited (five create requests per minute per tenant at the time of writing), which matters only if you script many pipelines.