Building a Private SAP Knowledge Base with DeepSeek and RAG

Large language models (LLMs) like ChatGPT can sometimes produce inaccurate or nonsensical responses, a phenomenon known as "hallucination." This occurs because LLMs generate text based on learned patterns and statistical probabilities from vast datasets, which may lead to logical inconsistencies or inaccuracies. To mitigate this, Retrieval-Augmented Generation (RAG) offers a powerful solution.

This article explores how to leverage the DeepSeek model with RAG to create a private knowledge base for SAP-related information. This setup allows users to provide their own SAP documentation, ensuring accurate and relevant responses tailored to their specific needs, without the risk of data leaving their environment. We will use the following tools:

• DeepSeek: A powerful open-source large language model.
• Ollama: A tool for running LLMs locally.
• AnythingLLM: A platform for building applications using LLMs, with built-in RAG support.

Understanding RAG: Augmenting LLMs with Real-Time Knowledge

RAG enhances LLMs by enabling them to access and integrate information from external knowledge sources. When a user poses a question, the system performs these steps:

1. Retrieval: A retriever module searches the external knowledge base for relevant documents.
2. Augmentation: The retrieved documents are combined with the original question.
3. Generation: The augmented prompt is fed to the LLM, which generates an answer based on both the original query and the external knowledge.

Unlike the static knowledge stored within the LLM's parameters, the external knowledge base can be updated in real-time, ensuring the model provides accurate and up-to-date information.

Step-by-Step Implementation Guide

Here's a practical guide to building your own private SAP knowledge base using DeepSeek and RAG.

1. Installing and Running DeepSeek with Ollama

Ollama simplifies running LLMs locally, similar to how Docker manages containers.

• Installation: Download and install Ollama from the official Ollama website.
• Running DeepSeek: Open your terminal and execute the command ollama run deepseek-r1:1.5b. This downloads and runs the 1.5 billion parameter version of DeepSeek. Smaller models like this are ideal for local testing and resource-constrained environments.

Once the download is complete, a think prompt appears in the terminal indicating that DeepSeek is ready. Typing "who are you?" should elicit a response from DeepSeek, confirming its proper operation.

2. Preparing SAP Documentation as Knowledge Base Material

To create a specialized SAP knowledge base, you need to provide DeepSeek with relevant SAP documentation.

• Sourcing Documents: Search for SAP-related PDF documents using search engines like Bing, using a query like site:sap.com filetype:pdf.
• Privacy: This approach ensures data privacy as sensitive information remains within your local environment.

3. Setting Up a Workspace in AnythingLLM

AnythingLLM simplifies connecting user-provided documents to an AI model.

• Downloading and Installing AnythingLLM: Instructions can be found in the AnythingLLM documentation.
• Creating a Workspace: Similar to setting up a project in an IDE, create a new workspace in AnythingLLM to house your SAP knowledge base.
• Connecting to DeepSeek: In the workspace settings, select the DeepSeek R1 model managed by Ollama as the LLM provider. This allows you to interact with DeepSeek through AnythingLLM's graphical interface.

4. Embedding Documents and Utilizing a Vector Database

Vector databases are crucial for efficient storage and retrieval of document embeddings.

• Uploading Documents: Use the upload icon in AnythingLLM to upload the SAP PDF documents you collected earlier.
• Saving and Embedding: Click "Save and Embed" to convert the document content into vector embeddings and store them in the vector database.

A vector database stores data as high-dimensional vectors, enabling efficient similarity searches. This is critical for RAG, where the system needs to find the most relevant text snippets based on the user's query. Instead of manually uploading files, AnythingLLM supports importing from various data sources, such as GitHub repositories, using access tokens for secure access.

5. Verifying the Setup

Finally, it's time to test the knowledge base.

• Querying DeepSeek: In the AnythingLLM chat window, ask a question related to the uploaded SAP documentation, such as "What is embedded EWM?".
• Analyzing the Response: DeepSeek's answer should be based on the content of the uploaded PDF file. The "Citation" section in the response confirms which document was used to generate the answer, demonstrating the RAG workflow.

The Importance of Vector Databases

Vector databases are optimized for storing and querying high-dimensional vectors, which represent the meaning of text, images, and other types of data. Key benefits include.

• Efficient Similarity Search: Find the most relevant information quickly, even within large datasets.
• Support for Unstructured Data: Ideal for storing and retrieving text, images, audio, and video.

Model Deployment Options

While this article focuses on local deployment, enterprise users have the option to deploy privately on services like Tencent Cloud HAI for benefits like better performance, customization and data security. An article about deploying DeepSeek on Tencent Cloud HAI is avaliable for deeper details on that process. Private deployment ensures data does not leave the company infrastructure.

Conclusion

By leveraging DeepSeek, Ollama, and AnythingLLM, anyone can create a private and customized SAP knowledge base. This RAG-based approach offers a powerful solution for accessing accurate and relevant SAP information, addressing the limitations of general-purpose LLMs and ensuring data privacy.