Setting Up a Local DeepSeek and Dify Knowledge Base for AI Agents

In today's rapidly evolving AI landscape, creating intelligent agents that can access and utilize vast amounts of information is crucial. This article guides you through setting up DeepSeek and a Dify knowledge base locally, allowing you to build powerful and secure AI agents. This local setup offers unparalleled data privacy and control, making it ideal for businesses and individuals concerned about data security.

Prerequisites

Before diving into the setup, ensure you have the following:

• Hardware: A machine with at least 2 CPU cores and 16GB of RAM (recommended).
• Software:
  • Ollama: A cross-platform tool for managing and deploying large language models (LLMs).
  • Dify: An open-source AI application development platform.
  • Docker: A containerization platform for simplified deployment (if you are planning to use docker).

Step 1: Deploying DeepSeek-R1 with Ollama

Ollama simplifies the process of running large language models like DeepSeek locally.

1. Install Ollama:

   • Visit the Ollama website and download the version compatible with your operating system (MacOS, Windows, or Linux).
   • Follow the installation instructions provided on the website.

2. Verify Installation: Open your terminal and run ollama -v. This command should display the installed Ollama version.

3. Install DeepSeek Model: Use Ollama to download and run the DeepSeek-r1 model:

   ollama run deepseek-r1:7b
   

   For a less restricted version:

   ollama run huihui_ai/deepseek-r1-abliterated:7b
   

   Note: Choose the appropriate DeepSeek model size based on your machine's specifications. A 7B model requires approximately 4.7GB of data.

4. Test DeepSeek: Once the installation is complete, ask the model a question in the terminal. You should receive a response from the DeepSeek model.

Step 2: Deploying Dify Community Edition

Dify provides a user-friendly platform for building AI applications. Here's how to deploy the community edition:

1. Download the Code:

   • Using Git:

     git clone https://github.com/langgenius/dify.git
     

   • Web Download: Visit Dify's GitHub repository and download the source code as a ZIP file.

2. Navigate to the Docker Directory: Open your terminal and navigate to the dify/docker directory within the downloaded Dify folder.

3. Copy the Configuration File:

   cp .env.example .env
   

4. Configure Docker Proxy (Optional): If you're behind a proxy, configure Docker to use it. For example edit /etc/docker/daemon.json and add these lines:

   {
     "registry-mirrors": [
         "https://dockerpull.org",
         "https://docker.1panel.dev",
         "https://docker.foreverlink.love",
         ...
     ]
   }
   

5. Run Dify with Docker Compose:

   docker compose up -d
   

   This command will download the necessary Docker images and start the Dify application in detached mode.

Step 3: Integrating DeepSeek with Dify

Now that both DeepSeek and Dify are running, let's connect them:

1. Access Dify: Open your web browser and navigate to http://127.0.0.1. You should be greeted with the Dify setup page.

2. Initialize Admin Information: Create an administrator account for your Dify instance.

3. Configure the Model Provider:

   • Click on your profile icon (usually in the top right corner) and go to "Settings."
   • Navigate to "Model Providers" and select "Ollama."

4. Set Model Information: Configure the DeepSeek model details:

   • Model Name: deepseek-r1:7b
   • Base URL: http://<your_machine_ip>:11434 (If using containerized Dify, use host.docker.internal or your machine's IP instead of 127.0.0.1).
   • Maximum Token Limit: 32768

5. Create a Knowledge Base:

   • Upload your data by dragging and dropping files or by scraping content from websites.

6. Create an Agent: Design your AI agent by defining its purpose, instructions, and the knowledge base it should use.

   • Navigate in Dify to Dataset and create one, then upload your documents that will compose the knowledge base.
   • Then navigate to Application and create a new Agent that will use the previously created Dataset.

Benefits of Local Deployment

• Data Privacy: All data remains on your local machine, ensuring complete privacy and control.
• Security: Eliminates the risk of data breaches associated with cloud-based services.
• Customization: Tailor the AI agent to your specific needs and data without external dependencies.
• Cost-Effectiveness: Avoid recurring subscription fees for cloud-based AI platforms.

Conclusion

By following these steps, you can successfully set up a local DeepSeek and Dify knowledge base. This setup empowers you to build intelligent AI agents that offer enhanced privacy, security, and customization. Embrace the power of local AI and unlock new possibilities for your projects and businesses.

Further Reading:

• Setting up Dify on MAC with Docker: a guide
• Local Dify setup guide
• Integrating local Ollama models with Dify
• Uncensored LLMs: A discussion
• Cherry Studio model integration for DeepSeek