Run DeepSeek-R1 Locally: A Comprehensive Guide

DeepSeek-R1, developed by the Chinese AI company DeepSeek, has been creating quite a stir in the AI community because of its capabilities and open-source availability. It is a compelling alternative to other models due to its proficiency in coding, problem-solving, and logical reasoning. This guide will walk you through the process of downloading and running DeepSeek-R1 on your computer using Ollama, allowing you to harness its power locally.

Why DeepSeek-R1?

DeepSeek-R1 is emerging as a strong contender in the AI model landscape for several key reasons:

• Cost-Effectiveness: Cheaper than many other models with similar capabilities.
• Exceptional Reasoning: Excels in complex problem-solving and logical reasoning tasks.
• Coding Prowess: Demonstrates impressive abilities in coding-related tasks.
• Chain-of-Thought Reasoning: Enhanced efficiency through built-in chain-of-thought reasoning and makes it superior to other AI models.
• Open Source: Allows for local use, customization, and experimentation.

Understanding Ollama

Ollama is a free and open-source tool designed to make running Natural Language Processing (NLP) models on your local machine as straightforward as possible. It simplifies the process of downloading, managing, and running models like DeepSeek-R1, abstracting away much of the underlying complexity.

Getting Started: Running DeepSeek-R1 in 3 Minutes

The following steps describe how you can quickly get started with DeepSeek-R1.

Step 1: Install Ollama

1. Go to the Ollama website .
2. Download the appropriate version for your operating system.
3. Follow the installation instructions provided on the website.

Step 2: Download DeepSeek-R1

A key feature of Ollama is that it enables you to use different DeepSeek-R1 parameters, according to your needs. Details about requirements can be found here. It is important to note that the choice of a higher parameter comes with increased hardware requirements: 1.5b, 7b, 8b, 14b, 32b, 70b, 671b.

Once Ollama is installed, type this command into your terminal to download the DeepSeek-R1 model:

ollama run deepseek-r1

This command will initiate the download. The duration depends on the speed of your internet connection.

Step 3: Model Verification

After the download, type the following command in the terminal:

ollama list

You should now see deepseek-r1 in the list of available models.

Step 4: Run DeepSeek-R1

Start the model by using the following command.

ollama run deepseek-r1

Step 5: Time for a Query

Once the installation and setup are successful, you can now start using the model. You can use it for a variety of tasks, including chain-of-thought reasoning and coding.

Building a RAG Application with DeepSeek and SingleStore

To expand your knowledge and create a simple RAG application, you can follow the steps below.

In this tutorial, the DeepSeek API key will be configured using NVIDIA NIM microservice. NVIDIA NIM (Inference Microservices) offers microservices that facilitate the deployment of AI models across different environments. We will use LangChain as our LLM framework and SingleStore as our vector database.

Prerequisites

• Free SingleStore Account
• Free DeepSeek API key from NVIDIA NIM

Step-by-step Guide

1. Set Up SingleStore:

   • Create a free SingleStore account.
   • Create a workspace
   • Create a database

2. Create a Notebook: SingleStore has an integrated Notebook feature. You can access it from the Data Studio.

   • Create a new Notebook.
   • Select your workspace and database from the dropdown menu.

3. Install Dependencies:

   !pip install langchain --quiet
   !pip install pdf2image --quiet
   !pip install pdfminer.six --quiet
   !pip install singlestoredb --quiet
   !pip install tiktoken --quiet
   !pip install --upgrade unstructured==0.10.14 --quiet
   !pip install -qU pypdf langchain_community
   !pip install langchain-nvidia-ai-endpoints --quiet
   !pip install langchain-deepseek-official --quiet
   

4. Import Libraries:

   from langchain.document_loaders import PyPDFLoader
   from langchain_nvidia_ai_endpoints import ChatNVIDIA
   from langchain.embeddings.openai import OpenAIEmbeddings
   from langchain.text_splitter import RecursiveCharacterTextSplitter
   from langchain.chains import RetrievalQA
   from langchain.vectorstores import SingleStoreDB
   import os
   

5. Load Documents:

   file_path = "https://unctad.org/system/files/official-document/wesp2023_en.pdf"
   loader = PyPDFLoader(file_path)
   data = loader.load()
   

6. Split Documents:

   text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=0)
   texts = text_splitter.split_documents(data)
   

7. Set Up Embeddings:

   os.environ["OPENAI_API_KEY"] = "Add your OpenAI API key"
   embedding = OpenAIEmbeddings()
   

8. Store Embeddings in SingleStore:

   docsearch = SingleStoreDB.from_documents(
       texts, embedding,
       table_name="deepseek_rag",  # Replace table name with any name
       host="admin:password@host_url:3306/database_name",  # Replace with your SingleStore connection
       port=3306
   )
   

   • Replace "Add your OpenAI API key" with your actual OpenAI API key.
   • Ensure you replace "admin:password@host_url:3306/database_name" with your SingleStore connection details.

9. Initialize DeepSeek through NVIDIA NIM:

   • Get your DeepSeek-R1 API Key for free from NVIDIA NIM microservice. Obtain it HERE.

   client = ChatNVIDIA(
       model="deepseek-ai/deepseek-r1",
       api_key="Add your DeepSeek-R1 API key you received from NVIDIA NIM microservice",  # Replace with your NVIDIA API key
       temperature=0.7,
       top_p=0.8,
       max_tokens=4096
   )
   

10. Create RAG Chain:

    qa_chain = RetrievalQA.from_chain_type(
        llm=client,
        chain_type="stuff",
        retriever=docsearch.as_retriever(search_kwargs={"k": 3}),
        return_source_documents=True
    )
    

11. Query the RAG System:

    query = "What India's GDP growth is projected to be?"
    result = qa_chain.invoke({"query": query})
    

12. Display Results:

    print("Answer:", result["result"])
    print("\nSources:")
    for doc in result["source_documents"]:
        print(f"- Page {doc.metadata['page']}: {doc.page_content[:100]}...")
    

A complete code repository is available on GitHub.

Conclusion

This guide has explained the process of setting up and running DeepSeek-R1 on your local machine using Ollama. This powerful setup lets you leverage local AI models, experiment with AI integration, and build AI applications.