DeepSeek-R1, developed by the Chinese AI company DeepSeek, has been creating quite a stir in the AI community because of its capabilities and open-source availability. It is a compelling alternative to other models due to its proficiency in coding, problem-solving, and logical reasoning. This guide will walk you through the process of downloading and running DeepSeek-R1 on your computer using Ollama, allowing you to harness its power locally.
DeepSeek-R1 is emerging as a strong contender in the AI model landscape for several key reasons:
Ollama is a free and open-source tool designed to make running Natural Language Processing (NLP) models on your local machine as straightforward as possible. It simplifies the process of downloading, managing, and running models like DeepSeek-R1, abstracting away much of the underlying complexity.
The following steps describe how you can quickly get started with DeepSeek-R1.
Step 1: Install Ollama
Step 2: Download DeepSeek-R1
A key feature of Ollama is that it enables you to use different DeepSeek-R1 parameters, according to your needs. Details about requirements can be found here. It is important to note that the choice of a higher parameter comes with increased hardware requirements: 1.5b, 7b, 8b, 14b, 32b, 70b, 671b.
Once Ollama is installed, type this command into your terminal to download the DeepSeek-R1 model:
ollama run deepseek-r1
This command will initiate the download. The duration depends on the speed of your internet connection.
Step 3: Model Verification
After the download, type the following command in the terminal:
ollama list
You should now see deepseek-r1 in the list of available models.
Step 4: Run DeepSeek-R1
Start the model by using the following command.
ollama run deepseek-r1
Step 5: Time for a Query
Once the installation and setup are successful, you can now start using the model. You can use it for a variety of tasks, including chain-of-thought reasoning and coding.
To expand your knowledge and create a simple RAG application, you can follow the steps below.
In this tutorial, the DeepSeek API key will be configured using NVIDIA NIM microservice. NVIDIA NIM (Inference Microservices) offers microservices that facilitate the deployment of AI models across different environments. We will use LangChain as our LLM framework and SingleStore as our vector database.
Set Up SingleStore:
Create a Notebook: SingleStore has an integrated Notebook feature. You can access it from the Data Studio.
Install Dependencies:
!pip install langchain --quiet
!pip install pdf2image --quiet
!pip install pdfminer.six --quiet
!pip install singlestoredb --quiet
!pip install tiktoken --quiet
!pip install --upgrade unstructured==0.10.14 --quiet
!pip install -qU pypdf langchain_community
!pip install langchain-nvidia-ai-endpoints --quiet
!pip install langchain-deepseek-official --quiet
Import Libraries:
from langchain.document_loaders import PyPDFLoader
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain.vectorstores import SingleStoreDB
import os
Load Documents:
file_path = "https://unctad.org/system/files/official-document/wesp2023_en.pdf"
loader = PyPDFLoader(file_path)
data = loader.load()
Split Documents:
text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=0)
texts = text_splitter.split_documents(data)
Set Up Embeddings:
os.environ["OPENAI_API_KEY"] = "Add your OpenAI API key"
embedding = OpenAIEmbeddings()
Store Embeddings in SingleStore:
docsearch = SingleStoreDB.from_documents(
texts, embedding,
table_name="deepseek_rag", # Replace table name with any name
host="admin:password@host_url:3306/database_name", # Replace with your SingleStore connection
port=3306
)
"Add your OpenAI API key" with your actual OpenAI API key."admin:password@host_url:3306/database_name" with your SingleStore connection details.Initialize DeepSeek through NVIDIA NIM:
client = ChatNVIDIA(
model="deepseek-ai/deepseek-r1",
api_key="Add your DeepSeek-R1 API key you received from NVIDIA NIM microservice", # Replace with your NVIDIA API key
temperature=0.7,
top_p=0.8,
max_tokens=4096
)
Create RAG Chain:
qa_chain = RetrievalQA.from_chain_type(
llm=client,
chain_type="stuff",
retriever=docsearch.as_retriever(search_kwargs={"k": 3}),
return_source_documents=True
)
Query the RAG System:
query = "What India's GDP growth is projected to be?"
result = qa_chain.invoke({"query": query})
Display Results:
print("Answer:", result["result"])
print("\nSources:")
for doc in result["source_documents"]:
print(f"- Page {doc.metadata['page']}: {doc.page_content[:100]}...")
A complete code repository is available on GitHub.
This guide has explained the process of setting up and running DeepSeek-R1 on your local machine using Ollama. This powerful setup lets you leverage local AI models, experiment with AI integration, and build AI applications.
