DeepSeek R1 has rapidly gained recognition for its advanced reasoning capabilities. The ability to run such a powerful AI model offline, especially with limited resources, offers significant advantages, particularly from a privacy perspective. This article provides a detailed guide on installing and using DeepSeek AI on a Linux system, covering everything from CUDA setup to Python integration.
By processing data locally, organizations can maintain sensitive information on-premises, significantly mitigating the risk of data breaches. Let's dive into how you can set up DeepSeek, use a GUI for seamless interaction, and integrate it into your Python scripts.
This step is specific to users leveraging the Windows Subsystem for Linux (WSL). If you have a native Linux installation with CUDA already configured then you may proceed to the next step. CUDA is a parallel computing platform and programming model developed by Nvidia for use with its GPUs. Here's how to enable CUDA within Ubuntu on WSL:
sudo apt update
sudo apt install build-essential
wget https://developer.download.nvidia.com/compute/cuda/12.6.2/local_installers/cuda_12.6.2_560.35.03_linux.run
sudo sh cuda_12.6.2_560.35.03_linux.run
After installation, verify CUDA is correctly installed using the nvidia-smi command. This command provides details about the installed NVIDIA drivers and GPU devices. You can further confirm the installation success by running the /usr/local/cuda-12.6/extras/demo_suite/deviceQuery script.
Ollama simplifies the process of creating, running, and deploying Large Language Model (LLM) applications. It provides flexible deployment options and makes managing and scaling applications easier. To install Ollama, use the following command:
curl -fsSL https://ollama.com/install.sh | sh
This command downloads and executes the Ollama installation script, setting up the necessary components to run DeepSeek.
DeepSeek-R1 offers multiple distilled models derived from Qwen and Llama architectures, tailored for different performance and resource requirements. The model sizes vary, with the 1.5b model around 2.3 GB, the 7b model approximately 4.7 GB, and the larger 70b model exceeding 40 GB.
You can find all available models on the Ollama library. To download a model, use the following command, replacing <model_name> with your desired version (e.g., deepseek-r1):
ollama pull <model_name>
Ollama will immediately switch to a prompt after downloading the model, allowing you to start interacting with DeepSeek right away.
Chatbox provides a user-friendly GUI for interacting with AI models. Once installed, configure Chatbox by selecting "OLLAMA API" as the Model Provider in the settings. Chatbox should automatically recognize the installed DeepSeek models.
With Chatbox, you can easily send prompts and receive responses from DeepSeek. The GUI makes it easier to manage conversations and refine your interactions with the model.
DeepSeek's API aligns well with the ChatGPT API, making integration with Python straightforward. You can use pip3 install openai, or alternatively, use the ollama Python package for a more direct approach.
First you should set up a virtual environment to manage dependencies:
python3 -m venv venv
source venv/bin/activate
pip3 install ollama
Here's a basic example of how to interact with DeepSeek using the ollama Python package:
import ollama
model_name = "deepseek-r1" # Or alternate model if preferred
response = ollama.chat(model=model_name, messages=[
{
'role': 'user',
'content': 'List five cybersecurity best practices.',
},
])
print(response['message']['content'])
This script sends a prompt to DeepSeek asking for cybersecurity best practices and prints the model’s response. This showcases how DeepSeek can be easily integrated into Python-based applications.
Artificial Intelligence models are becoming increasingly integral to our daily lives. The ability to run a model offline, even with limited computational resources, presents a significant advantage over relying solely on cloud-based solutions.
This capability opens up numerous possibilities for both cybersecurity defenders and attackers. Defenders can leverage it for learning, enhancing scripts, and identifying malicious activities, while attackers might explore generating ransomware or crafting sophisticated phishing campaigns. The future of AI, especially in the context of local and offline usage, is incredibly promising and warrants close attention.
