In our previous coverage, we discussed installing both Ollama and OpenWebUI within Docker containers. This streamlines the setup process significantly. So with that being said, I'll assume you already have Ollama and Open WebUI installed alongside a decent GPU.

With all the buzz around DeepSeek R1 going around, I wanted to touch base on this topic and share how you can also run DeepSeek on lower-grade hardware.

What is DeepSeek R1?

DeepSeek R1 is an advanced artificial intelligence model developed by Chinese company (DeepSeek) that aims to create powerful, versatile AI systems. R1, released in January of 2025, boasts capabilities in natural language understanding, reasoning, and problem-solving, making it a notable milestone for AGI research.

First off, yes, the Ollama DeepSeek R1 models are indeed distillation versions of their larger counterparts , resulting in more compact and efficient language models that preserve the essential features while reducing computational demands, allowing for faster execution on smaller hardware configurations. As such, you can find various "DeepSeek-R1-Distill" variants with varying levels of complexity available on Ollama to cater to different requirements.

This is the description on the Ollama website for DeepSeek R1:

DeepSeek's first-generation of reasoning models with comparable performance to OpenAI-o1, including six dense models distilled from DeepSeek-R1 based on Llama and Qwen.

That being said, all you need to do to install deepseek-r1 is type in the model name in the search box at the top of Open WebUI like so:

Then click on Pull "model-name" from Ollama.com and it will begin to download the model to your server so you can use it!

What Hardware do I need to Run DeepSeek?

Currently, these Ollama distilled versions of DeepSeek can be ran on lower consumer grade hardware. For example, I can run the deepseek-r1:8b version on my NVIDIA 2080ti at just under 80 tokens per second. I was also able to run the 14b version at 47 tokens per second.

What does the "b" mean in a Model Name?

The "b" represents billions, denoting the number of parameters in a model. A 7b model has 7 billion parameters. Larger models generally perform better than smaller ones, (in terms of response and output) but larger models can use more system RAM and VRAM causing slower response times or less tokens per second.

Why does DeepSeek use the term "Thinking"?

The DeepSeek model uses the term "thinking" because it shows how it comes up with answers by breaking down its thought process into small steps, just like humans do when solving problems. This helps make its thinking process more clear for everyone to see.

Can I run the OG DeepSeek R1?

Alright, if you're really sure you want to do this... be warned: waiting for responses can be a real drag. But hey, science isn't always about comfort, right? The Unsloth crew has got a workflow that might just keep you entertained (or driven mad). Just don't say I didn't warn you...

Hosthatch

High performance compute VMs, with dedicated CPU allocations and extremely high IOPS NVMe disks. Deploy in 16 edge locations, and deliver consistently high-performance to your users, regardless of where they are located.

Starting at $4/mo

Final Notes and Thoughts

DeepSeek R1 is a significant breakthrough in artificial intelligence research. As an open-source model, DeepSeek R1 is also making its technology accessible to researchers and developers worldwide, accelerating the development of AGI systems. With its strong academic ties and focus on general purpose AI, DeepSeek is poised to make a lasting impact on the tech industry. By sharing its research freely, the company is fostering a community driven approach to innovation, which holds promise for a wide range of possibilities.