How to Run Llama 3.3 and DeepSeek-R1 Locally with Ollama

Table of Contents

• How to Run Llama 3.3 and DeepSeek-R1 Locally with Ollama
  • Why Local LLM Deployment Matters in Production
  • Prerequisites and Environment Setup
    • Step 1: Install Ollama [8]
• Linux/macOS
• Verify installation
• Expected output: ollama [9] version 0.5.7 (or later)
• Start the Ollama service
  • Step 2: Pull and Run Llama 3.3
• Pull the 8B model (4.9GB download)

📺 Watch: Neural Networks Explained

Video by 3Blue1Brown

Last Updated: May 15, 2026

Running large language models locally has transitioned from a niche hobby to a production-viable deployment strategy. With Ollama's streamlined tooling, you can deploy Llama 3.3 (70B) or DeepSeek-R1 (671B) on consumer hardware in under five minutes. This tutorial walks through the exact steps, architecture decisions, and edge cases you'll encounter when running these models locally.

Why Local LLM Deployment Matters in Production

The shift toward local LLM inference isn't just about privacy—it's about latency, cost control, and data sovereignty. According to recent research published on ArXiv, quantized DeepSeek models show only a 2-4% performance degradation at 4-bit quantization while reducing memory requirements by 75% [1]. This makes running 70B+ parameter models feasible on a single RTX 4090 or dual A6000 setup.

Consider the production use case: a medical AI system processing patient queries. A multi-agent framework leverag [4]ing fine-tuned LLaMA and DeepSeek R1 demonstrated that local inference eliminates the 200-500ms network latency of API calls while maintaining HIPAA compliance [2]. Similarly, Python performance profiling tools like Scalene have integrated DeepSeek-R1 and LLaMA 3.2 for real-time code optimization suggestions, proving that local LLMs can enhance developer workflows without cloud dependencies [3].

Prerequisites and Environment Setup

Before diving into deployment, ensure your system meets these requirements:

Hardware Requirements:

• Minimum: 16GB RAM, 8GB VRAM (for 7B models)
• Recommended: 32GB+ RAM, 24GB+ VRAM (for 70B models)
• Optimal: 64GB+ RAM, 48GB+ VRAM (for 671B DeepSeek-R1)

Software Requirements:

• Linux (Ubuntu 22.04+), macOS 14+, or Windows 11 with WSL2
• Python 3.10+
• NVIDIA drivers 545+ (for GPU acceleration)
• Docker (optional, for containerized deployment)

Step 1: Install Ollama

Ollama provides a single binary that handles model downloading, quantization, and inference. Install it with:

# Linux/macOS
curl -fsSL https://ollama.com/install.sh | sh

# Verify installation
ollama --version
# Expected output: ollama version 0.5.7 (or later)

# Start the Ollama service
ollama serve

The ollama serve command starts a REST API on localhost:11434. This is the backbone for all model interactions.

Step 2: Pull and Run Llama 3.3

Llama 3.3 is Meta's latest open-weight model, available in 8B and 70B parameter variants. For local deployment, the 8B quantized version runs on most consumer GPUs:

# Pull the 8B model (4.9GB download)
ollama pull llama3.3:8b

# Run interactive chat
ollama run llama3.3:8b

For production workloads, use the 70B model with 4-bit quantization:

# Pull the 70B quantized model (35GB download)
ollama pull llama3.3:70b-q4_K_M

# Run with specific GPU configuration
OLLAMA_GPU_LAYERS=35 ollama run llama3.3:70b-q4_K_M

The OLLAMA_GPU_LAYERS environment variable controls how many transformer layers are offloaded to GPU. Setting it to 35 (out of 80 total layers for 70B) balances VRAM usage with inference speed.

Step 3: Pull and Run DeepSeek-R1

DeepSeek-R1 is a 671B mixture-of-experts (MoE) model that activates only 37B parameters per token. This makes it surprisingly efficient for its size:

# Pull the quantized DeepSeek-R1 (32GB download)
ollama pull deepseek-r1:671b-q4_K_M

# Run with memory optimization
OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 ollama run deepseek-r1:671b-q4_K_M

The MoE architecture means DeepSeek-R1 uses approximately 40GB VRAM at 4-bit quantization, compared to 140GB for a dense 671B model. This is why it runs on dual RTX 4090s (48GB total VRAM) while Llama 3.3 70B requires only 24GB.

Production-Grade Inference with Python

For programmatic access, use the Ollama Python library. This is essential for integrating local LLMs into your application pipeline:

import ollama
import json
from typing import Dict, List, Optional
import time

class LocalLLMInference:
    """Production-grade wrapper for Ollama inference with error handling and retry logic."""

    def __init__(self, model_name: str = "llama3.3:70b-q4_K_M", 
                 timeout: int = 120,
                 max_retries: int = 3):
        self.model_name = model_name
        self.timeout = timeout
        self.max_retries = max_retries
        self.client = ollama.Client(host='http://localhost:11434')

    def generate(self, prompt: str, 
                 system_prompt: Optional[str] = None,
                 temperature: float = 0.7,
                 max_tokens: int = 2048) -> Dict:
        """
        Generate text with retry logic and performance monitoring.

        Args:
            prompt: User input text
            system_prompt: Optional system-level instructions
            temperature: Sampling temperature (0.0-1.0)
            max_tokens: Maximum tokens to generate

        Returns:
            Dictionary with response, timing, and token usage
        """
        messages = []
        if system_prompt:
            messages.append({"role": "system", "content": system_prompt})
        messages.append({"role": "user", "content": prompt})

        for attempt in range(self.max_retries):
            try:
                start_time = time.time()

                response = self.client.chat(
                    model=self.model_name,
                    messages=messages,
                    options={
                        "temperature": temperature,
                        "num_predict": max_tokens,
                        "stop": ["<|eot_id|>", "<|end_of_text|>"]
                    }
                )

                elapsed = time.time() - start_time

                return {
                    "response": response['message']['content'],
                    "tokens_generated": response.get('eval_count', 0),
                    "tokens_per_second": response.get('eval_count', 0) / elapsed if elapsed > 0 else 0,
                    "elapsed_seconds": elapsed,
                    "model": self.model_name
                }

            except ollama.ResponseError as e:
                if e.status_code == 503:  # Model loading
                    print(f"Model loading, retrying in 5s (attempt {attempt + 1})")
                    time.sleep(5)
                elif e.status_code == 429:  # Rate limit
                    print(f"Rate limited, retrying in 10s (attempt {attempt + 1})")
                    time.sleep(10)
                else:
                    raise

        raise Exception(f"Failed after {self.max_retries} retries")

# Usage example
inference = LocalLLMInference(model_name="deepseek-r1:671b-q4_K_M")

result = inference.generate(
    prompt="Explain the concept of mixture-of-experts in transformer models.",
    system_prompt="You are a technical AI researcher. Provide concise, accurate explanations.",
    temperature=0.3,
    max_tokens=1024
)

print(f"Response ({result['tokens_per_second']:.1f} tok/s):")
print(result['response'][:500])

This wrapper handles three critical production concerns:

1. Graceful degradation through retry logic for transient failures
2. Performance monitoring via token-per-second tracking
3. Resource management with configurable timeouts

Architecture Decisions for Multi-Model Deployment

Running multiple models simultaneously requires careful resource planning. Here's a production architecture that handles both Llama 3.3 and DeepSeek-R1:

import asyncio
import aiohttp
from dataclasses import dataclass
from typing import Optional

@dataclass
class ModelConfig:
    """Configuration for each deployed model."""
    name: str
    min_vram_gb: int
    max_concurrent: int
    priority: int  # Lower number = higher priority

class OllamaRouter:
    """
    Intelligent router for multi-model Ollama deployment.
    Handles model switching, VRAM management, and request queuing.
    """

    def __init__(self):
        self.models = {
            "llama3.3:70b-q4_K_M": ModelConfig(
                name="llama3.3:70b-q4_K_M",
                min_vram_gb=24,
                max_concurrent=2,
                priority=1
            ),
            "deepseek-r1:671b-q4_K_M": ModelConfig(
                name="deepseek-r1:671b-q4_K_M",
                min_vram_gb=40,
                max_concurrent=1,
                priority=2
            )
        }
        self.active_model: Optional[str] = None
        self.request_queue = asyncio.Queue()

    async def route_request(self, model_name: str, prompt: str) -> Dict:
        """
        Route request to appropriate model, handling model switching.

        Edge case: If switching from DeepSeek to Llama, we must unload
        DeepSeek first to free VRAM.
        """
        if self.active_model and self.active_model != model_name:
            # Unload current model
            async with aiohttp.ClientSession() as session:
                await session.post(
                    "http://localhost:11434/api/generate",
                    json={"model": self.active_model, "keep_alive": "0s"}
                )
            self.active_model = None
            # Wait for VRAM to be freed
            await asyncio.sleep(2)

        # Load target model if needed
        if not self.active_model:
            async with aiohttp.ClientSession() as session:
                await session.post(
                    "http://localhost:11434/api/generate",
                    json={"model": model_name, "keep_alive": "5m"}
                )
            self.active_model = model_name

        # Send inference request
        async with aiohttp.ClientSession() as session:
            async with session.post(
                "http://localhost:11434/api/chat",
                json={
                    "model": model_name,
                    "messages": [{"role": "user", "content": prompt}],
                    "stream": False
                }
            ) as response:
                return await response.json()

    async def health_check(self) -> Dict:
        """Check status of all configured models."""
        async with aiohttp.ClientSession() as session:
            async with session.get("http://localhost:11434/api/tags") as response:
                models = await response.json()

        return {
            "active_model": self.active_model,
            "available_models": [m['name'] for m in models.get('models', [])],
            "queue_size": self.request_queue.qsize()
        }

# Usage
router = OllamaRouter()

async def main():
    # Route to Llama 3.3 for general queries
    result = await router.route_request(
        "llama3.3:70b-q4_K_M",
        "Write a Python function for binary search."
    )
    print(result['message']['content'][:200])

    # Route to DeepSeek-R1 for complex reasoning
    result = await router.route_request(
        "deepseek-r1:671b-q4_K_M",
        "Prove that the square root of 2 is irrational."
    )
    print(result['message']['content'][:200])

asyncio.run(main())

This router addresses a critical edge case: VRAM fragmentation. When switching between models, Ollama's keep_alive parameter controls how long a model stays loaded. Setting it to "0s" forces immediate unloading, preventing memory leaks.

Performance Optimization and Edge Cases

Memory Management

The most common failure mode in local LLM deployment is out-of-memory (OOM) errors. Here's how to handle them:

import psutil
import GPUtil
import subprocess

def monitor_resources() -> Dict:
    """
    Monitor system resources and provide optimization recommendations.
    """
    # CPU and RAM
    cpu_percent = psutil.cpu_percent(interval=1)
    memory = psutil.virtual_memory()

    # GPU
    gpus = GPUtil.getGPUs()
    gpu_info = []
    for gpu in gpus:
        gpu_info.append({
            "name": gpu.name,
            "memory_used_mb": gpu.memoryUsed,
            "memory_total_mb": gpu.memoryTotal,
            "utilization": gpu.load * 100
        })

    recommendations = []

    # Check if we're close to OOM
    if memory.percent > 90:
        recommendations.append(
            "CRITICAL: System RAM at {}%. Consider using a smaller model "
            "or increasing swap space.".format(memory.percent)
        )

    for gpu in gpu_info:
        if gpu['memory_used_mb'] / gpu['memory_total_mb'] > 0.95:
            recommendations.append(
                "WARNING: GPU {} VRAM at {:.1f}%. "
                "Reduce batch size or use more aggressive quantization.".format(
                    gpu['name'],
                    gpu['memory_used_mb'] / gpu['memory_total_mb'] * 100
                )
            )

    return {
        "cpu_percent": cpu_percent,
        "ram_percent": memory.percent,
        "gpu_info": gpu_info,
        "recommendations": recommendations
    }

# Run before inference
resources = monitor_resources()
if resources['recommendations']:
    for rec in resources['recommendations']:
        print(rec)

Quantization Trade-offs

According to the ArXiv analysis of DeepSeek quantization, 4-bit quantization (Q4_K_M) provides the best balance of quality and memory efficiency [1]. The performance drop is measurable but acceptable:

• Q8_0 (8-bit): <1% quality loss, 2x memory reduction
• Q4_K_M (4-bit): 2-4% quality loss, 4x memory reduction
• Q2_K (2-bit): 8-12% quality loss, 8x memory reduction

For production systems, start with Q4_K_M and only move to Q8_0 if quality metrics demand it.

Handling Long Contexts

Both Llama 3.3 and DeepSeek-R1 support 128K token contexts. However, attention computation scales quadratically with sequence length. For long documents:

def chunked_inference(model: str, long_text: str, chunk_size: int = 4096) -> str:
    """
    Process long texts by chunking and summarizing.

    Edge case: DeepSeek-R1's MoE architecture handles long contexts
    more efficiently than dense models like Llama 3.3.
    """
    chunks = [long_text[i:i+chunk_size] for i in range(0, len(long_text), chunk_size)]
    summaries = []

    for i, chunk in enumerate(chunks):
        response = ollama.chat(
            model=model,
            messages=[{
                "role": "user",
                "content": f"Summarize this chunk {i+1}/{len(chunks)}: {chunk}"
            }],
            options={"num_predict": 512}
        )
        summaries.append(response['message']['content'])

    # Final synthesis
    final = ollama.chat(
        model=model,
        messages=[{
            "role": "user",
            "content": f"Synthesize these summaries into a coherent response: {' '.join(summaries)}"
        }]
    )
    return final['message']['content']

Conclusion

Deploying Llama 3.3 and DeepSeek-R1 locally with Ollama is production-ready today. The key takeaways:

1. Start with quantized models (Q4_K_M) for the best memory-performance trade-off
2. Use the Python client for programmatic access with proper error handling
3. Implement resource monitoring to prevent OOM failures
4. Consider MoE architectures like DeepSeek-R1 for complex reasoning tasks

The research community has validated that quantized models maintain 96-98% of their original quality while being deployable on consumer hardware [1]. For sensitive applications like medical AI, local deployment eliminates data transfer risks while maintaining inference quality [2].

What's Next

• Explore fine-tuning [2] these models for domain-specific tasks using LoRA adapters
• Implement a model caching layer to reduce cold-start latency
• Set up monitoring with Prometheus and Grafana for production observability
• Consider multi-node deployment for models that exceed single-GPU VRAM

The local LLM ecosystem is evolving rapidly. As of May 2026, Ollama supports over 100 models with automatic quantization and GPU acceleration. The five-minute deployment promise is real—start with the commands above and iterate from there.