> ## Documentation Index
> Fetch the complete documentation index at: https://wwwpoc.ibm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama on Linux

> Learn how to run Granite models with Ollama on Linux.

## Overview

In this guide, we'll use [Ollama](https://ollama.com/), an open-source tool that makes it easy to download and run AI models locally.

***

## 1. Install Ollama

Install [Ollama for Linux](https://ollama.com/download/linux) with:

```bash theme={null}
curl -fsSL https://ollama.com/install.sh | sh
```

This will install Ollama on your system and set up a `systemd` service named `ollama.service` to run the server in the background.

<Warning>
  The automatic installation script requires root access. For manual
  installation (without root), see the [manual installation
  instructions](https://github.com/ollama/ollama/blob/main/docs/linux.md).
</Warning>

To manage the service manually:

```bash theme={null}
# Start the Ollama service
systemctl --user start ollama

# Enable Ollama to start automatically on login
systemctl --user enable ollama

# Check Ollama status
systemctl --user status ollama
```

***

## 2. Download Granite Models

Ollama supports a range of [IBM Granite models](https://ollama.com/ibm). Larger models provide better results but require more resources.

To download Granite 4:

```bash theme={null}
ollama pull ibm/granite4
```

***

## 3. Run Granite

To start chatting with Granite:

```bash theme={null}
ollama run ibm/granite4
```

If you want to use a different variant, replace the model name (e.g., `ibm/granite4`).

***

## 4. Notes on Context Length

<Note>
  By default, Ollama runs models with a short context length to save memory.
  For longer conversations, you can adjust it by setting:

  ```
  /set parameter num_ctx <desired_context_length>
  ```

  The largest supported context for Granite 4.0 models is `131072` (128k).
</Note>

***

## 5. Using the API

You can also interact with Granite programmatically using Ollama’s OpenAI-compatible API:

```bash theme={null}
curl -X POST http://localhost:11434/v1/chat/completions   -H "Content-Type: application/json"   -d '{
        "model": "ibm/granite4",
        "messages": [{"role": "user", "content": "How are you today?"}]
      }'
```

***

## 6. Adding documents

You can add documents to your context using the special `"document <title>"` role:

```bash theme={null}
curl -X POST http://localhost:11434/v1/chat/completions   -H "Content-Type: application/json"   -d '{
        "model": "ibm/granite4",
        "messages": [
          {"role": "document me.txt", "content": "I live in Denver"},
          {"role": "document weather.txt", "content": "It is currently snowing in Denver and sunny in Austin."},
          {"role": "user", "content": "What is the weather like outside my house?"}
        ]
      }'
```