AI

By rbosaz , 16 November, 2024

A Poem for the Times - November 2024

The wind is leaning, whispers dark and low,
Across the fields of anger, seeds of woe.
A nation fractured, branches bent and brown,
Where leaves of unity have fallen down.

The frost of cynicism on every face,
Apathy's pale blanket in this empty space.
The river of dissent, a roaring tide,
Reflecting shadows where the eagles glide.

The hungry ghosts of conflicts past reappear,
Memories of summers, fields of blood and fear.
A nation built on stories, some of them untrue,
Haunted by shadows, whispering shadows due.

Tags

By rbosaz , 12 September, 2024

Local Ollama LLM

The following was taken from here. All below steps assume a Linux O/S. I used Debian 12.

Setup Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Running Model:

ollama run llama3

To use the ollama API, you'll need to open appropriate ports and perform the following extracted from here:

We want our API endpoint to be reachable by the rest of the LAN. For ollama, this means setting OLLAMA_HOST=0.0.0.0 in the ollama.service.

  • Run the following command to edit the service:

    systemctl edit ollama.service

Find the [Service] section and add Environment="OLLAMA_HOST=0.0.0.0" under it. It should look like this:

[Service] Environment="OLLAMA_HOST=0.0.0.0"

  • Save and exit.

  • Reload the environment.

    systemctl daemon-reload systemctl restart ollama

Example Ollama API is as follows:

curl http://192.168.1.179:11434/api/generate -d '{
 "model": "codellama",
 "prompt": "Why is the sky blue?",
 "stream": false
}'

Adding a web UI

One of the easiest ways to add a web UI is to use a project called Open UI. With Open UI, you can add an eerily similar web frontend as used by OpenAI.

You can run the web UI using the OpenUI project inside of Docker. According to the official documentation from Open WebUI, you can use the following command if Ollama is on the same computer:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

URL to connect from remote computer: <host_name/ip_address>:3000

docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui

Keeping Your Docker Installation Up-to-Date

In case you want to update your local Docker installation to the latest version, you can do it with Watchtower:

docker run --rm --volume /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once open-webui

Tags

AI