Ollama

Ollama#

ollama website

Visit https://ollama.com and install it.

DOcumentation: ollama/ollama

Terminal commands:

# help 
ollama -h 

# run (or download) a model 
ollama run [MODELNAME]

We’ll start with Mistral (7B) https://ollama.com/library/mistral

ollama run mistral
>>> what is the perfect pet for kids of age 10?

This returns for example a list of 7 animals (Hamster, Guine Pig, Fish, Stick Insects, Turtles, Reptiles, Dog) with some explanaition.

System Prompt#

The responses of large language models are generated from a perspective. Of course, this is hugely influenced by the training data already. After that, it’s shaped by the System prompt or System message. They are a bridge between the possibilities of the llm and the actual use case.

Examples#

  • role: “You are a pirate”, “You are a plant”, “You are a friendly assistant”

  • language: “Speak in easy language, appropriate for children”

  • tone guideline: “Be always polite”

  • customized output: “Return your response in JSON-format”, “Write only a few sentences”, “Return just one option”, “Return python code”

  • narrowing down the expertise: “If a user asks about …, answer politely that you can’t answer that question.”

Example prompts by Anthropic AI

# open the help documentation inside the ollama chat
>>> /?
# open help for set (Set session variables)
>>> /? set
# change the system message
>>> /set system "return only one option"
>>> What is the perfect pet for kids of age 10?

  1. Hamster: Hamsters are low-maintenance pets that are easy for children aged 10 to care for and provide hours of entertainment. They require a small cage, food, water, bedding, and daily cleaning.

>>> /set system "you are a funny pirate. respond to questions with only one option."
>>> What is the perfect pet for kids of age 10?

  1. Hamster

# Next try
>>> /set system "respond to questions with only one option. speak as a pirate"
>>> What is the perfect pet for kids of age 10?

  1. Parrot, matey! Pirates love parrots, and so shall children of age 10! A parrot will provide endless entertainment with its clever tricks and talkative nature. Arrrr!

# After starting a new chat session with /bye:
>>> /set system "respond to questions with only one option. you are a vegan"
>>> what is the perfect pet for kids of age 10?

A dog or cat (as long as they are rescued from shelters and not purchased from breeders) would make great pets for kids aged 10, provided that proper care and responsibility are taught. However, it’s essential to remember that these animals have specific dietary needs and may require veterinary care, which can be costly over time. Alternatively, consider adopting a herbivorous pet such as a guinea pig or hamster if your family prefers non-traditional pets.

Modelfile#

As seen, the variables set with /set are temporal. If we wan’t to preserve them, we have to create a new model based on a model file.

Documentation: ollama/ollama

Get the modelfile of the model you want to modify

ollama show --modelfile mistral

# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM mistral:latest

FROM /usr/share/ollama/.ollama/models/blobs/sha256-ff82381e2bea77d91c1b824c7afb83f6fb73e9f7de9dda631bcdbca564aa5435
TEMPLATE """{{- if .Messages }}
{{- range $index, $_ := .Messages }}
{{- if eq .Role "user" }}
{{- if and (eq (len (slice $.Messages $index)) 1) $.Tools }}[AVAILABLE_TOOLS] {{ $.Tools }}[/AVAILABLE_TOOLS]
{{- end }}[INST] {{ if and $.System (eq (len (slice $.Messages $index)) 1) }}{{ $.System }}

{{ end }}{{ .Content }}[/INST]
{{- else if eq .Role "assistant" }}
{{- if .Content }} {{ .Content }}
{{- else if .ToolCalls }}[TOOL_CALLS] [
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}]
{{- end }}</s>
{{- else if eq .Role "tool" }}[TOOL_RESULTS] {"content": {{ .Content }}} [/TOOL_RESULTS]
{{- end }}
{{- end }}
{{- else }}[INST] {{ if .System }}{{ .System }}

{{ end }}{{ .Prompt }}[/INST]
{{- end }} {{ .Response }}
{{- if .Response }}</s>
{{- end }}"""
PARAMETER stop [INST]
PARAMETER stop [/INST]
LICENSE """  
...

Copy this content to a new plain text file, for example modelfile.txt. Or write the terminal output to that file with >:

# Syntax: ollama show --modelfile [MODELNAME] > [MODELFILE]
ollama show --modelfile mistral > modelfile.txt

Modify this file, for example insert SYSTEM "respond to questions with only one option. you are a vegan" below PARAMETERS.

Create a new modelfile with

# Syntax: ollama create [MODELNAME] -f [MODELFILE]
ollama create mymodel -f modelfile.txt

Then we can run the custom model with

ollama run mymodel
>>> What should I put on my pizza?

Vegan cheese, tomato sauce, vegetables, and plant-based pepperoni.

Removing a model#

# Syntax: ollama rm [MODELNAME]
ollama rm mymodel
Contents