However you install Ollama from ollama.com website, if you install it as a service, therefore you may want to do some simple configuration to the systemd ollama service textfile so the 30 billion tokenized llm models can run and perhaps others, if your computer system has enough RAM.

Location of ollama service textfile, /etc/systemd/system/ollama.service

There is the service area in brackets, the following configuration can be changed to the ollama.service textfile.

[Service]
Environment="OLLAMA_LLM_LIBRARY=cpu"
Environment="CUDA_VISIBLE_DEVICES="
Environment="OLLAMA_NUM_THREAD=35"

This simply allows cpu processing, not the gpu processing if gpu RAM is an issue.

Now, your system can run the 30B and perhaps other llm models.

By admin

Leave a Reply

Your email address will not be published. Required fields are marked *