File: gettext.info,  Node: Installing an LLM,  Next: msgpre Invocation,  Up: Pretranslating

8.1 Installing a Large Language Model
=====================================

   We don't recommend to use machine translation through a web service
in the cloud, controlled by someone else than yourself.  Such a machine
translation service would be have major drawbacks (it could go away any
time, it could be used to spy on you or manipulate you, or the costs
could go up beyond your control); see
.
Additionally, such a service typically has some cost (between $10 and
$25 per megabyte, as of 2025).

   Instead, we recommend a Large Language Model execution engine that
runs on hardware under your control.  This can be a desktop computer, or
for instance a single-board computer in your local network.

   At this point (in 2025), a Large Language Model execution engine that
is Free Software is ‘ollama’, that can be downloaded from
.

   Next, you will need to pick a Large Language Model.  There are two
properties to watch out for:
   • The license under which the LLM is released.  While some LLMs are
     under a Free Software license (most often the Apache 2.0 license)
     and are thus "open weights" models (see
     ), others have licenses
     with usage restrictions.
   • The size of the LLM. Generally, it should be smaller than the
     available RAM size of your machine on which you will run the LLM
     execution engine.

   Together with an LLM of reasonable quality, such as the model
‘ministral-3:14b’, the system requirements are as follows:
   • RAM: 16 GB.
   • Disk space: 10 GB (1 GB for ‘ollama’, 9 GB for the model).
   • GPU or TPU: A GPU (Graphics Processing Unit) or TPU (Tensor
     Processing Unit) is not needed.  As of 2025, a high-end GPU from
     certain vendors can be used by ‘ollama’ to provide an optional
     speedup.

   Additional configuration:
   • If you are running ‘ollama’ on your computer directly, no further
     configuration is needed.
   • If you are running ‘ollama’ on a separate machine, and want to make
     it accessible from all machines in the LAN: Edit the file
     ‘/etc/systemd/system/ollama.services’, adding a line:
     ‘Environment="OLLAMA_HOST=0.0.0.0"’.  See
     .
   • If you are running ‘ollama’ in a virtual machine, make the port
     11434 accessible through port forwarding.