Install Local LLM Support

Installs llama-cpp-python for running local GGUF models with local_AIGENIE. On Apple Silicon Macs, this includes Metal acceleration support for fast inference.

Usage

install_local_llm_support()

Value

Invisible TRUE on success.

Details

After installation, you can use any GGUF model file with local_AIGENIE(). Download GGUF models from HuggingFace (search for "GGUF" format).

Popular model recommendations:

• Llama 3 8B: Good balance of quality and speed

• Mistral 7B: Fast with good quality

• Qwen 2.5: Strong multilingual support

See also

local_AIGENIE, reinstall_python_env.

Examples

if (FALSE) { # \dontrun{
# Install local LLM support
install_local_llm_support()

# Then download a GGUF model and use with local_AIGENIE
results <- local_AIGENIE(
  item.attributes = my_traits,
  model.path = "~/models/llama-3-8b.Q4_K_M.gguf",
  embedding.model = "bert-base-uncased"
)
} # }