See this page for details on installing Python and the minimum libraries required for your development environment:
Install Python

Create and activate a Python virtual environment for isolated LLM development:
$ mkdir -p /home/mtitek/dev/llm $ cd /home/mtitek/dev/llm $ python3 -m venv .env $ source .env/bin/activate
To deactivate the virtual environment, run the following command:
$ cd /home/mtitek/dev/llm $ deactivate # if needed, you can permanently delete the virtual environment $ rm -rf .env

See this page for more details: https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

Hugging Face CLI: Access and download models from Hugging Face Hub.

Install Hugging Face CLI:
$ pip install "huggingface_hub[cli]"
Verify installation and check version:
$ huggingface-cli version huggingface_hub version: 0.30.2
You can use the CLI to download models:
$ huggingface-cli download microsoft/DialoGPT-small

See this page for more details: https://huggingface.co/docs/transformers/en/installation

Transformers: Main library for working with transformer models (BERT, GPT, etc.).

Install transformers:
$ pip install transformers
To install a CPU-only version of Transformers and a machine learning framework PyTorch, run the following command: $ pip install 'transformers[torch]'
To test if the installation was successful, run the following command. It should return a label and a score for the provided text: $ python3 -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('hugging face is the best'))" [{'label': 'POSITIVE', 'score': 0.999839186668396}]
Example: Running a model using Hugging Face Transformers:
$ vi llm.py from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small") tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small") print(model) print(tokenizer)
Run the python script:
$ python3 llm.py
Output:
GPT2TokenizerFast( name_or_path='microsoft/DialoGPT-small', vocab_size=50257, model_max_length=1024, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<|endoftext|>', 'eos_token': '<|endoftext|>', 'unk_token': '<|endoftext|>'}, clean_up_tokenization_spaces=True, added_tokens_decoder={50256: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=True),} ) GPT2LMHeadModel( (transformer): GPT2Model( (wte): Embedding(50257, 1024) (wpe): Embedding(1024, 1024) ... ) (lm_head): Linear(in_features=1024, out_features=50257, bias=False) )

See these pages for more details:
https://pypi.org/project/llama-cpp-python/
https://python.langchain.com/docs/integrations/llms/llamacpp/

llama-cpp-python: Run optimized LLMs locally with efficient inference.

Install llama-cpp-python:
$ pip install llama-cpp-python
Test lama-cpp-python:

Download a compatible model:
$ wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
Python code:
$ vi llm.py from llama_cpp import Llama model = Llama(model_path="./Phi-3-mini-4k-instruct-q4.gguf")
Run the python script:
$ python3 llm.py
Output:
llama_model_loader: loaded meta data with 24 key-value pairs and 195 tensors from Phi-3-mini-4k-instruct-q4.gguf (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = phi3 ... Using gguf chat template: {{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<|user|>' + ' ' + message['content'] + '<|end|>' + ' ' + '<|assistant|>' + ' '}}{% elif (message['role'] == 'assistant') %}{{message['content'] + '<|end|>' + ' '}}{% endif %}{% endfor %} Using chat eos_token: <|endoftext|> Using chat bos_token: <s>

See this page for more details: https://python.langchain.com/docs/how_to/installation/

LangChain: Framework for building applications powered by language models.

To install the main LangChain package:
$ pip install langchain
To install the LangChain core package:
$ pip install langchain-core
To install the LangChain community package:
$ pip install langchain-community
To install the LangChain command Line Interface (CLI) package:
$ pip install langchain-cli
To test the installation of the LangChain CLI package:
$ langchain-cli --version langchain-cli 0.0.36