See this page for more details: https://huggingface.co/docs/transformers/en/installation

Make sure that both python and pip packages are installed.

Create a directory to use for your tests: /home/mtitek/dev/llm

Create and activate a virtual environment in your project directory with venv:
$ cd /home/mtitek/dev/llm $ python -m venv .env $ source .env/bin/activate
Install Transformers with pip in your newly created virtual environment:
$ pip install transformers
To install a CPU-only version of Transformers and a machine learning framework PyTorch, run the following command: $ pip install 'transformers[torch]'
To test if the installation was successful, run the following command. It should return a label and a score for the provided text: $ python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('hugging face is the best'))" [{'label': 'POSITIVE', 'score': 0.999839186668396}]
To deactivate the virtual environment in your project directory and clean it up:
$ cd /home/mtitek/dev/llm $ deactivate $ rm -rf .env

See this page for more details: https://huggingface.co/docs/huggingface_hub/main/en/guides/cli

To install the Hugging Face CLI package: $ pip install -U "huggingface_hub[cli]"
You can use the CLI to download models:
$ huggingface-cli download microsoft/DialoGPT-small

Example:
from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-small") tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-small") print(model) print(tokenizer)
Output:
GPT2TokenizerFast( name_or_path='microsoft/DialoGPT-small', vocab_size=50257, model_max_length=1024, is_fast=True, padding_side='right', truncation_side='right', special_tokens={'bos_token': '<|endoftext|>', 'eos_token': '<|endoftext|>', 'unk_token': '<|endoftext|>'}, clean_up_tokenization_spaces=True, added_tokens_decoder={50256: AddedToken("<|endoftext|>", rstrip=False, lstrip=False, single_word=False, normalized=True, special=True),} ) GPT2LMHeadModel( (transformer): GPT2Model( (wte): Embedding(50257, 1024) (wpe): Embedding(1024, 1024) (drop): Dropout(p=0.1, inplace=False) (h): ModuleList( (0-23): 24 x GPT2Block( (ln_1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) (attn): GPT2Attention( (c_attn): Conv1D(nf=3072, nx=1024) (c_proj): Conv1D(nf=1024, nx=1024) (attn_dropout): Dropout(p=0.1, inplace=False) (resid_dropout): Dropout(p=0.1, inplace=False) ) (ln_2): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) (mlp): GPT2MLP( (c_fc): Conv1D(nf=4096, nx=1024) (c_proj): Conv1D(nf=1024, nx=4096) (act): NewGELUActivation() (dropout): Dropout(p=0.1, inplace=False) ) ) ) (ln_f): LayerNorm((1024,), eps=1e-05, elementwise_affine=True) ) (lm_head): Linear(in_features=1024, out_features=50257, bias=False) )

See this page for more details: https://python.langchain.com/docs/how_to/installation/

To install the main langchain package: $ pip install langchain
To install the langchain core package: $ pip install langchain-core
To install the langchain community package: $ pip install langchain-community

See these pages for more details:
https://pypi.org/project/llama-cpp-python/
https://python.langchain.com/docs/integrations/llms/llamacpp/

To install the llama-cpp-python package: $ pip install llama-cpp-python

Download this model:
$ wget https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf
Python code:
from langchain_community.llms import LlamaCpp model = LlamaCpp(model_path="Phi-3-mini-4k-instruct-q4.gguf", verbose=False) print(model)
Output:
llama_init_from_model: n_batch is less than GGML_KQ_MASK_PAD - increasing to 64 llama_init_from_model: n_ctx_per_seq (512) < n_ctx_train (4096) -- the full capacity of the model will not be utilized LlamaCpp Params: { 'model_path': 'Phi-3-mini-4k-instruct-q4.gguf', 'suffix': None, 'max_tokens': 256, 'temperature': 0.8, 'top_p': 0.95, 'logprobs': None, 'echo': False, 'stop_sequences': [], 'repeat_penalty': 1.1, 'top_k': 40 }