Python Libraries for LLMs using Docker

Notes for NVIDIA LLM Certification Exam

Here’s a more detailed explanation of the Python libraries for LLMs with Dockerized examples to make it easier for you to experiment with models without installing anything locally. The Docker setup will also install required packages directly during the build process to keep things simple.

1. Transformers (by Hugging Face) with Docker

Overview:

The Transformers library is the most widely used Python library for interacting with Large Language Models (LLMs). You can load pre-trained models like GPT, BERT, or T5 and fine-tune them or use them out of the box for tasks like text generation, classification, and summarization.

Dockerfile Example:

This Dockerfile will install Python, transformers, and torch so you can run LLM tasks like text generation.

# Dockerfile for Hugging Face Transformers with PyTorch

FROM python:3.9-slim

# Install required Python packages
RUN pip install transformers torch

# Set the working directory
WORKDIR /app

# Copy your script (if you have one) to the container
COPY text_generation.py /app/

# Run the script (for demo purposes, it runs automatically)
CMD ["python", "text_generation.py"]

text_generation.py Example:

from transformers import pipeline

# Initialize the GPT-2 model from Hugging Face
generator = pipeline('text-generation', model='gpt2')

# Prompt and generate
prompt = "The future of AI is"
output = generator(prompt, max_length=50)

# Print the generated text
print(output[0]['generated_text'])

Build and Run the Docker Container:

# Build the Docker image
docker build -t transformers-demo .

# Run the container
docker run transformers-demo

2. Tokenizers by Hugging Face with Docker

Overview:

Tokenizers by Hugging Face is a fast tokenization library, ideal for working with large datasets that need to be fed into LLMs. The library is written in Rust but provides Python bindings for seamless integration.

Dockerfile Example:

# Dockerfile for Tokenizers by Hugging Face

FROM python:3.9-slim

# Install required packages
RUN pip install tokenizers

# Set the working directory
WORKDIR /app

# Copy your tokenization script to the container
COPY tokenize.py /app/

# Run the tokenization script
CMD ["python", "tokenize.py"]

tokenize.py Example:

from tokenizers import BertWordPieceTokenizer

# Initialize the tokenizer
tokenizer = BertWordPieceTokenizer()

# Tokenize a sentence
sentence = "Large language models are powerful tools for AI."
output = tokenizer.encode(sentence)

# Output the tokens
print(output.tokens)

Build and Run the Docker Container:

# Build the Docker image
docker build -t tokenizer-demo .

# Run the container
docker run tokenizer-demo

3. PyTorch with Docker for LLMs

Overview:

PyTorch is essential for training and fine-tuning large language models. Many deep learning models are implemented using PyTorch, and this Docker setup ensures that everything needed is installed in the container.

Dockerfile Example:

# Dockerfile for PyTorch

FROM python:3.9-slim

# Install required packages
RUN pip install torch

# Set the working directory
WORKDIR /app

# Copy your PyTorch script to the container
COPY simple_nn.py /app/

# Run the script
CMD ["python", "simple_nn.py"]

simple_nn.py Example:

import torch
import torch.nn as nn

# Define a simple feedforward neural network
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

# Initialize model and input
model = SimpleNN()
input_data = torch.randn(1, 10)

# Perform forward pass
output = model(input_data)
print(output)

Build and Run the Docker Container:

# Build the Docker image
docker build -t pytorch-demo .

# Run the container
docker run pytorch-demo

4. TensorFlow and Keras with Docker

Overview:

TensorFlow is another widely used deep learning framework, especially for large-scale LLM training. The Keras API simplifies model building, allowing for easy experimentation.

Dockerfile Example:

# Dockerfile for TensorFlow and Keras

FROM python:3.9-slim

# Install TensorFlow
RUN pip install tensorflow

# Set the working directory
WORKDIR /app

# Copy your script to the container
COPY tensorflow_script.py /app/

# Run the script
CMD ["python", "tensorflow_script.py"]

tensorflow_script.py Example:

import tensorflow as tf

# Define a simple sequential model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(128, activation='relu', input_shape=(10,)),
    tf.keras.layers.Dense(1)
])

# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error')

# Generate dummy data
import numpy as np
X = np.random.rand(100, 10)
y = np.random.rand(100, 1)

# Train the model
model.fit(X, y, epochs=5)

Build and Run the Docker Container:

# Build the Docker image
docker build -t tensorflow-demo .

# Run the container
docker run tensorflow-demo

5. NVIDIA Triton Inference Server for LLMs

Overview:

NVIDIA Triton Inference Server is designed to simplify the deployment of deep learning models in production, particularly for serving large language models efficiently.

Docker Command to Run Triton Server:

docker run -d --gpus all -v /path/to/models:/models nvcr.io/nvidia/tritonserver:22.03-py3 tritonserver --model-repository=/models

Steps:

  1. Ensure you have an LLM model stored in /path/to/models.
  2. Run the above command to serve the model using Triton Inference Server.
  3. You can now send inference requests to the model using gRPC or HTTP.

Example of Model Inference Request (Using Python):

import requests

url = "http://localhost:8000/v2/models/model_name/infer"
data = {
  "inputs": [
    {
      "name": "input_name",
      "shape": [1, 10],
      "datatype": "FP32",
      "data": [0.5] * 10
    }
  ]
}

response = requests.post(url, json=data)
print(response.json())

Explanation:

Summary of Dockerized Examples:

These setups will allow you to experiment with different libraries and frameworks in isolated environments while ensuring compatibility with LLM workflows.