Lit container registry

Centralize, store, manage and deploy Docker containers. With Lightning container registry gone are the days of having to manage and configure your own Docker registry. Instead allow Lightning AI to do the manual labor for you freeing up time for you to spend on model development.

Benefits

✅ One platform: Enhanced interoperability between all Lightning AI products you use on a daily basis such as Lightning Deploy.

✅ Instantly scale: Containers stored on Lightning AI can be deployed and scaled with a few clicks of a button.

✅ Ready when you are: The second you navigate to the containers page you will be able to push containers to container registry.

✅ Security, by default: All containers are stored encrypted at rest with AES-256 encryption.

✅ Data privacy when needed: With BYOC, all containers are stored in private AWS S3 storage. Push, deploy and scale all within your VPC, no AWS certifications needed. Note: only AWS is supported for now.

Container images in the registry count toward your teamspace's storage, which is billed above the first 10 GB. Images stored in your own AWS account with BYOC don't count. See Storage for rates and limits.

Upload via Lightning CLI

First, install the Lightning CLI

pip install -U lightning-sdk

From Studio

Build and push containers from Studio. Once pushed, hit Deploy!

To illustrate, let's build and push the vLLM container. First define the Dockerfile

FROM python:3.10-slim

# Install vLLM
RUN pip install --upgrade pip && \
    pip install vllm

# Expose the vLLM default port
EXPOSE 8000

# Start the vLLM server with a Hugging Face model
CMD ["python", "-m", "vllm.entrypoints.api_server", "--model", "meta-llama/Llama-2-7b-chat-hf"]

Build and push the container

# build
docker build -t my-vllm-server .

# push
lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest

From Mac

To push from the Mac, install and start Docker:

# install
brew install docker

[Run Docker] Make sure Docker is running
# start
open -a Docker

# upload
lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest

From Linux

To push a container from Linux, install and start Docker and push

# install
curl -fsSL https://get.docker.com | bash

# start
sudo systemctl restart docker

# upload
lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest

From Windows

To push a container from Windows, first install Docker. Once it's installed, push the container.

lightning container upload my-vllm-server --teamspace org-or-user-name/teamspace-name --tag latest

Deploy container

Containers stored on Lightning AI's native registry can be deployed in one-click. Upload the container, select the container and click "Deploy".

Upload via Docker CLI

For those used to working with Docker, Lightning AI supports direct Docker CLI uploads.

First, login to the Lightning container registry (get your keys here), tag the container and push.

# login
export LIGHTNING_API_KEY=XXXXXXXXX-XXXXXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXX
echo $LIGHTNING_API_KEY | docker login litcr.io --username=your-lightning-username --password-stdin

docker build -t my-vllm-server .

# tag
docker tag my-vllm-server:latest litcr.io/lit-container/org-or-user-name/teamspace-name/my-vllm-server

# push
docker push litcr.io/lit-container/org-or-user-name/teamspace-name/my-vllm-server