automation

Run AI on Your Own Server with ODS

Name: Run AI on Your Own Server with ODS
Author: Light-Heart-Labs

Founders, host AI locally with ODS, for LLM inference and more, backed by 2.4k+ GitHub stars

2,414 stars368 forksShellQuality 9/10Updated 6/30/2026100% free · open source

What it does

ODS turns your local machine into an AI server for hosting large language models and other AI capabilities, allowing for inference, chat interfaces, and more

Install / run

git clone https://github.com/Light-Heart-Labs/ODS.git && cd ODS

When to use it

•You need to run AI models locally for privacy or security reasons
•You want to prototype AI-powered applications without incurring cloud costs
•You need to integrate AI capabilities into an existing local application or workflow

Quick start

1Run the setup script with `./setup.sh` to configure the environment
2Start the ODS server with `docker-compose up` to begin hosting AI models
3Access the ODS web interface at `http://localhost:8000` to interact with models
4Use the `ods-cli` command-line tool to manage models, agents, and workflows
5Edit the `config.yaml` file to customize ODS settings and model configurations

Ready-to-paste prompt

Use the `ods-cli` to run a sample language model inference with `ods-cli infer --model-name ' llama ' --prompt 'Tell me about the benefits of local AI deployment'`

Heads up: Ensure you have Docker and Docker Compose installed on your system, as ODS relies on these tools to manage its containerized environment

Saves to your device