design

Zinc: Local LLM Inference

Name: Zinc: Local LLM Inference
Author: zolotukhin

Run large language models locally with Zinc. For founders needing local AI inference on AMD GPUs and Apple Silicon.

456 stars19 forksZigQuality 9/10Updated 7/9/2026100% free · open source

What it does

Zinc allows founders to run large language models locally on their own hardware, specifically AMD GPUs and Apple Silicon, for private and efficient AI inference.

Install / run

git clone https://github.com/zolotukhin/zinc.git

When to use it

•You need to keep your AI data and models private and offline.
•Your application requires low-latency and high-throughput AI inference.
•You want to develop and test AI models on your local machine before deploying them to a cloud or production environment.

Quick start

1Build Zinc by running the command `zig build -fstage1` in the cloned repository.
2Create a configuration file, such as `config.toml`, to specify your model and hardware settings.
3Use the `zig-out/bin/zinc` command to run your local language model, referencing your configuration file.
4Test your setup with a sample prompt or input to verify that Zinc is working correctly.
5Optimize your configuration and model settings for your specific use case and hardware.

Ready-to-paste prompt

Run the command `zig-out/bin/zinc -c config.toml -p 'What is the meaning of life?'` to test your local language model with a sample prompt.

Heads up: Zinc requires a compatible Zig compiler version, specifically Zig 0.10.0 or later, to build and run correctly, so ensure you have the correct version installed before attempting to use Zinc.

Saves to your device