local-ai

Run LLMs with llama.cpp

Name: Run LLMs with llama.cpp
Author: ggml-org

Get fast LLM inference with llama.cpp, for founders using C++.

115,047 stars19,266 forksC++Quality 9/10Updated 6/7/2026100% free · open source

What it does

Llama.cpp enables fast and efficient inference of large language models in C/C++ applications, allowing developers to integrate AI capabilities into their projects

Install / run

git clone https://github.com/ggml-org/llama.cpp.git

When to use it

•When you need to deploy a language model in a resource-constrained environment
•When you want to integrate a language model into a C/C++ application
•When you need to achieve high-performance inference for large language models

Quick start

1Compile the project using the command 'mkdir build && cd build && cmake .. && cmake --build .'
2Download the pre-trained model weights using the provided script 'download-weights.sh'
3Create a C++ application that includes the 'llama.cpp' header file and links against the 'libllama.so' library
4Initialize the model using the 'LlamaModel' class and load the pre-trained weights
5Use the 'generate' function to generate text based on a given prompt

Ready-to-paste prompt

cout << LlamaModel::generate("Tell me a story about a character who", 100) << endl;

Heads up: Ensure that you have the necessary dependencies installed, including a C++ compiler and the CMake build system, and that your system meets the minimum requirements for the pre-trained models, including at least 4GB of RAM

Saves to your device

Topics

ggml

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

top-level files

folders

400.7M

repo size

MIT

license

Key files

.editorconfig

.pre-commit-config.yaml

AGENTS.md

pyrightconfig.json

README.md

requirements.txt

File tree

.devops/

.gemini/

.github/

.pi/

app/

benches/

ci/

cmake/

common/

conversion/

docs/

examples/

ggml/

gguf-py/

grammars/

include/

licenses/

media/

models/

pocs/

requirements/

scripts/

src/

tests/

Quick Actions

Details

Creator

ggml-org

Language

C++

Related skills

More local-ai tools founders pair with this one.

local-ai★ 15,386

MNN: Fast On-Device AI

Get high-performance Edge AI with MNN, battle-tested by Alibaba, for startup founders, with 15k+ GitHub stars

local-ai★ 218

Xybrid: AI on-device

Build AI-powered apps with Xybrid, for startup founders, with on-device AI capabilities.