llama.cpp for Android — On-Device LLM Inference AAR

Run LLMs on Android with one Gradle line. No cloud, no API key, no per-token billing. Your users' conversations never leave the phone.

A prebuilt llama.cpp AAR with a clean Kotlin coroutine API — chat completion, embeddings, GGUF models. No NDK, no CMake, no Python. Wraps llama.cpp build b9878.

Install

A) Maven Central (recommended)

dependencies {
    implementation("dev.ffmpegkit-maintained:llama-android:0.1.1")
}

B) JitPack

// settings.gradle.kts
dependencyResolutionManagement {
    repositories { google(); mavenCentral(); maven(url = uri("https://jitpack.io")) }
}
// build.gradle.kts
dependencies {
    implementation("com.github.ffmpegkit-maintained:llama-android:v0.1.1")
}

Quick start

import dev.ffmpegkit.llama.Llama
import dev.ffmpegkit.llama.LlamaConfig

lifecycleScope.launch {
    // 1. Load a GGUF model you shipped or downloaded (see "Models" below).
    val model = Llama.loadModel(
        modelPath = File(getExternalFilesDir("models"), "model.gguf").absolutePath,
        config = LlamaConfig(contextSize = 2048, threads = 4),
    )

    // 2. Chat completion.
    val result = Llama.complete(
        model,
        prompt = "Explain gravity to a 5-year-old.",
        systemPrompt = "You are a friendly teacher.",
        maxTokens = 256,
    )
    println(result.text)
    println("${result.tokensPerSecond} tok/s")

    // 3. Embeddings (text → vector) for search / RAG.
    val vector = Llama.embed(model, "on-device AI")

    // 4. Free native memory.
    Llama.releaseModel(model)
}

All heavy calls are suspend functions — call them from a coroutine.

Models (not bundled — you download them)

LLM weights are 400 MB–8 GB, far too large to bundle in an AAR. Download a GGUF model once (ship it in your app or fetch on first run):

Model	Size (Q4_K_M)	Notes	Download
Qwen2.5 0.5B	~400 MB	ultra-fast, basic	HF
Gemma 2 2B	~1.6 GB	compact & capable	HF
Llama 3.2 3B	~2.0 GB	best for chat	HF
Phi-3.5 Mini 3.8B	~2.2 GB	strong reasoning	HF

Rule of thumb: a Q4_K_M model needs roughly (model size + ~20%) of free RAM. On a 6 GB phone, stick to ≤ 3B models. See the wiki for a full guide.

What's inside

Engine	llama.cpp (b9878) + ggml, CPU/NEON
API	Chat completion, embeddings, system prompt, chat templates (auto)
Models	any GGUF (Q4_0, Q4_K_M, Q5_K_M, …)
ABI	arm64-v8a
Min SDK	API 24 (Android 7.0) · 16 KB pages (Android 15 ready)

Free vs Pro

	Free (this)	Pro
Chat completion, embeddings	✅	✅
Streaming tokens (Flow)	✗	✅
Multiple concurrent sessions (per-session KV cache)	✗	✅
Vulkan GPU acceleration	✗ (CPU/NEON)	✅
ABI	arm64-v8a	arm64-v8a + x86_64
Channel	Maven Central + JitPack + Release	Jokobee

Pro adds:

• Streaming tokens (Kotlin Flow) — render responses word by word
• Multiple concurrent sessions with per-session KV cache
• Vulkan GPU acceleration (in addition to CPU/NEON)
• x86_64 ABI, in addition to arm64-v8a

→ Get llama.cpp Pro on jokobee.com

Works with Whisper — voice AI, fully on-device

Chain the Jokobee on-device AI stack for a private voice assistant:

FFmpegKit (decode audio) → Whisper (speech → text) → llama.cpp (text → answer) → Android TTS (answer → speech). No data leaves the device.

Building from source

git clone --recursive https://github.com/ffmpegkit-maintained/llama-android.git
cd llama-android
./gradlew :library:assembleRelease

Requires NDK r27c (27.2.12479018) and CMake 3.22.1. llama.cpp is a pinned git submodule (b9878).

License

MIT — see LICENSE. llama.cpp is MIT (© the ggml authors).

---

Maintained by Jokobee · contact@jokobee.com

Third-Party Notices

Third-party components bundled or built into this project retain their original licenses. See THIRD-PARTY-NOTICES.txt, which is also copied into the AAR assets at build time (readable at runtime via context.assets.open("THIRD-PARTY-NOTICES.txt")).