shipping-a-model-in-an-android-applisted

Put a local language model inside a native Android app written in Kotlin or Java. Covers MediaPipe LLM Inference and LiteRT-LM for .task and .litertlm models, llama.cpp through the NDK for GGUF, ML Kit GenAI and Gemini Nano where a built-in model is enough, GPU and NNAPI delegation, Play asset packs, and whether to bundle the model or download it on first run. Use when the project is a native Android app and someone wants on-device or offline AI or local inference. Not for React Native or Expo projects, not for iOS native Swift projects, not for Flutter, and not for models running on a server.
ErtasAI/open-model-skills · ★ 2 · AI & Automation · score 76

Install: claude install-skill ErtasAI/open-model-skills

# Shipping a model in an Android app Android splits into two runtime families that take different artifact shapes, plus a third option that ships no model at all. Which of the two runtimes fits depends on what file is already in hand; which of the three paths fits depends on whether a built-in model is actually good enough for the task. Settle both before installing anything. ## Which artifact shape this needs | You are holding | Package | What it eats | |---|---|---| | A `.task` or `.litertlm` file | MediaPipe LLM Inference, or LiteRT-LM's own Android API | A Task Bundle or a LiteRT LM file, produced by `litert-torch` | | A single `.gguf` file | llama.cpp through the NDK | GGUF, self-contained | These are two separate native stacks with two separate build paths. A `.litertlm` file does not load into a llama.cpp-based wrapper, and a `.gguf` file does not load into MediaPipe's `LlmInference` or LiteRT-LM's `Engine`. **If what you are holding is neither of these,** a merged Hugging Face checkpoint (`config.json` + `model*.safetensors`) or a PEFT adapter directory (`adapter_config.json` + `adapter_model.safetensors`), it is not shippable into a native Android app as is. Run **inspecting-a-model-bundle** first to confirm which shape you actually have, then convert: - To `.litertlm`: `litert-torch export_hf`, covered below and in `references/mediapipe-litert-path.md`. - To GGUF: `convert_hf_to_gguf.py` then `llama-quantize`, covered below and in `references/llamacpp-jni-