RAG Implementation (Android Environment)

1. Query Input & Tokenization

  • The user inputs a query within the app.
  • HuggingFace Tokenizers (Java) + Tokenizers Native for Android (AAR)
    → Converts the input text into a sequence of tokens readable by the model.
    → Fast and efficient tokenizing is possible using android native library based JNI

2. Query Embedding Generation

  • ONNX Runtime (onnxruntime-android)
    → Loads a pre-trained embedding model(예: MiniLM, BERT based Sentence Embedding Model) exported to ONNX format.
    → Converts the tokenized query into a high-dimensional vector representation.
    → This vector serves as the baseline for semantic similarity search.

3. Knowledge Base Retrieval (Retrieval Stage)

  • ObjectBox (Local Vector DB)
    → Stores in-app documents, FAQs, or ingested external texts alongside their precomputed embeddings.
    → Computes cosine similarity or dot products between the query embedding and stored document vectors to retrieve the most relevant passages.
    → Supplies the precise context required to ground the model's response.

4. Context Augmentation (Augmentation Stage)

  • Combines the retrieved context with the original user query into a single structured prompt.

Example:

  • User Query: "What is RAG in Android?"

  • Retrieved Document: "RAG stands for Retrieval-Augmented Generation, an architecture combining retrieval mechanisms with generative models."

  • Model Input: "Question: What is RAG in Android? Context: RAG stands for Retrieval-Augmented Generation..."


5. Response Generation (Generation Stage)

  • DJL (Deep Java Library) BOM + HuggingFace Model
    → Loads HuggingFace-compatible language models via DJL.
    → Feeds the augmented input (query + retrieved context) to generate the final response.
    → DJL provides runtime flexibility on Android by supporting various native backends, including ONNX, PyTorch, and TensorFlow.

6. 결과 반환

  • Show answer generated by Model
  • It can prove accuracy and rliability because of answer based document

🔗 End-to-End Pipeline Summary

  1. User Query → Tokenization
  2. Query Embedding Generation via ONNX Runtime
  3. Relevant Context Retrieval from ObjectBox Vector DB
  4. Prompt Augmentation (Query + Retrieved Context)
  5. Response Generation via DJL + HuggingFace Model
  6. Final Output Delivered to the user

👉 This architecture eliminates sole reliance on a model's static training weights. By querying a local vector store (ObjectBox) or dynamic data sources, the app can supply up-to-date, domain-specific information on device. Combining retrieval-grounded precision with generative fluency forms the core advantage of an on-device RAG pipeline.