AI Engineering Internship: Building a Document Q&A Bot with RAG

1. Objective

Large Language Models (LLMs) are trained on massive public datasets, giving them a broad, general understanding of world knowledge. However, they suffer from two major limitations:

1. Knowledge Cutoffs & Lack of Private Data: They do not know about your private files, recent enterprise reports, or proprietary documents.
2. Hallucinations: When asked about details outside their training data, they often make up realistic-sounding but completely incorrect answers.

Retrieval-Augmented Generation (RAG) solves this problem. Instead of retraining or fine-tuning a massive model, we supplement the user's query with relevant context retrieved from a custom document library.

                                 +--------------------+
                                 |  Custom Documents  |
                                 | (PDF, DOCX, TXT)   |
                                 +---------+----------+
                                           |
                                           v
+-------------------+            +---------+----------+            +------------------+
|    User Query     +----------->|   Vector Database  +----------->| Retrieved Chunks |
+---------+---------+            |  (Semantic Search) |            +--------+---------+
          |                      +--------------------+                     |
          |                                                                 v
          |                      +--------------------+            +--------+---------+
          +--------------------->| Prompt Formulator  |<-----------+  Source Citations|
                                 +---------+----------+            +------------------+
                                           |
                                           v
                                 +---------+----------+
                                 |  Language Model    |
                                 |   (Gemini/LLM)     |
                                 +---------+----------+
                                           |
                                           v
                                 +---------+----------+
                                 | Grounded Answer &  |
                                 |    Citations       |
                                 +--------------------+

2. Technical Requirements

To build a robust and clean python-based RAG system, you will need the following technical stack.

Language & Runtime

• Python 3.11 or higher: Offers the best compatibility with modern vector stores and generative AI SDKs.

Core Libraries & Packages

• Document Processing: * pypdf: For robust and fast text extraction from PDF files.
  • python-docx: For reading and parsing Microsoft Word (.docx) documents.
• Vector Storage & Embeddings:
  • chromadb (or faiss-cpu): A lightweight, disk-persistent vector database that operates entirely locally without requiring server instances.
• LLM & Embeddings Integration:
  • google-generativeai: To access Google Gemini models (gemini-2.5-flash-preview-09-2025 for text generation and text-embedding-004 for creating dense vector representations).
• Environment & Utility Management:
  • python-dotenv: To securely load and manage sensitive API keys from a .env file.
  • tqdm: To provide progress bars during ingestion and indexing.

Prerequisites

• A valid API Key (e.g., Google Gemini API Key).
• A standard terminal/command line interface.
• Optional: streamlit (version 1.30+) if you choose to build a simple, interactive web-based UI as a bonus.