rag-document-qa
What it does
rag-document-qa is a full-stack Retrieval-Augmented Generation application. Users upload documents, ask natural-language questions about them, and receive answers that stream into the page token by token, each grounded in the actual text of the uploaded files rather than the model's training data alone. Source citations — chunk number and page — appear alongside the answer so the user can verify what was retrieved.
How it works
Why ingestion is asynchronous
When a document is uploaded, the API immediately returns a 202 Accepted response with a document ID rather than waiting for the full extraction and embedding pipeline to complete. The frontend polls the document's status (PROCESSING, READY, FAILED) until the background job finishes. This keeps the upload endpoint fast and decouples the slow work — OCR, chunking, embedding — from the HTTP response cycle.
How a scanned page still gets text
PDF extraction uses a three-strategy waterfall. The first attempt uses Apache PDFBox to read the digital text layer directly; this is fast and accurate for born-digital PDFs. If a page yields fewer than a minimum number of characters — a sign that it is a scanned image — the extractor falls through to Tesseract OCR via Tess4J. If Tesseract also fails, GPT-4o Vision is the last resort, treating the page as an image and asking the model to transcribe it. No page is silently skipped.
How answers stream token by token
Retrieval uses a native SQL cosine-similarity query on a pgvector IVFFlat index (100 lists) to find the five nearest chunks to the embedded question. Those chunks are assembled into a prompt and sent to GPT-4o. The response is streamed back using Spring WebFlux's Flux pipeline over Server-Sent Events, flushing each token past Tomcat's response buffer immediately so the user sees words appearing as the model generates them. Confidence is the average cosine similarity of the five retrieved chunks.
Design decisions
- Token-aware chunking with JTokkit and OpenAI's CL100K_BASE tokenizer produces 500-token chunks with 50-token overlap, matching the tokenizer used by the embedding model.
- IVFFlat indexing with 100 lists is a deliberate trade-off: faster approximate search at the cost of a small recall drop, acceptable for document Q&A where retrieval latency matters.
- Cross-document search is implemented purely in SQL with no schema change; the same vector table holds embeddings from all documents.
- The three-strategy extraction waterfall is implemented behind a DocumentHandler interface, making it straightforward to add new file types or swap out strategies.
- SSE streaming via Flux<ServerSentEvent> flushes tokens past the 8 KB buffer threshold that would otherwise batch them before delivery.
Stack
| Layer | Technology |
|---|---|
| Framework | Spring Boot 3.3 |
| Framework | Spring MVC |
| AI/ML | Spring AI 1.0 |
| AI/ML | OpenAI GPT-4o |
| Library | text-embedding-3-small (1536 dimensions) |
| Database | PostgreSQL 16 |
| Database | pgvector |
| Spring | Spring Data JPA (Hibernate) |
| Database | Flyway |
| Library | Project Reactor |
| Spring | Spring WebFlux |
| Processing | Apache PDFBox |
| Processing | Tesseract via Tess4J |
| Library | GPT-4o Vision |
| Processing | Apache POI |
| Processing | JTokkit |
| Library | Lombok |
| Library | SpringDoc OpenAPI |
| Library | Vanilla HTML/CSS/JS |
| Infra | Docker Compose |
| Language | Java 21 |