AI systems
RefundIt
A multi-agent assistant for tax-refund paperwork
A Hebrew conversational assistant, built by a three-student course team, that reads tax documents, extracts and checks their fields, and tells a salaried worker what is still missing for a refund claim. It is not tax advice.
Software system Status: Prototype Status: Completed Academic · team of 3
Architecture diagram
Purpose
Salaried workers in Israel are often owed a tax refund, but gathering the right documents (Form 106, payslips, deduction receipts) and checking eligibility is tedious. RefundIt lets a user upload the documents and talk to an assistant that says what is missing and what the documents support.
My Role & Leadership
Project Owner · Technical Lead · Developer. Owned the project and led its technical direction in a three-student course team, and worked hands-on in its development.
- Led the architecture: three agents behind one API (a chat orchestrator, a document agent and an assessment agent), with every step traced.
- Set the engineering guardrails: deterministic extraction before the language model, allow-listed queries instead of generated SQL, and a hard budget of six model calls per request.
What the team built
- A chat orchestrator that classifies each message into one of six fixed intents and routes it to document processing, retrieval or validation, then writes one Hebrew reply.
- A document pipeline: PDF text with an OCR fallback, document-type classification, schema-driven field extraction (deterministic synonym matching first, the language model only for what remains) and per-field confidence.
- Retrieval over a tax-reference index and over the user's own uploaded documents, kept in separate namespaces.
- Read-only, allow-listed queries over case data: a keyword router picks one of three fixed, parameterised queries instead of letting a model write SQL.
- An assessment agent that checks coverage and cross-document consistency, computes a readiness score and a simplified refund estimate, and drafts a report.
- A FastAPI service with token authentication, a PostgreSQL case store and a single-page Hebrew interface.
How it works
Three agents behind one API. The orchestrator handles the conversation and routing, the document agent turns uploads into validated fields and indexed text, and the assessment agent audits the case and drafts a report. Every step is traced with its prompt and response.
The user uploads a document or sends a message through the Hebrew interface.
A document is read (PDF text, or OCR for scans), classified, and its fields extracted, normalised and scored for confidence, then indexed.
A message is classified by intent and answered from the tax references and the user's own documents.
On request, the assessment agent checks coverage and consistency, scores readiness and drafts a report with a simplified estimate.
Design decisions
- Deterministic extraction before the language model: fields that synonym rules can match never cost a model call.
- Allow-listed queries instead of generated SQL: the assistant can read case status, documents and missing items, and nothing else.
- A hard budget of six model calls per request, so cost and latency stay bounded however a conversation goes.
- A trace of every step (module, prompt, response), so each answer can be inspected afterwards.
Key technologies
Outcomes and limitations
Known limits
A course prototype, not a deployed service, and not tax advice: the estimate uses a simplified tax-bracket table and a fixed credit-point value. Hebrew only. No evaluation was recorded.
What was provided
- The language and embedding models, through a university gateway.
- Pinecone, Supabase, Tesseract, PyMuPDF and FastAPI, as third-party services and libraries.
- The course's mandatory API endpoints.
Evidence
- The team's code: agents, service, schemas and interface, and the team's own architecture diagram.
- No accuracy, user or latency figure was recorded, so none is claimed.