Legal AI Assistant

High-level Overview

An AI-driven legal assistant that provides accurate, location-specific legal information. It leverages location-aware search and trusted legal sources to deliver clear, cited responses, and guides users to official documentation for further reference.

Role & Responsibilities

Search & Retrieval System

• Designed and implemented the query augmentation system for the citation engine (Module that generates the citations), deciding how queries should be adapted for each task to get the best results.
• Improved the vector search process to enhance citation accuracy by improving language model prompts, filtering out problematic data, and optimizing how final answers are generated using those citations.

Data Infrastructure

• Designed the logic for storing vector data, including pre-processing steps, what metadata to store with each vector, and regional separation by country and state.
• Designed a logging system to track every website the web scraper visited, failed to access, was blocked from, or ignored — all saved in a database for future reference.
• Migrated away from a locally run vector database (FAISS) setup to Supabase with pg_vector (a postgresql extension that allows storing vector data) to address scalability, maintenance concerns and metadata filtering.

System Performance & Architecture

• Optimized the AI’s response generation process to reduce latency by evaluating tools, external services, and choosing between AI agents with tools support and custom-built approach.

Key Achievements

• Advocated for transitioning away from the ReAct-based AI agent in favor of a custom retrieval workflow. Led the implementation, resulting in a 67% reduction in response times (from ~90 seconds to ~30 seconds).
• Implemented multithreading for parallel execution of independent API calls. This was one of the contributing factors to the overall response time reduction.
• Identified problematic data sources (e.g., non-informative web pages) by analyzing vector database responses. Created a block list to exclude these sources, resulting in a significant improvement in citation accuracy.
• Discovered instability caused by using a single query augmentation method for both vector search and final response generation, developed a separate augmentation module for each process. This improved citation accuracy noticeably.
• Found that inconsistent search results were caused by a vector search settings prioritized speed over relevance. Adjusted the search depth, improving citation accuracy with only a small increase in response time.Together, these enhancements led to an overall 60% improvement in citation accuracy.

Design Decisions and Trade-offs

• Why Supabase over self-hosted vector database (FAISS)?
  Chose Supabase for better horizontal scalability, metadata filtering, and reduced maintenance complexity.
• Why custom-built retrieval workflow over AI agent with tools (ReAct agent)?
  AI agents like ReAct (an AI agent combining reasoning and external tool use) introduced unnecessary latency and unpredictable control flow. We opted for a deterministic workflow with clear separation of search and generation phases, allowing better control and debugging.
• Why two separate query augmentation functions?
  Using a single prompt for both vector search and LLM response generation degraded citation quality. Splitting these concerns allowed us to optimize each independently.

Challenges & Constraints

• Limited legal data availability in certain states, requiring custom scraping pipelines and fallback logic.
• No access to full legal databases due to licensing constraints — had to rely on public legal portals.
• Latency expectations under 30s total response time despite complex document retrieval and parsing.

Mistakes & Learnings

1. Accidental Deletion of Admin Panel Data

• Issue: While temporarily handling DevOps responsibilities during a deployment, ran a volume-wiping command on the development server, which accidentally deleted all admin panel data including chat history, settings, and preferences.
• Root Cause: Misunderstood the environment architecture — unlike production, the development setup did not isolate admin data from core services.
• Resolution: Migrated admin data in development to a separate database, matching the production architecture to prevent similar issues in the future.
• What I learned: Always confirm the purpose and scope of infrastructure commands. Never assume — verify before executing.

2. Broken Vector Index During Data Scraping

• Issue: During a routine weekly data scrape, the service crashed mid-process. After the crash, vector search in the development environment returned incomplete or missing legal citations, causing a noticeable drop in citation engine performance compared to production.
• Troubleshooting:
  • Observed inconsistent results between development and production for the same legal queries.
  • Noticed that citation data retrieved from the dev vector DB was missing key entries.
  • Checked vector DB size on disk — it was about half the expected size.
  • Correlated findings with logs showing that the scraping service had crashed partway through the run.
• Root Cause: The scraper was writing directly to the live development vector database. When the service crashed, it left the vector index incomplete, degrading search accuracy.
• Resolution: Added a staging vector database that fully completes scraping and indexing before replacing the live DB. This ensures the live system only receives validated data.
• What I learned: Never write directly to live query systems. Always stage, validate, and then promote — especially for critical indexing tasks. Keeping ingestion and query layers separate protects system reliability.

Technologies Used

Python, Django, PostgreSQL, pg_vector, Supabase, Faiss (legacy), Docker, AWS EC2, AWS RDS, Langchain (legacy), langgraph (legacy)

Confidentiality Note

"Details are generalized due to NDA and confidentiality obligations."