POLICY GUARDIAN: A HYBRID TWO-TIER AI FRAMEWORK FOR AUTOMATED LEGAL DOCUMENT DISCOVERY, RISK CLASSIFICATION, AND SEMANTIC SIMPLIFICATION

Computer Science & Engineering

Every day, billions of internet users click "I Agree" on Terms of Service and Privacy Policy agreements without reading a single word. This is not a matter of carelessness — it is a structural impossibility imposed by documents deliberately written in dense legal language that is practically inaccessible to ordinary people. The consequences are significant: users routinely and unknowingly waive their rights to legal recourse, consent to unlimited data sharing, and accept liability terms they would reject if they understood them. Policy Guardian is a real-time, AI-powered browser extension engineered to close this transparency gap. The system deploys a novel Hybrid Two-Tier AI architecture: a locally fine-tuned DistilBERT transformer model performs high-speed, privacy-preserving classification of individual legal clauses directly on the user's machine, while a cloud-based large language model (Llama 3.3 70B via the Groq API) generates consolidated, plain-English verdicts for the flagged risky clauses only. The system also implements a Non-Destructive DOM Cloning scraper and a semantic Auto-Finder algorithm to automatically locate and extract legal documents from any website without user intervention. The fine-tuned Policy-Guardian-BERT model was trained on a rigorously cleaned subset of the lex_glue/unfair_tos benchmark dataset, achieving a classification accuracy of 96.33% and a macro-averaged F1-score of 0.9545 on a held-out validation set. The complete end-to-end pipeline — from link discovery and text extraction through risk classification to plain-English verdict generation — completes in approximately 1.75 seconds, making it entirely practical for everyday browsing. The system was deployed as a Chrome Extension using the Manifest V3 specification, connected to a Flask REST API backend, and validated across real-world websites including Reddit, Pinterest, and a purpose-built demonstration platform. By performing primary classification locally, ninety percent of the user's legal text is analyzed with zero data transmission to external servers, realizing a genuine Privacy by Design architecture. This paper presents the complete system design, training methodology, evaluation results, and a discussion of the social and ethical implications of deploying AI as a real-time consumer protection tool in the legal domain.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.