Real-Time Multi-Modal Traffic Violation Detection and Alert System – Final Year Project

Abstract

The rapid increase in urban vehicular traffic has amplified the incidence of traffic rule violations, leading to higher accident rates and congestion. This project proposes a real‑time, camera‑based system that integrates object detection, vehicle tracking, and behavior analysis to automatically detect common traffic violations such as red‑light crossing, illegal lane changes, and unauthorized pedestrian crossings. Leveraging state‑of‑the‑art deep learning models (YOLOv8 for detection, DeepSORT for tracking, and MediaPipe Pose for pedestrian intent), the system generates instant alerts to traffic control centers and provides visual evidence for enforcement. The solution is designed to run on edge devices (NVIDIA Jetson Nano) to ensure low latency and scalability across city‑wide deployments.

Project Overview

Urban traffic management increasingly relies on intelligent video analytics to supplement human monitoring. Traditional surveillance systems merely record footage, requiring manual review for violation detection, which is time‑consuming and error‑prone. This project bridges that gap by delivering an end‑to‑end pipeline that ingests live video streams, detects vehicles and pedestrians, tracks their trajectories, and applies rule‑based logic to flag violations. The system architecture comprises four core modules: (1) Video acquisition and preprocessing, (2) Multi‑class object detection, (3) Multi‑object tracking, and (4) Violation inference and alert generation. A web‑based dashboard visualizes live feeds, violation statistics, and provides playback capabilities for forensic analysis.

Problem Statement

Cities worldwide face challenges in enforcing traffic regulations due to limited manpower and the sheer volume of road intersections. Manual monitoring cannot keep pace with the dynamic nature of traffic flow, resulting in delayed or missed violation detection. Existing automated solutions often focus on a single violation type (e.g., red‑light detection) and lack robust multi‑modal analysis, leading to high false‑positive rates. There is a need for a comprehensive, low‑cost, and scalable system capable of detecting multiple violation categories simultaneously, with high accuracy and real‑time performance.

Proposed Solution

The proposed solution employs a hybrid deep learning framework:

1. Object Detection: YOLOv8 (trained on a custom dataset combining COCO vehicle classes and traffic sign annotations) identifies vehicles, traffic lights, and pedestrians.
2. Object Tracking: DeepSORT maintains consistent IDs across frames, enabling trajectory analysis.
3. Pose Estimation: MediaPipe Pose extracts pedestrian gait cues to infer crossing intent.
4. Rule Engine: A Python‑based engine evaluates temporal and spatial constraints (e.g., vehicle entering intersection after red light, pedestrian crossing when signal is green for pedestrians) to flag violations.
5. Edge Deployment: The pipeline is containerized with Docker and optimized using TensorRT for execution on NVIDIA Jetson Nano, achieving ~30 FPS on 720p streams.
6. Alert System: Violations trigger MQTT messages to a central server, which logs events and pushes notifications to a web dashboard built with Flask and React.

Expected Outcomes

• A fully functional prototype capable of detecting at least three violation types with >85% precision on a held‑out test set.
• Real‑time performance (≥25 FPS) on edge hardware.
• A web dashboard displaying live video, annotated detections, and violation logs.
• Comprehensive documentation, including dataset preparation scripts, model training notebooks, and deployment guides.
• Evaluation report comparing the system against baseline methods (single‑task detection) demonstrating reduced false‑positive rates.

Research Papers & References

• Real‑Time Traffic Violation Detection Using YOLOv5 and DeepSORT
• Multi‑Modal Fusion for Pedestrian Intent Prediction in Urban Environments
• Edge‑Optimized Deep Learning for Smart City Surveillance
• A Survey on Traffic Signal Violation Detection Using Computer Vision
• TensorRT Acceleration of YOLOv8 for Embedded Systems