Abstract
We present a distributed framework of understanding, indexing, and searching complex events from large amounts of surveillance video content. Video events and relationships between scene entities are represented by Spatio-Temporal And-Or Graphs (ST-AOG) and inferred in a distributed computing system using a bottom-up top-down strategy. We propose a method for sub-graph indexing of ST-AOGs of the recognized events for robust retrieval and quick search. Plain text reports of the scene are automatically generated to describe scene entities' relationships, contextual information, as well as events of interest. When a query is provided as keywords, plain text, voice, or a video clip, the query is parsed and the closest events are extracted utilizing text description and sub-graph matching.