FWF-STRR - Space-Time Representation and Recognition in Computer Vision

This project shall investigate spatial and temporal relations in video in a principled manner. We will develop a novel representation, called ``space-time volume of interest'' VOI, that will be used consistently to represent various entities in video data, for instance independent foreground motion, background motion patterns, and repetitive patterns in space-time. These entities will be described by extended space-time oriented energy filters, and the descriptions will be utilized to categorize video textures, dynamic scenes, types of camera motion, independently moving foreground objects, and activities. The basic, individual representational units, i.e., individual VOIs, shall be integrated to complex, compound space-time models that cast votes for particular spatial and temporal locations as well as scales. The project is coarsely structured into four work packages, addressing the VOI representation itself, extraction of local space-time descriptors, learning and long-term space-time description, and online detection/categorization in videos. With this research project, we expect to contribute to a better representation and understanding of the ``deep structure'' of visual space-time. The novel concepts developed in this project will initiate a variety of further basic research in this field, as well as encourage new applications in video analysis, surveillance, and autonomous systems.