Exploratory Video Visual Interface

The above video analysis produces rich and complex patterns over time. We are developing a variety of analysis techniques to attack this ever-changing stream of topics, allowing the user to quickly get an overview of what is being discussed and then home in on the most relevant stories associated with topics of interest. The top figure shows clusters of topics over a period of time, in this case one month. For this time period, the pixel grid for each topic looks like a monthly calendar with importance of the topic for a given day given by saturation of the color. The clustering significantly enriches the sense of topic context, and the pixel grid gives details of similarity or difference among clustered topics. The bottom figure provides an alternate ThemeRiver-like view showing topic behavior over time in a continuous and scalable fashion. Each band of color shows how topics rise and fall over time. At the bottom are key frames for selected times and topics. In both views in the figure, the user can select a pixel or a topic band, respectively, to bring up a display such as the video hot topics view above, positioned for the selected days and set of topics. Interlinked and interacting views such as these will be very important for fast and effective exploration of broadcast video or other multimedia collections. This work is led by Jing Yang and Mohammad Ghoniem.