Face Mining of Star Trek

Our face mining technology enables new video viewing experiences:

* Clickable navigation to scenes of interest

* Search for specific people

* Hyperlinking scenes of interest

* Embedding scenes of interest in other media (e.g blogs)

* Adding commentary synchronized with play points of interests

* Summarizing occurrences of single characters

* Summarizing co-occurrences of characters

Here we illustrate the face mining concept for the TV series Star Trek. Specifically, we applied our state-of-the art algorithms in face detection, face tracking and face recognition to 67 Star Trek episodes over three seasons. This process automatically extracts all visible face tracks, and clusters these into a small number of same-person groupings. Currently, we recognize frontal or near-frontal tracks. In the near future, we will extend our results to non-frontal tracks as well.

Given our face mining output, it took a person less than five minutes to assign names to all the main characters (Kirk, Spock, Mccoy, Uhura, Scotty, Sulu and Chekov) and a couple of minor ones (Janice, Nurse Chapel) across all 67 episodes. The above graph visualizes the relative face time for each of the main characters across all three seasons.

via Pittsburgh Pattern Recognition.