Why Re-invent Search?
Search information is a fundamental requirement nowadays. Information is carried over various mediums in different circumstances. In the realm of Internet, the most prevalent information carriers are text, audio, image and video. The search over text has been intensively researched and commercialized such as Google, Bing or Yahoo Search. However, the search over other carriers, especially video, is still lackluster compared with its text peer and there is nearly no mature commercialized application in the market. People and organizations have invested a lot over times to make the search over audio, image and video more practical thanks to the achievements of artificial intelligence in the recent years, but still, we are yet close to a generic, efficient, robust and scalable multi-media search approach. Consequently, filling this gap becomes our mission. We choose video, the most natural and information rich carrier as our initial target, to re-define the search.
People may ask: I can search video right now, just use YouTube search as an example. Correct, there are tons of ways to search the video you are looking for, but nearly all of the existing search approaches are based on the title, keywords or tags of the video. For example, if you search “elk” via YouTube search, you may notice that all the results returned either have elk in its title, keywords, or anything in the text description. That’s because the video search is still looking at the text description of videos rather than looking into the video itself! Assume you captured a video introducing different types of mammals, including elk, who live in North America. You gave 1 minute to the show of elk. But for some reasons, the video is titled as “Mammals in North America”, and unfortunately, elk is not mentioned at all in the text description of the video. Guess what? Your video very likely won’t appear in the search result of elk over the Internet. The traditional search engine won’t look into your video content, but only go through the text description around your video. That’s a big problem, and that’s what we are trying to solve. We would like the search engine to look into your video, understand what’s inside the video and where it is in the video.
When human watches video, the brain tries to get answers of two questions: what’s in the video and where is it? We are here to answer these two basic questions. Given the huge amount of video over the Internet, we know that it’s impossible to ask human to watch them all and write down the content descriptions, so we ask computer to watch them, video by video, frame by frame, and write down what the computer saw and where it is inside the video. Then when user asks for elk, the search engine will look into the results computer generated so your video “Mammals in North America” will appear in the search result.
Going further, even though somebody knows that your video “Mammals in North America” contains a scene of an elk, but he doesn’t know at what time the scene begins and ends. What he should do? Probably the doable, and the only approach is watching the video from the very beginning and get finger crossed he could find the elk sooner than later. Sounds stupid and tedious… But we have a way better solution for now! Our search engine will not only return “Mammals in North America” to you but also tell you the start and end time of the scene of elk! You don’t need to explore the entire video to find what you are searching for, but just dive into the video fragment where your search object exists.
We are here trying to facilitate the way how people search content. We expand the scope of search engine from text to video. We are trying to understand the content of video, not missing any significant object or scene, and then present what we have known to the user. Furthermore, we would like to reshape the search engine as a novel interface between human and computer, where the computer could perceive or even understand the world mimicking human being. In short, we are just answering two simple questions: What and Where?