User Guide

Overview

We aim to provide a simple interface to users for searching the content from videos over the internet. Before diving deeper into the details of the guidance, let’s firstly get a sense of how our system works as a whole. The streamki search engine is backed by a sophisticated video understanding system, which crawls videos from the internet, then feeds the video into the AI engine. The AI engine watches the video, analyzes it, notes down what has been recognized into a documentation system, where the search engine can query the result from. In short, the search engine is on top of the video pool that the video understanding system has watched. Based on what the user requests for, the search engine will look for it from the video pool. The larger the pool size is, the more results the user will get.

Our search engine not only can help the user find the videos which contain the object that the user is searching for, but is also capable to tell the time range within the video when the object shows up. For example, if a user would like to search for “locomotive”, the search engine will populate the locomotive videos into the result list, and going further, extract the time range when the locomotive shows up, then presents the time ranges to the user.

The UI

The UI of streamki search looks very similar to all the existing mainstream search engines. As Fig. 1 illustrates, there is one edit box in the center of the main page, where the user could input their search target. It’s worthy to point out that we post the total length of the video our backend system has watched, to just give a sense what the size of our video pool is. The video pool is growing day by day, hopefully we could accelerate the growth rate by investing more to our computing infrastructure so the user can have a wider scope to the video resource over the internet.

Fig. 1 The main page of streamki search engine

The search will execute when the user hits “Go!” button. The search result, which is a list of videos, will be presented to the user shortly. Fig. 2 is a sample result page of searching for “locomotive”. The result page presents three types of information around the video to the user: the thumbnail of the video, the video fragment list and the source URL of the video. The thumbnail is a frame extracted from the video representing the overall video content. The video fragment list consists of the time ranges where the “locomotive” exists. Each time range is a hyperlink redirecting the user to playback that specific portion of the video. The source URL of the video, which can be used as an extended resource, could redirect the user to the original web page containing the video.

Fig. 2 A search result sample

How to phrase your search?

Conceptually, the user can input any text to the search engine. However, there are some constraints regarding the phrasing of the input text. Before elaborating the constraints, let’s firstly introduce WordNet. WordNet is a lexical database of English. It groups nouns, verbs, adjectives and adverbs into sets of cognitive synonyms (synsets), each expressing a distinct concept. Our search engine currently only focuses on the nouns part. For example, if a user inputs “American Black Bear” in the search box, the search engine will firstly translate it into the corresponding synset, then query the synset from our video database for the related video fragments. The list of video fragments which contains “American Black Bear” will be analyzed, ranked and then presented to the user. In principle, the WordNet synset is the “common language” throughout the entire system, thus if a particular input text can be successfully translated into a synset and this synset is currently supported by our system, this input is accepted. For example, if a user inputs something like “the 3 year old black bear on my driveway looking for food”, this search will fail because the input words is still not in the WordNet synset pool.

We are currently support ~3500 synsets, which cover the objects people are familiar with in everyday life. We know these are still a tiny piece, and we are working hard to expand the scope of our supported synsets. For the entire list of supported objects, please refer to “supported objects” page.

1. English only;

2. No typo;

3. A word or a sequence of words representing a noun, no adjective, no verb, and no adverb;

4. Be a synset in our 3500 supported synset list.

Interaction with the search engine

The best way to check if the user’s input is supported is sending the input then the search engine will tell the result. There are five types of result the user can expect:

1. The search input is accepted and the records are found. Fig. 2 is an example.

2. The search input is accepted but no record is found as Fig. 3 illustrates. The search engine will pop the message at the end of the web page as the red box outlined in Fig. 3.


Fig.3 A search result page showing no record is found

3. The search input has typo so the search engine rejects it. For example, if the user input “lllocomotive” where the prefix two “l” are redundant, it will be rejected and the search engine will guess what should be the correct input as the red box outlined in Fig. 4.


Fig. 4 A search result page showing typo of the input

4. The search input is not in the 3500 synsets list so the search engine still rejects it. For example, if the user input is “Canadian National Railway locomotive”, since it’s not a synset of WordNet, the search engine will reject it as the red box outlined in Fig. 5.


Fig. 5 A search result page showing the input object is not supported

5. The search input is supported by our system, but it is too generic so covers a huge amount of synsets, which may overload our system and can potentially yield an annoyingly large list of result candidates. In order to mitigate that, we currently don’t allow the input string we thought too generic. For example, if you would like to search for “goldfish”, you should input “goldfish” rather than “fish” because “fish” is too generic and will trigger the protection mechanism. The search engine will reject “fish” as the red box outlined in Fig. 6.


Fig.6 A search result page showing the input object is too generic

Want to know more?

It the user would like to get a better understanding of our system, please refer to “Technology” subpage of our website.