[ad_1]
\n<\/p>\n
The internet contains an enormous amount of publicly available videos that we can learn from. You can watch a person make a gorgeous presentation, a digital artist draw a beautiful sunset, and a Minecraft player build an intricate house. However, these videos only provide a record of\u00a0what<\/em>\u00a0happened but not precisely\u00a0how<\/em>\u00a0it was achieved, i.e., you will not know the exact sequence of mouse movements and keys pressed. If we would like to build large-scale\u00a0foundation models<\/a>\u00a0in these domains as we\u2019ve done in language with\u00a0GPT<\/a>, this lack of action labels poses a new challenge not present in the language domain, where \u201caction labels\u201d are simply the next words in a\u00a0sentence.<\/p>\n In order to utilize the wealth of unlabeled video data available on the internet, we introduce a novel, yet simple, semi-supervised imitation learning method: Video PreTraining (VPT). We start by gathering a small dataset from contractors where we record not only their video, but also the actions they took, which in our case are keypresses and mouse movements. With this data we train an inverse dynamics model (IDM), which predicts the action being taken at each step in the video. Importantly, the IDM can use past\u00a0and future<\/em>\u00a0information to guess the action at each step. This task is much easier and thus requires far less data than the behavioral cloning task of predicting actions given\u00a0past video frames only<\/em>, which requires inferring what the person wants to do and how to accomplish it. We can then use the trained IDM to label a much larger dataset of online videos and learn to act via behavioral\u00a0cloning.
<\/p>\n<\/div>\n<\/div>\n