{"id":68,"date":"2023-01-24T12:17:35","date_gmt":"2023-01-24T12:17:35","guid":{"rendered":"https:\/\/todaysainews.com\/index.php\/2023\/01\/24\/learning-to-play-minecraft-with-video-pretraining-vpt\/"},"modified":"2025-04-27T07:36:22","modified_gmt":"2025-04-27T07:36:22","slug":"learning-to-play-minecraft-with-video-pretraining-vpt","status":"publish","type":"post","link":"https:\/\/todaysainews.com\/index.php\/2023\/01\/24\/learning-to-play-minecraft-with-video-pretraining-vpt\/","title":{"rendered":"Learning to Play Minecraft with Video PreTraining (VPT)"},"content":{"rendered":"

[ad_1]
\n<\/p>\n

\n <\/p>\n
\n

We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data. With fine-tuning, our model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Our model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents.<\/p>\n<\/div>\n

Read Paper<\/a><\/p>\n
View Code and model weights<\/a><\/p>\n
MineRL Competition<\/a>
\n<\/section>\n

The internet contains an enormous amount of publicly available videos that we can learn from. You can watch a person make a gorgeous presentation, a digital artist draw a beautiful sunset, and a Minecraft player build an intricate house. However, these videos only provide a record of what<\/em> happened but not precisely how<\/em> it was achieved, i.e. you will not know the exact sequence of mouse movements and keys pressed. If we would like to build large-scale foundation models<\/a> in these domains as we\u2019ve done in language with GPT<\/a>, this lack of action labels poses a new challenge not present in the language domain, where \u201caction labels\u201d are simply the next words in a sentence.<\/p>\n

In order to utilize the wealth of unlabeled video data available on the internet, we introduce a novel, yet simple, semi-supervised imitation learning method: Video PreTraining (VPT). We start by gathering a small dataset from contractors where we record not only their video, but also the actions they took, which in our case are keypresses and mouse movements. With this data we train an inverse dynamics model (IDM), which predicts the action being taken at each step in the video. Importantly, the IDM can use past and future<\/em> information to guess the action at each step. This task is much easier and thus requires far less data than the behavioral cloning task of predicting actions given past video frames only<\/em>, which requires inferring what the person wants to do and how to accomplish it. We can then use the trained IDM to label a much larger dataset of online videos and learn to act via behavioral cloning.<\/p>\n

<\/p>\n

\n
\n <\/div>\n
\n <\/div>
VPT method overview<\/figcaption><\/figure>\n

VPT Zero-Shot Results<\/h2>\n

We chose to validate our method in Minecraft because it (1) is one of the most actively played video games in the world and thus has a wealth of freely available video data and (2) is open-ended with a wide variety of things to do, similar to real-world applications such as computer usage. Unlike prior<\/a> works<\/a> in Minecraft that use simplified action spaces aimed at easing exploration, our AI uses the much more generally applicable, though also much more difficult, native human interface: 20Hz framerate with the mouse and keyboard.<\/p>\n

Trained on 70,000 hours of IDM-labeled online video, our behavioral cloning model (the \u201cVPT foundation model\u201d) accomplishes tasks in Minecraft that are nearly impossible to achieve with reinforcement learning from scratch. It learns to chop down trees to collect logs, craft those logs into planks, and then craft those planks into a crafting table; this sequence takes a human proficient in Minecraft approximately 50 seconds or 1,000 consecutive game actions.<\/p>\n

<\/p>\n

\n
\n <\/div>\n
\n <\/div>
Sequence of items required to craft a crafting table, labeled with the median time it takes proficient humans to reach each step<\/figcaption><\/figure>\n