{"id":304,"date":"2023-01-30T15:56:02","date_gmt":"2023-01-30T15:56:02","guid":{"rendered":"https:\/\/todaysainews.com\/index.php\/2023\/01\/30\/software-and-tasks-for-continuous-control\/"},"modified":"2025-04-27T07:35:22","modified_gmt":"2025-04-27T07:35:22","slug":"software-and-tasks-for-continuous-control","status":"publish","type":"post","link":"https:\/\/todaysainews.com\/index.php\/2023\/01\/30\/software-and-tasks-for-continuous-control\/","title":{"rendered":"Software and Tasks for Continuous Control"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div>\n<h4>Overview<\/h4>\n<p>A public colab notebook with a tutorial for dm_control software is available <a href=\"https:\/\/colab.sandbox.google.com\/github\/deepmind\/dm_control\/blob\/master\/tutorial.ipynb\">here<\/a>.<\/p>\n<h5>Infrastructure<\/h5>\n<ul role=\"list\">\n<li>An autogenerated MuJoCo Python wrapper provides full access to the underlying engine.<\/li>\n<li>PyMJCF is a Document Object Model, wherein a hierarchy of Python <em>Entity <\/em>objects corresponds to MuJoCo model elements.<\/li>\n<li>Composer is the high-level \u201cgame engine\u201d which streamlines the composing of Entities into scenes and the defining observations, rewards, terminations and general game logic.<\/li>\n<li>The Locomotion framework introduces several abstract Composer entities such as the Arena and Walker, facilitating locomotion-like tasks.<\/li>\n<\/ul>\n<h5>Environments<\/h5>\n<ul role=\"list\">\n<li>The <a href=\"https:\/\/www.youtube.com\/watch?v=rAai4QzcYbs\">Control Suite<\/a>, including a new <a href=\"https:\/\/www.youtube.com\/watch?v=RhRLjbb7pBE&amp;t=5s\">quadruped<\/a> and <a href=\"https:\/\/www.youtube.com\/watch?v=i0_OjDil0Fg\">dog<\/a> environment.<\/li>\n<li>Several locomotion tasks, including soccer.<\/li>\n<li>Single arm robotic manipulation tasks using snap-together bricks.<\/li>\n<\/ul>\n<h4>Highlights<\/h4>\n<h5>Named Indexing<\/h5>\n<p>Exploiting MuJoCo\u2019s support of <em>names<\/em> for all model elements, we allow strings to index and slice into arrays. So instead of writing:<\/p>\n<p>&#8220;fingertip_height = physics.data.geom_xpos[7, 2]&#8221;<\/p>\n<p>&#8230;using obscure, fragile numerical indexing, you can write:<\/p>\n<p>&#8220;fingertip_height = physics.named.data.geom_xpos[&#8216;fingertip&#8217;, &#8216;z&#8217;]&#8221;<\/p>\n<p>leading to a much more robust, readable codebase.<\/p>\n<h4>PyMJCF<\/h4>\n<p>The PyMJCF library creates a Python object hierarchy with 1:1 correspondence to a MuJoCo model. It introduces the attach() method which allows models to be attached to one another. For example, in our tutorial we create procedural multi-legged creatures by attaching legs to bodies and creatures to the scene.<\/p>\n<figure style=\"padding-bottom:33.723653395784545%\" class=\"w-richtext-align-center w-richtext-figure-type-video\">\n<p><iframe loading=\"lazy\" title=\"PyMJCF &quot;creatures&quot;\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/0Lw_77PErjg?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/figure>\n<h5>Composer<\/h5>\n<p>Composer is the \u201cgame engine\u201c framework, which defines a particular order of runtime function calls, and abstracts the affordances of <em>reward<\/em>, <em>termination<\/em> and <em>observation<\/em>. These abstractions allowed us to create useful submodules:<\/p>\n<p>composer.Observable: An abstract observation wrapper which can add noise, delays, buffering and filtering to any sensor.<\/p>\n<p>composer.Variation: A set of tools for randomising simulation quantities, allowing for agent robustification and sim-to-real via model variation.<\/p>\n<figure class=\"w-richtext-figure-type- w-richtext-align-center\">\n<div><img decoding=\"async\" src=\"https:\/\/assets-global.website-files.com\/621e749a546b7592125f38ed\/6231ee766a323a1793376c3b_diagram3.svg\" alt=\"\"\/><\/div><figcaption>Diagram showing the life-cycle of Composer callbacks. Rounded rectangles represent callbacks that Tasks and Entities may implement. Blue rectangles represent built-in Composer operations.<\/figcaption><\/figure>\n<h4>Locomotion<\/h4>\n<p>The Locomotion framework introduced the abstractions:<\/p>\n<p>Walker: A controllable entity with common locomotion-related methods, like projection of vectors into an egocentric frame.<\/p>\n<p>Arena: A self-scaling randomised scene, in which the walker can be placed and given a task to perform.<\/p>\n<p>For example, using just 4 function calls, we can instantiate a humanoid walker, a WallsCorridor arena and combine them in a RunThroughCorridor task.<\/p>\n<figure style=\"padding-bottom:45%\" class=\"w-richtext-align-center w-richtext-figure-type-video\">\n<p><iframe loading=\"lazy\" title=\"CMU Humanoid solving the RunThroughCorridor task\" width=\"640\" height=\"480\" src=\"https:\/\/www.youtube.com\/embed\/UfSHdOg-bOA?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/figure>\n<h4>New Control Suite domains<\/h4>\n<h5>Quadruped<\/h5>\n<ul role=\"list\">\n<li>A generic quadruped domain with a passively stable body.<\/li>\n<li>Several pure locomotion tasks (e.g. walk, run).<\/li>\n<li>An escape task requiring rough terrain navigation.<\/li>\n<li>A fetch task requiring ball dribbling.<\/li>\n<\/ul>\n<figure style=\"padding-bottom:45%\" class=\"w-richtext-align-center w-richtext-figure-type-video\">\n<p><iframe loading=\"lazy\" title=\"Control Suite Quadruped\" width=\"640\" height=\"480\" src=\"https:\/\/www.youtube.com\/embed\/RhRLjbb7pBE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/figure>\n<h5>Dog<\/h5>\n<ul role=\"list\">\n<li>An elaborate model based on a skeleton commissioned from <a href=\"https:\/\/www.turbosquid.com\/Search\/Artists\/leo3Dmodels\">leo3Dmodels<\/a>.<\/li>\n<li>A challenging ball-fetching task that requires precision grasping with the mouth.<\/li>\n<\/ul>\n<figure style=\"padding-bottom:33.723653395784545%\" class=\"w-richtext-align-center w-richtext-figure-type-video\">\n<p><iframe loading=\"lazy\" title=\"Control Suite dog domain\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/i0_OjDil0Fg?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/figure>\n<h5>Showcase<\/h5>\n<p>A fast-paced montage of dm_control based tasks from DeepMind:<\/p>\n<figure style=\"padding-bottom:33.723653395784545%\" class=\"w-richtext-align-center w-richtext-figure-type-video\">\n<p><iframe loading=\"lazy\" title=\"dm_control showcase\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/CMjoiU482Jk?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<\/figure>\n<\/div>\n<p>[ad_2]<br \/>\n<br \/><a href=\"https:\/\/www.deepmind.com\/blog\/dm-control-software-and-tasks-for-continuous-control\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] Overview A public colab notebook with a tutorial for dm_control software is available here. Infrastructure An autogenerated<\/p>\n","protected":false},"author":2,"featured_media":305,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[],"class_list":["post-304","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-deepmind-ai"],"_links":{"self":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/comments?post=304"}],"version-history":[{"count":1,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/304\/revisions"}],"predecessor-version":[{"id":2915,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/304\/revisions\/2915"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/media\/305"}],"wp:attachment":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/media?parent=304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/categories?post=304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/tags?post=304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}