{"id":49,"date":"2023-01-24T06:37:33","date_gmt":"2023-01-24T06:37:33","guid":{"rendered":"https:\/\/todaysainews.com\/index.php\/2023\/01\/24\/new-and-improved-content-moderation-tooling\/"},"modified":"2025-04-27T07:36:22","modified_gmt":"2025-04-27T07:36:22","slug":"new-and-improved-content-moderation-tooling","status":"publish","type":"post","link":"https:\/\/todaysainews.com\/index.php\/2023\/01\/24\/new-and-improved-content-moderation-tooling\/","title":{"rendered":"New and Improved Content Moderation Tooling"},"content":{"rendered":"<p> [ad_1]<br \/>\n<\/p>\n<div>\n        <!--kg-card-begin: markdown--><\/p>\n<div class=\"js-excerpt\">\n<p>We are introducing a new and improved content moderation tool: The <a href=\"https:\/\/beta.openai.com\/docs\/api-reference\/moderations\">Moderation endpoint<\/a> improves upon our previous content filter, and is available for free today to OpenAI API developers.<\/p>\n<\/div>\n<p>To help developers protect their applications against possible misuse, we are introducing the faster and more accurate <a href=\"https:\/\/beta.openai.com\/docs\/api-reference\/moderations\">Moderation endpoint<\/a>. This endpoint provides OpenAI API developers with free access to <a href=\"https:\/\/openai.com\/blog\/customized-gpt-3\/\">GPT-based<\/a> classifiers that detect undesired content\u2014an instance of <a href=\"https:\/\/openai.com\/blog\/critiques\/\">using AI systems<\/a> to assist with human supervision of these systems. We have also released both a <a href=\"https:\/\/arxiv.org\/abs\/2208.03274\">technical paper<\/a> describing our methodology and the <a href=\"https:\/\/github.com\/openai\/moderation-api-release\">dataset<\/a> used for evaluation.<\/p>\n<p>When given a text input, the Moderation endpoint assesses whether the content is sexual, hateful, violent, or promotes self-harm\u2014content prohibited by our <a href=\"https:\/\/beta.openai.com\/docs\/usage-guidelines\/content-policy\">content policy<\/a>. The endpoint has been trained to be quick, accurate, and to perform robustly across a range of applications. Importantly, this reduces the chances of products \u201csaying\u201d the wrong thing, even when deployed to users at-scale. As a consequence, AI can unlock benefits in sensitive settings, like education, where it could not otherwise be used with confidence.<\/p>\n<div class=\"wide\">\n<div class=\"animation js-animation\">\n<div class=\"animation-endpoint\">\n<div class=\"animation-endpoint-internals\">\n<p>Violence<\/p>\n<p>Self-harm<\/p>\n<p>Hate<\/p>\n<p>Sexual<\/p>\n<\/p><\/div>\n<p>      <span class=\"animation-endpoint-name\">Moderation endpoint<\/span>\n    <\/div>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"animation-arrow animation-arrow--down js-arrow\" width=\"10\" height=\"43\" viewbox=\"0 0 10 43\" fill=\"none\"><path d=\"M5 43L9.33013 35.5H0.669873L5 43ZM4.25 0V36.25H5.75V0H4.25Z\" fill=\"black\"\/><\/svg><br \/>\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"116\" height=\"10\" fill=\"none\" class=\"animation-arrow animation-arrow--right js-arrow\"><path d=\"m115.314 5-7.5-4.33v8.66l7.5-4.33ZM.5 5.75h108.064v-1.5H.5v1.5Z\" fill=\"#000\"\/><\/svg>\n  <\/div>\n<\/div>\n<p>The Moderation endpoint helps developers to benefit from our infrastructure investments. Rather than build and maintain their own classifiers\u2014an extensive process, as we document in our <a href=\"https:\/\/arxiv.org\/abs\/2208.03274\">paper<\/a>\u2014they can instead access accurate classifiers through a single API call.<\/p>\n<p>As part of OpenAI\u2019s <a href=\"https:\/\/openai.com\/charter\/\">commitment<\/a> to <a href=\"https:\/\/openai.com\/blog\/best-practices-for-deploying-language-models\/\">making the AI ecosystem safer<\/a>, we are providing this endpoint to allow free moderation of all OpenAI API-generated content. For instance, <a href=\"https:\/\/www.inworld.ai\/\">Inworld<\/a>, an OpenAI API customer, uses the Moderation endpoint to help their AI-based virtual characters remain appropriate for their audiences. By leveraging OpenAI\u2019s technology, Inworld can focus on their core product: creating memorable characters. We currently do not support monitoring of third-party traffic.<\/p>\n<hr\/>\n<p>Get started with the Moderation endpoint by checking out <a href=\"https:\/\/beta.openai.com\/docs\/guides\/moderation\/overview\">the documentation<\/a>. More details of the training process and model performance are available in our <a href=\"https:\/\/arxiv.org\/abs\/2208.03274\">paper<\/a>. We have also released an <a href=\"https:\/\/github.com\/openai\/moderation-api-release\">evaluation dataset<\/a>, featuring Common Crawl data labeled within these categories, which we hope will spur further research in this area.<\/p>\n<p><!--kg-card-end: markdown--><\/div>\n<p>[ad_2]<br \/>\n<br \/><a href=\"https:\/\/openai.com\/blog\/new-and-improved-content-moderation-tooling\/\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>[ad_1] We are introducing a new and improved content moderation tool: The Moderation endpoint improves upon our previous<\/p>\n","protected":false},"author":2,"featured_media":50,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-49","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tools"],"_links":{"self":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/49","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/comments?post=49"}],"version-history":[{"count":1,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/49\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/posts\/49\/revisions\/56"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/media\/50"}],"wp:attachment":[{"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/media?parent=49"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/categories?post=49"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/todaysainews.com\/index.php\/wp-json\/wp\/v2\/tags?post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}