{"blog_title":"Seitaro Shinagawa\u306e\u96d1\u8a18\u5e33","title":"OpenAI\u306ePPO\u8a00\u8a9e\u751f\u6210\u8ad6\u6587\u3092\u8aad\u3080\u2460  ","author_name":"snowman_88888","height":"190","author_url":"https://blog.hatena.ne.jp/snowman_88888/","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fsnowman-88888.hatenablog.com%2Fentry%2F2020%2F12%2F24%2F235532\" title=\"OpenAI\u306ePPO\u8a00\u8a9e\u751f\u6210\u8ad6\u6587\u3092\u8aad\u3080\u2460   - Seitaro Shinagawa\u306e\u96d1\u8a18\u5e33\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","version":"1.0","width":"100%","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/s/snowman_88888/20201224/20201224100713.png","categories":[],"provider_url":"https://hatena.blog","url":"https://snowman-88888.hatenablog.com/entry/2020/12/24/235532","published":"2020-12-24 23:55:32","provider_name":"Hatena Blog","blog_url":"https://snowman-88888.hatenablog.com/","description":"\u3053\u306e\u8a18\u4e8b\u306f\u5f37\u5316\u5b66\u7fd2\u82e6\u624b\u306e\u4f1a Advent Calendar 2020\u306e24\u65e5\u76ee\u306e\u8a18\u4e8b\u3067\u3059\u3002 2020.12.28 \u8ffd\u8a18 \u89e3\u91c8\u304c\u81f4\u547d\u7684\u306b\u30df\u30b9\u3063\u3066\u307e\u3057\u305f\u30fb\u30fb\u30fb\"To keep $\\pi$ from moving too far from $\\rho$\"\u306a\u306e\u3067\u300c\u4e21\u5206\u5e03\u304c\u96e2\u308c\u3059\u304e\u306a\u3044\u3088\u3046\u306b\u300d\u3067\u3059\u3002 \uff08\u8aa4\uff09\u5b66\u7fd2\u3059\u308b\u65b9\u7b56$\\pi$\u304c\u3001\u521d\u671f\u65b9\u7b56$ \\rho$\u3088\u308a\u3082\u96e2\u308c\u305f\u5206\u5e03\u3068\u306a\u308b\u3088\u3046\u306b\u3001KL divergence\u304c\u5927\u304d\u304f\u306a\u308b\u3088\u3046\u306b\u3059\u308b\u5f79\u5272\u3067\u3059\u3002 \uff08\u6b63\uff09\u5b66\u7fd2\u3059\u308b\u65b9\u7b56$\\pi$\u304c\u3001\u521d\u671f\u65b9\u7b56$ \\rho$\u3088\u308a\u3082\u96e2\u308c\u3059\u304e\u306a\u3044\u3088\u3046\u306b\u3001\u4e21\u5206\u5e03\u306eKL divergence\u304c\u5927\u304d\u3044\u3068\u30da\u30ca\u30eb\u30c6\u30a3\u3092\u304b\u3051\u308b\u5f79\u5272\u3067\u3059\u3002 2020.\u2026","type":"rich"}