{"categories":[],"url":"https://nishiohirokazu.hatenadiary.org/entry/20140208/1391825340","description":"\u30ea\u30ab\u30ec\u30f3\u30c8\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8(RNN)\u306b\u9577\u8ddd\u96e2\u76f8\u95a2\u3092\u5b66\u7fd2\u3055\u305b\u308b\u306e\u306f\u96e3\u3057\u3044\u554f\u984c\u3060\u3063\u305f\u304c\u3001Hessian-Free\u3092\u4f7f\u3063\u305f\u3089\u3067\u304d\u305f\u3001\u3068\u3044\u3046\u8a71\u3002RNN\u306fBack Propagation Through Time(BPTT)\uff0b\u78ba\u7387\u7684\u52fe\u914d\u6cd5\u3067\u7c21\u5358\u306b\u8a08\u7b97\u3067\u304d\u308b\u3053\u3068\u304c\u9577\u6240\u3068\u3055\u308c\u3066\u3044\u308b\u304c\u300110\u30bf\u30a4\u30e0\u30b9\u30c6\u30c3\u30d7\u307b\u3069\u96e2\u308c\u305f\u76f8\u95a2\u306f1\u6b21\u306e\u52fe\u914d\u6cd5\u3067\u306f\u5168\u7136\u5b66\u7fd2\u3067\u304d\u306a\u3044\u305d\u306e\u539f\u56e0\u306f\u201cvanishing/exploding gradients\u201d\u3002\u9577\u8ddd\u96e2\u76f8\u95a2\u306fBPTT\u3067\u4f55\u5ea6\u3082\u524d\u306e\u6642\u523b\u306e\u5c64\u3078\u4f1d\u642c\u3055\u308c\u308b\u305f\u3081\u3001\u8aa4\u5dee\u4fe1\u53f7\u304c\u3059\u3050\u306b\u6e1b\u8870\u3057\u3066\u6d88\u3048\u3066\u3057\u307e\u3046\u3002\u6700\u8fd1\u751f\u307e\u308c\u305fHessian-Free(HF) \u307e\u305f\u306e\u540d\u3092truncated-Newton, \u2026","blog_url":"https://nishiohirokazu.hatenadiary.org/","author_name":"nishiohirokazu","image_url":null,"provider_url":"https://hatena.blog","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fnishiohirokazu.hatenadiary.org%2Fentry%2F20140208%2F1391825340\" title=\"Deep Learning\u8ad6\u6587\u7d39\u4ecb\u300cLearning Recurrent Neural Networks with Hessian-Free Optimization\u300d - \u897f\u5c3e\u6cf0\u548c\u306e\u306f\u3066\u306a\u30c0\u30a4\u30a2\u30ea\u30fc\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","blog_title":"\u897f\u5c3e\u6cf0\u548c\u306e\u306f\u3066\u306a\u30c0\u30a4\u30a2\u30ea\u30fc","provider_name":"Hatena Blog","version":"1.0","width":"100%","published":"2014-02-08 11:09:00","height":"190","title":"Deep Learning\u8ad6\u6587\u7d39\u4ecb\u300cLearning Recurrent Neural Networks with Hessian-Free Optimization\u300d","type":"rich","author_url":"https://blog.hatena.ne.jp/nishiohirokazu/"}