{"author_name":"derwind","categories":["machine_learning"],"version":"1.0","blog_title":"\u3089\u3093\u3060\u3080\u306a\u8a18\u61b6","url":"https://randommemory.hatenablog.com/entry/2022/03/03/002243","blog_url":"https://randommemory.hatenablog.com/","type":"rich","provider_url":"https://hatena.blog","published":"2022-03-03 00:22:43","height":"190","image_url":null,"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Frandommemory.hatenablog.com%2Fentry%2F2022%2F03%2F03%2F002243\" title=\"Attention (1) - \u3089\u3093\u3060\u3080\u306a\u8a18\u61b6\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","width":"100%","title":"Attention (1)","author_url":"https://blog.hatena.ne.jp/derwind/","provider_name":"Hatena Blog","description":"\u3061\u3087\u3063\u3068\u6c17\u6301\u3061\u3092\u5165\u308c\u66ff\u3048\u3066\u3001\u30bc\u30ed\u3064\u304f 2 (17) - \u3089\u3093\u3060\u3080\u306a\u8a18\u61b6 \u4ee5\u6765 100 \u65e5\u307b\u3069\u5bdd\u304b\u305b\u305f\u30bc\u30ed\u3064\u304f 2 \u3092\u984c\u6750\u306b attention \u306b\u3064\u3044\u3066\u6574\u7406\u3092\u3057\u3066\u304a\u304d\u305f\u3044\u3002\u3056\u3063\u3068 7 \u7ae0\u300cRNN \u306b\u3088\u308b\u6587\u7ae0\u751f\u6210\u300d\u304b\u3089\u304a\u3055\u3089\u3044\u3057\u305f\u3044\u3002[Embedding]->[LSTM]->[Affine]->[Softmax] \u3068\u3044\u3046\u30ec\u30a4\u30e4\u304c\u91cd\u306a\u3063\u3066\u3053\u308c\u304c\u6642\u7cfb\u5217\u30c7\u30fc\u30bf\u3067\u547c\u3070\u308c\u308b\u306e\u3067\u3042\u3063\u305f\u3002\u753b\u50cf\u5206\u985e\u30e2\u30c7\u30eb\u3068\u306e\u6bd4\u8f03\u3067\u5f37\u5f15\u306b\u66f8\u304f\u3068 Embedding \u2194\ufe0e \u30b9\u30d1\u30fc\u30b9\u904e\u304e\u306a\u3044\u30d9\u30af\u30c8\u30eb\u5316 \u2194\ufe0e \u753b\u50cf\u306e $[0, 255]^N$ \u3067\u306e\u89e3\u91c8 LSTM \u2194\ufe0e \u7279\u5fb4\u91cf\u306e\u62bd\u51fa \u2194\ufe0e \u7573\u307f\u8fbc\u307f\u5c64 Affine \u2194\ufe0e \u5206\u985e\u5668 \u2194\ufe0e \u2026"}