{"version":"1.0","height":"190","width":"100%","type":"rich","author_name":"inarizuuuushi","provider_url":"https://hatena.blog","provider_name":"Hatena Blog","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Finarizuuuushi.hatenablog.com%2Fentry%2F2022%2F06%2F20%2F090000\" title=\"Transformer\u306b\u3088\u308b\u7ffb\u8a33\u30b7\u30b9\u30c6\u30e0\u81ea\u4f5c; part3 Multi-head Attention - Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","url":"https://inarizuuuushi.hatenablog.com/entry/2022/06/20/090000","categories":["Transformer\u81ea\u4f5c"],"title":"Transformer\u306b\u3088\u308b\u7ffb\u8a33\u30b7\u30b9\u30c6\u30e0\u81ea\u4f5c; part3 Multi-head Attention","published":"2022-06-20 09:00:00","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/i/inarizuuuushi/20220612/20220612105107.png","author_url":"https://blog.hatena.ne.jp/inarizuuuushi/","blog_title":"Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38","blog_url":"https://inarizuuuushi.hatenablog.com/","description":"\u672c\u7a3f\u3067\u306fMulti-head Attention\u306b\u3064\u3044\u3066\u5b9f\u88c5\u3092\u884c\u3044\u307e\u3059\u3002 Multi-head Attention Q, K, V\u3092\u5206\u5272\u3057\u3066\u305d\u308c\u305e\u308c\u3067scaled dot-product attention\u3092\u5b9f\u884c\u3001\u7d50\u679c\u3092\u96c6\u7d04(concat) \u3053\u3061\u3089\u306e\u65b9\u304c\u7cbe\u5ea6\u304c\u826f\u3044(\u7406\u7531\u306f\u5206\u304b\u3089\u3093\u3051\u3069\u3082; \u8907\u6570\u306e\u6587\u8108\u3092\u53d6\u308a\u51fa\u305b\u308b\u3068\u3044\u3046\u52b9\u679c\u3082) \u305d\u308c\u305e\u308c\u306eattention\u3092\u4e26\u5217\u5b9f\u884c\u3067\u304d\u308b\u3068\u3044\u3046\u5229\u70b9 \u30b3\u30fc\u30c9 attention mask attention\u306e\u306b\u5bfe\u3059\u308b\u7279\u5fb4\u91cf\u7b97\u51fa\u5f0f\u3092\u601d\u3044\u51fa\u305d\u3046\u3002(\u53c2\u8003: Attention - Speaker Deck ) mask\u751f\u6210\u90e8\u5206\u306e\u30b3\u30fc\u30c9 transformer\u5185\u306eforw\u2026"}