{"author_url":"https://blog.hatena.ne.jp/Ninhydrin/","url":"https://ninhydrin.hatenablog.com/entry/2021/09/16/101038","published":"2021-09-16 10:10:38","height":"190","type":"rich","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/N/Ninhydrin/20210915/20210915094335.png","blog_title":"\u306b\u3093\u3072\u3069\u308a\u3093\u306f\u3093\u306e\u3046","provider_name":"Hatena Blog","description":"arxiv.org \u3042\u304f\u307e\u3067\u500b\u4eba\u7684\u30e1\u30e2\u30ec\u30d9\u30eb \u6982\u8981 ViT\u306e\u3088\u3046\u306b\u8fd1\u5e74\u3067\u306fTransformer\u306eCV\u5206\u91ce\u3078\u306e\u5fdc\u7528\u304c\u6d3b\u767a\u3060\u304c\u3001\u753b\u50cf\u8a8d\u8b58\u306b\u304a\u3044\u3066self-attention\u304c\u9ad8\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u5f97\u308b\u305f\u3081\u306e\u9375\u306a\u306e\u304b\u8abf\u67fb\u3057\u3001\u6700\u8fd1\u307e\u305f\u518d\u71c3\u3057\u3066\u3044\u308bMLP\u30e2\u30c7\u30eb\u3092\u6539\u826f\u3057\u305fattention-free\u306asMLPNet\u3092\u63d0\u6848\u3002 \u5c0e\u5165 ViT\u3084DeiT\u306a\u3069\u3001\u9069\u5207\u306a\u4e8b\u524d\u5b66\u7fd2\u3092\u3057\u305ftransformer\u7cfb\u306e\u30d3\u30b8\u30e7\u30f3\u30e2\u30c7\u30eb\u306f\u753b\u50cf\u8a8d\u8b58\u3067SOTA\u3092\u9054\u6210\u3057\u305f\u3002 DeiT\u306a\u3069\u306e\u7573\u307f\u8fbc\u307f\u306e\u306a\u3044ViT\u306f2\u3064\u306e\u56fa\u5b9a\u89b3\u5ff5\u7684\u306a\u3082\u306e\u3092\u751f\u307f\u51fa\u3057\u305f 1) \u30b0\u30ed\u30fc\u30d0\u30eb\u306a\u4f9d\u5b58\u95a2\u4fc2\u306e\u30e2\u30c7\u30ea\u30f3\u30b0\u304c\u91cd\u8981\u3067\u3001\u7573\u307f\u8fbc\u307f\u306e\u3088\u3046\u306a\u30ed\u30fc\u30ab\u30eb\u306a\u4f9d\u5b58\u95a2\u4fc2\u3092\u7f6e\u304d\u63db\u3048\u3089\u2026","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fninhydrin.hatenablog.com%2Fentry%2F2021%2F09%2F16%2F101038\" title=\" [\u8ad6\u6587\u30e1\u30e2] Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? - \u306b\u3093\u3072\u3069\u308a\u3093\u306f\u3093\u306e\u3046\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","provider_url":"https://hatena.blog","blog_url":"https://ninhydrin.hatenablog.com/","version":"1.0","categories":["Attention","ViT","\u8ad6\u6587\u8aad\u307f","MLP"],"width":"100%","title":" [\u8ad6\u6587\u30e1\u30e2] Sparse MLP for Image Recognition: Is Self-Attention Really Necessary?","author_name":"Ninhydrin"}