{"image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/T/TadaoYamaoka/20240605/20240605215041.png","published":"2024-06-05 22:16:40","blog_title":"TadaoYamaoka\u306e\u958b\u767a\u65e5\u8a18","blog_url":"https://tadaoyamaoka.hatenablog.com/","author_url":"https://blog.hatena.ne.jp/TadaoYamaoka/","description":"\u524d\u56de\u307e\u3067\u306f\u3001\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0\u306b\u5b66\u7fd2\u53ef\u80fd\u306a\u7d76\u5bfe\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0\u3092\u4f7f\u7528\u3057\u3066\u3044\u305f\u3002 \u4eca\u56de\u306f\u3001\u76f8\u5bfe\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0\u3092\u8a66\u3059\u3002 \u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0 Transformer\u306f\u3001\u30c8\u30fc\u30af\u30f3\u304c\u5165\u529b\u306e\u4f55\u756a\u76ee\u306b\u3042\u308b\u304b\u306b\u3088\u3089\u305a\u7b49\u4fa1\u306b\u6271\u3046\u3002 \u305d\u306e\u305f\u3081\u3001\u5165\u529b\u306e\u4f4d\u7f6e\u304c\u610f\u5473\u3092\u6301\u3064\u5834\u5408\u306f\u3001\u4f4d\u7f6e\u3092\u4f55\u3089\u304b\u306e\u5f62\u3067\u30a8\u30f3\u30b3\u30fc\u30c9\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b\u3002 \u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u3067\u306f\u3001\u6587\u7ae0\u3092\u30c8\u30fc\u30af\u30f3\u5217\u306b\u3057\u305f\u5834\u5408\u3001\u4f55\u756a\u76ee\u306e\u30c8\u30fc\u30af\u30f3\u304b\u3092Sin\u3068Cos\u3067\u30a8\u30f3\u30b3\u30fc\u30c9\u3059\u308b\u3053\u3068\u304c\u884c\u308f\u308c\u308b\u3002Llama2\u3067\u306f\u3001rotary positional embeddings(RoPE)\u3068\u3044\u3046\u3001\u30c8\u30fc\u30af\u30f3\u306e\u76f8\u5bfe\u4f4d\u7f6e\u3092\u30a8\u30f3\u30b3\u30fc\u30c9\u3059\u308b\u624b\u6cd5\u304c\u4f7f\u308f\u308c\u3066\u3044\u308b\u3002 \u5c06\u68cb\u306e\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0 \u5c06\u68cb\u306f\u3001\u76e4\u4e0a\u306e\u7d76\u5bfe\u7684\u306a\u4f4d\u7f6e\u2026","type":"rich","width":"100%","version":"1.0","height":"190","provider_url":"https://hatena.blog","provider_name":"Hatena Blog","author_name":"TadaoYamaoka","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Ftadaoyamaoka.hatenablog.com%2Fentry%2F2024%2F06%2F05%2F221640\" title=\"\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\u3067\u5c06\u68cbAI\u3092\u4f5c\u308b \u305d\u306e\uff16\uff08\u76f8\u5bfe\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0\uff09 - TadaoYamaoka\u306e\u958b\u767a\u65e5\u8a18\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","title":"\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\u3067\u5c06\u68cbAI\u3092\u4f5c\u308b \u305d\u306e\uff16\uff08\u76f8\u5bfe\u4f4d\u7f6e\u30a8\u30f3\u30b3\u30fc\u30c0\uff09","url":"https://tadaoyamaoka.hatenablog.com/entry/2024/06/05/221640","categories":["\u30b3\u30f3\u30d4\u30e5\u30fc\u30bf\u5c06\u68cb","Transformer"]}