{"type":"rich","version":"1.0","published":"2025-09-24 13:00:00","title":"\u69d8\u3005\u306a\u30b3\u30f3\u30c6\u30ad\u30b9\u30c8\u9577\u306b\u304a\u3051\u308b LLM \u306e Self-Attention \u306e Query \u3068 Key \u306e\u5206\u6790","height":"190","blog_title":"ABEJA Tech Blog","provider_url":"https://hatena.blog","width":"100%","url":"https://tech-blog.abeja.asia/entry/longcontext-llm-massive-values-202509","categories":["LLM"],"provider_name":"Hatena Blog","author_name":"flatton","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Ftech-blog.abeja.asia%2Fentry%2Flongcontext-llm-massive-values-202509\" title=\"\u69d8\u3005\u306a\u30b3\u30f3\u30c6\u30ad\u30b9\u30c8\u9577\u306b\u304a\u3051\u308b LLM \u306e Self-Attention \u306e Query \u3068 Key \u306e\u5206\u6790 - ABEJA Tech Blog\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","author_url":"https://blog.hatena.ne.jp/flatton/","description":"ABEJA\u3067\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u3092\u3057\u3066\u3044\u308b\u85e4\u539f\u3067\u3059\u3002 \u4eca\u56de\u306f LLM \u306e\u30ed\u30f3\u30b0\u30b3\u30f3\u30c6\u30ad\u30b9\u30c8\u8a00\u8a9e\u51e6\u7406\uff08Long-context language modeling; LCLM\uff09\u306b\u95a2\u9023\u3059\u308b\u30d6\u30ed\u30b0\u306b\u306a\u308a\u307e\u3059\u3002\u8fd1\u5e74\u306e LLM \u3067\u306f\u30aa\u30fc\u30d7\u30f3\u30fb\u30af\u30ed\u30fc\u30ba\u30c9\u554f\u308f\u305a\u3088\u308a\u9577\u5927\u306a\u30b3\u30f3\u30c6\u30ad\u30b9\u30c8\u3092\u6b63\u78ba\u306b\u6271\u3048\u308b\u30e2\u30c7\u30eb\u304c\u5897\u3048\u3066\u304d\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u3088\u3046\u306b\u30ed\u30f3\u30b0\u30b3\u30f3\u30c6\u30ad\u30b9\u30c8LLM\u304c\u5897\u3048\u3066\u304f\u308b\u3068\u3001\u3069\u306e\u3088\u3046\u306b\u30e2\u30c7\u30eb\u306eLCLM\u6027\u80fd\u3092\u8a55\u4fa1\u3059\u3079\u304d\u304b\u304c\u8ab2\u984c\u3068\u306a\u308a\u307e\u3059\u3002 \u3057\u304b\u3057\u306a\u304c\u3089\u3001\u65e5\u672c\u8a9e\u306e LCLM \u8a55\u4fa1\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3084\u30d9\u30f3\u30c1\u30de\u30fc\u30af\u306f\u73fe\u72b6\u306a\u304f\u3001\u81ea\u4f5c\u3059\u308b\u306b\u3057\u3066\u3082 128k \u30c8\u30fc\u30af\u30f3\u3082\u3042\u308b\u6587\u7ae0\u3092\u4eba\u9593\u304c\u8aad\u3093\u3067\u54c1\u8cea\u3092\u62c5\u4fdd\u3059\u308b\u306e\u306f\u304b\u306a\u308a\u96e3\u3057\u3044\u554f\u984c\u3060\u2026","blog_url":"https://tech-blog.abeja.asia/","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/f/flatton/20250911/20250911211315.png"}