{"type":"rich","author_name":"satojkovic","blog_title":"stMind","blog_url":"https://stmind.hatenablog.com/","description":"2024\u5e74\u306e\u6700\u521d\u306e\u30a8\u30f3\u30c8\u30ea\u30fc\u306fGPT\u3067\u3059\u3002 GPT\u30e2\u30c7\u30eb\u3092\u81ea\u4f5c\u3057\u3066\u3001OpenAI\u304c\u516c\u958b\u3057\u3066\u3044\u308b\u5b66\u7fd2\u6e08\u307f\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u30ed\u30fc\u30c9\u3001\u30c6\u30ad\u30b9\u30c8\u751f\u6210\u307e\u3067\u306e\u4e00\u9023\u306e\u51e6\u7406\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002 \u30e2\u30c7\u30eb \u6b63\u78ba\u306b\u306fGPT2\u306eTransformer\u30d6\u30ed\u30c3\u30af\u3092\u81ea\u4f5c\u3057\u307e\u3059\u3002 \u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u306e\u5927\u90e8\u5206\u306fGPT\u3068\u540c\u3058\u3067\u3059\u304c\u3001\u4ee5\u4e0b\u306e\u5909\u66f4\uff08pre-norm\uff09\u304c\u884c\u308f\u308c\u3066\u3044\u307e\u3059\u3002 LayerNorm\u306fAttention\u3068MLP\u306e\u524d\u3067\u9069\u7528 \u8ffd\u52a0\u306eLayerNorm\u3092Transformer\u30d6\u30ed\u30c3\u30af\u306e\u5f8c\u3067\u9069\u7528 Transformer\u30d6\u30ed\u30c3\u30af\u3092\u9664\u304fText & Position\u57cb\u3081\u8fbc\u307f\u3068Next Token\u751f\u6210\u306f\u3001 picoGPT\u306e\u30b3\u30fc\u30c9\u3092\u5229\u7528\u3057\u307e\u3059\u2026","height":"190","title":"GPT\u3092\u81ea\u4f5c\u3057\u3066\u5b66\u7fd2\u6e08\u307f\u30d1\u30e9\u30e1\u30fc\u30bf\u3067\u30c6\u30ad\u30b9\u30c8\u751f\u6210","provider_url":"https://hatena.blog","width":"100%","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/s/satojkovic/20240108/20240108142513.png","categories":[],"url":"https://stmind.hatenablog.com/entry/2024/01/08/162734","provider_name":"Hatena Blog","author_url":"https://blog.hatena.ne.jp/satojkovic/","published":"2024-01-08 16:27:34","version":"1.0","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fstmind.hatenablog.com%2Fentry%2F2024%2F01%2F08%2F162734\" title=\"GPT\u3092\u81ea\u4f5c\u3057\u3066\u5b66\u7fd2\u6e08\u307f\u30d1\u30e9\u30e1\u30fc\u30bf\u3067\u30c6\u30ad\u30b9\u30c8\u751f\u6210 - stMind\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>"}