{"version":"1.0","blog_title":"Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38","url":"https://inarizuuuushi.hatenablog.com/entry/2022/09/19/090000","author_url":"https://blog.hatena.ne.jp/inarizuuuushi/","provider_url":"https://hatena.blog","type":"rich","blog_url":"https://inarizuuuushi.hatenablog.com/","description":"NVIDIA\u304c\u63d0\u6848\u3059\u308bTransformer\u3092\u30d9\u30fc\u30b9\u3068\u3059\u308b\u8a00\u8a9e\u51e6\u7406\u30e2\u30c7\u30eb\u306e\u4e26\u5217\u5316\u5b9f\u88c5\u3002\u30b5\u30fc\u30d9\u30a4\u8a18\u4e8b\u306f\u3053\u3061\u3089\u2193\u2193\u2193 Github\u306e\u30ec\u30dd\u30b8\u30c8\u30ea\u306b\u306f Data Preprocessing \uff08\u30c7\u30fc\u30bf\u524d\u51e6\u7406\uff09 Pretraining\uff08\u4e8b\u524d\u5b66\u7fd2\uff09 Evaluation and Tasks\uff08\u8a55\u4fa1\uff09 \u306e\u30b3\u30fc\u30c9\u304c\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002 \u4e8b\u524d\u5b66\u7fd2\u306b\u3064\u3044\u3066\u306f BERT GPT T5 \u306e3\u3064\u306e\u30e2\u30c7\u30eb\u306b\u3064\u3044\u3066\u3001\u5206\u6563\u7248\u3068\u305d\u3046\u3067\u306a\u3044\u3082\u306e\u304c\u305d\u308c\u305e\u308c\u7528\u610f\u3055\u308c\u3066\u3044\u307e\u3059\u306d\u3002 \u8ad6\u6587\u3092\u8aad\u3080\u3068\u5206\u304b\u308a\u3084\u3059\u3044\u3067\u3059\u304c\u3001Megatron-LM\u3067\u63d0\u6848\u3057\u3066\u3044\u308b\u306e\u306f\u30e2\u30c7\u30eb\u306e\u5782\u76f4\u7684\u306a\u5206\u5272\u306b\u3088\u308b\u30e2\u30c7\u30eb\u4e26\u5217\u3068Attention\u90e8\u5206\u306e\u6c34\u5e73\u5206\u5272\u306b\u3088\u308b\u4e26\u5217\u51e6\u7406\u306e\u5408\u308f\u305b\u2026","image_url":null,"height":"190","width":"100%","provider_name":"Hatena Blog","published":"2022-09-19 09:00:00","categories":["\u5206\u6563\u6df1\u5c64\u5b66\u7fd2"],"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Finarizuuuushi.hatenablog.com%2Fentry%2F2022%2F09%2F19%2F090000\" title=\"Megatron-LM\u306e\u30bd\u30fc\u30b9\u30b3\u30fc\u30c9\u3092\u8aad\u3080 - Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","author_name":"inarizuuuushi","title":"Megatron-LM\u306e\u30bd\u30fc\u30b9\u30b3\u30fc\u30c9\u3092\u8aad\u3080"}