{"blog_url":"https://klmquasi.hatenablog.com/","provider_url":"https://hatena.blog","description":"\u305d\u3082\u305d\u3082AutoTokenizer.from_pretrained(trust_remote_code=True) \u306e\u3088\u3046\u306a\u6307\u5b9a\u306a\u3057\u306b\u65e5\u672c\u8a9e\u306e\u4e3b\u8981\u306a\u5f62\u614b\u7d20\u89e3\u6790\u5668\u3092\u30c8\u30fc\u30af\u30ca\u30a4\u30b6\u3068\u3057\u3066\u4f7f\u3048\u308b\u306a\u3089\u3053\u3093\u306a\u82e6\u52b4\u3082\u7121\u3044\u308f\u3051\u3067\u3001\u3001\u3001BertJapaneseTokenizer\u3067sudachipy\u3084jumanpp\u3082\u6271\u3048\u308b\u3088\u3046\u62e1\u5f35\u3059\u308b\u30d7\u30eb\u30ea\u30af\u3082\u9032\u884c\u4e2d\u3067\u3059\uff01https://t.co/3KGPYimFCA\u2014 hiroshi matsuda (@hmtd223) 2022\u5e749\u670828\u65e5","title":"\u304d\u3087\u3046\u306e\u3088\u3057\u306a\u3057\u3054\u3068","author_name":"klmquasi","author_url":"https://blog.hatena.ne.jp/klmquasi/","categories":["twitter"],"width":"100%","version":"1.0","blog_title":"Quasi - Log","provider_name":"Hatena Blog","published":"2022-09-28 00:00:00","height":"190","url":"https://klmquasi.hatenablog.com/entry/2022/09/28/000000","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fklmquasi.hatenablog.com%2Fentry%2F2022%2F09%2F28%2F000000\" title=\"\u304d\u3087\u3046\u306e\u3088\u3057\u306a\u3057\u3054\u3068 - Quasi - Log\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","type":"rich","image_url":null}