{"description":"KoichiYasuoka\u3055\u3093\u304b\u3089Hatena Blog\u306b\u30b3\u30e1\u30f3\u30c8\u3092\u9802\u304d\u307e\u3057\u305f\u3002tokenizer\u306b\u5de5\u592b\u3092\u3057\u3066\u306f\uff1f\u3068\u3044\u3046\u3082\u306e\u3067\u3057\u305f\u3002\u300c\u65e5\u672c\u306e\u9996\u76f8\u300d\u304c\u300c\u65e5\u672c\u306e \u9996\u76f8\u300d\u306b\u5206\u304b\u3061\u66f8\u304d\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u3053\u308c\u3067\u306f\u300c\u65e5\u672c\u300d\u3092\u53d6\u308a\u51fa\u305b\u306a\u3044\u3002\u3064\u307e\u308a\u3001 \u4eca\u307e\u3067\uff1a\u65e5\u672c\u306e \u9996\u76f8 \u6539\u5584 \uff1a\u65e5\u672c \u306e \u9996\u76f8 \u3068\u3057\u305f\u3044\u3002\u7d0d\u5f97\u3067\u3059\u3002Yasuoka\u3055\u3093\u3001\u3042\u308a\u304c\u3068\u3046\u3054\u3056\u3044\u307e\u3059\u3002 \u30b5\u30f3\u30d7\u30eb\u30b3\u30fc\u30c9\u3082\u4ed8\u3051\u3066\u3044\u305f\u3060\u3044\u305f\u306e\u3067\u3001\u3055\u3063\u305d\u304f\u8a66\u3057\u3066\u307f\u307e\u3057\u305f\u3002F1-score\u306f0.929362\u3068\u3001\u524d\u56de\u306e0.870399\u3088\u308a\u9ad8\u304f\u306a\u3063\u3066\u3044\u307e\u3059\u3002\u671f\u5f85\u901a\u308a\u306e\u7d50\u679c\u3068\u306a\u308a\u307e\u3057\u305f\u3002\u3053\u306etokenizer\u306e\u4ed5\u69d8\u306f\u3082\u3063\u305f\u3044\u306a\u3044\u3067\u3059\u306d\u3002\u4eca\u56de\u3001\u5229\u7528\u3057\u305fTokenizer\u6539\u3060\u2026","published":"2025-02-20 22:03:23","height":"190","version":"1.0","image_url":null,"provider_url":"https://hatena.blog","title":"Tokenizer\u3092\u898b\u76f4\u3057\u3066modernBERT\u3067NER\u3092Colab\u3067\u305f\u3081\u3057\u3066\u307f\u308b\u3002","width":"100%","type":"rich","author_url":"https://blog.hatena.ne.jp/bwgift/","blog_url":"https://bwgift.hatenadiary.jp/","provider_name":"Hatena Blog","author_name":"bwgift","blog_title":"\u5730\u5e73\u7dda\u307e\u3067\u884c\u3063\u3066\u304f\u308b\u3002","url":"https://bwgift.hatenadiary.jp/entry/2025/02/20/220323","categories":["LLM","Colaboratory","BERT","Python","\u81ea\u7136\u8a00\u8a9e\u51e6\u7406"],"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fbwgift.hatenadiary.jp%2Fentry%2F2025%2F02%2F20%2F220323\" title=\"Tokenizer\u3092\u898b\u76f4\u3057\u3066modernBERT\u3067NER\u3092Colab\u3067\u305f\u3081\u3057\u3066\u307f\u308b\u3002 - \u5730\u5e73\u7dda\u307e\u3067\u884c\u3063\u3066\u304f\u308b\u3002\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>"}