{"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fcrumbjp.hateblo.jp%2Fentry%2F20130802%2F1375417544\" title=\"\u719f\u8a9e\u89e3\u6790\u304c\u96e3\u3057\u3044\u3002\u3002\u3002\uff08\u3082\u306f\u3084\u8d70\u308a\u66f8\u304d\u30e1\u30e2\uff09 - \u4e2d\u5e74engineer\u306e\u72ec\u308a\u8a00 - crumbjp\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","url":"https://crumbjp.hateblo.jp/entry/20130802/1375417544","width":"100%","categories":[],"author_url":"https://blog.hatena.ne.jp/hiroppon/","height":"190","provider_name":"Hatena Blog","published":"2013-08-02 13:25:44","image_url":null,"provider_url":"https://hatena.blog","type":"rich","title":"\u719f\u8a9e\u89e3\u6790\u304c\u96e3\u3057\u3044\u3002\u3002\u3002\uff08\u3082\u306f\u3084\u8d70\u308a\u66f8\u304d\u30e1\u30e2\uff09","author_name":"hiroppon","blog_title":"\u4e2d\u5e74engineer\u306e\u72ec\u308a\u8a00 - crumbjp","description":"\u4fee\u6b63\uff1a x: unigram => o: bigram \u52e2\u3044\u3067\u66f8\u304f\u3068\u3053\u3046\u3060\u3088\u30fb\u30fb\u30fb \u5f62\u614b\u7d20\u89e3\u6790\u3067Tokenize\u5f8c\u306bN-gram\u3067\u719f\u8a9e\u89e3\u6790\u3057\u3088\u3046\u3068\u3057\u305f\u304c\u3001\u3061\u3068\u56f0\u3063\u305f\u3002\u3002 \u65e5\u672c \u7dad\u65b0 \u306e \u4f1a\u3053\u308c\u3092bigram\u3059\u308b\u3068 \u65e5\u672c\u7dad\u65b0 \u7dad\u65b0\u306e \u306e\u4f1a\u3068\u306a\u3063\u3066\u3001\u3053\u308c\u3092\u4f7f\u3063\u3066\u3082\u3046\u4e00\u56deTokenize\u3059\u308b\u3068 \u65e5\u672c\u7dad\u65b0 \u306e\u4f1a\u66f4\u306bbigram\u3057\u3066\u3084\u3063\u3068\u5b8c\u6210\u3059\u308b\u3002 \u65e5\u672c\u7dad\u65b0\u306e\u4f1a\u3053\u3053\u304b\u3089\u3001\u3082\u3046\u4e00\u56deTokenize\u3059\u308b\u3068\u671b\u3080\u7d50\u679c\u306b\u306a\u308b\u304c\u3001\u30013\u56de\u3082Tokenize\u3057\u305f\u3089\u65e5\u304c\u66ae\u308c\u308b\u3058\u3083\u6e08\u307e\u306a\u3044\u3088\u30fb\u30fb\u30fb \u8ffd\u8a18 \u3068\u308a\u3042\u3048\u305a\u3001\u4e00\u6c17\u306b4-gram, 3-gram , 2-gram \u3068\u304b\u3051\u3066\u307f\u308b\u4e8b\u306b\u3057\u305f\u3002 \u8f9e\u66f8\u306b\u7121\u99c4\u306a\u767b\u9332\u3092\u3057\u306a\u3044\u69d8\u306b\u3001\u2026","version":"1.0","blog_url":"https://crumbjp.hateblo.jp/"}