{"version":"1.0","provider_url":"https://hatena.blog","title":"dropless MoE(Mixture of Experts)\u3092\u8a66\u3059 \u305d\u306e2(ONNX\u30a8\u30af\u30b9\u30dd\u30fc\u30c8)","blog_title":"TadaoYamaoka\u306e\u958b\u767a\u65e5\u8a18","url":"https://tadaoyamaoka.hatenablog.com/entry/2026/05/24/225641","published":"2026-05-24 22:56:41","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/T/TadaoYamaoka/20260524/20260524224728.png","author_url":"https://blog.hatena.ne.jp/TadaoYamaoka/","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Ftadaoyamaoka.hatenablog.com%2Fentry%2F2026%2F05%2F24%2F225641\" title=\"dropless MoE(Mixture of Experts)\u3092\u8a66\u3059 \u305d\u306e2(ONNX\u30a8\u30af\u30b9\u30dd\u30fc\u30c8) - TadaoYamaoka\u306e\u958b\u767a\u65e5\u8a18\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","width":"100%","description":"\u524d\u56de\u3001PyTorch\u3067\u5b9f\u88c5\u3057\u305fMoE\u5bfe\u5fdc\u306eSwinTransformer\u30e2\u30c7\u30eb\u3092TensorRT\u3067\u63a8\u8ad6\u3067\u304d\u308b\u3088\u3046\u306b\u3001ONNX\u306b\u30a8\u30af\u30b9\u30dd\u30fc\u30c8\u3059\u308b\u3002\u6700\u4f4e\u9650\u3001grouped_mm\u306e\u307f\u3092\u30ab\u30b9\u30bf\u30e0\u30aa\u30da\u30ec\u30fc\u30bf\u3068\u3059\u308c\u3070\u3088\u3044\u304c\u3001router + dispatch + expert + combine\u3092\u5206\u96e2\u3059\u308b\u3068TensorRT\u306e\u30b0\u30e9\u30d5\u3068plugin\u306e\u5165\u51fa\u529b\u304c\u8907\u96d1\u306b\u306a\u308b\u305f\u3081\u3001MoE\u5316\u3057\u305fMLP\u3092\u307e\u3068\u3081\u3066\u30d7\u30e9\u30b0\u30a4\u30f3\u5316\u3059\u308b\u3002 \u30bd\u30fc\u30b9\u3067\u306f\u3001DroplessMoEMlp\u3092\u30ab\u30b9\u30bf\u30e0\u30ce\u30fc\u30c9\u306b\u3059\u308b\u3002 \u30ab\u30b9\u30bf\u30e0\u30ce\u30fc\u30c9 \u4ee5\u4e0b\u306e\u3088\u3046\u306aforward\u304c\u7a7a\u306e\u30ce\u30fc\u30c9\u3092\u5b9a\u7fa9\u3057\u3066\u3001symbolic\u306bONNX\u306e\u30aa\u30da\u30ec\u30fc\u30bf\u306e\u30e1\u30bf\u60c5\u5831\u3092\u5b9a\u7fa9\u3059\u308b\u3002\u2026","type":"rich","height":"190","author_name":"TadaoYamaoka","blog_url":"https://tadaoyamaoka.hatenablog.com/","categories":["MoE"],"provider_name":"Hatena Blog"}