{"width":"100%","blog_url":"https://inarizuuuushi.hatenablog.com/","title":"DistributedDataParallel (pytorch) \u306e\u5185\u90e8\u30c7\u30b6\u30a4\u30f3","categories":["pytorch","\u5206\u6563\u6df1\u5c64\u5b66\u7fd2","DistributedDataParallel"," python "],"provider_url":"https://hatena.blog","blog_title":"Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38","author_name":"inarizuuuushi","image_url":"https://user-images.githubusercontent.com/16999635/72401724-d296d880-371a-11ea-90ab-737f86543df9.png","description":"https://pytorch.org/docs/stable/notes/ddp.html#internal-design \u306e\u65e5\u672c\u8a9e\u8a33 + \u304a\u307e\u3051\u306e\u811a\u6ce8 pytorch \u306fv1.12 Internal Design \u3053\u3053\u3067\u306f\u3001torch.nn.parallel.DistributedDataParallel\u304c\u3069\u306e\u3088\u3046\u306b\u52d5\u4f5c\u3057\u3066\u3044\u308b\u304b\u3092\u30011\u3064\u306e\u53cd\u5fa9\u51e6\u7406\u306e\u5404\u30b9\u30c6\u30c3\u30d7\u306e\u8a73\u7d30\u306b\u8e0f\u307f\u8fbc\u3093\u3067\u660e\u3089\u304b\u306b\u3059\u308b\u3002 \u524d\u63d0\u6761\u4ef6 DDP\u306fc10d ProcessGroup\u306b\u4f9d\u5b58\u3057\u3066\u901a\u4fe1\u3092\u884c\u3044\u307e\u3059\u3002\u3057\u305f\u304c\u3063\u3066\u3001\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u306fDDP\u3092\u69cb\u7bc9\u3059\u308b\u524d\u306bProcessGroup\u306e\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3092\u4f5c\u6210\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002 \u69cb\u7bc9 \u2026","type":"rich","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Finarizuuuushi.hatenablog.com%2Fentry%2F2022%2F10%2F17%2F090000\" title=\"DistributedDataParallel (pytorch) \u306e\u5185\u90e8\u30c7\u30b6\u30a4\u30f3 - Sabrou-mal \u30b5\u30d6\u30ed\u30a6\u4e38\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","height":"190","author_url":"https://blog.hatena.ne.jp/inarizuuuushi/","url":"https://inarizuuuushi.hatenablog.com/entry/2022/10/17/090000","provider_name":"Hatena Blog","version":"1.0","published":"2022-10-17 09:00:00"}