{"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fhtn20190109.hatenablog.com%2Fentry%2F2026%2F05%2F03%2F125742\" title=\"\u6700\u9069\u5316\u624b\u6cd5\u306e\u66f4\u65b0\u5f0f - HTN20190109\u306e\u65e5\u8a18\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","published":"2026-05-03 12:57:42","author_url":"https://blog.hatena.ne.jp/HTN20190109/","blog_url":"https://htn20190109.hatenablog.com/","author_name":"HTN20190109","image_url":null,"provider_url":"https://hatena.blog","provider_name":"Hatena Blog","description":"\u30fb\u78ba\u7387\u7684\u52fe\u914d\u964d\u4e0b\u6cd5 SGD \u30fb\u30e2\u30fc\u30e1\u30f3\u30bf\u30e0 Momentum \u30fb\u30cd\u30b9\u30c6\u30ed\u30d5 Nesterov \u5b9f\u88c5\u7528\u306e\u66f4\u65b0\u5f0f \u3068\u304a\u304f\u3068 \u3068\u306a\u308b\u3002 \u307e\u305f\u3001 \u306e\u4e21\u8fba\u306b \u3092\u8db3\u3059\u3002 \u4e0b\u8a18\u306f\u6bd4\u8f03\u7528 \u306eMomentum Momuntum\u3068\u6bd4\u3079\u3066\u7b2c2\u9805\u306f\u03b1\u500d\u3001\u7b2c3\u9805\u306b(1+\u03b1)\u500d \u3055\u3089\u306b\u3001 \u3092\u4ee3\u5165\u3059\u308b\u3068\u5b9f\u88c5\u3067\u3088\u304f\u4f7f\u308f\u308c\u308b\u6a19\u6e96\u5f62\u3068\u306a\u308b \u30fbAdaGrad \u52fe\u914d\u306e\u4e8c\u4e57\u30fbRMSProp \u52fe\u914d\u306e\u4e8c\u4e57\u306e\u79fb\u52d5\u5e73\u5747 \u30fbAdaDelta \u52fe\u914d\u306e\u4e8c\u4e57\u306e\u79fb\u52d5\u5e73\u5747\u3068\u66f4\u65b0\u91cf\u306e\u4e8c\u4e57\u306e\u79fb\u52d5\u5e73\u5747 \u30fbAdam \u30fbAdamW Adam\u3068\u306f\u7570\u306a\u308a\u3001L2\u6b63\u5247\u5316\u3092\u52fe\u914d\u306b\u6df7\u305c\u306a\u3044\u3067\u8a08\u7b97\u3059\u308b\u3002L2\u6b63\u5247\u5316\u306f\u6700\u5f8c\u306b\u52a0\u7b97\u3059\u308b","width":"100%","type":"rich","categories":["DL"],"version":"1.0","url":"https://htn20190109.hatenablog.com/entry/2026/05/03/125742","title":"\u6700\u9069\u5316\u624b\u6cd5\u306e\u66f4\u65b0\u5f0f","height":"190","blog_title":"HTN20190109\u306e\u65e5\u8a18"}