{"provider_name":"Hatena Blog","title":"\u30ab\u30eb\u30d0\u30c3\u30af\u30fb\u30e9\u30a4\u30d6\u30e9\u30fc\u60c5\u5831\u91cf","provider_url":"https://hatena.blog","width":"100%","image_url":null,"blog_title":"\u3089\u3093\u3060\u3080\u306a\u8a18\u61b6","description":"Jensen\u306e\u4e0d\u7b49\u5f0f - \u3089\u3093\u3060\u3080\u306a\u8a18\u61b6\u3067\u89e6\u308c\u305f\u30ab\u30eb\u30d0\u30c3\u30af\u30fb\u30e9\u30a4\u30d6\u30e9\u30fc\u60c5\u5831\u91cf\u306b\u3064\u3044\u3066\u7279\u5b9a\u306e\u78ba\u7387\u5206\u5e03\u306e\u6642\u306b\u8a08\u7b97\u3057\u3066\u307f\u305f\u3044\u3002PRML \u306e 1.6. Relative entropy and mutual information \u306b\u3066 relative entropy \u6216\u3044\u306f Kullback-Leibler divergence \u3068\u3044\u3046\u540d\u3067\u77e5\u3089\u308c\u308b\u3082\u306e\u3068\u3057\u3066\\begin{align} \\mathrm{KL}(p\\|q) &= - \\int p(X) \\ln q(x) dx - \\left(- \\int p(x) \\ln p(x) dx\\right) \\\\ &= - \\int p(x) \\ln \\l\u2026","url":"https://randommemory.hatenablog.com/entry/2020/06/28/231318","type":"rich","published":"2020-06-28 23:13:18","author_name":"derwind","author_url":"https://blog.hatena.ne.jp/derwind/","version":"1.0","blog_url":"https://randommemory.hatenablog.com/","categories":["math-other","statistics","machine_learning"],"html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Frandommemory.hatenablog.com%2Fentry%2F2020%2F06%2F28%2F231318\" title=\"\u30ab\u30eb\u30d0\u30c3\u30af\u30fb\u30e9\u30a4\u30d6\u30e9\u30fc\u60c5\u5831\u91cf - \u3089\u3093\u3060\u3080\u306a\u8a18\u61b6\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","height":"190"}