{"author_url":"https://blog.hatena.ne.jp/yohei-a/","blog_title":"ablog","title":"Presto \u3067 Parquet \u306b\u30af\u30a8\u30ea\u3059\u308b\u3068\u30d5\u30a1\u30a4\u30eb\u4e2d\u306e\u5fc5\u8981\u306a Column chunk \u306e\u307f\u3092\u8aad\u3093\u3067\u3044\u308b\u304b","author_name":"yohei-a","type":"rich","description":"Presto \u304b\u3089\u898b\u308b\u3068 (parquet\u306e)page\u5358\u4f4d\u3067 IO \u3057\u3066\u3001HDFS \u306e API \u3092\u53e9\u3044\u3066\u3001HDFS \u306f DSDataInputStream \u3068\u304b\u3067\u8aad\u3093\u3067\u3001OS\u30ec\u30a4\u30e4\u30fc\u304b\u3089\u307f\u308b\u3068 sendfile(2) \u3067 xfs \u306a\u3069\u306e\u30d5\u30a1\u30a4\u30eb\u30b7\u30b9\u30c6\u30e0\u306e\u30d5\u30a1\u30a4\u30eb\u3092\u8aad\u3093\u3067\u3068\u3044\u3046\u3053\u3068\u306b\u306a\u3063\u3066\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u63a8\u5bdf\u3002\u2014 yohei.az (@yoheia) 2018\u5e7410\u67089\u65e5 1)flame graph\u3067hdfs\u306eio\u30b7\u30b9\u30c6\u30e0\u30b3\u30fc\u30eb\u3092\u767a\u884c\u5143\u30b3\u30fc\u30eb\u30b9\u30bf\u30c3\u30af\u3092\u7279\u5b9a2)strace\u3067io\u3092\u767a\u884c\u3057\u305f\u30d5\u30a1\u30a4\u30eb\u3068io\u30b5\u30a4\u30ba\u3001\u30b7\u30fc\u30b1\u30f3\u30b7\u30e3\u30ebor\u30e9\u30f3\u30c0\u30e0\u3092\u78ba\u8a8d3)blktrace+btt\u3067io\u7bc4\u56f2(bno)\u3092\u78ba\u8a8d4\u2026","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/y/yohei-a/20181021/20181021053908.jpg","height":"190","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fyohei-a.hatenablog.jp%2Fentry%2F20181020%2F1540064600\" title=\"Presto \u3067 Parquet \u306b\u30af\u30a8\u30ea\u3059\u308b\u3068\u30d5\u30a1\u30a4\u30eb\u4e2d\u306e\u5fc5\u8981\u306a Column chunk \u306e\u307f\u3092\u8aad\u3093\u3067\u3044\u308b\u304b - ablog\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","version":"1.0","width":"100%","provider_name":"Hatena Blog","published":"2018-10-20 04:43:20","provider_url":"https://hatena.blog","categories":["Presto","Parquet"],"blog_url":"https://yohei-a.hatenablog.jp/","url":"https://yohei-a.hatenablog.jp/entry/20181020/1540064600"}