{"height":"190","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fsoh335.hatenablog.com%2Fentry%2F2018%2F05%2F02%2F110016\" title=\"athena \u3067\u691c\u7d22\u3059\u308b\u305f\u3081\u306bkinesis firehose \u3067 s3 \u306b\u7f6e\u3044\u305f\u3082\u306e\u3092 glue \u306e crawler \u3067\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3092\u66f4\u65b0\u3059\u308b - soh335 memo\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","provider_name":"Hatena Blog","width":"100%","published":"2018-05-02 11:00:16","blog_title":"soh335 memo","version":"1.0","provider_url":"https://hatena.blog","categories":["tech"],"author_name":"soh335","title":"athena \u3067\u691c\u7d22\u3059\u308b\u305f\u3081\u306bkinesis firehose \u3067 s3 \u306b\u7f6e\u3044\u305f\u3082\u306e\u3092 glue \u306e crawler \u3067\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3092\u66f4\u65b0\u3059\u308b","author_url":"https://blog.hatena.ne.jp/soh335/","url":"https://soh335.hatenablog.com/entry/2018/05/02/110016","description":"kinesis firehose \u3067 s3 \u306b\u7f6e\u304f\u3068 s3://bucket/prefix/$year/$month/$day/$hour/xxx.gz \u306e\u5f62\u5f0f\u306b\u306a\u308b\u3002 glue \u306e crawler \u3092\u4f5c\u308b\u3068\u3001s3 \u306e\u30d1\u30b9\u3092\u6307\u5b9a\u3057\u3066 athena \u306e\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u3092\u4f5c\u3063\u3066\u304f\u308c\u308b\u3002\u4e0a\u306e\u30d1\u30b9\u5f62\u5f0f\u3060\u3068 $year, $month, $day, $hour \u306e 4 \u3064\u3092\u4f7f\u3063\u305f\u3082\u306e\u306b\u306a\u308b\u3002 \u3068\u3053\u308d\u3067\u3001athena \u306e\u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u306f 20,000 \u304c\u30c7\u30d5\u30a9\u30eb\u30c8\u3067\u306f\u4e0a\u9650\u306b\u306a\u3063\u3066\u3044\u308b\u3002\u306a\u306e\u3067\u3053\u308c\u3060\u3068\u3042\u307e\u308a\u4f5c\u308c\u306a\u3044\u3002 \u306a\u306e\u3067\u3001s3 \u306e put object event \u3092\u898b\u308b lambda \u3092\u4f5c\u308a s3://\u2026","image_url":null,"type":"rich","blog_url":"https://soh335.hatenablog.com/"}