{"provider_url":"https://hatena.blog","provider_name":"Hatena Blog","url":"https://yohei-a.hatenablog.jp/entry/20201217/1608210420","version":"1.0","title":"AWS Glue \u306e Spark \u30b8\u30e7\u30d6\u3067 spark.conf.set() \u3059\u308b\u65b9\u6cd5","author_name":"yohei-a","blog_url":"https://yohei-a.hatenablog.jp/","image_url":"https://cdn-ak.f.st-hatena.com/images/fotolife/y/yohei-a/20201217/20201217220104.png","height":"190","author_url":"https://blog.hatena.ne.jp/yohei-a/","categories":["AWS"],"blog_title":"ablog","published":"2020-12-17 22:07:00","width":"100%","html":"<iframe src=\"https://hatenablog-parts.com/embed?url=https%3A%2F%2Fyohei-a.hatenablog.jp%2Fentry%2F20201217%2F1608210420\" title=\"AWS Glue \u306e Spark \u30b8\u30e7\u30d6\u3067 spark.conf.set() \u3059\u308b\u65b9\u6cd5 - ablog\" class=\"embed-card embed-blogcard\" scrolling=\"no\" frameborder=\"0\" style=\"display: block; width: 100%; height: 190px; max-width: 500px; margin: 10px 0px;\"></iframe>","type":"rich","description":"[\u30b8\u30e7\u30d6\u306e\u7de8\u96c6]-[\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u8a2d\u5b9a\u3001\u30b9\u30af\u30ea\u30d7\u30c8\u30e9\u30a4\u30d6\u30e9\u30ea\u304a\u3088\u3073\u30b8\u30e7\u30d6\u30d1\u30e9\u30e1\u30fc\u30bf (\u4efb\u610f)]-[\u30b8\u30e7\u30d6\u30d1\u30e9\u30e1\u30fc\u30bf] \u3067\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u8a2d\u5b9a\u3059\u308b\u3002 \u30ad\u30fc: --conf \u5024: spark.sql.parquet.enableVectorizedReader=false \u95a2\u9023 Glue Spark \u30b8\u30e7\u30d6\u3067 dynamic_frame \u304b\u3089 Parquet \u3092\u8aad\u3082\u3046\u3068\u3059\u308b\u3068 \"Unsupported encoding: DELTA_BINARY_PACKED\" \u3068\u6012\u3089\u308c\u308b - ablog"}