Snowflake launches support for Apache Iceberg, a table format for demanding Analytics in production environments.
Apache Iceberg is in the spotlight. In late 2021, AWS announced that Amazon EMR (big data and analytics platform) supports Iceberg. “The table format is suitable for fast queries in large tables and managing complex datasets in data lakes”, the organization described. Snowflake shares the opinion. In late January 2022, Snowflake introduced support for Iceberg in the Snowflake Data Cloud. This enables Snowflake Data Cloud to perform queries in data structured through Iceberg.
Iceberg’s popularity is growing. The table format was developed in 2018 by Netflix and Apple engineers. They were in the market for an open source alternative to Apache Hive’s table format. Resultingly, Apache Iceberg was designed to process data in Parquet, ORC and Avro with multiple users and frameworks without corrupting the data.
AWS and Snowflake have multiple reasons for embracing the format. As mentioned earlier, AWS is optimistic about the speed of queries within Iceberg. Snowflake emphasizes the wide range of file types and metadata that the format supports. Although it’s too early to speak of an industry norm, Iceberg’s future is promising.