Build, Optimize & Scale Your Data Lakehouse with Qlik Open Lakehouse

An open lakehouse is a data lakehouse built on open standards (open table formats like Apache Iceberg and open catalogs), so your data lives in your own object storage and any compatible engine can query it, rather than being locked to one vendor's platform. It combines the low-cost storage of a data lake with the reliability and performance of a data warehouse, while keeping the data portable across tools. This openness lets organizations mix and match query engines and avoid vendor lock-in.

An open table format is a metadata layer that sits on top of raw files in object storage and makes them behave like a reliable database table, adding ACID transactions, schema evolution, and time travel. It's distinct from a file format like Parquet, which defines how bytes are stored: the file format holds the data, while the table format tracks which files belong to a table and manages changes over time. The three leading open table formats are Apache Iceberg, Delta Lake, and Apache Hudi.

Apache Iceberg and Delta Lake are both open table formats that bring ACID transactions, schema evolution, and time travel to data lakes, but they come from different origins. Iceberg (created at Netflix) was designed as a vendor-neutral specification with the broadest multi-engine support, so many query engines can read and write the same tables independently; Delta Lake (created at Databricks) grew out of Apache Spark and offers the deepest Spark integration. Iceberg has become the industry standard for cross-engine, vendor-neutral lakehouses.

An Iceberg catalog is the service that tracks the metadata and current state of Iceberg tables, so query engines can reliably discover which tables exist and read the correct, up-to-date version. It's a key part of the open lakehouse because it decouples tables from any single engine: examples include AWS Glue, Snowflake Open Catalog, and Apache Polaris, often accessed through the open Iceberg REST catalog standard. A shared catalog is what lets many tools query the same tables consistently.

The small files problem happens when frequent or streaming writes produce many tiny files in a data lake, which slows queries because engines must open and scan far more files than necessary. Compaction solves this by merging small files into larger, optimized ones, cutting fragmentation and speeding up reads. Well-managed lakehouses automate compaction along with expiring old snapshots and removing orphaned files to keep tables fast and cost-efficient.

Vendor lock-in is when your data or workloads become so dependent on a single vendor's proprietary format or engine that switching later is costly and disruptive. Open table formats like Apache Iceberg help avoid it by storing data in an open standard on your own object storage, so multiple engines can access the same tables and you can change tools without migrating or duplicating data. Supporting multiple engines and catalogs keeps an architecture flexible over time.

Zero-copy data mirroring makes data available to another system (such as a cloud data warehouse) without physically copying or duplicating it, so there's a single source of truth instead of multiple divergent copies. This cuts storage costs and avoids the drift and latency that come with maintaining redundant datasets. With open formats, the same governed data can be queried across platforms without duplication.

Object storage is a way of storing data as discrete objects (each with the data, metadata, and a unique identifier) in a flat, massively scalable pool, rather than in the folders of a file system or the rows of a database. It's inexpensive, durable, and virtually unlimited in scale, which makes it the standard storage layer for data lakes and lakehouses; common services include Amazon S3, Azure Data Lake Storage, and Google Cloud Storage. It decouples cheap storage from the compute that queries it.

Apache Parquet is an open-source, columnar file format designed for efficient storage and fast analytical queries on large datasets. Because it stores data by column rather than by row, it compresses well and lets query engines read only the columns they need, which dramatically cuts scan time and cost. Parquet is the standard file format underneath open table formats like Apache Iceberg, which add a metadata layer to turn collections of Parquet files into reliable, transactional tables.

A data lake stores raw files of any type cheaply in object storage but has no built-in structure, transactions, or consistency guarantees, which can lead to an unreliable "data swamp." A data lakehouse adds an open table format on top of that same storage, bringing warehouse-grade features like ACID transactions, schema enforcement, and fast queries while keeping the low cost and flexibility of the lake. In short, a lakehouse turns a data lake into something you can trust for BI, analytics, and AI.