AI

Agentic AI Just Rewrote the Data Engineer's Job Description. Here's What IT Leaders Need to Know.

Headshot of blog author Dan Potter. He has short hair, wearing a black suit jacket over a light blue checkered shirt, smiling at the camera.

Dan Potter

5 minutes

Gartner predicts that 70% of today's data engineering tasks will be fully automated by 2030. I put that number to Tim Garrod, Qlik's Head of Product Management for data integration and quality, on a recent Qlik Insider session, and his answer is the one every CIO, CDO, and VP of IT should sit with: automation doesn't make the data engineer obsolete, it makes the good ones ten times more valuable. AI amplifies skilled judgment. It doesn't replace it.

That distinction matters because it reframes the real question in front of technology leaders right now. It isn't “how much of data engineering can we automate.” It's “what does our data need to look like before we let autonomous agents act on it.” Agentic AI doesn't just consume data, it makes decisions and takes action on your organization's behalf. Garbage in, garbage out used to mean a bad report. Now it means a bad business decision executed at machine speed with no human in the loop to catch it.

That's the shift behind what Qlik calls Agentic Data Engineering, and it's worth understanding in plain terms because it changes what you should expect from your platform and your team.

From moving data to earning trust

Data engineers have always built the pipelines that move, transform, and store data. That work isn't going away, but it's no longer the whole job. The new mandate is providing the context, semantics, cost visibility, and governance that let AI agents know which data to trust, for which use case, and why. Tim's phrase for this stuck with me: data engineers are becoming architects of intent, not just builders of pipelines.

Three shifts worth putting on your roadmap

The first is a move to intent driven design. This is easy to confuse with vibe coding, where you tell an AI what to build and it generates an entire codebase with no real governance behind it. Intent driven design is the opposite instinct. The data engineer defines the goals, the constraints, and the quality rules as a governed specification, and AI builds on top of that durable foundation. The output has to meet the same trust bar as something built by hand, whether a person or an agent wrote it. Practically, this shows up as declarative, YAML based pipelines that are readable enough for AI to build and precise enough for a human to govern.

The second is autonomous operations. Think about the 3 a.m. pipeline failure that used to mean waking up an on call engineer. A meaningful share of that troubleshooting and remediation can now be handled by AI. Tim was candid that this is a trust journey, not a light switch. The question every IT leader is actually asking is how much autonomy you're comfortable extending to an agent making changes to business critical pipelines while everyone is asleep, and that comfort level will rightly increase in stages.

The third, and the one I'd argue matters most to your AI strategy overall, is building a genuinely trusted data foundation. This is the “AI ready data” question: does your data carry the quality, lineage, and business context an agent needs to use it correctly, without a human there to catch a bad assumption? Qlik's approach here is data products, domain oriented packages of data that carry their own trust score, semantics, and lineage, so an agent isn't scanning a metadata repository with thousands of tables and guessing. It's pulling from something already vetted.

Why this is also a cost conversation

If you're watching AI spend climb the way cloud costs once did, this connects directly to your TCO. Poor context and low trust data don't just produce worse answers, they burn tokens through repeated iteration and hallucination correction. Better semantics and quality up front measurably reduce that consumption. Pair that with a scalable, low cost storage foundation like an open lakehouse built on Iceberg, and you're addressing both sides of the cost equation: compute efficiency and AI efficiency.

It also isn't only a data engineering conversation anymore. As trust and semantics become the currency, business analysts, data product owners, and BI teams are stepping directly into work that used to require deep technical skill. That's a genuine expansion of who can contribute to your data strategy, not a threat to any one role in it.

Where to start

Tim's advice, and mine, is not to wait for a perfect strategy before you move. Don't try to fix all your data at once. Pick one business initiative where trusted, AI ready data would clearly move the needle, and build from there. The tooling to raise data quality, add semantics, and extend governance without a heavy technical lift already exists. The organizations that get comfortable with agentic data engineering now will be the ones setting the pace on agentic AI more broadly. The ones that wait will be catching up on both fronts at once.

If you want to see the full conversation, including where Qlik is headed next with intent driven design and autonomous operations, watch the Qlik Insider session on demand.

In this article:

AI

Ready to get started?