Prefer watching? Watch "What Does a Data Engineer Actually Do in the AI Era?" on YouTube. The video is narrated in English.
Sometimes my work really does look like typing a prompt. Build this ingestion pipeline. Add this aggregation. Follow the pattern in this repository. Then code appears.
From the outside, I understand why that looks like the difficult part has disappeared. Are data engineers still writing SQL, building ETL scripts, and fixing pipelines, or are we just asking an assistant to do those things now?
I still do all of that. But across my previous and current workplaces, the way I spend some of my time has changed. Less of it goes into typing the first version of the code. More goes into making the environment understandable, checking the result, and deciding what the assistant should be allowed to touch.
This is my experience, not a claim that every company works this way. I am keeping both employers anonymous. The interesting part is the work around the prompt, not anyone's internal systems.
Making the repository understandable to AI
At my previous company, I was working as a lead when the company started using AI to speed up development. One of my tasks was to make our existing workflow easier for AI to work with. Not replace the whole thing. Make what we already had understandable.
An assistant can write a pipeline without knowing how our team writes a pipeline. It does not automatically know where files belong, which naming conventions matter, how tests run, or what changes between development and production.
A lot of that knowledge used to live in people's heads. I started moving it into instructions inside the repository, including an AGENTS.md file: how the project works, which patterns to follow, and what not to do.
Those instructions do not make generated code correct by themselves. They give the assistant a better starting point and give the person reviewing its work something concrete to compare against.
I also built an MCP server so the assistant could connect to our data warehouse through tools, instead of needing me to supply the context manually each time. MCP gives an assistant a way to use tools; it is not permission to access everything. The actual capabilities depend on how those tools are implemented and authorized.
And yes, AI helped me build the instructions and the server too. I was using AI to make the environment easier for AI to use.
A short prompt can hide a lot of engineering
After that work, someone on the team could ask for a new ingestion or aggregation pipeline and get a useful starting point faster. The prompt could be short because much of the explanation was already in the environment.
That distinction matters. A few magic words had not replaced the engineering. Some of the engineering had moved into the repository conventions, instructions, and tools surrounding those words.
At my current workplace, that connected workflow goes further in other areas. Timesheets, GitLab, and Jira are already connected to AI. Instead of opening each application separately, I can ask about my tasks, work on a change, and ask the assistant to update the relevant ticket.
It starts to feel like: what have I not finished today? Okay, help me finish it.
That is convenient. But a ticket marked done is not the same as work that is correct, and a pipeline that runs is not automatically a pipeline we should trust. Faster coordination and code generation do not settle either question.
Why the data itself is still disconnected
There is an important difference between the two workplaces. At the previous one, I could build that warehouse connection. At my current one, the data itself is not connected in the same way.
A big part of the reason is compliance. I do not mean that as an obstacle we should quietly work around. There are real questions behind that boundary:
- What data would leave the environment, and where would it be processed?
- Who is authorized to see it?
- What ends up in logs?
- Can the assistant only read, or can it change something too?
Even a read-only connection can expose information that should not be exposed. Connecting a tool does not answer those questions for us.
Depending on the company's rules, approved metadata, masked samples, or a development environment might be a starting point. Those are possible approaches, not a description of a deployment at my current employer. Any such access still needs approval, limits, and a record of what happened.
Some data may need to remain disconnected. That is not necessarily a failure of the technology. Sometimes it is the correct boundary. The job includes deciding what access should be allowed, not only making access technically possible.
The query can run and still be wrong
Experience changes how useful an assistant feels. Someone senior often already has a picture of what the answer should look like. They notice missing context, know which question to ask next, or recognize a suspicious piece of generated SQL.
Not because seniors are always right. They have had more opportunities to see things go wrong.
Consider a hypothetical join, not an incident from either workplace. An AI joins a revenue table to another table that contains several rows for the same customer. The query runs successfully, but the join multiplies rows and inflates the total. Nothing crashes. The dashboard is confidently wrong.
Checking that requires understanding the grain of the data: what one row represents, whether the join keys are unique, and what result we expected. Successful execution is only one part of that check.
Learning to check the work, not just request it
That leaves a difficult question for someone starting out. If AI does most of the writing before a fresh graduate learns how to check it, where does that judgment come from?
I do not think the answer is to avoid AI. I would rather use it as something I can question: explain this join, list your assumptions, show me a small example where it breaks.
Then I would check the example myself. Read the SQL. Predict the result before running it. Compare row counts. Try duplicate records, missing values, and a failed run that needs to be retried.
The point is to stay involved in the reasoning, not to prove that learning only counts without help. And the team has a responsibility here too. We cannot give juniors an assistant, demand faster delivery, and then act surprised that they still need mentoring.
What my job looks like now
I still care about the data arriving, the numbers making sense, and the pipeline behaving properly in production. Those responsibilities have not disappeared because producing code has become faster.
I spend less time typing some of that code and more time making context clear, reviewing results, and setting access boundaries. I do not know exactly where the role is going, and the places I have worked are not moving at the same speed.
But when someone asks whether my job is just prompting now, I understand the question. The prompt is the part they can see. There is still quite a lot behind it.
How far has AI reached in your workplace: writing code, updating tickets, or accessing data too?
Watch the full reflection: What Does a Data Engineer Actually Do in the AI Era?.