A quick heads-up before you dive in: the video version of this topic is in Indonesian, and it's less of a structured breakdown than it is a keluh kesah — an unfiltered venting session about a platform migration I've been living through at work. So I wrote this article as the polished, English-language companion for international readers who want the full story without the language barrier. Watch the video on YouTube: Why Big Companies Keep Migrating Their Data Stacks
If you've worked in data long enough, you've probably lived through it. One day the company is all-in on Google BigQuery. Two years later, leadership announces a strategic shift to Snowflake. A few years after that, someone in a boardroom decides Databricks is the future, and your team starts planning another migration.
From the outside, and honestly from inside the engineering team too, these moves look baffling. The current platform works. The pipelines run. So why does this keep happening?
When I talk to fellow data engineers, the assumption is almost always the same: the migration must have been driven by technical superiority. Better performance, better pricing model, better features. We assume somebody ran the benchmarks and BigQuery lost.
But here's what I've learned watching these migrations play out at large companies: the decision is rarely about the tech. It's about business politics and strategic partnerships.
The Real Driver Behind Enterprise Deals
Let's start with the first big reason: enterprise deals.
Cloud data platform vendors operate in an extremely competitive market. Snowflake, BigQuery, Databricks, Azure Synapse, Redshift — they're all fighting for the same enterprise budgets. And when the deal size is measured in millions of dollars per year, vendors don't compete on feature checklists alone. They compete with money.
What does that look like in practice? When a vendor smells a potential enterprise customer, they come to the table with offers like:
- Heavy discounts on list pricing that no public documentation will ever show you.
- Cloud credits — large pools of free usage that make the first year or two look almost free on paper.
- Subsidized migration support, including dedicated solution architects and professional services teams who help you move your workloads.
That last point matters more than people realize. Migration is expensive, and one of the biggest barriers to switching platforms is the engineering cost of actually doing it. When the vendor offers to subsidize that cost, the financial equation flips completely. Suddenly the "crazy" idea of moving everything becomes very attractive to whoever owns the budget.
Strategic Partnerships: The 5-Year Commitment Game
The second reason runs deeper: strategic partnerships.
Large companies don't just buy software licenses; they sign long-term agreements. A typical arrangement might be a five-year commitment with a platform vendor, negotiated as part of a broader cloud partnership.
Here's how the math works out. Under one of these agreements:
- The annual fees drop significantly compared to pay-as-you-go list prices.
- Migration support is subsidized, so the vendor effectively pays down your switching cost.
- The total cost of ownership over five years ends up lower than staying put, even after accounting for the disruption of migrating.
So when a CIO compares "stay on the current platform at full price" versus "move to a new platform under a subsidized five-year deal," the spreadsheet often favors moving. Not because Snowflake is magically faster than BigQuery, but because the commercial package was designed to make leaving more rational than staying.
And this creates a cycle. Contracts expire, new vendors show up with aggressive offers, and the whole dance begins again. Five years later, the company migrates somewhere else — sometimes even back to a platform they left years earlier.
Yes, It's Frustrating for Engineers
Now let's be honest about the other side of this story, because it affects real people.
Every one of these migrations means that somebody has to do the actual work. Rebuilding ingestion pipelines. Rewriting SQL dialect differences. Re-testing dbt models. Re-validating data quality checks. Migrating BI dashboards and downstream consumers. That heavy lifting lands squarely on the data engineering team — usually while they're still expected to keep the existing platform running and deliver the regular roadmap.
I completely understand the frustration. You spend months mastering a platform's quirks, harden your pipelines around its behavior, and then a contract negotiation you weren't part of resets everything. Your Snowflake-specific expertise gets devalued overnight, and you start from zero on a new stack.
It can feel like your technical judgment doesn't matter. And in a sense, at the moment the deal is signed, it genuinely didn't — because the decision was never made in a technical forum.
The Silver Lining: This Is Why Data Engineering Jobs Exist
But here's the twist that emerged once I finished venting, and it's what I want to leave you with.
As annoying as these constant migrations are, they are exactly what keeps the data engineering job market alive.
Think about it. If every company picked a data platform once in 2015 and never touched it again, the industry would need far fewer data engineers. Pipelines would be built once, stable for a decade, and maintenance would be a small team's side project.
Instead, every enterprise deal, every strategic partnership, every five-year contract renewal cycle generates fresh demand for people who know how to move data at scale. Migration projects are some of the most reliable sources of data engineering work out there — and companies pay well for them, precisely because the work is painful and high-stakes.
So the same business politics that frustrate us are also what keeps our skills valuable. Vendors keep competing, deals keep getting signed, and platforms keep getting swapped — and every swap needs someone who understands both the source and the target.
Wrapping Up
The next time your company announces yet another data platform migration, resist the urge to ask "which benchmark did the new tool win?" The answer is usually none of them.
Ask instead: whose budget got approved, which vendor offered the sweetest deal, and how many years is the commitment?
Understanding this dynamic won't make the migration itself any less tedious. But it will make you a smarter engineer — one who understands that data platforms are bought in boardrooms, not chosen in benchmarks. And it might even change how you position your own career: the ability to migrate systems safely is not a distraction from your job. In this industry, it practically is the job security.
👉 Watch the original rant (in Indonesian): Why Companies Keep Migrating Their Data Stacks