The Data Engineering Stack Employers Actually Ask For: October 2026

Most “top data engineering skills” lists count mentions. A mention isn’t a requirement, though: a job that says “Spark required, Kafka a plus” names both tools but treats them very differently. So this report splits each tool in two. How many jobs require it, and how many list it only as nice-to-have?

We looked at every data engineering job its employer posted between 1 and 31 October 2026 that passed our moderation: 1,738 jobs at 730 companies, collected from company career pages, ATS boards, job boards and Hacker News, with duplicates merged. 90% of them name at least one technology. Those that do name about five, on average three required and two nice-to-have.

The short version

• Spark is the one hard gate. 593 jobs name it, and 78% of those require it. That’s 463 jobs, more than any other tool and 27% of the whole market.
• dbt overtakes Airflow once you count requirements. Airflow is named more often (518 jobs against 484), but dbt is required in more (315 against 309). Four in ten Airflow mentions are nice-to-have.
• Databricks is required more often than Snowflake (70% against 64% of the jobs naming each), and in nearly half again as many jobs (309 against 212).
• Kafka and Iceberg are the “plus” tools of the seven. Kafka is required in just over half of its jobs (54%), and Iceberg in half (51%). Among jobs that clearly separate the two lists, both drop to 40% or lower.
• The seven tools form two stacks. One is Spark, Databricks and Delta Lake; the other is dbt, Airflow and Snowflake. Only 3% of Databricks jobs mention Iceberg.

The ranking

Ranked by the number of jobs that require each tool. “Named in” counts every job that mentions the tool. “Required when named” is the share of those jobs that list it as a must. The strict view repeats that share using only the 985 jobs that list at least one nice-to-have, so we know they separate the two lists (more on that below).

Rank	Tool	Required	Nice-to-have	Named in	Required when named	Strict view
1	Spark	463	130	593 (34%)	78%	66%
2	dbt	315	169	484 (28%)	65%	51%
3	Airflow	309	209	518 (30%)	60%	44%
3	Databricks	309	135	444 (26%)	70%	57%
5	Snowflake	212	117	329 (19%)	64%	51%
6	Kafka	178	152	330 (19%)	54%	40%
7	Iceberg	61	59	120 (7%)	51%	34%

Taken together, 1,007 jobs (58%) require at least one of the seven, and 1,261 (73%) name at least one.

How often a tool is named, and how often it’s a must

Mentions and requirements don’t move together. The chart below plots the 28 most-named technologies. The further right a tool sits, the more jobs name it; the higher up, the more often those jobs require it.

Four groups stand out:

• Top right: Spark. It’s both the most-named tool and one of the most often required. No other tool comes close on both counts.
• Top left: platform commitments. Microsoft Fabric (77%), Delta Lake (74%), Unity Catalog (70%) and Azure Data Factory (68%) appear in fewer jobs. When they do appear, they’re usually required, because they describe the platform the team already runs on. You’d need to know it from day one.
• Bottom: the tools next to the job. BI tools (Power BI 44%, Looker 43%, Tableau 41%), Fivetran (39%) and MLflow (42%) are mostly nice-to-have. Employers see them as tools you can pick up, not ones you’ll be screened on.
• Git is required in 85% of the jobs that name it, more than any tool in the chart. Few jobs mention it (18%), probably because most employers take it for granted.

Do employers really separate “required” from “nice-to-have”?

Not always. Some jobs put everything in one list of requirements, and we count every tool in such a list as required. That inflates the required share. So we repeated the count using only the 985 jobs (63% of those naming any technology) that list at least one nice-to-have. In those jobs we know the employer drew the line.

Every share falls by 12 to 17 points, but the order barely changes. Spark is still the only tool required in a clear majority of its jobs (66%), and Databricks still beats Snowflake. In this stricter view, Airflow (44%), Kafka (40%) and Iceberg (34%) are named more often as nice-to-have than as required.

Head to head, by category

Orchestration. Airflow appears in five times as many jobs as Dagster (518 against 107) and more than ten times as many as Prefect (48). Dagster rarely replaces Airflow outright: 80 of its 107 jobs also name Airflow, often as an alternative to it. Azure Data Factory is required in 68% of its jobs, more often than Airflow, because it comes with the Azure platform.

Warehouse and lakehouse. Databricks leads Snowflake in jobs named (444 against 329) and in how often it’s required (70% against 64%). BigQuery is third with 224 jobs and is required as often as Snowflake (67%). Redshift trails (142 jobs) and is required in only 54% of them, the lowest share among the big four. Microsoft Fabric is small (95 jobs) but required in 77% of them, the highest share in this group.

Streaming. Kafka has no real competitor (330 jobs), but it’s one of the most optional tools in this report: 152 jobs list it as nice-to-have. Flink (103) is required a bit more often (58%). Pub/Sub is required in 79% of its 61 jobs, which fits a tool that comes with a team that has already chosen GCP.

Table formats. This is the clearest split in the report. Delta Lake is required in 74% of its jobs and Iceberg in 51%. Delta comes with Databricks, so a Databricks shop needs it. Iceberg is more often a “familiarity with open table formats” bullet. Hudi barely registers (11 jobs).

Two stacks, not one

Which tools appear together says more than any single ranking.

• The lakehouse stack: Spark, Databricks, Delta Lake. 62% of Databricks jobs also name Spark. The single most common combination of the seven is “Spark and Databricks, nothing else” (139 jobs).
• The warehouse and dbt stack: dbt, Airflow, Snowflake. 57% of Snowflake jobs name dbt, and 54% of dbt jobs name Airflow. dbt appears with Snowflake (188 jobs) almost twice as often as with Databricks (103).
• Few teams run both platforms. Only 119 jobs name Databricks and Snowflake together, 27% of Databricks jobs.
• Kafka and Iceberg sit in the middle. Both lean toward Spark (64% and 67% of their jobs name it) and Airflow (54% and 56%). Iceberg is close to absent from Databricks jobs (14 of 444) and only somewhat more common in Snowflake jobs (30 of 329). For now, employers name it mostly alongside Spark, Airflow and Kafka, not with either vendor’s platform.

If you’re choosing what to learn next, pick a side first. A Spark and Databricks CV and a dbt and Snowflake CV fit different jobs, and few jobs ask for both.

What each kind of role requires

The mix depends heavily on the role. Here’s the share of jobs in each role category that require each tool:

• Core data engineering jobs require Spark most (39%), then Databricks (23%) and Airflow (22%).
• Analytics engineering is a dbt job. 46% of analytics engineering jobs require dbt, and 61% name it. Nothing else comes close: Airflow is next at 18%, and Kafka (3%) and Spark (10%) hardly matter.
• Platform, DataOps and MLOps jobs require less of the seven, because they ask for infrastructure tools instead: Kubernetes, Terraform and cloud services. Kafka matters here about as much as in core roles (12%).

By seniority, the differences are smaller than you might expect. Junior jobs require Spark about as often as senior ones, but dbt much less:

Required in…	Jobs	Spark	dbt	Airflow	Databricks	Snowflake	Kafka	Iceberg
Junior and intern	132	28%	5%	14%	11%	6%	10%	2%
Mid	295	26%	16%	20%	18%	14%	8%	2%
Senior	773	31%	22%	19%	20%	13%	13%	5%
Lead, staff, principal	216	25%	21%	16%	21%	19%	11%	6%

The junior row rests on 132 jobs, so treat it as rough. Still, dbt and Snowflake requirements rise with seniority. Iceberg is a senior-level ask: it’s required in 2% of junior and mid jobs and 5–6% at senior level and above.

Languages and clouds

SQL is the most universal requirement in data engineering. It’s named in 66% of jobs and required in 98% of those. Python is named more often (71%) and required in 86% of those jobs. Java and Scala come next and are required in about two-thirds of the jobs that name them, mostly Spark and streaming jobs. Rust is the opposite: of the 47 jobs that mention it, 60% list it as nice-to-have.

Cloud platforms are required less often than you might think: 57–61% of the jobs that name them. A lot of jobs say “AWS, GCP or Azure”, and we count all three as named. Being fluent in one cloud is usually enough.

AI coding assistants enter the requirements

For the first time, AI coding assistants show up in enough jobs to count: 114 jobs (7%) name Claude Code, Cursor or Claude. These are mostly senior roles (45 of the 114), and the list is led by DoorDash (9 jobs) and Samsara (5). Most treat them as a plus: 43 jobs require one. That’s small, but we’ll track it month by month.

Does any of this pay more?

Not in a way we can see yet. Only 176 US jobs state a salary (median $180k), so tool-level samples are small. Jobs naming Snowflake or Kafka sit at the US median, and jobs naming Spark, dbt, Airflow or Databricks sit somewhat below it. Iceberg jobs have a higher median ($200k), but only 11 of them state a salary. In our October market report, level explained far more of the pay difference than any tool did. We’ll come back to this when there’s more data.

What to take from this

• If you learn one tool, make it Spark. It’s the only tool required in a clear majority of the jobs that name it, even by the strict count.
• Pick a stack. Spark and Databricks or dbt and Snowflake, with Airflow as a broadly useful extra on either side.
• Treat Kafka and Iceberg as differentiators. Many jobs mention them and fewer require them, so they’ll set you apart without often being the reason you’re screened out.
• Don’t skip SQL. It’s in two-thirds of jobs and nearly always required.

How we counted

• What’s included: jobs whose employer-stated posting date falls in October 2026 and that passed our moderation as genuine data engineering roles (data engineering, analytics engineering, data platform and infrastructure, DataOps and MLOps). Jobs that have since closed are included. A job posted on several sites counts once.
• Required or nice-to-have: we read each job description and record every technology it names, marked as required or nice-to-have according to the description. Tools in “preferred”, “bonus” or “a plus” sections count as nice-to-have. When a job puts all its tools in one list, we count them all as required. The strict view therefore uses only the 985 jobs that list at least one nice-to-have.
• Normalization: “PySpark” and “Apache Spark” both count as Spark, “Apache Iceberg” as Iceberg, and so on. A job counts once per tool however many times it mentions it.
• Coverage: these are the jobs our sources reach: company career pages, ATS boards, job boards and Hacker News “Who’s Hiring”. It isn’t the whole market. LinkedIn and Indeed, for example, aren’t included. We added sources during October, so we’ll be careful when comparing months.
• Data date: read on 9 November 2026. Job descriptions are sometimes re-processed after edits, so a count can differ by one or two from our October market report.

We’ll publish this ranking every month, so next month we can show which tools are moving. To get new data engineering jobs every week, with the ones that state their salary and stack first, subscribe to the weekly Digest. Or browse jobs by tool: Spark, dbt, Airflow, Databricks, Snowflake, Kafka and Iceberg.