Migration 0101_3_2_0_ui_improvements_for_deadlines upgrade is slow on deployments with large deadline and serialized_dag tables.

[vatsrahul1001]

Apache Airflow version

main (development)

If "Other Airflow 3 version" selected, which one?

No response

What happened?

Migration 0101_3_2_0_ui_improvements_for_deadlines upgrade is slow on deployments with large deadline and serialized_dag tables. With 10M deadline rows and 100K serialized_dags, the migration took ~16 minutes.

The UPDATE phase holds row-level locks on all deadline rows for ~7 minutes, and the subsequent ALTER COLUMN ... SET NOT NULL acquires ACCESS EXCLUSIVE lock which blocks all reads and writes to the deadline table.

What you think should happen instead?

No response

How to reproduce

Steps to reproduce:

1. Start Airflow on 3.1.8 with PostgreSQL backend
2. Accumulate deadline rows and serialized_dag entries (100K+ deadlines to see noticeable impact)
3. Upgrade to 3.2.0 (airflow db migrate)
4. Observe migration 0101 duration scales linearly with deadline row count

Operating System

MAc

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[bittoby]

@vatsrahul1001 Can I pick this up? I can do it correctly. I'd like to contribute

[henry3260]

@vatsrahul1001 Can I pick this up? I can do it correctly. I'd like to contribute

Thanks for your interest! You can directly raise a pr. Refer to https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#issue-reporting-and-resolution-process