Hello, I noticed that on the first start with a PostgreSQL database engine, composer-dev start always fails with:
psycopg2.OperationalError: could not translate host name
"composer-local-dev-db-" to address: Name or service not known
Second composer-dev start was then always successful when I tried a few times.
EDIT: at first I was misled and thought the issue was caused by a race condition. But the proposed fix didn't solve the issue, so I dig more to find the real cause, and then updated this issue and PR with a fix.
Root cause
The Airflow container's entrypoint.sh (line 110) runs sudo chmod -R o+xrw $PGDATA when $PGDATA is set in the environment. The problem is that PGDATA was being passed to the Airflow container via **default_db_variables spread in get_default_environment_variables():
|
"GCP_PROJECT": self.project_id, |
|
"GOOGLE_CLOUD_PROJECT": self.project_id, |
|
**default_db_variables, |
|
**self.get_environment_variables_for_image_version(), |
Postgres requires strict permissions (0700 or 0750) on its data directory. The chmod corrupts those permissions, Postgres refuses to start, its container exits, its hostname disappears from Docker DNS -- which produces the misleading DNS resolution error I shared above.
This only happens on the first start because postgresql_data/ does not exist yet. Postgres initializes the data directory during container startup, and the Airflow entrypoint's chmod corrupts permissions in that window. On subsequent starts the data directory is already initialized, so Postgres starts instantly before the chmod can do damage.
Fix
Filter default_db_variables to only pass AIRFLOW__*-prefixed vars to the Airflow container. Postgres-specific vars (PGDATA, POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB) are not needed by Airflow.
In composer_local_dev/environment.py (line ~697):
# Before:
**default_db_variables,
# After:
**{k: v for k, v in default_db_variables.items() if k.startswith("AIRFLOW__")},
Steps to reproduce
- Fresh environment (empty postgresql_data/ directory)
- composer-dev start
Expected behavior
Environment starts successfully on first attempt.
Actual behavior
Environment fails to start.
Traceback:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not
translate host name "composer-local-dev-db-<env-name>" to address:
Name or service not known
Workaround
Run composer-dev start a second time. Subsequent starts succeed because postgresql_data/ already exists and Postgres initialises instantly, so the chmod no longer has time to corrupt permissions before Postgres finishes starting.
Hello, I noticed that on the first start with a PostgreSQL database engine,
composer-dev startalways fails with:psycopg2.OperationalError: could not translate host name
"composer-local-dev-db-" to address: Name or service not known
Second
composer-dev startwas then always successful when I tried a few times.EDIT: at first I was misled and thought the issue was caused by a race condition. But the proposed fix didn't solve the issue, so I dig more to find the real cause, and then updated this issue and PR with a fix.
Root cause
The Airflow container's
entrypoint.sh(line 110) runssudo chmod -R o+xrw $PGDATAwhen$PGDATAis set in the environment. The problem is thatPGDATAwas being passed to the Airflow container via**default_db_variablesspread inget_default_environment_variables():composer-local-dev/composer_local_dev/environment.py
Lines 695 to 698 in c29a1ac
Postgres requires strict permissions (
0700or0750) on its data directory. The chmod corrupts those permissions, Postgres refuses to start, its container exits, its hostname disappears from Docker DNS -- which produces the misleading DNS resolution error I shared above.This only happens on the first start because
postgresql_data/does not exist yet. Postgres initializes the data directory during container startup, and the Airflow entrypoint's chmod corrupts permissions in that window. On subsequent starts the data directory is already initialized, so Postgres starts instantly before the chmod can do damage.Fix
Filter
default_db_variablesto only passAIRFLOW__*-prefixed vars to the Airflow container. Postgres-specific vars (PGDATA,POSTGRES_USER,POSTGRES_PASSWORD,POSTGRES_DB) are not needed by Airflow.In
composer_local_dev/environment.py(line ~697):Steps to reproduce
Expected behavior
Environment starts successfully on first attempt.
Actual behavior
Environment fails to start.
Traceback:
Workaround
Run
composer-dev starta second time. Subsequent starts succeed because postgresql_data/ already exists and Postgres initialises instantly, so the chmod no longer has time to corrupt permissions before Postgres finishes starting.