Improve documentation on variable precedence with compose file interpolation

[kennethso168]

Is this a docs issue?

• My issue is about the documentation content or website

Type of issue

Information is incorrect

Description

In this doc, precedence of variable interpolation is explained as below:

Ways to set variables with interpolation

Docker Compose can interpolate variables into your Compose file from multiple sources.

Note that when the same variable is declared by multiple sources, precedence applies:

1. Variables from your shell environment
2. If --env-file is not set, variables set by an .env file in local working directory (PWD)
3. Variables from a file set by --env-file or an .env file in project directory

However, consider the following example project with the following structure and contents. The project is located at /home/kenneth/test-docker-compose-env. Commands are run with shell with current directory at /home/kenneth/test-docker-compose-env

$ tree -a

.
├── .env
└── postgres
    ├── .env
    └── compose.yml

.env

DB_NAME=db_name_defined_by_outer_env
OUTER_ENV_IS_SOURCED=true

postgres\.env

DB_NAME=db_name_defined_by_inner_env
INNER_ENV_IS_SOURCED=true

postgres\compose.yml

---
services:
  db:
    image: postgres:18
    environment:
      - POSTGRES_DB=${DB_NAME:?}
      - OUTER_ENV_IS_SOURCED=${OUTER_ENV_IS_SOURCED:-false}
      - INNER_ENV_IS_SOURCED=${INNER_ENV_IS_SOURCED:-false}

If we run docker compose -f postgres/compose.yml config, we got the following result:

name: postgres
services:
  db:
    environment:
      INNER_ENV_IS_SOURCED: "true"
      OUTER_ENV_IS_SOURCED: "false"
      POSTGRES_DB: db_name_defined_by_inner_env
    image: postgres:18
    networks:
      default: null
networks:
  default:
    name: postgres_default

$ docker compose -f postgres/compose.yml config --environment (only relevant environment variables shown)

DB_NAME=db_name_defined_by_inner_env
INNER_ENV_IS_SOURCED=true
PWD=/home/kenneth/test-docker-compose-env

In this case, the .env file located at the shell's working directory (PWD) are not sourced at all.

We got different output when we defined the compose file with environment variable instead

$ COMPOSE_FILE=postgres/compose.yml docker compose config

name: postgres
services:
  db:
    environment:
      INNER_ENV_IS_SOURCED: "true"
      OUTER_ENV_IS_SOURCED: "true"
      POSTGRES_DB: db_name_defined_by_outer_env
    image: postgres:18
    networks:
      default: null
networks:
  default:
    name: postgres_default

Location

https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/

Suggestion

Let's have a look at the compose cli documentation as below:

Options

Option	Default	Description
--all-resources		Include all resources, even those not used by services
--ansi	auto	Control when to print ANSI control characters ("never"
--compatibility		Run compose in backward compatibility mode
--dry-run		Execute command in dry run mode
--env-file		Specify an alternate environment file
-f, --file		Compose configuration files
--parallel	-1	Control max parallelism, -1 for unlimited
--profile		Specify a profile to enable
--progress		Set type of progress output (auto, tty, plain, json, quiet)
--project-directory		Specify an alternate working directory (default: the path of the, first specified, Compose file)
-p, --project-name		Project name

From this, we can infer that if we use a -f flag, the working directory is treated to be "the path of the, first specified, Compose file)" (as seen by the description of the --project-directory). Only when both --project-directory and -f flags are not set, the working directory is treated to be the shell's current directory (PWD) Therefore, mentioning about PWD in "2. If --env-file is not set, variables set by an .env file in local working directory (PWD)" is misleading.

I would suggest changing the precedence part as below:

Note that when the same variable is declared by multiple sources, precedence applies:

1. Variables from your shell environment
2. Variables from a file set by --env-file
3. If --env-file is not set, variables set by an .env file in the working directory, which is defined in order as either the directory set by --project-directory, the path of the first specified compose file with -f/--file, or shell's current directory (PWD)
4. If --env-file, --project-directory or --file are not set, variables set by an .env file in the directory of compose file defined by the COMPOSE_FILE pre-defined variable (See local .env file versus <project directory> .env file)

And the section "local .env file versus <project directory> .env file" would warrant a rewrite as well.

Original:

When executed without an explicit --env-file flag, Compose searches for an .env file in your working directory (PWD) and loads values both for self-configuration and interpolation. If the values in this file define the COMPOSE_FILE pre-defined variable, which results in a project directory being set to another folder,
Compose will load a second .env file, if present. This second .env file has a lower precedence.

My suggestion:

When executed without explicit --env-file, --project-directory or --file flags, Compose searches for an .env file in the your working directory (PWD) and loads values both for self-configuration and interpolation. If the COMPOSE_FILE pre-defined variable is defined, which results in a project directory being set to another folder,
Compose will load a second .env file, if present. This second .env file has a lower precedence.