Environment variables not getting passed through with php-fpm and ECS

I have a task definition that looks like this (truncated for readability). And I need to get the host IP address dynamically available inside the PHP container. It's required for Datadog to work.

[
{
...
"entryPoint": [
"/usr/local/bin/entrypoint.sh"
],
"portMappings": [
{
"hostPort": 0,
"protocol": "tcp",
"containerPort": 9000
}
],
...

This is a container that runs a Symfony application, with php-fpm. So my entrypoint.sh looks like this:

#!/bin/sh
export TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
export DD_AGENT_HOST=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4)
echo "Datadog agent host: ${DD_AGENT_HOST}"
php-fpm -F

The echo debug prints the correct value in the cloudwatch logs. However when I log into this container after it starts en check the environment variables with php -i, the variable is not in there.

I made sure clear_env = no is in my www.conf. But no success.

Here's my Dockerfile, for reference:

# Multi stage build. We first run composer install, and later copy it over in the 2nd image
FROM composer:2 as vendor
WORKDIR "/app/platform"
COPY ["./app/composer.json", "./app/composer.lock*", "/app/platform/"]
RUN composer install --no-interaction --no-dev --prefer-dist --ignore-platform-reqs
FROM php:8.1-fpm-buster as platform
RUN \
# Symfony needs write access to some folders. We need to pre generate these
mkdir -p "/app/platform/var" \
&& chmod 777 -R "/app/platform/var" \
# Install necessary binaries and packages
&& apt-get update \
&& apt-get install -y gnupg g++ procps openssl git unzip zlib1g-dev libzip-dev libfreetype6-dev libpng-dev libjpeg-dev libicu-dev  libonig-dev libxslt1-dev acl libjpeg62-turbo-dev libmcrypt-dev zip vim \
# Install php extensions
#&& pecl install redis \
#&& docker-php-ext-enable redis \
&& docker-php-ext-install pdo pdo_mysql zip xsl gd intl opcache exif mbstring soap \
# Install Composer
&& curl -sS https://getcomposer.org/installer | php -- --install-dir="/usr/local/bin" --filename=composer && chmod +x "/usr/local/bin/composer"
# Datadog
ARG DD_ENV_ARG=placeholder
ENV DD_ENV=$DD_ENV_ARG
ENV DD_VERSION=1
ENV DD_SERVICE=platform-aws
ENV DD_PROFILING_ENABLED=true
COPY "./.docker-prod/platform/datadog-setup.php" "/app/platform/datadog-setup.php"
RUN php /app/platform/datadog-setup.php --php-bin=all --enable-profiling
COPY "./app" "/app/platform"
RUN perl -pi -e 's#^(?=access\.log\b)#;#' /usr/local/etc/php-fpm.d/docker.conf
COPY "./.docker-prod/platform/php.ini" "/usr/local/etc/php/conf.d/docker-php-config.ini"
COPY "./.docker-prod/platform/www.conf" "/usr/local/etc/php-fpm.d/www.conf"
COPY --from=vendor "/app/platform/vendor" "/app/platform/vendor"
WORKDIR "/app/platform"
RUN \
composer dump-autoload --optimize --classmap-authoritative \
&& composer run-script post-install-cmd
# Entrypoint
COPY "./.docker-prod/platform/entrypoint.sh" "/usr/local/bin/entrypoint.sh"
RUN chmod +x "/usr/local/bin/entrypoint.sh"

I honestly tried everything to make this work, but no success. Even the documentation of datadog using the same kind of approach, but using that exact example also didn't work: https://docs.datadoghq.com/containers/amazon_ecs/apm/?tab=ec2metadataendpoint&code-lang=php

Anyone any ideas?

Let me first repeat your question, what my understanding of it is, and then, provide a solution and explanation:

> When I log into this container [FROM php:8.1-fpm-buster, and] after it starts [, when I] check the environment variables with php -i, the variable is not in there.

That's correct, the php binary invoked by docker-exec(1) in the container doesn't have the environment of php-fpm(1) invoked in the entrypoint.

For php-fpm(1) you wrote you've solved the original problem of DD_AGENT_HOST parameter unavailability by doing two things:

1. You configured the php-fpm pool with the clear_env directive setting it to "no" (clear_env = no). Please note that this takes in the whole environment, I'd highly recommend to set each environment parameter explicitly[¹][¹], I'd consider this is because you wanted to share how you were able to enable it for this example material and you're aware of the pool configuration directives, and you're interested in the exemplary DD_AGENT_HOST parameter only.
2. You exported the DD_AGENT_HOST parameter in the entrypoint with the export shell built-in (export PARAMETER=VALUE).

For php(1), the binary of the PHP CLI SAPI, you were not yet able to make the DD_AGENT_HOST parameter available.

So much for my understanding of your question, the answer to it is that your entrypoint is not yet complete.
Which is good news, because it means you were already close and can easily go on at that place.

While all sub-processes of the entrypoint (you should use exec for php-fpm btw., compare with the original entrypoint) will have the DD_AGENT_HOST parameter bootstrapped, php(1) (CLI) does not have it yet, docker-exec will not add it there.

Therefore, you need to bootstrap the php process with the environment whenever you invoke it, and that bootstrapping must be seeded within the entrypoint so that the container is prepared completely (technically, you could do it during the build of the container, but for the property you're looking for it likely does not exist then).

This works by replacing php(1) with a stub that takes care to initialize the environment and then passing it over to the original binary.

I created a git repository that has a complete test-stand and shows the production for the Dockerfile, here an excerpt for that relevant part for a php:8.1-fpm-buster base image, quoted from the COPY[²][²] section of the [entrypoint docker build³][³]. The example is in the sh syntax, I added some callouts:

> ~~~Shell
> ## HK: setup /usr/local/bin/php for PHP docker exec default environment
> mv -n /usr/local/bin/php /usr/local/bin/php-cli.bin # <1>
> <<STUB cat > /usr/local/bin/php
> #!/bin/bash
> export DD_AGENT_HOST="${DD_AGENT_HOST:?}" # <2>
> exec -a "$0" /usr/local/bin/php-cli.bin "$@" # <3>
> STUB
> chmod --reference=/usr/local/bin/php-cli.bin /usr/local/bin/php
> ~~~

^{<1> The original binary is moved, as it will be started again}

^{<2> Parameters need to be "re-exported", the :? parameter substitution is used to exit early by giving a precise parameter error when unset or empty. You can find this also in some of my answers in the CI collective, I highly recommend this for builds and CI pipelines of different kinds.}

^{Note: If the required parameter is non-empty but with a wrong or of a faulty value, you may want to add additional checks before you apply these changes as this can quickly become cumbersome when you rely on it otherwise. That is, do your own build checks.}

^{<3> This is where some magic comes in, note that this form of exec constitutes a bash requirement and buster has you covered – just FYI.}

So that's all the magic. If we now run the tests with diverse forms of invocation, we can see it working (omissions for readability, you can run the full tests yourself):

$ make clean test
...
./build.sh: now building "build"...
...
./build.sh: container-id: 81dc410de4a3 ...
./build.sh: [XX-XXX-XXXX XX:XX:XX] NOTICE: datadog agent host DD_AGENT_HOST: "FAKE-IT"
./build.sh: [XX-XXX-XXXX XX:XX:XX] NOTICE: fpm is running, pid 1
./build.sh: [XX-XXX-XXXX XX:XX:XX] NOTICE: ready to handle connections
./build.sh: done build "build" (id: 81dc410de4a3); entrypoint: /usr/local/bin/app-docker-php-entrypoint
...
./test.sh: initializing and running 5 test/s in "build"...

[1/5] php -i (non-interactive)
+ docker exec build php -i
  [1] DD_AGENT_HOST => FAKE-IT
  [1] $_SERVER['DD_AGENT_HOST'] => FAKE-IT
  [1] $_ENV['DD_AGENT_HOST'] => FAKE-IT

[2/5] php -i (non-interactive, within PHP itself)
+ docker exec build php -r 'passthru("php -i");'
  [2] DD_AGENT_HOST => FAKE-IT
  ...

[3/5] /bin/sh (non-interactive)
+ docker exec build /bin/sh -c 'php -i'
  [3] DD_AGENT_HOST => FAKE-IT
  ...

[4/5] /bin/sh (interactive; no tty)
+ docker exec build /bin/sh -ic 'php -i'
/bin/sh: 0: can't access tty; job control turned off
  [4] DD_AGENT_HOST => FAKE-IT
  ...

[5/5] /bin/sh (interactive)
+ docker exec -t build /bin/sh -ic 'php -i'
  [5] DD_AGENT_HOST => FAKE-IT
  ...

As the build tests show, with the modified entrypoint the parameter is available within the PHP CLI SAPI in the diverse forms of invocations available with docker-exec(1).

The value FAKE-IT is just a test value for the parameter to show it's a solution independent to Amazon Web Services and relies on standard Debian GNU Linux.

¹ ^{Cf. [Example #1 Passing environment variables and PHP settings to a pool][¹]}

² ^{docker build as docker buildx version: github.com/docker/buildx v0.11.2 9872040}

³ ^{Cf. [build.sh:L35-L66 Docker full ENTRYPOINT production in Dockerfile][³]}

[¹]: https://www.php.net/manual/en/install.fpm.configuration.php#example-24 "Passing environment variables and PHP settings to a pool in List of pool directives – Example in: Configuration – FastCGI Process Manager (FPM) – Installation and Configuration – PHP Manual"

[²]: https://localhost/ "docker build as docker buildx version: github.com/docker/buildx v0.11.2 9872040"

[³]: https://github.com/hakre/so76831368/blob/master/build.sh#L35-L66 "Docker full ENTRYPOINT production in Dockerfile"

> However, when I log into this container after it starts en check the environment variables with php -i, the variable is not in there.

But... when you run php -i (or any PHP script) directly from the command line, you are invoking the PHP CLI (Command Line Interface) binary, not the PHP-FPM process. Environment variables set in php-fpm configurations (like /usr/local/etc/php-fpm.d/www.conf) will not necessarily be available in this context.

If you are setting environment variables specifically for PHP-FPM, you will need to test them in the context of a PHP script that is being served by PHP-FPM and accessed via a web request.

So, to truly check if DD_AGENT_HOST is available to PHP-FPM, create a PHP script like this:

<?php

$envName = 'DD_AGENT_HOST';
$envValue = getenv($envName);

echo "Environment Variable Name: {$envName}\n";
echo "Value: '{$envValue}'\n";

if (empty($envValue)) {
    echo "The value appears to be empty or not set.";
} else {
    echo "The value is set and not empty.";
}
?>

Then serve it using PHP-FPM and your web server (Nginx, Apache, etc.).
And Access it through a web request.

That will give you a definitive answer whether the environment variable is available in the PHP-FPM context.

>However when I log into this container after it starts en check the environment variables

well since it's a separate login session, with separate enviorment variables from whatever session is running your entrypoint.sh, maybe adding DD_AGENT_HOST to /etc/profile.d would fix it?

#!/bin/sh

export TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
export DD_AGENT_HOST=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/local-ipv4)
echo "export DD_AGENT_HOST=$DD_AGENT_HOST" > /etc/profile.d/dd_agent_host.sh

echo "Datadog agent host: ${DD_AGENT_HOST}"

php-fpm -F