Dockerized Elasticsearch: How to configure `reindex.remote.whitelist` elastic.yml with variables inside .env file

First I should say I have searched a lot and read many things about the problem but non worked for me. I need to reindex an index from a remote server as follows

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://THE.GIVEN.IP.ADDRESS:9200"
    },
    "index": "TEST"
  },
  "dest": {
    "index": "TEST"
  }
}

But I get the error

[THE.GIVEN.IP.ADDRESS:9200] not whitelisted in reindex.remote.whitelist

I need to set reindex.remote.whitelist property in elasticsearch.yml of an ES instance running via docker. This is the elastic search service of the docker file

  elasticSearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.0
    container_name: elasticSearch
    environment:
      - discovery.type=single-node
      - reindex.remote.whitelist=${REINDEX_REMOTE_WHITELIST}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elasticdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - backend

The REINDEX_REMOTE_WHITELIST is set inside a .env file and the command

> docker compose config

gives me this variable correctly. However it is not working and the reindex command reports error

[THE.GIVEN.IP.ADDRESS:9200] not whitelisted in reindex.remote.whitelist

After googling, I find out I need to include the URL in reindex.remote.whitelist property of elasticsearch.yml file that is located in /usr/share/elasticsearch/config directory of the container. To do so I modified the volume of docker-compose file as follows

  elasticSearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.9.0
    container_name: elasticSearch
    environment:
      - discovery.type=single-node
      - reindex.remote.whitelist=${REINDEX_REMOTE_WHITELIST}
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - elasticdata:/usr/share/elasticsearch/data
      - ./docker/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
     ports:
      - 9200:9200
    networks:
      - backend

and added ./docker/elasticsearch/elasticsearch.yml file with the following content

cluster.name: "docker-cluster"
network.host: 0.0.0.0
reindex.remote.whitelist: "http://THE.GIVEN.IP.ADDRESS:9200"

restarted the container, however, it still not working. and I get the same error. I have two things need to solve:

1. How to successfully reindex the data?
2. How to read the remove IP address from the .env file? Is this a good practice to hard the IP inside the elasticsearch.yml of the project directory?

Thanks in advance.

you should add only hostname without "http" in your docker-compose settings. Here is my config:

<pre>
version: '3'

volumes:
elasticsearch_data: {}

services:
elasticsearch:
restart: always
image: elasticsearch:7.17.10
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms4096m -Xmx4096m"
- reindex.remote.whitelist=my_remote_hostname:443
volumes:
- elasticsearch_data:/usr/share/elasticsearch/data
ports:
- "9200:9200"
</pre>