MongoDB in Azure App Service, via Docker-Compose

I'll keep this very simple as I've gone back to basics for this - my container configuration is a little more complex, but this demonstrates the problem really quickly.

I have the following docker-compose for Azure App Service:

version: '3.4'
services:


  mongo:
    image: myrepo/myrepo:mongo-1
    restart: always
    volumes:
      - mongo:/var/mydata
    environment:
      MONGO_INITDB_ROOT_USERNAME: xxx
      MONGO_INITDB_ROOT_PASSWORD: xxx
    ports:
      - "27017:27107"
    networks:
      - app-network

volumes:
  mongo:
    driver: azure_file
    driver_opts:
      share_name: mongo
      storage_account_name: xxx
      storage_account_key: xxx

networks:
  app-network:

Nothing fancy. I've then created a custom Mongo image as follows:

FROM mongo
EXPOSE 27017
 
ENTRYPOINT ["mongod", "--dbpath=/var/mydata", "--bind_ip_all"]

It all seems to be accepted. I've read elsewhere that it's a bad idea to mount to /data/db as it causes issues - so I've stuck to those guidelines.

The Azure File Share seems to be configured correctly. However, when I start the container I get repeated error messages during Mongo startup that it cannot open WiredTiger.wt - and attempts to create new ones repeatedly. I can start with a completely empty file share - and it still does this. Files ARE being created by Mongo in the share.

I'm at my wits end as I've tried loads of suggestions (too many to mention here), but I cannot figured this out at all.

Anything I could be missing would be greatly received. File permissions seems to be fine as I can touch and/or remove a file from the share no problem - it just seems to be Mongo with the issue.

Many thanks.

Update

Here's some of the output from the container when it runs - you can see it's tried a few times with the WiredTiger file:

2023-02-26T20:41:30.809321991Z {"t":{"$date":"2023-02-26T20:41:30.809+00:00"},"s":"I",  "c":"WT",       "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":{"ts_sec":1677444090,"ts_usec":809021,"thread":"14:0x7dfcbbc19cc0","session_name":"connection","category":"WT_VERB_BLOCK","category_id":3,"verbose_level":"NOTICE","verbose_level_id":-1,"msg":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.7"}}}
2023-02-26T20:41:30.840231616Z {"t":{"$date":"2023-02-26T20:41:30.839+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":1,"message":{"ts_sec":1677444090,"ts_usec":839910,"thread":"14:0x7dfcbbc19cc0","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/var/mydata/WiredTiger.wt: handle-open: open","error_str":"Operation not permitted","error_code":1}}}
2023-02-26T20:41:31.002537945Z {"t":{"$date":"2023-02-26T20:41:31.002+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":17,"message":{"ts_sec":1677444091,"ts_usec":2211,"thread":"14:0x7dfcbbc19cc0","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/var/mydata/WiredTiger.wt: handle-open: open","error_str":"File exists","error_code":17}}}
2023-02-26T20:41:31.050284867Z {"t":{"$date":"2023-02-26T20:41:31.050+00:00"},"s":"I",  "c":"WT",       "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":{"ts_sec":1677444091,"ts_usec":50047,"thread":"14:0x7dfcbbc19cc0","session_name":"connection","category":"WT_VERB_BLOCK","category_id":3,"verbose_level":"NOTICE","verbose_level_id":-1,"msg":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.8"}}}
2023-02-26T20:41:31.080410844Z {"t":{"$date":"2023-02-26T20:41:31.080+00:00"},"s":"E",  "c":"WT",       "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":1,"message":{"ts_sec":1677444091,"ts_usec":80113,"thread":"14:0x7dfcbbc19cc0","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":9,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__posix_open_file:805:/var/mydata/WiredTiger.wt: handle-open: open","error_str":"Operation not permitted","error_code":1}}}
2023-02-26T20:41:31.097099487Z {"t":{"$date":"2023-02-26T20:41:31.096+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
2023-02-26T20:41:31.097778289Z {"t":{"$date":"2023-02-26T20:41:31.097+00:00"},"s":"F",  "c":"STORAGE",  "id":28595,   "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"1: Operation not permitted"}}
2023-02-26T20:41:31.098377890Z {"t":{"$date":"2023-02-26T20:41:31.097+00:00"},"s":"F",  "c":"ASSERT",   "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":708}}

It's almost as if there's some kind of permissions issue perhaps? I'm obviously using the correct file share key etc. otherwise my test command wouldn't work. I've tried chown/chgrp to root just in case - makes no difference (not that I though it would) - using the following:

CMD [ "sh", "-c", "ls -l /var; touch /var/mydata/test.txt; rm /var/mydata/test.txt; chmod -R 777 /var/mydata; chown -R root /var/mydata; chgrp root /var/mydata; mongod --dbpath=/var/mydata --bind_ip_all" ]

Just to reiterate - it doesn't matter if the Azure File Share is empty or not - it will always keep retrying the WiredTiger file around 8 or so times until the container gives up and terminates.

Update #2

I also tried creating a custom image, copying a Mongo configuration file containing the db path etc. rather than using commandline arguments or environment variables for docker-compose - still no joy.

I experienced the same issue as you when trying to set up persistent storage using an Azure storage account and have finally found a solution.

It seems that Azure File Shares are incompatible with MongoDB's underlying storage driver, WiredTiger. I do not know the exact reason why, but I was also unable to solve it by changing permissions, creating a custom image, etc.

The solution is to use Azure Cosmos DB for MongoDB. The MongoDB API in Cosmos allows you to take advantage of existing Mongo libraries, so you should be able to use it with minimal code changes. All I had to do was replace my mongo connection string with the connection string provided by Cosmos.

Here are the docs to get you started: https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/introduction

I hope this helps, and good luck!