Application restart automation on Linux servers based on condition

We have a requirement to start our JFrog tools services hosted on Linux machines given certain conditions as noted below.

We have JFrog Artifactory and JFrog Xray hosted on separate servers. Both of these applications have databases hosted in a PostgreSQL database server.

We need to ensure that the database server is up and accessible before the two JFrog applications start. The order of operations needs to be that the database server comes online and is accessible, then the Artifactory service needs to be started, and once Artifactory is up, we need to have the Xray service started.

We have tried using a shell script along with an Azure DevOps pipeline, but we got blocked as we cant bring our JFrog production servers up. The Azure DevOps agents are not properly checking the status of the services and running the necessary startup scripts.

We use Ansible Tower in our organization. The question, is this something that Ansible playbooks would handle nicely, or should we be looking at other approaches?

> ... start our ... services hosted ... on certain conditions ...
>
> ... if the DB server is up and the DB is accessible from the App servers ... the Artifactory service need to be started and once Artifactory is up, Xray service need to be started ...
>
> ... is this something that Ansible playbooks would handle nicely ...

Yes, of course, this can simply be achieved by

• Using Inventory basics: formats, hosts, and groups, Organizing the inventory accordingly and group by

• Using Playbook execution

  > Playbooks with multiple ‘plays’ can orchestrate multi-machine deployments, running one play on your webservers, then another play on your database servers, then a third play on your network infrastructure, and so on.

• And using wait_for module – Waits for a condition before continuing for the connection to the the PostgreSQL Database Backed and uri module – Interacts with webservices to wait_for_http for the connection to the JFrog Artifactory REST API

For example

- hosts: artifactory
  become: false
  gather_facts: false

  tasks:

  - name: Test connection
    wait_for:
      host: postgresql.example.com
      port: 5432
      state: drained         # Port should be open
      delay: 0               # No wait before first check (sec)
      timeout: 3             # Stop checking after timeout (sec)
      active_connection_states: SYN_RECV

  - debug:
      msg: "Several other tasks, in example starting Artifactory"

- hosts: xray
  become: false
  gather_facts: false

  tasks:

  - name: Test REST API connection
    URI:
      url: https://repository.example.com/artifactory/system/ping
      method: GET
      status_code: 200
      return_content: true
    register: result
    until: result.status == 200 and result.content == 'OK'
    retries: 10              # Retry n times
    delay: 30                # Pause between each call (sec)

  - debug:
      msg: "Several other tasks ..."

It is recommended to have a look at all examples of wait_for_http and define before which state, endpoint and result of JFrog Artifactory REST API is considered as GOOD/STARTED/UP. So, in example, one could use the endpoint system/ping which just returns an OK for a How to use Artifactory Health Check.

Have you reviewed the documentation at https://jfrog.com/help/r/jfrog-installation-setup-documentation/enable-postgresql-connectivity-from-remote-servers?

If you use the script route, you can use something like pg_isready -h someremotehost to determine status of the db server (see https://www.postgresql.org/docs/current/app-pg-isready.html for more).

To determine when artifactory is up you could curl the endpoint (see https://jfrog.com/knowledge-base/artifactory-how-to-use-artifactory-health-check/ for more.)

Here is one of the provided examples:

curl -uadminuser:Password  -XGET http://localhost:8081/artifactory/api/system/ping -H 'Content-Type: text/plain'

Output Response : OK

There are some potentially helpful hints in this thread https://stackoverflow.com/questions/35069027/docker-wait-for-postgresql-to-be-running