How do I target endpoints with specific ports in prometheus

I'm trying to configure prometheus to target kubernetes pods in a namespace, but only port 8081, even though the pods expose both 8080 and 8081. With this configuration:

      - job_name: 'api'
        metrics_path: "/actuator/prometheus"
        kubernetes_sd_configs:
          - role: endpoints
            namespaces:
              names:
                - "api"

I get two targets for each pod - one for each port. I can't figure out how to narrow down the targets further to just the 8081 ports. Thanks for any help with this!

You can use the regex in kubernete_sd_configs to match the port and endpoint you are trying to target.
Example :

- job_name: my_job
  metrics_path: /metrics
  kubernetes_sd_configs:
   - roles: endpoints
     namespaces:
      name:
       - api
  relable_configs:
   - source_labels: [_meta_kubernetes_service_name,_meta_kubernetes_endpoint_port_name]
     regex: my_service;8081

Here we are using regex to match the endpoints with the service name called my_service and port number 8081. If you need to scrape the metrics from different endpoints then you need to modify the regular expression accordingly.

Make sure you mentioned appropriate service discovery annotations on your kubernetes services to ensure the prometheus can discover them using kubernetes_sd_configs.

For more information have a glance at this blog written by Lou Marvin Caraig.

Check this prometheus forum discussion for more inputs.