Nginx负载均衡配置

huangapple
go评论97阅读模式
英文:

Nginx loadbalancing configuration

问题

我有一个位于CentOS 9上的nginx.config文件，我想在其中配置两个upstream http块之间的负载均衡。问题出在位置。负载均衡在"presto_resource_managers"上完美执行，但在"presto_coordinators"上没有执行。

nginx版本：nginx/1.22.1

尝试1：

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  6. # 加载动态模块。请参考 /usr/share/doc/nginx/README.dynamic。
  7. include /usr/share/nginx/modules/*.conf;
  9. events {
  10.     worker_connections 1024;
  11. }
  13. http {
  14.     upstream presto_coordinators {
  15.         server 192.168.33.10:8080;
  16.         server 192.168.33.11:8080;
  17.     }
  19.     upstream presto_resource_managers {
  20.         server 192.168.33.10:8083;
  21.         server 192.168.33.11:8083;
  22.     }
  24.     server {
  25.         listen 8085;
  26.         location /coordinators/ { ## 我尝试过不使用斜杠和各种组合。
  27.             proxy_pass http://presto_coordinators;
  28.         }
  29.         location / {
  30.             proxy_pass http://presto_resource_managers;
  31.         }
  32.     }
  34.     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  35.                       '$status $body_bytes_sent "$http_referer" '
  36.                       '"$http_user_agent" "$http_x_forwarded_for"';
  38.     access_log  /var/log/nginx/access.log  main;
  40.     sendfile            on;
  41.     tcp_nopush          on;
  42.     keepalive_timeout   65;
  43.     types_hash_max_size 4096;
  45.     include             /etc/nginx/mime.types;
  46.     default_type        application/octet-stream;
  48.     # 从 /etc/nginx/conf.d 目录加载模块化配置文件。
  49.     # 更多信息，请参考 http://nginx.org/en/docs/ngx_core_module.html#include
  50.     include /etc/nginx/conf.d/*.conf;
  52.     server {
  53.         listen       80;
  54.         listen       [::]:80;
  55.         server_name  _;
  56.         root         /usr/share/nginx/html;
  58.         # 为默认服务器块加载配置文件。
  59.         include /etc/nginx/default.d/*.conf;
  61.         error_page 404 /404.html;
  62.         location = /404.html {
  63.         }
  65.         error_page 500 502 503 504 /50x.html;
  66.         location = /50x.html {
  67.         }
  68.     }
  69. }

尝试2：

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  6. events {
  7.     worker_connections 1024;
  8. }
  10. http {
  11.     upstream presto_coordinators {
  12.         server 192.168.33.10:8080;
  13.         server 192.168.33.11:8080;
  14.     }
  16.     upstream presto_resource_managers {
  17.         server 192.168.33.10:8083;
  18.         server 192.168.33.11:8083;
  19.     }
  21.     server {
  22.         listen 80 default_server;
  23.         listen [::]:80 default_server;
  24.         server_name _;
  26.         location /coordinators {
  27.             proxy_pass http://presto_coordinators;
  28.         }
  30.         location / {
  31.             proxy_pass http://presto_resource_managers;
  32.         }
  33.     }
  35.     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  36.                       '$status $body_bytes_sent "$http_referer" '
  37.                       '"$http_user_agent" "$http_x_forwarded_for"';
  39.     access_log  /var/log/nginx/access.log  main;
  41.     sendfile            on;
  42.     tcp_nopush          on;
  43.     keepalive_timeout   65;
  44.     types_hash_max_size 4096;
  46.     include             /etc/nginx/mime.types;
  47.     default_type        application/octet-stream;
  49.     include /etc/nginx/conf.d/*.conf;
  50. }

尝试3：

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  6. events {
  7.     worker_connections 1024;
  8. }
  10. http {
  11.     upstream presto_coordinators {
  12.         server 192.168.33.10:8080;
  13.         server 192.168.33.11:8080;
  14.     }
  16.     upstream presto_resource_managers {
  17.         server 192.168.33.10:8083;
  18.         server 192.168.33.11:8083;
  19.     }
  21.     server {
  22.         listen 80 default_server;
  23.         listen [::]:80 default_server;
  24.         server_name _;
  26.         location /coordinators {
  27.             proxy_pass http://presto_coordinators;
  28.             proxy_set_header Host $host;
  29.             proxy_set_header X-Real-IP $remote_addr;
  30.         }
  31.         location / {
  32.             proxy_pass http://presto_resource_managers;
  33.             proxy_set_header Host $host;
  34.             proxy_set_header X-Real-IP $remote_addr;
  35.         }
  36.     }
  38.     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  39.                       '$status $body_bytes_sent "$http_referer" '
  40.                       '"$http_user_agent" "$http_x_forwarded_for"';
  42.     access_log  /var/log/nginx/access.log  main;
  44.     sendfile            on;
  45.     tcp_nopush          on;
  46.     keepalive_timeout   65;
  47. }

这些尝试中没有一个对协调器的负载均衡起作用。

对于熟悉Presto的人，这是协调器的配置文件：

  1. coordinator=true
  2. node-scheduler.include-coordinator=false
  3. http-server.http.port=8080
  4. query.max-memory=50GB
  5. query.max-memory-per-node=1GB
  6. discovery.uri=http://localhost:8085
  7. resource-manager-enabled=true

discovery.uri指向负载均衡器的presto_resource_manager，以便协调器可以被宣布给资源管理器。节点本身（服务器）暴露给192.168.33.10:8080，以便负载均衡器可以获取它。

英文:

I have an nginx.config file on CentOS 9 in which i want to configure loadbalancing between two upstream http blocks. The problem is the location. Loadbalancing is being executed perfectly on the "presto_resource_managers", but not the presto_coordinators as well.
Ive tried multiple configurations.

nginx version: nginx/1.22.1

Attempt 1:

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  5. # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
  6. include /usr/share/nginx/modules/*.conf;
  7. events {
  8. worker_connections 1024;
  9. }
  10. http {
  11. upstream presto_coordinators {
  12. server 192.168.33.10:8080;
  13. server 192.168.33.11:8080;
  14. }
  15. upstream presto_resource_managers {
  16. server 192.168.33.10:8083;
  17. server 192.168.33.11:8083;
  18. }
  19. server {
  20. listen 8085;
  21. location /coordinators/ { ## I have tried without the slashes and combinations.
  22. proxy_pass http://presto_coordinators;
  23. }
  24. location / {
  25. proxy_pass http://presto_resource_managers;
  26. }
  27. }
  28. log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  29. '$status $body_bytes_sent "$http_referer" '
  30. '"$http_user_agent" "$http_x_forwarded_for"';
  31. access_log  /var/log/nginx/access.log  main;
  32. sendfile            on;
  33. tcp_nopush          on;
  34. keepalive_timeout   65;
  35. types_hash_max_size 4096;
  36. include             /etc/nginx/mime.types;
  37. default_type        application/octet-stream;
  38. # Load modular configuration files from the /etc/nginx/conf.d directory.
  39. # See http://nginx.org/en/docs/ngx_core_module.html#include
  40. # for more information.
  41. include /etc/nginx/conf.d/*.conf;
  42. server {
  43. listen       80;
  44. listen       [::]:80;
  45. server_name  _;
  46. root         /usr/share/nginx/html;
  47. # Load configuration files for the default server block.
  48. include /etc/nginx/default.d/*.conf;
  49. error_page 404 /404.html;
  50. location = /404.html {
  51. }
  52. error_page 500 502 503 504 /50x.html;
  53. location = /50x.html {
  54. }
  55. }
  56. # Settings for a TLS enabled server.
  57. #    server {
  58. #        listen       443 ssl http2;
  59. #        listen       [::]:443 ssl http2;
  60. #        server_name  _;
  61. #        root         /usr/share/nginx/html;
  62. #
  63. #        ssl_certificate "/etc/pki/nginx/server.crt";
  64. #        ssl_certificate_key "/etc/pki/nginx/private/server.key";
  65. #        ssl_session_cache shared:SSL:1m;
  66. #        ssl_session_timeout  10m;
  67. #        ssl_ciphers PROFILE=SYSTEM;
  68. #        ssl_prefer_server_ciphers on;
  69. #
  70. #        # Load configuration files for the default server block.
  71. #        include /etc/nginx/default.d/*.conf;
  72. #
  73. #        error_page 404 /404.html;
  74. #        location = /404.html {
  75. #        }
  76. #
  77. #        error_page 500 502 503 504 /50x.html;
  78. #        location = /50x.html {
  79. #        }
  80. #    }
  81. }

Attempt 2:

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  5. events {
  6. worker_connections 1024;
  7. }
  8. http {
  9. upstream presto_coordinators {
  10. server 192.168.33.10:8080;
  11. server 192.168.33.11:8080;
  12. }
  13. upstream presto_resource_managers {
  14. server 192.168.33.10:8083;
  15. server 192.168.33.11:8083;
  16. }
  17. server {
  18. listen 80 default_server;
  19. listen [::]:80 default_server;
  20. server_name _;
  21. location /coordinators {
  22. proxy_pass http://presto_coordinators;
  23. }
  24. location / {
  25. proxy_pass http://presto_resource_managers;
  26. }
  27. }
  28. log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  29. '$status $body_bytes_sent "$http_referer" '
  30. '"$http_user_agent" "$http_x_forwarded_for"';
  31. access_log  /var/log/nginx/access.log  main;
  32. sendfile            on;
  33. tcp_nopush          on;
  34. keepalive_timeout   65;
  35. types_hash_max_size 4096;
  36. include             /etc/nginx/mime.types;
  37. default_type        application/octet-stream;
  38. include /etc/nginx/conf.d/*.conf;
  39. }

Attempt 3:

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  5. events {
  6. worker_connections 1024;
  7. }
  8. http {
  9. upstream presto_coordinators {
  10. server 192.168.33.10:8080;
  11. server 192.168.33.11:8080;
  12. }
  13. upstream presto_resource_managers {
  14. server 192.168.33.10:8083;
  15. server 192.168.33.11:8083;
  16. }
  17. server {
  18. listen 80 default_server;
  19. listen [::]:80 default_server;
  20. server_name _;
  21. location /coordinators {
  22. proxy_pass http://presto_coordinators;
  23. proxy_set_header Host $host;
  24. proxy_set_header X-Real-IP $remote_addr;
  25. }
  26. location / {
  27. proxy_pass http://presto_resource_managers;
  28. proxy_set_header Host $host;
  29. proxy_set_header X-Real-IP $remote_addr;
  30. }
  31. }
  32. log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  33. '$status $body_bytes_sent "$http_referer" '
  34. '"$http_user_agent" "$http_x_forwarded_for"';
  35. access_log  /var/log/nginx/access.log  main;
  36. sendfile            on;
  37. tcp_nopush          on;
  38. keepalive_timeout   65;

None of these attempts worked for the coordinators's loadbalancing.

For those who are familiar with presto, this is the coordinator's config file:

  1. coordinator=true
  2. node-scheduler.include-coordinator=false
  3. http-server.http.port=8080
  4. query.max-memory=50GB
  5. query.max-memory-per-node=1GB
  6. discovery.uri=http://localhost:8085
  7. resource-manager-enabled=true

The discovery.uri points to the Loadbalancer's presto_resource_manager, so that the coordinator can be announced to the ResourceManagers. The node itself (the server) is exposed to 192.168.33.10:8080 so it can be fetched by the load balancer.

答案1

得分: 0

所以，我找到了解决方案。这是我的文件现在的样子：

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  6. # 加载动态模块。参见 /usr/share/doc/nginx/README.dynamic。
  7. include /usr/share/nginx/modules/*.conf;
  9. events {
  10.     worker_connections 1024;
  11. }
  13. http {
  14.     upstream presto_coordinators {
  15.         server 192.168.33.10:8080;
  16.         server 192.168.33.11:8080;
  17.     }
  19.     upstream presto_resource_managers {
  20.         server 192.168.33.10:8083;
  21.         server 192.168.33.11:8083;
  22.     }
  24.     server {
  25.         listen 8085;
  26.         location /coordinators/ {
  27.             proxy_pass http://presto_coordinators/;
  28.         }
  29.         location / {
  30.             proxy_pass http://presto_resource_managers;
  31.         }
  32.     }
  34.     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  35.                       '$status $body_bytes_sent "$http_referer" '
  36.                       '"$http_user_agent" "$http_x_forwarded_for"';
  38.     access_log  /var/log/nginx/access.log  main;
  40.     sendfile            on;
  41.     tcp_nopush          on;
  42.     keepalive_timeout   65;
  43.     types_hash_max_size 4096;
  45.     include             /etc/nginx/mime.types;
  46.     default_type        application/octet-stream;
  48.     # 从 /etc/nginx/conf.d 目录加载模块化配置文件。
  49.     # 更多信息请参见 http://nginx.org/en/docs/ngx_core_module.html#include
  50.     include /etc/nginx/conf.d/*.conf;
  52.     server {
  53.         listen       80;
  54.         listen       [::]:80;
  55.         server_name  _;
  56.         root         /usr/share/nginx/html;
  58.         # 加载默认服务器块的配置文件。
  59.         include /etc/nginx/default.d/*.conf;
  61.         error_page 404 /404.html;
  62.         location = /404.html {
  63.         }
  65.         error_page 500 502 503 504 /50x.html;
  66.         location = /50x.html {
  67.         }
  68.     }
  70.     # 启用 TLS 的服务器设置。
  71.     #    server {
  72.     #        listen       443 ssl http2;
  73.     #        listen       [::]:443 ssl http2;
  74.     #        server_name  _;
  75.     #        root         /usr/share/nginx/html;
  76.     #
  77.     #        ssl_certificate "/etc/pki/nginx/server.crt";
  78.     #        ssl_certificate_key "/etc/pki/nginx/private/server.key";
  79.     #        ssl_session_cache shared:SSL:1m;
  80.     #        ssl_session_timeout  10m;
  81.     #        ssl_ciphers PROFILE=SYSTEM;
  82.     #        ssl_prefer_server_ciphers on;
  83.     #
  84.     #        # 加载默认服务器块的配置文件。
  85.     #        include /etc/nginx/default.d/*.conf;
  86.     #
  87.     #        error_page 404 /404.html;
  88.     #        location = /404.html {
  89.     #        }
  90.     #
  91.     #        error_page 500 502 503 504 /50x.html;
  92.     #        location = /50x.html {
  93.     #        }
  94.     #    }
  95. }
英文:

So, I found the solution. It was a syntax error
This is how my file looks like now:

  1. user nginx;
  2. worker_processes auto;
  3. error_log /var/log/nginx/error.log notice;
  4. pid /run/nginx.pid;
  5. # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
  6. include /usr/share/nginx/modules/*.conf;
  7. events {
  8. worker_connections 1024;
  9. }
  10. http {
  11. upstream presto_coordinators {
  12. server 192.168.33.10:8080;
  13. server 192.168.33.11:8080;
  14. }
  15. upstream presto_resource_managers {
  16. server 192.168.33.10:8083;
  17. server 192.168.33.11:8083;
  18. }
  19. server {
  20. listen 8085;
  21. location /coordinators/ {
  22. proxy_pass http://presto_coordinators/;
  23. }
  24. location / {
  25. proxy_pass http://presto_resource_managers;
  26. }
  27. }
  28. log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
  29. '$status $body_bytes_sent "$http_referer" '
  30. '"$http_user_agent" "$http_x_forwarded_for"';
  31. access_log  /var/log/nginx/access.log  main;
  32. sendfile            on;
  33. tcp_nopush          on;
  34. keepalive_timeout   65;
  35. types_hash_max_size 4096;
  36. include             /etc/nginx/mime.types;
  37. default_type        application/octet-stream;
  38. # Load modular configuration files from the /etc/nginx/conf.d directory.
  39. # See http://nginx.org/en/docs/ngx_core_module.html#include
  40. # for more information.
  41. include /etc/nginx/conf.d/*.conf;
  42. server {
  43. listen       80;
  44. listen       [::]:80;
  45. server_name  _;
  46. root         /usr/share/nginx/html;
  47. # Load configuration files for the default server block.
  48. include /etc/nginx/default.d/*.conf;
  49. error_page 404 /404.html;
  50. location = /404.html {
  51. }
  52. error_page 500 502 503 504 /50x.html;
  53. location = /50x.html {
  54. }
  55. }
  56. # Settings for a TLS enabled server.
  57. #    server {
  58. #        listen       443 ssl http2;
  59. #        listen       [::]:443 ssl http2;
  60. #        server_name  _;
  61. #        root         /usr/share/nginx/html;
  62. #
  63. #        ssl_certificate "/etc/pki/nginx/server.crt";
  64. #        ssl_certificate_key "/etc/pki/nginx/private/server.key";
  65. #        ssl_session_cache shared:SSL:1m;
  66. #        ssl_session_timeout  10m;
  67. #        ssl_ciphers PROFILE=SYSTEM;
  68. #        ssl_prefer_server_ciphers on;
  69. #
  70. #        # Load configuration files for the default server block.
  71. #        include /etc/nginx/default.d/*.conf;
  72. #
  73. #        error_page 404 /404.html;
  74. #        location = /404.html {
  75. #        }
  76. #
  77. #        error_page 500 502 503 504 /50x.html;
  78. #        location = /50x.html {
  79. #        }
  80. #    }
  81. }

huangapple
  • 本文由 发表于 2023年6月5日 19:49:55
  • 转载请务必保留本文链接：https://go.coder-hub.com/76406131.html
  • centos
  • nginx
  • nginx-config
  • nginx-reverse-proxy
  • presto
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定