2023年6月5日 19:49:55go评论63阅读模式

英文:

Nginx loadbalancing configuration

问题

我有一个位于CentOS 9上的nginx.config文件，我想在其中配置两个upstream http块之间的负载均衡。问题出在位置。负载均衡在"presto_resource_managers"上完美执行，但在"presto_coordinators"上没有执行。

nginx版本：nginx/1.22.1

尝试1：

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;

# 加载动态模块。请参考 /usr/share/doc/nginx/README.dynamic。
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    upstream presto_coordinators {
        server 192.168.33.10:8080;
        server 192.168.33.11:8080;
    }

    upstream presto_resource_managers {
        server 192.168.33.10:8083;
        server 192.168.33.11:8083;
    }

    server {
        listen 8085;
        location /coordinators/ { ## 我尝试过不使用斜杠和各种组合。
            proxy_pass http://presto_coordinators;
        }
        location / {
            proxy_pass http://presto_resource_managers;
        }
    }

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # 从 /etc/nginx/conf.d 目录加载模块化配置文件。
    # 更多信息，请参考 http://nginx.org/en/docs/ngx_core_module.html#include
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80;
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # 为默认服务器块加载配置文件。
        include /etc/nginx/default.d/*.conf;

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }
}

尝试2：

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    upstream presto_coordinators {
        server 192.168.33.10:8080;
        server 192.168.33.11:8080;
    }

    upstream presto_resource_managers {
        server 192.168.33.10:8083;
        server 192.168.33.11:8083;
    }

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;

        location /coordinators {
            proxy_pass http://presto_coordinators;
        }

        location / {
            proxy_pass http://presto_resource_managers;
        }
    }

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    include /etc/nginx/conf.d/*.conf;
}

尝试3：

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;

events {
    worker_connections 1024;
}

http {
    upstream presto_coordinators {
        server 192.168.33.10:8080;
        server 192.168.33.11:8080;
    }

    upstream presto_resource_managers {
        server 192.168.33.10:8083;
        server 192.168.33.11:8083;
    }

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;

        location /coordinators {
            proxy_pass http://presto_coordinators;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
        location / {
            proxy_pass http://presto_resource_managers;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   65;
}

这些尝试中没有一个对协调器的负载均衡起作用。

对于熟悉Presto的人，这是协调器的配置文件：

coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=50GB
query.max-memory-per-node=1GB
discovery.uri=http://localhost:8085
resource-manager-enabled=true

discovery.uri指向负载均衡器的presto_resource_manager，以便协调器可以被宣布给资源管理器。节点本身（服务器）暴露给192.168.33.10:8080，以便负载均衡器可以获取它。

英文:

I have an nginx.config file on CentOS 9 in which i want to configure loadbalancing between two upstream http blocks. The problem is the location. Loadbalancing is being executed perfectly on the "presto_resource_managers", but not the presto_coordinators as well.
Ive tried multiple configurations.

nginx version: nginx/1.22.1

Attempt 1:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
upstream presto_coordinators {
server 192.168.33.10:8080;
server 192.168.33.11:8080;
}
upstream presto_resource_managers {
server 192.168.33.10:8083;
server 192.168.33.11:8083;
}
server {
listen 8085;
location /coordinators/ { ## I have tried without the slashes and combinations.
proxy_pass http://presto_coordinators;
}
location / {
proxy_pass http://presto_resource_managers;
}
}
log_format  main  &#39;$remote_addr - $remote_user [$time_local] &quot;$request&quot; &#39;
&#39;$status $body_bytes_sent &quot;$http_referer&quot; &#39;
&#39;&quot;$http_user_agent&quot; &quot;$http_x_forwarded_for&quot;&#39;;
access_log  /var/log/nginx/access.log  main;
sendfile            on;
tcp_nopush          on;
keepalive_timeout   65;
types_hash_max_size 4096;
include             /etc/nginx/mime.types;
default_type        application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen       80;
listen       [::]:80;
server_name  _;
root         /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
error_page 404 /404.html;
location = /404.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
# Settings for a TLS enabled server.
#    server {
#        listen       443 ssl http2;
#        listen       [::]:443 ssl http2;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate &quot;/etc/pki/nginx/server.crt&quot;;
#        ssl_certificate_key &quot;/etc/pki/nginx/private/server.key&quot;;
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers PROFILE=SYSTEM;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        error_page 404 /404.html;
#        location = /404.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#        location = /50x.html {
#        }
#    }
}

Attempt 2:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
upstream presto_coordinators {
server 192.168.33.10:8080;
server 192.168.33.11:8080;
}
upstream presto_resource_managers {
server 192.168.33.10:8083;
server 192.168.33.11:8083;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
location /coordinators {
proxy_pass http://presto_coordinators;
}
location / {
proxy_pass http://presto_resource_managers;
}
}
log_format  main  &#39;$remote_addr - $remote_user [$time_local] &quot;$request&quot; &#39;
&#39;$status $body_bytes_sent &quot;$http_referer&quot; &#39;
&#39;&quot;$http_user_agent&quot; &quot;$http_x_forwarded_for&quot;&#39;;
access_log  /var/log/nginx/access.log  main;
sendfile            on;
tcp_nopush          on;
keepalive_timeout   65;
types_hash_max_size 4096;
include             /etc/nginx/mime.types;
default_type        application/octet-stream;
include /etc/nginx/conf.d/*.conf;
}

Attempt 3:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
upstream presto_coordinators {
server 192.168.33.10:8080;
server 192.168.33.11:8080;
}
upstream presto_resource_managers {
server 192.168.33.10:8083;
server 192.168.33.11:8083;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
location /coordinators {
proxy_pass http://presto_coordinators;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
proxy_pass http://presto_resource_managers;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
log_format  main  &#39;$remote_addr - $remote_user [$time_local] &quot;$request&quot; &#39;
&#39;$status $body_bytes_sent &quot;$http_referer&quot; &#39;
&#39;&quot;$http_user_agent&quot; &quot;$http_x_forwarded_for&quot;&#39;;
access_log  /var/log/nginx/access.log  main;
sendfile            on;
tcp_nopush          on;
keepalive_timeout   65;

None of these attempts worked for the coordinators's loadbalancing.

For those who are familiar with presto, this is the coordinator's config file:

coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=50GB
query.max-memory-per-node=1GB
discovery.uri=http://localhost:8085
resource-manager-enabled=true

The discovery.uri points to the Loadbalancer's presto_resource_manager, so that the coordinator can be announced to the ResourceManagers. The node itself (the server) is exposed to 192.168.33.10:8080 so it can be fetched by the load balancer.

答案1

得分: 0

所以，我找到了解决方案。这是我的文件现在的样子：

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;

# 加载动态模块。参见 /usr/share/doc/nginx/README.dynamic。
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 1024;
}

http {
    upstream presto_coordinators {
        server 192.168.33.10:8080;
        server 192.168.33.11:8080;
    }

    upstream presto_resource_managers {
        server 192.168.33.10:8083;
        server 192.168.33.11:8083;
    }

    server {
        listen 8085;
        location /coordinators/ {
            proxy_pass http://presto_coordinators/;
        }
        location / {
            proxy_pass http://presto_resource_managers;
        }
    }

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # 从 /etc/nginx/conf.d 目录加载模块化配置文件。
    # 更多信息请参见 http://nginx.org/en/docs/ngx_core_module.html#include
    include /etc/nginx/conf.d/*.conf;

    server {
        listen       80;
        listen       [::]:80;
        server_name  _;
        root         /usr/share/nginx/html;

        # 加载默认服务器块的配置文件。
        include /etc/nginx/default.d/*.conf;

        error_page 404 /404.html;
        location = /404.html {
        }

        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
        }
    }

    # 启用 TLS 的服务器设置。
    #    server {
    #        listen       443 ssl http2;
    #        listen       [::]:443 ssl http2;
    #        server_name  _;
    #        root         /usr/share/nginx/html;
    #
    #        ssl_certificate "/etc/pki/nginx/server.crt";
    #        ssl_certificate_key "/etc/pki/nginx/private/server.key";
    #        ssl_session_cache shared:SSL:1m;
    #        ssl_session_timeout  10m;
    #        ssl_ciphers PROFILE=SYSTEM;
    #        ssl_prefer_server_ciphers on;
    #
    #        # 加载默认服务器块的配置文件。
    #        include /etc/nginx/default.d/*.conf;
    #
    #        error_page 404 /404.html;
    #        location = /404.html {
    #        }
    #
    #        error_page 500 502 503 504 /50x.html;
    #        location = /50x.html {
    #        }
    #    }
}

英文:

So, I found the solution. It was a syntax error
This is how my file looks like now:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
upstream presto_coordinators {
server 192.168.33.10:8080;
server 192.168.33.11:8080;
}
upstream presto_resource_managers {
server 192.168.33.10:8083;
server 192.168.33.11:8083;
}
server {
listen 8085;
location /coordinators/ {
proxy_pass http://presto_coordinators/;
}
location / {
proxy_pass http://presto_resource_managers;
}
}
log_format  main  &#39;$remote_addr - $remote_user [$time_local] &quot;$request&quot; &#39;
&#39;$status $body_bytes_sent &quot;$http_referer&quot; &#39;
&#39;&quot;$http_user_agent&quot; &quot;$http_x_forwarded_for&quot;&#39;;
access_log  /var/log/nginx/access.log  main;
sendfile            on;
tcp_nopush          on;
keepalive_timeout   65;
types_hash_max_size 4096;
include             /etc/nginx/mime.types;
default_type        application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen       80;
listen       [::]:80;
server_name  _;
root         /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
error_page 404 /404.html;
location = /404.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
# Settings for a TLS enabled server.
#    server {
#        listen       443 ssl http2;
#        listen       [::]:443 ssl http2;
#        server_name  _;
#        root         /usr/share/nginx/html;
#
#        ssl_certificate &quot;/etc/pki/nginx/server.crt&quot;;
#        ssl_certificate_key &quot;/etc/pki/nginx/private/server.key&quot;;
#        ssl_session_cache shared:SSL:1m;
#        ssl_session_timeout  10m;
#        ssl_ciphers PROFILE=SYSTEM;
#        ssl_prefer_server_ciphers on;
#
#        # Load configuration files for the default server block.
#        include /etc/nginx/default.d/*.conf;
#
#        error_page 404 /404.html;
#        location = /404.html {
#        }
#
#        error_page 500 502 503 504 /50x.html;
#        location = /50x.html {
#        }
#    }
}

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

Nginx负载均衡配置

问题

答案1

Docker Laravel: 重定向到 https

我的while read命令没有打开一个while循环。

从确切的数值找过渡 SQL

Golang + nginx + https（安全套接层）

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论