错误套接字:在对Golang中的API进行基准测试时打开的文件太多。

huangapple go评论94阅读模式
英文:

Error socket: too many open files while benchmarking api in golang

问题

我正在构建一个系统,其中我使用rest(HTTP)和gRPC实现了API,主要是使用golang,以便将两种替代方案进行比较研究。但是,当我对rest API进行基准测试时,我遇到了socket: too many open files错误。使用resty htttp客户端:

package httpcardpayment

import (
	"encoding/json"
	"errors"
	"log"
	"net/http"
	"time"

	"github.com/bmviniciuss/gateway/src/core/card_payment"
	"github.com/go-resty/resty/v2"
)

type HttpCardPaymentService struct {
	Client *resty.Client
}

func NewHttpCardPaymentService() *HttpCardPaymentService {
	return &HttpCardPaymentService{
		Client: resty.New().SetTimeout(5 * time.Second),
	}
}

type CardPaymentRequest struct {
	ClientId    string                 `json:"client_id"`
	Amount      float64                `json:"amount"`
	PaymentType string                 `json:"payment_type"`
	PaymentDate string                 `json:"payment_date"`
	PaymentInfo CardPaymentRequestInfo `json:"payment_info"`
}

type CardPaymentRequestInfo struct {
	CardToken string `json:"card_token"`
}

type CardPaymentResult struct {
	Id          string          `json:"id"`
	ClientId    string          `json:"client_id"`
	Amount      float64         `json:"amount"`
	PaymentType string          `json:"payment_type"`
	PaymentDate string          `json:"payment_date"`
	PaymentInfo CardPaymentInfo `json:"payment_info"`
}

type CardPaymentInfo struct {
	MaskedNumber string `json:"masked_number"`
}

func (h *HttpCardPaymentService) CreatePayment(payment *card_payment.CardPayment) error {
	result := &CardPaymentResult{}

	b := &CardPaymentRequest{
		ClientId:    payment.ClientId,
		Amount:      payment.Amount,
		PaymentType: payment.PaymentType,
		PaymentDate: payment.PaymentDate,
		PaymentInfo: CardPaymentRequestInfo{
			CardToken: payment.PaymentInfo.CardToken,
		},
	}

	log.Printf("Calling HTTP Card Payment Microsservice with %+v\n", b)

	res, err := h.Client.R().
		SetHeader("Content-Type", "application/json").
		SetHeader("Connection", "close").
		SetBody(b).
		Post("http://0.0.0.0:5002/api/payment")

	if err != nil {
		log.Println("Error in request to process new card payment", err)
		return errors.New("An error occur while creating the card payment")
	}

	err = json.Unmarshal(res.Body(), &result)

	if err != nil {
		log.Println("Error while parsing request body")
		return errors.New("An error occur while creating the card payment")
	}

	if res.StatusCode() != http.StatusCreated {
		log.Println("The response was not expected", res.StatusCode(), string(res.Body()))
		return errors.New("An error occur while creating the card payment")
	}

	log.Println("Card payment created")
	payment.Id = result.Id
	payment.PaymentInfo.MaskedNumber = result.PaymentInfo.MaskedNumber

	return nil
}

完整代码:https://github.com/bmviniciuss/tcc/tree/bmviniciuss/devs-48
https://github.com/bmviniciuss/tcc/blob/bmviniciuss/devs-48/gateway-go/src/adapters/card_payment/http/htpp_card_payment_service.go

英文:

I'm building a system where I have implemented APIs with rest(HTTP) and gRPC - mostly in golang - in order to compare both alternatives as a study. But, when I'm benchmarking the rest APIs i'm getting a
socket: too many open files error. using the resty htttp client:

错误套接字:在对Golang中的API进行基准测试时打开的文件太多。

package httpcardpayment

import (
	"encoding/json"
	"errors"
	"log"
	"net/http"
	"time"

	"github.com/bmviniciuss/gateway/src/core/card_payment"
	"github.com/go-resty/resty/v2"
)

type HttpCardPaymentService struct {
	Client *resty.Client
}

func NewHttpCardPaymentService() *HttpCardPaymentService {
	return &HttpCardPaymentService{
		Client: resty.New().SetTimeout(5 * time.Second),
	}
}

type CardPaymentRequest struct {
	ClientId    string                 `json:"client_id"`
	Amount      float64                `json:"amount"`
	PaymentType string                 `json:"payment_type"`
	PaymentDate string                 `json:"payment_date"`
	PaymentInfo CardPaymentRequestInfo `json:"payment_info"`
}

type CardPaymentRequestInfo struct {
	CardToken string `json:"card_token"`
}

type CardPaymentResult struct {
	Id          string          `json:"id"`
	ClientId    string          `json:"client_id"`
	Amount      float64         `json:"amount"`
	PaymentType string          `json:"payment_type"`
	PaymentDate string          `json:"payment_date"`
	PaymentInfo CardPaymentInfo `json:"payment_info"`
}

type CardPaymentInfo struct {
	MaskedNumber string `json:"masked_number"`
}

func (h *HttpCardPaymentService) CreatePayment(payment *card_payment.CardPayment) error {
	result := &CardPaymentResult{}

	b := &CardPaymentRequest{
		ClientId:    payment.ClientId,
		Amount:      payment.Amount,
		PaymentType: payment.PaymentType,
		PaymentDate: payment.PaymentDate,
		PaymentInfo: CardPaymentRequestInfo{
			CardToken: payment.PaymentInfo.CardToken,
		},
	}

	log.Printf("Calling HTTP Card Payment Microsservice with %+v\n", b)

	res, err := h.Client.R().
		SetHeader("Content-Type", "application/json").
		SetHeader("Connection", "close").
		SetBody(b).
		Post("http://0.0.0.0:5002/api/payment")

	if err != nil {
		log.Println("Error in request to process new card payment", err)
		return errors.New("An error occur while creating the card payment")
	}

	err = json.Unmarshal(res.Body(), &result)

	if err != nil {
		log.Println("Error while parsing request body")
		return errors.New("An error occur while creating the card payment")
	}

	if res.StatusCode() != http.StatusCreated {
		log.Println("The response was not expected", res.StatusCode(), string(res.Body()))
		return errors.New("An error occur while creating the card payment")
	}

	log.Println("Card payment created")
	payment.Id = result.Id
	payment.PaymentInfo.MaskedNumber = result.PaymentInfo.MaskedNumber

	return nil
}

Full Code: https://github.com/bmviniciuss/tcc/tree/bmviniciuss/devs-48
https://github.com/bmviniciuss/tcc/blob/bmviniciuss/devs-48/gateway-go/src/adapters/card_payment/http/htpp_card_payment_service.go

答案1

得分: 1

关于基准测试,您可能需要增加最大打开文件描述符的数量。

尝试编辑以下文件 /etc/security/limits.conf 并更新以下行:

*         hard    nofile      1048576
*         soft    nofile      1048576

如果不够的话,您可以将数字改为更大的值。

英文:

As for benchmarking, you might need to increase maximum number of open file descriptors.

Try to edit this file /etc/security/limits.conf and update following lines:

*         hard    nofile      1048576
*         soft    nofile      1048576

You can change to a greater number if it's not enough.

huangapple
  • 本文由 发表于 2022年7月1日 09:13:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/72823484.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定