How to logging in Amazon Web Service ( AWS )?

huangapple go评论106阅读模式
英文:

How to logging in Amazon Web Service ( AWS )?

问题

我有一个使用Golang构建的项目,部署在AWS的Docker实例上。

在项目内部,我创建了一个日志文件,程序会将多个日志写入其中。

我该如何访问该日志文件?

还有其他正确的日志记录方式吗?

谢谢。

英文:

I have a project built in Golang and deployed on a Docker instance in AWS.

Internally I create a log file where the program write several logs.

How can I access that log file?

Is there another correct way to logging?

Thanks

答案1

得分: 1

你可以将容器的日志文件挂载到EC2主机上。在运行容器时,可以使用-v标志来实现这一点:

docker run -v /var/log/my_host_log_file.log:/var/log/your_container_log_file.log your-image

另外,你还可以配置你的应用程序将日志记录到标准输出,并将syslog作为日志驱动程序(使用--log-driver=syslog开关)。然后,容器的日志将被写入主机上的/var/log/messages文件中。

英文:

You could mount the log file from your container to your EC2 host. You can do this by using the -v flag when running your container:

docker run -v /var/log/my_host_log_file.log:/var/log/your_container_log_file.log your-image

Alternatively, you can configure your app to log to stdout and use syslog as your log driver (using the --log-driver=syslog switch). Your container logs will then be written to /var/log/messages on your host.

答案2

得分: 0

如果您使用AWS,我建议直接将日志发送到AWS CloudWatch。

首先,在AWS CloudWatch中创建一个新的日志组,例如"Production"。在您的Docker-Compose.yml文件中(或通过docker run...),添加AWS Logdriver:

logging:
  driver: "awslogs"
  options:
    awslogs-region: "eu-central-1"
    awslogs-group: "Production"
    awslogs-stream: "MyApp"

接下来,创建一个具有访问AWS CloudWatch权限的IAM用户,并将凭据添加到Docker主机。

示例IAM策略:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

对于使用systemd的Ubuntu系统:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

并将以下内容添加到文件中:

[Service]
Environment="AWS_ACCESS_KEY_ID=<aws_access_key_id>"
Environment="AWS_SECRET_ACCESS_KEY=<aws_secret_access_key>"

运行以下命令:

systemctl daemon-reload
service docker restart

现在,您的日志应该出现在AWS CloudWatch中。

英文:

If you use AWS, i would suggest to send Logs direct to AWS CloudWatch.

First create a new Log-Group in AWS Cloudwatch, for example "Production". In your Docker-Compose.yml (or via docker run..) add the AWS Logdriver:

    logging:
    driver: &quot;awslogs&quot;
    options:
      awslogs-region: &quot;eu-central-1&quot;
      awslogs-group: &quot;Production&quot;
      awslogs-stream: &quot;MyApp&quot;

Next creat a IAM user with Access to AWS Cloudwatch and add to the Dockerhost the credentials.

Example IAM Policy:

&quot;Version&quot; &quot;2012-10-17&quot; 
&quot;Statement&quot; 

"Action" "logs:CreateLogStream" "logs:PutLogEvents" "Effect" "Allow" "Resource"

On Ubuntu with systemd:

&quot;Version&quot; &quot;2012-10-17&quot; 
&quot;Statement&quot; 
  &quot;Action&quot; 
    &quot;logs:CreateLogStream&quot; 
    &quot;logs:PutLogEvents&quot; 
  &quot;Effect&quot; 
  &quot;Allow&quot; &quot;Resource&quot;

And add to the File:

[Service] Environment&quot;AWS_ACCESS_KEY_ID=&lt;aws_access_key_id&gt;&quot; 
Environment&quot;AWS_SECRET_ACCESS_KEY=&lt;aws_secret_access_key&gt;&quot;

Run:

systemctl daemon-reload
service docker restart

Now your logs should appear in AWS Cloudwatch.

答案3

得分: 0

感谢回复。

在寻找问题解决方案的一段时间后,我找到了解决方法!

首先,我需要将实例内部的文件挂载到docker主机上。

为此,我在项目的根目录中添加了一个名为Dockerrun.aws.json的Json文件(http://docs.aws.amazon.com/es_es/elasticbeanstalk/latest/dg/create_deploy_docker_image.html#create_deploy_docker_image_dockerrun)。

这个文件声明了共享文件夹(卷)(在docker主机和实例之间),我在其中保存我的日志文件。这一行相当于在docker run命令中添加-v标志(https://docs.docker.com/engine/tutorials/dockervolumes/#mount-a-host-directory-as-data-volume)。我这样做是因为我无法向正在运行的实例添加挂载点,也无法通过ssh停止它。

{
  "AWSEBDockerrunVersion": "1",
  "Volumes": [
  {
    "HostDirectory": "/var/log/",
    "ContainerDirectory": "/go/src/app/log"
  }
  ]
}

然后,为了告诉AWS在请求记录时下载我的日志文件(最后100行、捆绑或轮转),我将这些文件添加到项目目录中的.ebextension文件夹中(http://docs.aws.amazon.com/en_us/elasticbeanstalk/latest/dg/using-features.logging.html#health-logs-extend)。

Log_bundle.conf

Files:
"/opt/elasticbeanstalk/tasks/bundlelogs.d/log_bundle.conf":
    Mode: "000755"
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

Log_rotate.config

Files:
"/opt/elasticbeanstalk/tasks/bundlelogs.d/log_rotate.conf":
    Mode: "000755"
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

Log_tail.config

Files:
"/opt/elasticbeanstalk/tasks/publishlogs.d/log_tail.conf":
    Mode: "000755"
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

最后,我没有尝试Amazon Could Watch,但这是下一步。

祝好!

英文:

Thanks for reply.

After a while looking for the solution to the problem, I found it!

Firstly, I needed to mount the file that is inside the instance in the docker-host.

To do this I add a Json file in the root folder of my project called Dockerrun.aws.json
( http://docs.aws.amazon.com/es_es/elasticbeanstalk/latest/dg/create_deploy_docker_image.html#create_deploy_docker_image_dockerrun )

That is the file that declares the shared folder (volumes) (beetwen docker-host and instance) where I save my log file . This line is equivalent to adding -v flag in the docker run command (https://docs.docker.com/engine/tutorials/dockervolumes/#mount-a-host-directory-as-data-volume). I do this this way because I can not add mount to a running instance and i cant stop it by ssh.

{
  &quot;AWSEBDockerrunVersion&quot;: &quot;1&quot;,
  &quot;Volumes&quot;: [
  {
    &quot;HostDirectory&quot;: &quot;/var/log/&quot;,
    &quot;ContainerDirectory&quot;: &quot;/go/src/app/log&quot;
  }
  ]
}

Then to tell aws that I want to download my log file when I request records. (Tail (last 100 lines), bundle or rotate) I add these files to the .ebextension folder in my project directory. ( http://docs.aws.amazon.com/en_us/elasticbeanstalk/latest/dg/using-features.logging.html#health-logs-extend )

Log_bundle.conf

Files:
&quot;/opt/elasticbeanstalk/tasks/bundlelogs.d/log_bundle.conf&quot;:
    Mode: &quot;000755&quot;
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

Log_rotate.config

Files:
&quot;/opt/elasticbeanstalk/tasks/bundlelogs.d/log_rotate.conf&quot;:
    Mode: &quot;000755&quot;
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

Log_tail.config

Files:
&quot;/opt/elasticbeanstalk/tasks/publishlogs.d/log_tail.conf&quot;:
    Mode: &quot;000755&quot;
    Owner: root
    Group: root
    Content: |
      /var/log/application.log

Finally, I dont try Amazon Could Watch but is the next step.

Regards

答案4

得分: 0

如果你使用ELK(Elasticsearch,Logstash,Kibana),我建议使用"logrus"。

获取库

go get github.com/sirupsen/logrus

然后在你的项目中

package main

import (
    logrus "github.com/sirupsen/logrus"
)

var log = logrus.New()

func main() {
    conn, _ := net.Dial("tcp", "logstash-address")
    hook := logrustash.New(conn, logrustash.DefaultFormatter(logrus.Fields{"type": "my-app"}))
    log.Hooks.Add(hook)

    log.Info("Hello World!")
}
英文:

If you use ELK (Elasticsearch, Logstash, Kibana), I would suggest to use "logrus"

Get the library

> go get github.com/sirupsen/logrus

Then in your project

package main

import (

	logrus &quot;github.com/sirupsen/logrus&quot;
)

var log = logrus.New()

func main() {

	conn, _ := net.Dial(&quot;tcp&quot;, &quot;logstash-address&quot;)
	hook := logrustash.New(conn, logrustash.DefaultFormatter(logrus.Fields{&quot;type&quot;: &quot;my-app&quot;}))
	log.Hooks.Add(hook)


	log.Info(&quot;Hello World!&quot;)

}

huangapple
  • 本文由 发表于 2017年3月11日 02:10:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/42725090.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定