Sometimes we are asked to provide a different logging driver to the ECS stack. Most of them are wide-known solutions as Papertrail or Splunk.


Here is a quick how to to provision and change the app task definition to send data across Papertrail's logspout.


Papertrail provision:


  • The stack will create a new ECS service
  • The container running inside this new service will be responsible for sending the logs to papertrail


Check the file below to see an example of terraform code to deploy this:


resource "aws_ecs_service" "example_logspout" {
  cluster                 = module.ecs_apps.ecs_name
  name                    = "logspout"
  task_definition         = aws_ecs_task_definition.example_task_logspout.arn
  desired_count           = 1
  placement_constraints {
    type = "distinctInstance"
  }
}

data "template_file" "example_task_logspout" {
  template = file("./task-definitions/logspout.json")

  vars = {
    syslogHostname    = "example-${local.workspace["cluster_name"]}-${local.workspace["region"]}",
    name    = "${local.workspace["cluster_name"]}-${local.workspace["logspout_name"]}",
    image   = "gliderlabs/logspout:latest",
    command = local.workspace["logspout_command"]
  }
}

resource "aws_ecs_task_definition" "example_task_logspout" {
  family                = "${local.workspace["cluster_name"]}-${local.workspace["logspout_name"]}"
  container_definitions = data.template_file.example_task_logspout.rendered

  execution_role_arn = module.ecs_apps.ecs_task_iam_role_arn
  task_role_arn      = module.ecs_apps.ecs_task_iam_role_arn

  volume {
    name      = "dockersock"
    host_path = "/var/run/docker.sock"
  }
}


For this task specifically, we don't need an application-style repo to create a task definition. The task definition file can be placed in the ECS stack as well.


Check the file below to see an example of task-definition.json:


[
  {
    "essential": true,
    "name": "${name}",
    "image": "gliderlabs/logspout:latest",
    "command": ["${command}"],
    "cpu": 1,
    "memory": 128,
    "environment": [
      { "name": "SYSLOG_HOSTNAME", "value": "${syslogHostname}" }
    ],
    "mountPoints": [
      {
        "containerPath": "/var/run/docker.sock",
        "sourceVolume": "dockersock"
      }
    ]
  }
]


And the _variables.tf file:


 #Papertrail
 logspout_name = "logspout"
 logspout_command = "syslog+tls://logsXX.papertrailapp.com:XXXXX"


Application logging:


By default Cloudwatch is the log driver for all ECS applications. To set the Papertrail's logging container collector change the log driver to:


  "logConfiguration": {
         "logDriver": "json-file"
   },


Credits:

Jeremias Roma