diff --git a/walkthroughs/howto-access-log/README.md b/walkthroughs/howto-access-log/README.md new file mode 100644 index 00000000..d202b259 --- /dev/null +++ b/walkthroughs/howto-access-log/README.md @@ -0,0 +1,174 @@ +## Access Log Format Feature Background +Today App Mesh supports configuring access log file path (https://docs.aws.amazon.com/app-mesh/latest/userguide/envoy-logs.html) for virtual nodes (https://docs.aws.amazon.com/app-mesh/latest/userguide/virtual_nodes.html) and virtual gateways (https://docs.aws.amazon.com/app-mesh/latest/userguide/virtual_gateways.html). We apply the same configuration for all the listener filter chains in Envoy. If not specified, Envoy will output logs to /dev/stdout by default. + +In this feature, we add support for different logging format like text_format_source or json_format. Following the walkthrough, you will be able to deploy color app with customized logging focusing on things that you really care about. It would also help if you want to export the logging file to other tools which requires a specific format or pattern to do further analysis. + +## Step 1: Prerequisites + +1. Make sure you have version 1.18.172 or higher of the [AWS CLI v1](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv1.html) installed or you have version 2.0.62 or higher of the [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed. + +2. You'll need a keypair stored in AWS to access a bastion host. + If you do not already have one, you can create a keypair using the command below if you don't have one. See [Amazon EC2 Key Pairs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html). + + ```bash + aws ec2 create-key-pair --key-name color-app | jq -r .KeyMaterial > ~/.ssh/color-app.pem + chmod 400 ~/.ssh/color-app.pem + ``` + + This command creates an Amazon EC2 Key Pair with name `color-app` and saves the private key at `~/.ssh/color-app.pem`. + +4. Additionally, this walkthrough makes use of the unix command line utility `jq`. If you don't already have it, you can install it from [here](https://stedolan.github.io/jq/). +5. Install Docker. It is needed to build the demo application images. + +![](fargate.png) +## Step 2: Set Environment Variables and deploy color app +We need to set a few environment variables before provisioning the infrastructure. +Please set the value for `AWS_ACCOUNT_ID`, `KEY_PAIR_NAME`,'AWS_DEFAULT_REGION' and `ENVOY_IMAGE` below. + +```bash +export AWS_ACCOUNT_ID= +export KEY_PAIR_NAME= +export AWS_DEFAULT_REGION= +export ENVOY_IMAGE=840364872350.dkr.ecr..amazonaws.com/aws-appmesh-envoy:v1.22.2.1-prod +export AWS_PROFILE=default +export ENVIRONMENT_NAME=LOGGING +export MESH_NAME=appmesh-mesh-logging +export SERVICES_DOMAIN=logging.local +export GOPROXY=direct +export GO_PROXY=direct +``` +This example is based on color app with Virtual Node on both EC2 and Fargate instance. Follow the color app example in aws-app-mesh-examples/examples/apps/colorapp/ +then follow walkthrough in aws-app-mesh-examples/walkthroughs/fargate/ to the step where you have 2 active virtual nodes blue (on EC2) and green (on fargate). + +Or you can follow the following steps: + +We'll start by setting up the basic infrastructure for our services. + +1. create the VPC. + +```bash +./infrastructure/vpc.sh +``` +2. Create the Mesh + +```bash +./infrastructure/appmesh-mesh.sh +``` +3. create the ECS cluster + +```bash +./infrastructure/ecs-cluster.sh +``` +4. create mesh resources + +```bash +./colorapp-ecs/appmesh-colorapp.sh +``` +5. Deploy service to ECS + +```bash +aws ecr create-repository --repository-name=gateway +export COLOR_GATEWAY_IMAGE=$(aws ecr describe-repositories --repository-names=gateway --query 'repositories[0].repositoryUri' --output text) +./colorapp-ecs/gateway/deploy.sh +``` + +```bash +aws ecr create-repository --repository-name=colorteller +export COLOR_TELLER_IMAGE=$(aws ecr describe-repositories --repository-names=colorteller --query 'repositories[0].repositoryUri' --output text) +./colorapp-ecs/colorteller/deploy.sh +``` + +```bash +./colorapp-ecs/ecs-colorapp.sh +``` + +To test: + +```bash +colorapp=$(aws cloudformation describe-stacks --region=$AWS_DEFAULT_REGION --stack-name=$ENVIRONMENT_NAME-ecs-colorapp --query="Stacks[0].Outputs[?OutputKey=='ColorAppEndpoint'].OutputValue" --output=text) +curl $colorapp/color +``` +6. deploy new virtual node on fargate instance + +```bash +./colorapp-ecs/fargate/appmesh-colorapp.sh +./colorapp-ecs/fargate/fargate-colorteller.sh +``` + +To test: + +```bash +colorapp=$(aws cloudformation --region=$AWS_DEFAULT_REGION describe-stacks --stack-name=$ENVIRONMENT_NAME-ecs-colorapp --query="Stacks[0].Outputs[?OutputKey=='ColorAppEndpoint'].OutputValue" --output=text) +curl $colorapp/color +``` + +## Step 3: Update logging format and Verify + +Our next step is to update the logging format and test it out. + +1. check meshes is deployed + +```bash +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh list-meshes +``` +2. check virtual node green (on fargate) + +```bash +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh describe-virtual-node --virtual-node-name colorteller-green-vn --mesh-name $MESH_NAME +``` + +3. update virtual node with text logging format in it + +```bash +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-blue-vn --mesh-name $MESH_NAME --cli-input-json file://src/blue-text-format.json +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-green-vn --mesh-name $MESH_NAME --cli-input-json file://src/green-text-format.json +``` +If you go to ECS -> Green/Blue backend task -> log -> envoy and search for Green/Blue +You should see logs like this: + +``` +BlueTestLog::200:path=/ping +``` + +4. update virtual node with json logging format in it + +```bash +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-blue-vn --mesh-name $MESH_NAME --cli-input-json file://src/blue-json-format.json +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-green-vn --mesh-name $MESH_NAME --cli-input-json file://src/green-json-format.json +``` + +If you go to ECS -> Green/Blue backend task -> log -> envoy and search for Green/Blue +You should see logs like this: + +``` +{"BlueTestLog":200,"protocol":"HTTP/1.1"} +``` + +5. update virtual node with no logging format in it + +```bash +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-blue-vn --mesh-name $MESH_NAME --cli-input-json file://src/blue-no-format.json +aws --endpoint-url https://frontend.$AWS_DEFAULT_REGION.prod.lattice.aws.a2z.com --region $AWS_DEFAULT_REGION appmesh update-virtual-node --virtual-node-name colorteller-green-vn --mesh-name $MESH_NAME --cli-input-json file://src/green-no-format.json +``` +Search for `colorteller-blue.logging.local` in console you and should see the default format like following +``` +[2022-08-04T23:00:29.383Z] "GET /ping HTTP/1.1" 200 - 0 0 1 0 "-" "Envoy/HC" "05ba7454-fce2-9360-92e3-a4da997d4f44" "colorteller-blue.logging.local:9080" "127.0.0.1:9080" +``` +6. change between different formats or change the pattern to find bugs! + +## Step 4: Clean Up + + +If you want to keep the application running, you can do so, but this is the end of this walkthrough. +Run the following commands to clean up and tear down the resources that we’ve created. + +Delete the CloudFormation stacks: + +```bash +aws cloudformation delete-stack --stack-name $ENVIRONMENT_NAME-ecs-service +aws cloudformation delete-stack --stack-name $ENVIRONMENT_NAME-ecs-cluster +aws ecr delete-repository --force --repository-name $COLOR_TELLER_IMAGE_NAME +aws ecr delete-repository --force --repository-name $WRK_TOOL_IMAGE_NAME +aws cloudformation delete-stack --stack-name $ENVIRONMENT_NAME-ecr-repositories +aws cloudformation delete-stack --stack-name $ENVIRONMENT_NAME-vpc +``` diff --git a/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.sh b/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.sh new file mode 100755 index 00000000..1f454b9d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-appmesh-colorapp" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/appmesh-colorapp.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + ServicesDomain="${SERVICES_DOMAIN}" \ + AppMeshMeshName="${MESH_NAME}" diff --git a/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.yaml b/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.yaml new file mode 100644 index 00000000..fb0d4f3d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/appmesh-colorapp.yaml @@ -0,0 +1,200 @@ +--- +Parameters: + EnvironmentName: + Type: String + Description: Environment name that joins all the stacks + + ServicesDomain: + Type: String + Description: DNS namespace used by services e.g. default.svc.cluster.local + + AppMeshMeshName: + Type: String + Description: Name of mesh + +Resources: + + ColorTellerBlackVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-black-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-black.${ServicesDomain}" + + ColorTellerBlueVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-blue-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-blue.${ServicesDomain}" + + ColorTellerRedVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-red-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-red.${ServicesDomain}" + + ColorTellerWhiteVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-white-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller.${ServicesDomain}" + + ColorTellerVirtualRouter: + Type: AWS::AppMesh::VirtualRouter + Properties: + MeshName: !Ref AppMeshMeshName + VirtualRouterName: colorteller-vr + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + + ColorTellerRoute: + Type: AWS::AppMesh::Route + DependsOn: + - ColorTellerVirtualRouter + - ColorTellerWhiteVirtualNode + - ColorTellerRedVirtualNode + - ColorTellerBlueVirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualRouterName: colorteller-vr + RouteName: colorteller-route + Spec: + HttpRoute: + Action: + WeightedTargets: + - VirtualNode: colorteller-white-vn + Weight: 1 + - VirtualNode: colorteller-blue-vn + Weight: 1 + - VirtualNode: colorteller-red-vn + Weight: 1 + Match: + Prefix: "/" + + ColorTellerVirtualService: + Type: AWS::AppMesh::VirtualService + DependsOn: + - ColorTellerVirtualRouter + Properties: + MeshName: !Ref AppMeshMeshName + VirtualServiceName: !Sub "colorteller.${ServicesDomain}" + Spec: + Provider: + VirtualRouter: + VirtualRouterName: colorteller-vr + + TcpEchoVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: tcpecho-vn + Spec: + Listeners: + - PortMapping: + Port: 2701 + Protocol: tcp + HealthCheck: + Protocol: tcp + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "tcpecho.${ServicesDomain}" + + TcpEchoVirtualService: + Type: AWS::AppMesh::VirtualService + DependsOn: + - TcpEchoVirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualServiceName: !Sub "tcpecho.${ServicesDomain}" + Spec: + Provider: + VirtualNode: + VirtualNodeName: tcpecho-vn + + ColorGatewayVirtualNode: + Type: AWS::AppMesh::VirtualNode + DependsOn: + - ColorTellerVirtualService + - TcpEchoVirtualService + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorgateway-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + ServiceDiscovery: + DNS: + Hostname: !Sub "colorgateway.${ServicesDomain}" + Backends: + - VirtualService: + VirtualServiceName: !Sub "colorteller.${ServicesDomain}" + - VirtualService: + VirtualServiceName: !Sub "tcpecho.${ServicesDomain}" \ No newline at end of file diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorgateway-base-task-def.json b/walkthroughs/howto-access-log/colorapp-ecs/colorgateway-base-task-def.json new file mode 100644 index 00000000..c036bbf2 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorgateway-base-task-def.json @@ -0,0 +1,81 @@ +{ + "family": $NAME, + "proxyConfiguration":{ + "type": "APPMESH", + "containerName": "envoy", + "properties": [ + { + "name": "IgnoredUID", + "value": "1337" + }, + { + "name": "ProxyIngressPort", + "value": "15000" + }, + { + "name": "ProxyEgressPort", + "value": "15001" + }, + { + "name": "AppPorts", + "value": "9080" + }, + { + "name": "EgressIgnoredIPs", + "value": "169.254.170.2,169.254.169.254" + } + ] + }, + "containerDefinitions": [ + { + "name": "app", + "image": $APP_IMAGE, + "portMappings": [ + { + "containerPort": 9080, + "hostPort": 9080, + "protocol": "tcp" + } + ], + "environment": [ + { + "name": "SERVER_PORT", + "value": "9080" + }, + { + "name": "COLOR_TELLER_ENDPOINT", + "value": $COLOR_TELLER_ENDPOINT + }, + { + "name": "TCP_ECHO_ENDPOINT", + "value": $TCP_ECHO_ENDPOINT + }, + { + "name": "STAGE", + "value": $STAGE + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX_APP + } + }, + "essential": true, + "dependsOn": [ + { + "containerName": "envoy", + "condition": "HEALTHY" + } + ] + }, + $ENVOY_CONTAINER_JSON, + $XRAY_CONTAINER_JSON + ], + "taskRoleArn": $TASK_ROLE_ARN, + "executionRoleArn": $EXECUTION_ROLE_ARN, + "networkMode": "awsvpc", + "memory": "256" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller-base-task-def.json b/walkthroughs/howto-access-log/colorapp-ecs/colorteller-base-task-def.json new file mode 100644 index 00000000..58d6df4f --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller-base-task-def.json @@ -0,0 +1,77 @@ +{ + "family": $NAME, + "proxyConfiguration":{ + "type": "APPMESH", + "containerName": "envoy", + "properties": [ + { + "name": "IgnoredUID", + "value": "1337" + }, + { + "name": "ProxyIngressPort", + "value": "15000" + }, + { + "name": "ProxyEgressPort", + "value": "15001" + }, + { + "name": "AppPorts", + "value": "9080" + }, + { + "name": "EgressIgnoredIPs", + "value": "169.254.170.2,169.254.169.254" + } + ] + }, + "containerDefinitions": [ + { + "name": "app", + "image": $APP_IMAGE, + "portMappings": [ + { + "containerPort": 9080, + "hostPort": 9080, + "protocol": "tcp" + } + ], + "environment": [ + { + "name": "COLOR", + "value": $COLOR + }, + { + "name": "SERVER_PORT", + "value": "9080" + }, + { + "name": "STAGE", + "value": $STAGE + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX_APP + } + }, + "essential": true, + "dependsOn": [ + { + "containerName": "envoy", + "condition": "HEALTHY" + } + ] + }, + $ENVOY_CONTAINER_JSON, + $XRAY_CONTAINER_JSON + ], + "taskRoleArn": $TASK_ROLE_ARN, + "executionRoleArn": $EXECUTION_ROLE_ARN, + "networkMode": "awsvpc", + "memory": "256" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/.gitignore b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/.gitignore new file mode 100644 index 00000000..963c55f7 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/.gitignore @@ -0,0 +1 @@ +teller diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/Dockerfile b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/Dockerfile new file mode 100644 index 00000000..2bae9d4b --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/Dockerfile @@ -0,0 +1,37 @@ +FROM public.ecr.aws/amazonlinux/amazonlinux:2 AS builder +RUN yum update -y && \ + yum install -y ca-certificates unzip tar gzip git && \ + yum clean all && \ + rm -rf /var/cache/yum + +RUN curl -LO https://golang.org/dl/go1.17.1.linux-amd64.tar.gz && \ + tar -C /usr/local -xzvf go1.17.1.linux-amd64.tar.gz + +ENV PATH="${PATH}:/usr/local/go/bin" +ENV GOPATH="${HOME}/go" +ENV PATH="${PATH}:${GOPATH}/bin" + +ARG GO_PROXY=https://proxy.golang.org +WORKDIR /go/src/github.com/aws/aws-app-mesh-examples/colorapp/teller + +# go.mod and go.sum go into their own layers. +COPY go.mod . +COPY go.sum . + +# Set the proxies for the go compiler +RUN go env -w GOPROXY=${GO_PROXY} +# This ensures `go mod download` happens only when go.mod and go.sum change. +RUN go mod download + +COPY . . +RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix nocgo -o /aws-app-mesh-examples-colorapp-teller . + +FROM public.ecr.aws/amazonlinux/amazonlinux:2 +RUN yum update -y && \ + yum install -y ca-certificates && \ + yum clean all && \ + rm -rf /var/cache/yum + +COPY --from=builder /aws-app-mesh-examples-colorapp-teller /bin/aws-app-mesh-examples-colorapp-teller + +ENTRYPOINT ["/bin/aws-app-mesh-examples-colorapp-teller"] diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/deploy.sh b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/deploy.sh new file mode 100755 index 00000000..79be8887 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/deploy.sh @@ -0,0 +1,43 @@ +#!/bin/bash + +set -eo pipefail + +if [ -z $AWS_ACCOUNT_ID ]; then + echo "AWS_ACCOUNT_ID environment variable is not set." + exit 1 +fi + +if [ -z $AWS_DEFAULT_REGION ]; then + echo "AWS_DEFAULT_REGION environment variable is not set." + exit 1 +fi + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" +ECR_URL="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com" +COLOR_TELLER_IMAGE=${COLOR_TELLER_IMAGE:-"${ECR_URL}/colorteller"} +GO_PROXY=${GO_PROXY:-"https://proxy.golang.org"} +AWS_CLI_VERSION=$(aws --version 2>&1 | cut -d/ -f2 | cut -d. -f1) + +ecr_login() { + if [ $AWS_CLI_VERSION -gt 1 ]; then + aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | \ + docker login --username AWS --password-stdin ${ECR_URL} + else + $(aws ecr get-login --no-include-email) + fi +} + +describe_create_ecr_registry() { + local repo_name=$1 + local region=$2 + aws ecr describe-repositories --repository-names ${repo_name} --region ${region} \ + || aws ecr create-repository --repository-name ${repo_name} --region ${region} +} + +# build +docker build --build-arg GO_PROXY=$GO_PROXY -t $COLOR_TELLER_IMAGE ${DIR} + +# push +ecr_login +describe_create_ecr_registry colorteller ${AWS_DEFAULT_REGION} +docker push $COLOR_TELLER_IMAGE diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.mod b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.mod new file mode 100644 index 00000000..0b61deaf --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.mod @@ -0,0 +1,12 @@ +module github.com/aws/aws-app-mesh-examples/colorapp/teller + +go 1.12 + +require ( + github.com/DATA-DOG/go-sqlmock v1.3.3 // indirect + github.com/aws/aws-sdk-go v1.19.0 // indirect + github.com/aws/aws-xray-sdk-go v0.9.4 + github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf // indirect + github.com/pkg/errors v0.8.1 // indirect + github.com/prometheus/client_golang v1.1.0 +) diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.sum b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.sum new file mode 100644 index 00000000..8babf25b --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/go.sum @@ -0,0 +1,84 @@ +github.com/DATA-DOG/go-sqlmock v1.3.3 h1:CWUqKXe0s8A2z6qCgkP4Kru7wC11YoAnoupUKFDnH08= +github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM= +github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/aws/aws-sdk-go v1.19.0 h1:3d9Htr/dl/+8xJYx/fpjEifvfpabZB1YUu61i/WX87Q= +github.com/aws/aws-sdk-go v1.19.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aws/aws-xray-sdk-go v0.9.4 h1:3mtFCrgFR5IefmWFV5pscHp9TTyOWuqaIKJIY0d1Y4g= +github.com/aws/aws-xray-sdk-go v0.9.4/go.mod h1:XtMKdBQfpVut+tJEwI7+dJFRxxRdxHDyVNp2tHXRq04= +github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= +github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf h1:XI2tOTCBqEnMyN2j1yPBI07yQHeywUSCEf8YWqf0oKw= +github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf/go.mod h1:9d6lWj8KzO/fd/NrVaLscBKmPigpZpn5YawRPw+e3Yo= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs= +github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= +github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= +github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= +github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.1.0 h1:BQ53HtBmfOitExawJ6LokA4x8ov/z0SYYb0+HxJfRI8= +github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g= +github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90 h1:S/YWwWx/RA8rT8tKFRuGUZhuA90OyIBpPCXkcbwU8DE= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.6.0 h1:kRhiuYSXR3+uv2IbVbZhUxK5zVD/2pp3Gd2PpvPkpEo= +github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= +github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.3 h1:CTwfnzjQ+8dS6MhHHu4YswVAD99sL2wjPqP+VkURmKE= +github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980 h1:dfGZHvZk057jK2MCeWus/TowKpJ8y4AmooUzdBSR9GU= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3 h1:4y9KwBHBgBNwDbtu44R5o1fdOCQUEXhbk/P4A9WmJq0= +golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= diff --git a/walkthroughs/howto-access-log/colorapp-ecs/colorteller/main.go b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/main.go new file mode 100644 index 00000000..14dae9d2 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/colorteller/main.go @@ -0,0 +1,61 @@ +package main + +import ( + "fmt" + "log" + "net/http" + "os" + + "github.com/aws/aws-xray-sdk-go/xray" +) + +const defaultPort = "8080" +const defaultColor = "black" +const defaultStage = "default" + +func getServerPort() string { + port := os.Getenv("SERVER_PORT") + if port != "" { + return port + } + + return defaultPort +} + +func getColor() string { + color := os.Getenv("COLOR") + if color != "" { + return color + } + + return defaultColor +} + +func getStage() string { + stage := os.Getenv("STAGE") + if stage != "" { + return stage + } + + return defaultStage +} + +type colorHandler struct{} +func (h *colorHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + log.Println("color requested, responding with", getColor()) + fmt.Fprint(writer, getColor()) +} + +type pingHandler struct{} +func (h *pingHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + log.Println("ping requested, reponding with HTTP 200") + writer.WriteHeader(http.StatusOK) +} + +func main() { + log.Println("starting server, listening on port " + getServerPort()) + xraySegmentNamer := xray.NewFixedSegmentNamer(fmt.Sprintf("%s-colorteller-%s", getStage(), getColor())) + http.Handle("/", xray.Handler(xraySegmentNamer, &colorHandler{})) + http.Handle("/ping", xray.Handler(xraySegmentNamer, &pingHandler{})) + http.ListenAndServe(":"+getServerPort(), nil) +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-100.json b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-100.json new file mode 100644 index 00000000..89c23aeb --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-100.json @@ -0,0 +1,19 @@ +{ + "routeName": "colorteller-route", + "spec": { + "httpRoute": { + "action": { + "weightedTargets": [ + { + "virtualNode": "colorteller-black-vn", + "weight": 1 + } + ] + }, + "match": { + "prefix": "/" + } + } + }, + "virtualRouterName": "colorteller-vr" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-80-blue-20.json b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-80-blue-20.json new file mode 100644 index 00000000..4d294918 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-black-80-blue-20.json @@ -0,0 +1,23 @@ +{ + "routeName": "colorteller-route", + "spec": { + "httpRoute": { + "action": { + "weightedTargets": [ + { + "virtualNode": "colorteller-black-vn", + "weight": 8 + }, + { + "virtualNode": "colorteller-blue-vn", + "weight": 2 + } + ] + }, + "match": { + "prefix": "/" + } + } + }, + "virtualRouterName": "colorteller-vr" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-100.json b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-100.json new file mode 100644 index 00000000..fe3e436d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-100.json @@ -0,0 +1,19 @@ +{ + "routeName": "colorteller-route", + "spec": { + "httpRoute": { + "action": { + "weightedTargets": [ + { + "virtualNode": "colorteller-blue-vn", + "weight": 1 + } + ] + }, + "match": { + "prefix": "/" + } + } + }, + "virtualRouterName": "colorteller-vr" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-80-red-20.json b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-80-red-20.json new file mode 100644 index 00000000..a26d5b11 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-blue-80-red-20.json @@ -0,0 +1,23 @@ +{ + "routeName": "colorteller-route", + "spec": { + "httpRoute": { + "action": { + "weightedTargets": [ + { + "virtualNode": "colorteller-blue-vn", + "weight": 8 + }, + { + "virtualNode": "colorteller-red-vn", + "weight": 2 + } + ] + }, + "match": { + "prefix": "/" + } + } + }, + "virtualRouterName": "colorteller-vr" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-red-100.json b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-red-100.json new file mode 100644 index 00000000..4ea78b1e --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/config/update_routes/colorteller-route-red-100.json @@ -0,0 +1,19 @@ +{ + "routeName": "colorteller-route", + "spec": { + "httpRoute": { + "action": { + "weightedTargets": [ + { + "virtualNode": "colorteller-red-vn", + "weight": 1 + } + ] + }, + "match": { + "prefix": "/" + } + } + }, + "virtualRouterName": "colorteller-vr" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/create-task-defs.sh b/walkthroughs/howto-access-log/colorapp-ecs/create-task-defs.sh new file mode 100755 index 00000000..6269b9fb --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/create-task-defs.sh @@ -0,0 +1,131 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +if [ -z "${ENVOY_IMAGE}" ]; then + echo "ENVOY_IMAGE environment is not defined" + exit 1 +fi + +if [ -z "${COLOR_GATEWAY_IMAGE}" ]; then + echo "COLOR_GATEWAY_IMAGE environment is not defined" + exit 1 +fi + +if [ -z "${COLOR_TELLER_IMAGE}" ]; then + echo "COLOR_TELLER_IMAGE environment is not defined" + exit 1 +fi + +stack_output=$(aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation describe-stacks --stack-name "${ENVIRONMENT_NAME}-ecs-cluster" --output json \ + | jq '.Stacks[].Outputs[]') + +task_role_arn=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "TaskIamRoleArn") | .OutputValue')) + +execution_role_arn=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "TaskExecutionIamRoleArn") | .OutputValue')) + +ecs_service_log_group=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "ECSServiceLogGroup") | .OutputValue')) + +envoy_log_level="debug" + +generate_xray_container_json() { + app_name=$1 + xray_container_json=$(jq -n \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg AWS_LOG_STREAM_PREFIX "${app_name}-xray" \ + -f "${DIR}/xray-container.json") +} + +generate_envoy_container_json() { + app_name=$1 + envoy_container_json=$(jq -n \ + --arg ENVOY_IMAGE $ENVOY_IMAGE \ + --arg VIRTUAL_NODE "mesh/$MESH_NAME/virtualNode/${app_name}-vn" \ + --arg APPMESH_XDS_ENDPOINT "${APPMESH_XDS_ENDPOINT}" \ + --arg ENVOY_LOG_LEVEL $envoy_log_level \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg AWS_LOG_STREAM_PREFIX "${app_name}-envoy" \ + -f "${DIR}/envoy-container.json") +} + +generate_sidecars() { + app_name=$1 + generate_envoy_container_json ${app_name} + generate_xray_container_json ${app_name} +} + +generate_color_teller_task_def() { + color=$1 + task_def_json=$(jq -n \ + --arg NAME "$ENVIRONMENT_NAME-ColorTeller-${color}" \ + --arg STAGE "$APPMESH_STAGE" \ + --arg COLOR "${color}" \ + --arg APP_IMAGE $COLOR_TELLER_IMAGE \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_LOG_STREAM_PREFIX_APP "colorteller-${color}-app" \ + --arg TASK_ROLE_ARN $task_role_arn \ + --arg EXECUTION_ROLE_ARN $execution_role_arn \ + --argjson ENVOY_CONTAINER_JSON "${envoy_container_json}" \ + --argjson XRAY_CONTAINER_JSON "${xray_container_json}" \ + -f "${DIR}/colorteller-base-task-def.json") + task_def=$(aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + ecs register-task-definition \ + --cli-input-json "$task_def_json" \ + --output json) +} + +# Color Gateway Task Definition +generate_sidecars "colorgateway" +task_def_json=$(jq -n \ + --arg NAME "$ENVIRONMENT_NAME-ColorGateway" \ + --arg STAGE "$APPMESH_STAGE" \ + --arg COLOR_TELLER_ENDPOINT "colorteller.$SERVICES_DOMAIN:9080" \ + --arg TCP_ECHO_ENDPOINT "tcpecho.$SERVICES_DOMAIN:2701" \ + --arg APP_IMAGE $COLOR_GATEWAY_IMAGE \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_LOG_STREAM_PREFIX_APP "colorgateway-app" \ + --arg TASK_ROLE_ARN $task_role_arn \ + --arg EXECUTION_ROLE_ARN $execution_role_arn \ + --argjson ENVOY_CONTAINER_JSON "${envoy_container_json}" \ + --argjson XRAY_CONTAINER_JSON "${xray_container_json}" \ + -f "${DIR}/colorgateway-base-task-def.json") +task_def=$(aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + ecs register-task-definition \ + --cli-input-json "$task_def_json" \ + --output json ) +colorgateway_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) + +# Color Teller White Task Definition +generate_sidecars "colorteller-white" +generate_color_teller_task_def "white" +colorteller_white_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) + +# Color Teller Red Task Definition +generate_sidecars "colorteller-red" +generate_color_teller_task_def "red" +colorteller_red_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) + +# Color Teller Blue Task Definition +generate_sidecars "colorteller-blue" +generate_color_teller_task_def "blue" +colorteller_blue_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) + +# Color Teller Black Task Definition +generate_sidecars "colorteller-black" +generate_color_teller_task_def "black" +colorteller_black_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) \ No newline at end of file diff --git a/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.sh b/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.sh new file mode 100755 index 00000000..1ecb4351 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.sh @@ -0,0 +1,29 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +# ecs-colorapp.yaml expects "true" or "false" (default is "false") +# will deploy the TesterService, which perpetually invokes /color to generate history +: "${DEPLOY_TESTER:=false}" + +# Creating Task Definitions +source ${DIR}/create-task-defs.sh + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-ecs-colorapp" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/ecs-colorapp.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + ECSServicesDomain="${SERVICES_DOMAIN}" \ + AppMeshMeshName="${MESH_NAME}" \ + ColorGatewayTaskDefinition="${colorgateway_task_def_arn}" \ + ColorTellerWhiteTaskDefinition="${colorteller_white_task_def_arn}" \ + ColorTellerRedTaskDefinition="${colorteller_red_task_def_arn}" \ + ColorTellerBlueTaskDefinition="${colorteller_blue_task_def_arn}" \ + ColorTellerBlackTaskDefinition="${colorteller_black_task_def_arn}" \ + DeployTester="${DEPLOY_TESTER}" + diff --git a/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.yaml b/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.yaml new file mode 100644 index 00000000..cfcd8a80 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/ecs-colorapp.yaml @@ -0,0 +1,433 @@ +--- +Parameters: + EnvironmentName: + Type: String + Description: Environment name that joins all the stacks + + AppMeshMeshName: + Type: String + Description: Name of mesh + + ECSServicesDomain: + Type: String + Description: DNS namespace used by services e.g. default.svc.cluster.local + + ColorGatewayTaskDefinition: + Type: String + Description: Task definition for ColorGateway Service + + ColorTellerWhiteTaskDefinition: + Type: String + Description: Task definition for ColorTeller White Service + + ColorTellerRedTaskDefinition: + Type: String + Description: Task definition for ColorTeller Red Service + + ColorTellerBlueTaskDefinition: + Type: String + Description: Task definition for ColorTeller Blue Service + + ColorTellerBlackTaskDefinition: + Type: String + Description: Task definition for ColorTeller Black Service + + LoadBalancerPath: + Type: String + Default: "*" + Description: A path on the public load balancer that this service + should be connected to. Use * to send all load balancer + traffic to this service. + + DeployTester: + Type: String + Default: false + AllowedValues: + - true + - false + Description: Set to "true" to include the TesterService (to generate color history) + +Conditions: + ShouldDeployTester: + !Equals [true, !Ref DeployTester] + +Resources: + + ### colorteller.default.svc.cluster.local + ColorTellerWhiteServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorteller" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorTellerWhiteService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorTellerWhiteServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorTellerWhiteTaskDefinition } + + ### colorteller-blue.default.svc.cluster.local + ColorTellerBlueServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorteller-blue" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorTellerBlueService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorTellerBlueServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorTellerBlueTaskDefinition } + + ### colorteller-red.default.svc.cluster.local + ColorTellerRedServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorteller-red" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorTellerRedService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorTellerRedServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorTellerRedTaskDefinition } + + ### colorteller-black.default.svc.cluster.local + ColorTellerBlackServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorteller-black" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorTellerBlackService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorTellerBlackServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorTellerBlackTaskDefinition } + + ### colorgateway.default.svc.cluster.local + ColorGatewayServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorgateway" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorGatewayService: + Type: 'AWS::ECS::Service' + DependsOn: + - WebLoadBalancerRule + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorGatewayServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorGatewayTaskDefinition } + LoadBalancers: + - ContainerName: app + ContainerPort: 9080 + TargetGroupArn: !Ref WebTargetGroup + + ### tester + TesterService: + Type: 'AWS::ECS::Service' + Condition: ShouldDeployTester + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: TesterTaskDefinition } + + TesterTaskDefinition: + Type: 'AWS::ECS::TaskDefinition' + Condition: ShouldDeployTester + Properties: + ContainerDefinitions: + - Name: "app" + Image: "tstrohmeier/alpine-infinite-curl" + Essential: true + Command: + - !Sub "-h http://colorgateway.${ECSServicesDomain}:9080/color" + LogConfiguration: + LogDriver: "awslogs" + Options: + awslogs-group: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceLogGroup" + awslogs-region: { Ref: "AWS::Region" } + awslogs-stream-prefix: "tester-app" + ExecutionRoleArn: + 'Fn::ImportValue': !Sub "${EnvironmentName}:TaskExecutionIamRoleArn" + TaskRoleArn: + 'Fn::ImportValue': !Sub "${EnvironmentName}:TaskIamRoleArn" + NetworkMode: "awsvpc" + Memory: 256 + + ### tcpecho.default.svc.cluster.local + TcpEchoServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "tcpecho" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + TcpEchoService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: EC2 + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': TcpEchoServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: TcpEchoTaskDefinition } + + TcpEchoTaskDefinition: + Type: 'AWS::ECS::TaskDefinition' + Properties: + ContainerDefinitions: + - Name: "app" + Image: "cjimti/go-echo" + Essential: true + LogConfiguration: + LogDriver: "awslogs" + Options: + awslogs-group: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceLogGroup" + awslogs-region: { Ref: "AWS::Region" } + awslogs-stream-prefix: "tcpecho-app" + PortMappings: + - ContainerPort: 2701 + HostPort: 2701 + Protocol: "tcp" + Environment: + - Name: "TCP_PORT" + Value: "2701" + - Name: "NODE_NAME" + Value: !Sub "mesh/${AppMeshMeshName}/virtualNode/tcpecho-vn" + ExecutionRoleArn: + 'Fn::ImportValue': !Sub "${EnvironmentName}:TaskExecutionIamRoleArn" + TaskRoleArn: + 'Fn::ImportValue': !Sub "${EnvironmentName}:TaskIamRoleArn" + NetworkMode: "awsvpc" + Memory: 256 + + PublicLoadBalancerSG: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Access to the public facing load balancer + VpcId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + SecurityGroupIngress: + - CidrIp: 0.0.0.0/0 + IpProtocol: tcp + FromPort: 80 + ToPort: 80 + + # public ALB for color gateway + PublicLoadBalancer: + Type: AWS::ElasticLoadBalancingV2::LoadBalancer + Properties: + Scheme: internet-facing + LoadBalancerAttributes: + - Key: idle_timeout.timeout_seconds + Value: '30' + Subnets: + - { 'Fn::ImportValue': !Sub "${EnvironmentName}:PublicSubnet1" } + - { 'Fn::ImportValue': !Sub "${EnvironmentName}:PublicSubnet2" } + SecurityGroups: [!Ref 'PublicLoadBalancerSG'] + + WebTargetGroup: + Type: AWS::ElasticLoadBalancingV2::TargetGroup + Properties: + HealthCheckIntervalSeconds: 6 + HealthCheckPath: /ping + HealthCheckProtocol: HTTP + HealthCheckTimeoutSeconds: 5 + HealthyThresholdCount: 2 + TargetType: ip + Name: !Sub "${EnvironmentName}-web" + Port: 80 + Protocol: HTTP + UnhealthyThresholdCount: 2 + TargetGroupAttributes: + - Key: deregistration_delay.timeout_seconds + Value: 120 + VpcId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + + PublicLoadBalancerListener: + Type: AWS::ElasticLoadBalancingV2::Listener + DependsOn: + - PublicLoadBalancer + Properties: + DefaultActions: + - TargetGroupArn: !Ref WebTargetGroup + Type: 'forward' + LoadBalancerArn: !Ref 'PublicLoadBalancer' + Port: 80 + Protocol: HTTP + + WebLoadBalancerRule: + Type: AWS::ElasticLoadBalancingV2::ListenerRule + Properties: + Actions: + - TargetGroupArn: !Ref WebTargetGroup + Type: 'forward' + Conditions: + - Field: path-pattern + Values: [!Ref 'LoadBalancerPath'] + ListenerArn: !Ref PublicLoadBalancerListener + Priority: 1 + +Outputs: + + ColorAppEndpoint: + Description: Public endpoint for Color App service + Value: !Join ['', ['http://', !GetAtt 'PublicLoadBalancer.DNSName']] + diff --git a/walkthroughs/howto-access-log/colorapp-ecs/envoy-container.json b/walkthroughs/howto-access-log/colorapp-ecs/envoy-container.json new file mode 100644 index 00000000..91b84b38 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/envoy-container.json @@ -0,0 +1,69 @@ +{ + "name": "envoy", + "image": $ENVOY_IMAGE, + "user": "1337", + "essential": true, + "ulimits": [ + { + "name": "nofile", + "hardLimit": 15000, + "softLimit": 15000 + } + ], + "portMappings": [ + { + "containerPort": 9901, + "hostPort": 9901, + "protocol": "tcp" + }, + { + "containerPort": 15000, + "hostPort": 15000, + "protocol": "tcp" + }, + { + "containerPort": 15001, + "hostPort": 15001, + "protocol": "tcp" + } + ], + "environment": [ + { + "name": "APPMESH_VIRTUAL_NODE_NAME", + "value": $VIRTUAL_NODE + }, + { + "name": "ENVOY_LOG_LEVEL", + "value": $ENVOY_LOG_LEVEL + }, + { + "name": "APPMESH_XDS_ENDPOINT", + "value": $APPMESH_XDS_ENDPOINT + }, + { + "name": "ENABLE_ENVOY_XRAY_TRACING", + "value": "1" + }, + { + "name": "ENABLE_ENVOY_STATS_TAGS", + "value": "1" + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX + } + }, + "healthCheck": { + "command": [ + "CMD-SHELL", + "curl -s http://localhost:9901/server_info | grep state | grep -q LIVE" + ], + "interval": 5, + "timeout": 2, + "retries": 3 + } +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/README.md b/walkthroughs/howto-access-log/colorapp-ecs/fargate/README.md new file mode 100644 index 00000000..924c6acb --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/README.md @@ -0,0 +1,231 @@ +# Using AWS App Mesh with Fargate + +## Overview + +I gave a walkthrough in my [previous article] on how to deploy a simple microservice application to ECS and configure [AWS App Mesh] to provide traffic control and observability for it. In this article, we are going to start to explore what it means when we say that App Mesh is a service mesh that lets you control and monitor services spanning multiple AWS compute environments. We'll start with using Fargate as an ECS launch type to deploy a specific version of our `colorteller` service before we move on and explore distributing traffic across other environments, such as EC2 and EKS. + +This is intentionally meant to be a simple example for clarity, but in the real world there are many use cases where creating a service mesh that can bridge different compute environments becomes very useful. Fargate is a compute service for AWS that helps you run containerized tasks using the primitives (tasks and services) of an ECS application without the need to directly configure and manage EC2 instances. Our contrived example in this article demonstrates a scenario in which you already have a containerized application running on ECS, but want to shift your workloads to use [Fargate] so that you can evolve your application without the need to manage compute infrastructure directly. + +Our strategy will be to deploy a new version of our `colorteller` service with Fargate and begin shifting traffic to it. If all goes well, then we will continue to shift more traffic to the new version until it is serving 100% of all requests. For this demo, we'll use "blue" to represent the original version and "green" to represent the new version. + +As a refresher, this is what the programming model for the Color App looks like: + +![](img/appmesh-fargate-colorapp-demo-1.png) +

Figure 1. Programmer perspective of the Color App.

+ +In terms of App Mesh configuration, we will want to begin shifting traffic over from version 1 (represented by `colorteller-blue` in the following diagram) over to version 2 (represented by `colorteller-green`). Remember, in App Mesh, every version of a service is ultimately backed by actual running code somewhere (in this case ECS/Fargate tasks), so each service will have it's own *virtual node* representation in the mesh that provides this conduit. + +![](img/appmesh-fargate-colorapp-demo-2.png) +

Figure 2. App Mesh configuration of the Color App.

+ +Finally, there is the physical deployment of the application itself to a compute environment. In this demo, `colorteller-blue` runs on ECS using the EC2 launch type and `colorteller-green` will run on ECS using the Fargate launch type. Our goal is to test with a portion of traffic going to `colorteller-green`, ultimately increasing to 100% of traffic going to the new green version. + +![](img/appmesh-fargate-colorapp-demo-3.png) +

Figure 3. AWS deployment perspective of the Color App.

+ +## Prerequisites + +1. You have successfully set up the prerequisites and deployed the Color App as described in the previous [walkthrough]. + +## Deploy + +### Initial configuration + +Once you have deployed the Color App (see #prerequisites), configure the app so that 100% of traffic goes to `colorteller-blue` for now. The blue color will represent version 1 of our colorteller service. + +Log into the App Mesh console and drill down into "Virtual routers" for the mesh. Configure the HTTP route to send 100% of traffic to the `colorteller-blue` virtual node. + +![](../../examples/apps/colorapp/img/appmesh-colorteller-route-1.png) +

Figure 4. Routes in the App Mesh console.

+ +Test the service and confirm in X-Ray that the traffic flows through the `colorteller-blue` as expected with no errors. + +![](../../examples/apps/colorapp/img/appmesh-xray-tracing-1.png) +

Figure 5. Tracing the colorgateway virtual node.

+ +### Deploy the new colorteller to Fargate + +For this configuration, we will deploy `colorteller-green`, which represents version 2 of our colorteller service. Initally, we will only send 30% of our traffic over to it. If our monitoring indicates that the service is healthy, we'll increase it to 60%, then finally to 100%. In the real world, you might choose more granular increases with automated rollout (and rollback if issues are indicated), but we're keeping things simple for the demo. + +As part of the original [walkthrough] we pushed the `gateway` and `colorteller` images to ECR (see [Deploy Images]) and then launched ECS tasks with these images. We will now launch an ECS task using the Fargate launch type with the same `colorteller` and `envoy` images. When the task is deployed, the running `envoy` container will be a sidecar for the `colorteller` container. Even with the Fargate launch type where we don't manually configure EC2 instances, a sidecar container will always be co-located on the same physical instance and its lifecycle coupled to the lifecycle of the primary application container (see [Sidecar Pattern]). + +#### 1. Update the mesh configuration + +Our updated CloudFormation templates are located in the [repo] under `walkthroughs/fargate`. + +This updated mesh configuration adds a new virtual node (`colorteller-green-vn`) and updates the virtual router (`colorteller-vr`) for the `colorteller` virtual service, so that traffic will be distributed between the blue and green virtual nodes at a 2:1 ratio (i.e., the green node will receive one third of the traffic). + +``` +$ ./appmesh-colorapp.sh +... +Waiting for changeset to be created.. +Waiting for stack create/update to complete +... +Successfully created/updated stack - DEMO-appmesh-colorapp +$ +``` + +#### 2. Deploy the green task to Fargate + +The `fargate-colorteller.sh` script creates parameterized template definitions before deploying the `fargate-colorteller.yaml` CloudFormation template. The change to launch a colorteller task as a Fargate task is in `fargate-colorteller-task-def.json`. + +``` +$ ./fargate-colorteller.sh +... + +Waiting for changeset to be created.. +Waiting for stack create/update to complete +Successfully created/updated stack - DEMO-fargate-colorteller +$ +``` + +### Verify the Fargate deployment + +The endpoint for the ColorApp is one of the CloudFormation template's outputs. You can view it in the stack output in the CloudFormation console, or fetch it with the AWS CLI: + +``` +$ colorapp=$(aws cloudformation describe-stacks --stack-name=$ENVIRONMENT_NAME-ecs-colorapp --query="Stacks[0 +].Outputs[?OutputKey=='ColorAppEndpoint'].OutputValue" --output=text); echo $colorapp> ].Outputs[?OutputKey=='ColorAppEndpoint'].OutputValue" --output=text); echo $colorapp +http://DEMO-Publi-YGZIJQXL5U7S-471987363.us-west-2.elb.amazonaws.com +``` + +We assigned the endpoint to the `colorapp` environment variable so we can use it for a few curl requests: + +``` +$ curl $colorapp/color +{"color":"blue", "stats": {"blue":1}} +$ +``` + +Since the weight of blue to green is 2:1, the result is not unsurprising. Let's clear the histogram and run it a few times until we get a green result: + +``` +$ curl $colorapp/color/clear +cleared + +$ for ((n=0;n<200;n++)); do echo "$n: $(curl -s $colorapp/color)"; done + +0: {"color":"blue", "stats": {"blue":1}} +1: {"color":"green", "stats": {"blue":0.5,"green":0.5}} +2: {"color":"blue", "stats": {"blue":0.67,"green":0.33}} +3: {"color":"green", "stats": {"blue":0.5,"green":0.5}} +4: {"color":"blue", "stats": {"blue":0.6,"green":0.4}} +5: {"color":"gre +en", "stats": {"blue":0.5,"green":0.5}} +6: {"color":"blue", "stats": {"blue":0.57,"green":0.43}} +7: {"color":"blue", "stats": {"blue":0.63,"green":0.38}} +8: {"color":"green", "stats": {"blue":0.56,"green":0.44}} +... +199: {"color":"blue", "stats": {"blue":0.66,"green":0.34}} +``` + +So far so good: this looks like what we expected for a 2:1 ratio. + +Let's take a look at our X-Ray console: + +![](img/appmesh-fargate-xray-blue-green.png) +

Figure 5. X-Ray console map after initial testing.

+ +The results look good: 100% success, no errors. + +We can now increase the rollout of the new (green) version of our service running on Fargate. + +Since we're using CloudFormation to manage our stacks, which lets us keep our configuration under version control and simplifies the process of undeploying resources, we could update the virtual route in `appmesh-colorapp.yaml` and deploy the updated mesh configuration by running `appmesh-colorapp.sh`. + +For this demo, however, let's just use the App Mesh console to make the change. Navigate to "Virtual routers" for "appmesh-mesh", and edit the "colorteller-route". Change it so that the weighted ratio looks like this: + +![](img/appmesh-fargate-routing-blue-green-2.png) +

Figure 6. Modifying route weights with the App Mesh console.

+ +Let's run our simple verification test again: + +``` +$ curl $colorapp/color/clear +cleared +fargate $ for ((n=0;n<200;n++)); do echo "$n: $(curl -s $colorapp/color)"; done +0: {"color":"green", "stats": {"green":1}} +1: {"color":"blue", "stats": {"blue":0.5,"green":0.5}} +2: {"color":"green", "stats": {"blue":0.33,"green":0.67}} +3: {"color":"green", "stats": {"blue":0.25,"green":0.75}} +4: {"color":"green", "stats": {"blue":0.2,"green":0.8}} +5: {"color":"green", "stats": {"blue":0.17,"green":0.83}} +6: {"color":"blue", "stats": {"blue":0.29,"green":0.71}} +7: {"color":"green", "stats": {"blue":0.25,"green":0.75}} +... +199: {"color":"green", "stats": {"blue":0.32,"green":0.68}} +$ +``` + +The results look good and we can confirm in the X-Ray console that we see no errors. + +Finally, we can shift 100% of our traffic over to the new colorteller version. This time, we'll modify the mesh configuration template and redeploy it: + +`appmesh-colorteller.yaml` +``` + ColorTellerRoute: + Type: AWS::AppMesh::Route + DependsOn: + - ColorTellerVirtualRouter + - ColorTellerGreenVirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualRouterName: colorteller-vr + RouteName: colorteller-route + Spec: + HttpRoute: + Action: + WeightedTargets: + - VirtualNode: colorteller-green-vn + Weight: 1 + Match: + Prefix: "/" +``` + + +``` +$ ./appmesh-colorapp.sh +... +Waiting for changeset to be created.. +Waiting for stack create/update to complete +... +Successfully created/updated stack - DEMO-appmesh-colorapp +$ +``` + +Again, we will want to repeat our verification process to confirm that the new version of our service is running successfully. + +## Summary + +In this walkthrough, we rolled out an update from version 1 (blue) of our colorteller service to version 2 (green). We demonstrated that App Mesh supports a mesh spanning ECS services that we ran as EC2 tasks and as Fargate tasks. In our next walkthrough, we will demonstrate that App Mesh handles even uncontainerized services launched directly on EC2 instances, providing a uniform and powerful way to control and monitor our distributed microservice applications on AWS. + +## Resources + +[AWS App Mesh Documentation] + +[AWS CLI] + +[Color App] + +[Currently available AWS regions for App Mesh] + +[Envoy Image] + +[Envoy documentation] + + + +[A/B testing]: https://en.wikipedia.org/wiki/A/B_testing +[previous article]: ../../examples/apps/colorapp/README.md +[AWS App Mesh Documentation]: https://aws.amazon.com/app-mesh/getting-started/ +[AWS App Mesh]: https://aws.amazon.com/app-mesh/ +[AWS CLI]: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html +[Color App]: https://github.com/aws/aws-app-mesh-examples/tree/main/examples/apps/colorap +[Currently available AWS regions for App Mesh]: https://docs.aws.amazon.com/general/latest/gr/rande.html#appmesh_region +[Deploy Images]: https://medium.com/p/de3452846e9d#0d56 +[Envoy documentation]: https://www.envoyproxy.io/docs/envoy/latest +[Envoy Image]: https://docs.aws.amazon.com/app-mesh/latest/userguide/envoy.html +[Fargate]: https://aws.amazon.com/fargate/ +[repo]: https://github.com/aws/aws-app-mesh-examples +[Sidecar Pattern]: https://www.oreilly.com/library/view/designing-distributed-systems/9781491983638/ch02.html +[walkthrough]: ../../examples/apps/colorapp/README.md +[walkthrough prerequisites]: https://medium.com/containers-on-aws/aws-app-mesh-walkthrough-deploy-the-color-app-on-amazon-ecs-de3452846e9d#42cf diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.sh b/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.sh new file mode 100755 index 00000000..1f454b9d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-appmesh-colorapp" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/appmesh-colorapp.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + ServicesDomain="${SERVICES_DOMAIN}" \ + AppMeshMeshName="${MESH_NAME}" diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.yaml b/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.yaml new file mode 100644 index 00000000..5c242e9d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/appmesh-colorapp.yaml @@ -0,0 +1,218 @@ +--- +Parameters: + EnvironmentName: + Type: String + Description: Environment name that joins all the stacks + + ServicesDomain: + Type: String + Description: DNS namespace used by services e.g. default.svc.cluster.local + + AppMeshMeshName: + Type: String + Description: Name of mesh + +Resources: + + ColorTellerGreenVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-green-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-green.${ServicesDomain}" + + ColorTellerBlackVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-black-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-black.${ServicesDomain}" + + ColorTellerBlueVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-blue-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-blue.${ServicesDomain}" + + ColorTellerRedVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-red-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller-red.${ServicesDomain}" + + ColorTellerWhiteVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorteller-white-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + HealthCheck: + Protocol: http + Path: "/ping" + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "colorteller.${ServicesDomain}" + + ColorTellerVirtualRouter: + Type: AWS::AppMesh::VirtualRouter + Properties: + MeshName: !Ref AppMeshMeshName + VirtualRouterName: colorteller-vr + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + + ColorTellerRoute: + Type: AWS::AppMesh::Route + DependsOn: + - ColorTellerVirtualRouter + - ColorTellerBlueVirtualNode + - ColorTellerGreenVirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualRouterName: colorteller-vr + RouteName: colorteller-route + Spec: + HttpRoute: + Action: + WeightedTargets: + - VirtualNode: colorteller-blue-vn + Weight: 2 + - VirtualNode: colorteller-green-vn + Weight: 1 + Match: + Prefix: "/" + + ColorTellerVirtualService: + Type: AWS::AppMesh::VirtualService + DependsOn: + - ColorTellerVirtualRouter + Properties: + MeshName: !Ref AppMeshMeshName + VirtualServiceName: !Sub "colorteller.${ServicesDomain}" + Spec: + Provider: + VirtualRouter: + VirtualRouterName: colorteller-vr + + TcpEchoVirtualNode: + Type: AWS::AppMesh::VirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: tcpecho-vn + Spec: + Listeners: + - PortMapping: + Port: 2701 + Protocol: tcp + HealthCheck: + Protocol: tcp + HealthyThreshold: 2 + UnhealthyThreshold: 2 + TimeoutMillis: 2000 + IntervalMillis: 5000 + ServiceDiscovery: + DNS: + Hostname: !Sub "tcpecho.${ServicesDomain}" + + TcpEchoVirtualService: + Type: AWS::AppMesh::VirtualService + DependsOn: + - TcpEchoVirtualNode + Properties: + MeshName: !Ref AppMeshMeshName + VirtualServiceName: !Sub "tcpecho.${ServicesDomain}" + Spec: + Provider: + VirtualNode: + VirtualNodeName: tcpecho-vn + + ColorGatewayVirtualNode: + Type: AWS::AppMesh::VirtualNode + DependsOn: + - ColorTellerVirtualService + - TcpEchoVirtualService + Properties: + MeshName: !Ref AppMeshMeshName + VirtualNodeName: colorgateway-vn + Spec: + Listeners: + - PortMapping: + Port: 9080 + Protocol: http + ServiceDiscovery: + DNS: + Hostname: !Sub "colorgateway.${ServicesDomain}" + Backends: + - VirtualService: + VirtualServiceName: !Sub "colorteller.${ServicesDomain}" + - VirtualService: + VirtualServiceName: !Sub "tcpecho.${ServicesDomain}" diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/envoy-container.json b/walkthroughs/howto-access-log/colorapp-ecs/fargate/envoy-container.json new file mode 100644 index 00000000..91b84b38 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/envoy-container.json @@ -0,0 +1,69 @@ +{ + "name": "envoy", + "image": $ENVOY_IMAGE, + "user": "1337", + "essential": true, + "ulimits": [ + { + "name": "nofile", + "hardLimit": 15000, + "softLimit": 15000 + } + ], + "portMappings": [ + { + "containerPort": 9901, + "hostPort": 9901, + "protocol": "tcp" + }, + { + "containerPort": 15000, + "hostPort": 15000, + "protocol": "tcp" + }, + { + "containerPort": 15001, + "hostPort": 15001, + "protocol": "tcp" + } + ], + "environment": [ + { + "name": "APPMESH_VIRTUAL_NODE_NAME", + "value": $VIRTUAL_NODE + }, + { + "name": "ENVOY_LOG_LEVEL", + "value": $ENVOY_LOG_LEVEL + }, + { + "name": "APPMESH_XDS_ENDPOINT", + "value": $APPMESH_XDS_ENDPOINT + }, + { + "name": "ENABLE_ENVOY_XRAY_TRACING", + "value": "1" + }, + { + "name": "ENABLE_ENVOY_STATS_TAGS", + "value": "1" + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX + } + }, + "healthCheck": { + "command": [ + "CMD-SHELL", + "curl -s http://localhost:9901/server_info | grep state | grep -q LIVE" + ], + "interval": 5, + "timeout": 2, + "retries": 3 + } +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.json b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.json new file mode 100644 index 00000000..e460bed6 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.json @@ -0,0 +1,81 @@ +{ + "family": $NAME, + "proxyConfiguration":{ + "type": "APPMESH", + "containerName": "envoy", + "properties": [ + { + "name": "IgnoredUID", + "value": "1337" + }, + { + "name": "ProxyIngressPort", + "value": "15000" + }, + { + "name": "ProxyEgressPort", + "value": "15001" + }, + { + "name": "AppPorts", + "value": "9080" + }, + { + "name": "EgressIgnoredIPs", + "value": "169.254.170.2,169.254.169.254" + } + ] + }, + "containerDefinitions": [ + { + "name": "app", + "image": $APP_IMAGE, + "portMappings": [ + { + "containerPort": 9080, + "hostPort": 9080, + "protocol": "tcp" + } + ], + "environment": [ + { + "name": "COLOR", + "value": $COLOR + }, + { + "name": "SERVER_PORT", + "value": "9080" + }, + { + "name": "STAGE", + "value": $STAGE + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX_APP + } + }, + "essential": true, + "dependsOn": [ + { + "containerName": "envoy", + "condition": "HEALTHY" + } + ] + }, + $ENVOY_CONTAINER_JSON, + $XRAY_CONTAINER_JSON + ], + "taskRoleArn": $TASK_ROLE_ARN, + "executionRoleArn": $EXECUTION_ROLE_ARN, + "requiresCompatibilities": [ + "FARGATE" + ], + "networkMode": "awsvpc", + "cpu": "256", + "memory": "512" +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.sh b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.sh new file mode 100755 index 00000000..adc1f54f --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller-task-def.sh @@ -0,0 +1,55 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +stack_output=$(aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation describe-stacks --stack-name "${ENVIRONMENT_NAME}-ecs-cluster" \ + | jq '.Stacks[].Outputs[]') + +task_role_arn=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "TaskIamRoleArn") | .OutputValue')) + +execution_role_arn=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "TaskExecutionIamRoleArn") | .OutputValue')) + +ecs_service_log_group=($(echo $stack_output \ + | jq -r 'select(.OutputKey == "ECSServiceLogGroup") | .OutputValue')) + +envoy_log_level="debug" + +# Color Teller Green Task Definition +COLOR=green +envoy_container_json=$(jq -n \ + --arg ENVOY_IMAGE $ENVOY_IMAGE \ + --arg VIRTUAL_NODE "mesh/$MESH_NAME/virtualNode/colorteller-$COLOR-vn" \ + --arg APPMESH_XDS_ENDPOINT "${APPMESH_XDS_ENDPOINT}" \ + --arg ENVOY_LOG_LEVEL $envoy_log_level \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg AWS_LOG_STREAM_PREFIX "colorteller-$COLOR-envoy" \ + -f "${DIR}/envoy-container.json") +xray_container_json=$(jq -n \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg AWS_LOG_STREAM_PREFIX_ENVOY "colorteller-$COLOR-xray" \ + -f "${DIR}/xray-container.json") +task_def_json=$(jq -n \ + --arg NAME "$ENVIRONMENT_NAME-colorteller-$COLOR" \ + --arg STAGE "$APPMESH_STAGE" \ + --arg COLOR "$COLOR" \ + --arg APP_IMAGE $COLOR_TELLER_IMAGE \ + --arg AWS_REGION $AWS_DEFAULT_REGION \ + --arg ECS_SERVICE_LOG_GROUP $ecs_service_log_group \ + --arg AWS_LOG_STREAM_PREFIX_APP "colorteller-$COLOR-app" \ + --arg TASK_ROLE_ARN $task_role_arn \ + --arg EXECUTION_ROLE_ARN $execution_role_arn \ + --argjson ENVOY_CONTAINER_JSON "${envoy_container_json}" \ + --argjson XRAY_CONTAINER_JSON "${xray_container_json}" \ + -f "${DIR}/fargate-colorteller-task-def.json") +task_def=$(aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + ecs register-task-definition \ + --cli-input-json "$task_def_json") +colorteller_green_task_def_arn=($(echo $task_def \ + | jq -r '.taskDefinition | .taskDefinitionArn')) diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.sh b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.sh new file mode 100755 index 00000000..46da96fd --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +# Create colorteller task definition +source ${DIR}/fargate-colorteller-task-def.sh + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-fargate-colorteller" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/fargate-colorteller.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + ECSServicesDomain="${SERVICES_DOMAIN}" \ + AppMeshMeshName="${MESH_NAME}" \ + ColorTellerGreenTaskDefinition="${colorteller_green_task_def_arn}" diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.yaml b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.yaml new file mode 100644 index 00000000..b8eaaf58 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/fargate-colorteller.yaml @@ -0,0 +1,56 @@ +--- +Parameters: + EnvironmentName: + Type: String + Description: Environment name that joins all the stacks + + AppMeshMeshName: + Type: String + Description: Name of mesh + + ECSServicesDomain: + Type: String + Description: DNS namespace used by services e.g. default.svc.cluster.local + + ColorTellerGreenTaskDefinition: + Type: String + Description: Task definition for ColorTeller Green Service + +Resources: + + ### colorteller-green.default.svc.cluster.local + ColorTellerGreenServiceDiscoveryRecord: + Type: 'AWS::ServiceDiscovery::Service' + Properties: + Name: "colorteller-green" + DnsConfig: + NamespaceId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + DnsRecords: + - Type: A + TTL: 300 + HealthCheckCustomConfig: + FailureThreshold: 1 + + ColorTellerGreenService: + Type: 'AWS::ECS::Service' + Properties: + Cluster: + 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSCluster" + DeploymentConfiguration: + MaximumPercent: 200 + MinimumHealthyPercent: 100 + DesiredCount: 1 + LaunchType: FARGATE + ServiceRegistries: + - RegistryArn: + 'Fn::GetAtt': ColorTellerGreenServiceDiscoveryRecord.Arn + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: DISABLED + SecurityGroups: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + Subnets: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + TaskDefinition: { Ref: ColorTellerGreenTaskDefinition } diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-1.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-1.png new file mode 100644 index 00000000..ccd11fa3 Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-1.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-2.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-2.png new file mode 100644 index 00000000..cc9ca971 Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-2.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-3.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-3.png new file mode 100644 index 00000000..acb54975 Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-colorapp-demo-3.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-1.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-1.png new file mode 100644 index 00000000..d9cf6474 Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-1.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-2.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-2.png new file mode 100644 index 00000000..67934780 Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-routing-blue-green-2.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-xray-blue-green.png b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-xray-blue-green.png new file mode 100644 index 00000000..23a74fcb Binary files /dev/null and b/walkthroughs/howto-access-log/colorapp-ecs/fargate/img/appmesh-fargate-xray-blue-green.png differ diff --git a/walkthroughs/howto-access-log/colorapp-ecs/fargate/xray-container.json b/walkthroughs/howto-access-log/colorapp-ecs/fargate/xray-container.json new file mode 100644 index 00000000..35c0ae3c --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/fargate/xray-container.json @@ -0,0 +1,23 @@ +{ + "name": "xray-daemon", + "image": "public.ecr.aws/xray/aws-xray-daemon", + "user": "1337", + "essential": true, + "cpu": 32, + "memoryReservation": 256, + "portMappings": [ + { + "hostPort": 2000, + "containerPort": 2000, + "protocol": "udp" + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX_ENVOY + } + } +} \ No newline at end of file diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/.gitignore b/walkthroughs/howto-access-log/colorapp-ecs/gateway/.gitignore new file mode 100644 index 00000000..ad230ccf --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/.gitignore @@ -0,0 +1 @@ +gateway diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/Dockerfile b/walkthroughs/howto-access-log/colorapp-ecs/gateway/Dockerfile new file mode 100644 index 00000000..30b6e83e --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/Dockerfile @@ -0,0 +1,37 @@ +FROM public.ecr.aws/amazonlinux/amazonlinux:2 AS builder +RUN yum update -y && \ + yum install -y ca-certificates unzip tar gzip git && \ + yum clean all && \ + rm -rf /var/cache/yum + +RUN curl -LO https://golang.org/dl/go1.17.1.linux-amd64.tar.gz && \ + tar -C /usr/local -xzvf go1.17.1.linux-amd64.tar.gz + +ENV PATH="${PATH}:/usr/local/go/bin" +ENV GOPATH="${HOME}/go" +ENV PATH="${PATH}:${GOPATH}/bin" + +ARG GO_PROXY=https://proxy.golang.org +WORKDIR /go/src/github.com/aws/aws-app-mesh-examples/colorapp/gateway + +# go.mod and go.sum go into their own layers. +COPY go.mod . +COPY go.sum . + +# Set the proxies for the go compiler +RUN go env -w GOPROXY=${GO_PROXY} +# This ensures `go mod download` happens only when go.mod and go.sum change. +RUN go mod download + +COPY . . +RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix nocgo -o /aws-app-mesh-examples-colorapp-gateway . + +FROM public.ecr.aws/amazonlinux/amazonlinux:2 +RUN yum update -y && \ + yum install -y ca-certificates && \ + yum clean all && \ + rm -rf /var/cache/yum + +COPY --from=builder /aws-app-mesh-examples-colorapp-gateway /bin/aws-app-mesh-examples-colorapp-gateway + +ENTRYPOINT ["/bin/aws-app-mesh-examples-colorapp-gateway"] diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/deploy.sh b/walkthroughs/howto-access-log/colorapp-ecs/gateway/deploy.sh new file mode 100755 index 00000000..62e8699a --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/deploy.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +# vim:syn=sh:ts=4:sw=4:et:ai + +set -ex + +if [ -z $AWS_ACCOUNT_ID ]; then + echo "AWS_ACCOUNT_ID environment variable is not set." + exit 1 +fi + +if [ -z $AWS_DEFAULT_REGION ]; then + echo "AWS_DEFAULT_REGION environment variable is not set." + exit 1 +fi + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" +ECR_URL="${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com" +COLOR_GATEWAY_IMAGE=${COLOR_GATEWAY_IMAGE:-"${ECR_URL}/gateway"} +GO_PROXY=${GO_PROXY:-"https://proxy.golang.org"} +AWS_CLI_VERSION=$(aws --version 2>&1 | cut -d/ -f2 | cut -d. -f1) + +ecr_login() { + if [ $AWS_CLI_VERSION -gt 1 ]; then + aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | \ + docker login --username AWS --password-stdin ${ECR_URL} + else + $(aws ecr get-login --no-include-email) + fi +} + +describe_create_ecr_registry() { + local repo_name=$1 + local region=$2 + aws ecr describe-repositories --repository-names ${repo_name} --region ${region} \ + || aws ecr create-repository --repository-name ${repo_name} --region ${region} +} + +# build +docker build --build-arg GO_PROXY=$GO_PROXY -t $COLOR_GATEWAY_IMAGE ${DIR} + +# push +ecr_login +describe_create_ecr_registry gateway ${AWS_DEFAULT_REGION} +docker push $COLOR_GATEWAY_IMAGE diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.mod b/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.mod new file mode 100644 index 00000000..47345a1d --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.mod @@ -0,0 +1,14 @@ +module github.com/aws/aws-app-mesh-examples/colorapp/gateway + +go 1.12 + +require ( + github.com/DATA-DOG/go-sqlmock v1.3.3 // indirect + github.com/aws/aws-sdk-go v1.19.0 // indirect + github.com/aws/aws-xray-sdk-go v0.9.4 + github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.1 // indirect + github.com/pkg/errors v0.8.0 + github.com/prometheus/client_golang v1.1.0 +) diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.sum b/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.sum new file mode 100644 index 00000000..c7452816 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/go.sum @@ -0,0 +1,76 @@ +github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM= +github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/aws/aws-sdk-go v1.19.0 h1:3d9Htr/dl/+8xJYx/fpjEifvfpabZB1YUu61i/WX87Q= +github.com/aws/aws-sdk-go v1.19.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo= +github.com/aws/aws-xray-sdk-go v0.9.4 h1:3mtFCrgFR5IefmWFV5pscHp9TTyOWuqaIKJIY0d1Y4g= +github.com/aws/aws-xray-sdk-go v0.9.4/go.mod h1:XtMKdBQfpVut+tJEwI7+dJFRxxRdxHDyVNp2tHXRq04= +github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= +github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf h1:XI2tOTCBqEnMyN2j1yPBI07yQHeywUSCEf8YWqf0oKw= +github.com/cihub/seelog v0.0.0-20151216151435-d2c6e5aa9fbf/go.mod h1:9d6lWj8KzO/fd/NrVaLscBKmPigpZpn5YawRPw+e3Yo= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs= +github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM= +github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= +github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= +github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU= +github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.1.0 h1:BQ53HtBmfOitExawJ6LokA4x8ov/z0SYYb0+HxJfRI8= +github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g= +github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90 h1:S/YWwWx/RA8rT8tKFRuGUZhuA90OyIBpPCXkcbwU8DE= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.6.0 h1:kRhiuYSXR3+uv2IbVbZhUxK5zVD/2pp3Gd2PpvPkpEo= +github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc= +github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.3 h1:CTwfnzjQ+8dS6MhHHu4YswVAD99sL2wjPqP+VkURmKE= +github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= diff --git a/walkthroughs/howto-access-log/colorapp-ecs/gateway/main.go b/walkthroughs/howto-access-log/colorapp-ecs/gateway/main.go new file mode 100644 index 00000000..6f9fb33a --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/gateway/main.go @@ -0,0 +1,225 @@ +package main + +import ( + "bufio" + "encoding/json" + "fmt" + "io/ioutil" + "log" + "math" + "net" + "net/http" + "os" + "strings" + "sync" + + "github.com/aws/aws-xray-sdk-go/xray" + "github.com/pkg/errors" +) + +const defaultPort = "8080" +const defaultStage = "default" +const maxColors = 1000 + +var colors [maxColors]string +var colorsIdx int +var colorsMutext = &sync.Mutex{} + +func getServerPort() string { + port := os.Getenv("SERVER_PORT") + if port != "" { + return port + } + + return defaultPort +} + +func getStage() string { + stage := os.Getenv("STAGE") + if stage != "" { + return stage + } + + return defaultStage +} + +func getColorTellerEndpoint() (string, error) { + colorTellerEndpoint := os.Getenv("COLOR_TELLER_ENDPOINT") + if colorTellerEndpoint == "" { + return "", errors.New("COLOR_TELLER_ENDPOINT is not set") + } + return colorTellerEndpoint, nil +} + +type colorHandler struct{} + +func (h *colorHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + color, err := getColorFromColorTeller(request) + if err != nil { + writer.WriteHeader(http.StatusInternalServerError) + writer.Write([]byte("500 - Unexpected Error")) + return + } + + colorsMutext.Lock() + defer colorsMutext.Unlock() + + addColor(color) + statsJson, err := json.Marshal(getRatios()) + if err != nil { + fmt.Fprintf(writer, `{"color":"%s", "error":"%s"}`, color, err) + return + } + fmt.Fprintf(writer, `{"color":"%s", "stats": %s}`, color, statsJson) +} + +func addColor(color string) { + colors[colorsIdx] = color + + colorsIdx += 1 + if colorsIdx >= maxColors { + colorsIdx = 0 + } +} + +func getRatios() map[string]float64 { + counts := make(map[string]int) + var total = 0 + + for _, c := range colors { + if c != "" { + counts[c] += 1 + total += 1 + } + } + + ratios := make(map[string]float64) + for k, v := range counts { + ratio := float64(v) / float64(total) + ratios[k] = math.Round(ratio*100) / 100 + } + + return ratios +} + +type clearColorStatsHandler struct{} + +func (h *clearColorStatsHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + colorsMutext.Lock() + defer colorsMutext.Unlock() + + colorsIdx = 0 + for i := range colors { + colors[i] = "" + } + + fmt.Fprint(writer, "cleared") +} + +func getColorFromColorTeller(request *http.Request) (string, error) { + colorTellerEndpoint, err := getColorTellerEndpoint() + if err != nil { + return "-n/a-", err + } + + client := xray.Client(&http.Client{}) + req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("http://%s", colorTellerEndpoint), nil) + if err != nil { + return "-n/a-", err + } + + resp, err := client.Do(req.WithContext(request.Context())) + if err != nil { + return "-n/a-", err + } + + defer resp.Body.Close() + body, err := ioutil.ReadAll(resp.Body) + if err != nil { + return "-n/a-", err + } + + color := strings.TrimSpace(string(body)) + if len(color) < 1 { + return "-n/a-", errors.New("Empty response from colorTeller") + } + + return color, nil +} + +func getTCPEchoEndpoint() (string, error) { + tcpEchoEndpoint := os.Getenv("TCP_ECHO_ENDPOINT") + if tcpEchoEndpoint == "" { + return "", errors.New("TCP_ECHO_ENDPOINT is not set") + } + return tcpEchoEndpoint, nil +} + +type tcpEchoHandler struct{} + +func (h *tcpEchoHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + endpoint, err := getTCPEchoEndpoint() + if err != nil { + writer.WriteHeader(http.StatusInternalServerError) + fmt.Fprintf(writer, "tcpecho endpoint is not set") + return + } + + log.Printf("Dialing tcp endpoint %s", endpoint) + conn, err := net.Dial("tcp", endpoint) + if err != nil { + writer.WriteHeader(http.StatusInternalServerError) + fmt.Fprintf(writer, "Dial failed, err:%s", err.Error()) + return + } + defer conn.Close() + + strEcho := "Hello from gateway" + log.Printf("Writing '%s'", strEcho) + _, err = fmt.Fprintf(conn, strEcho) + if err != nil { + writer.WriteHeader(http.StatusInternalServerError) + fmt.Fprintf(writer, "Write to server failed, err:%s", err.Error()) + return + } + + reply, err := bufio.NewReader(conn).ReadString('\n') + if err != nil { + writer.WriteHeader(http.StatusInternalServerError) + fmt.Fprintf(writer, "Read from server failed, err:%s", err.Error()) + return + } + + fmt.Fprintf(writer, "Response from tcpecho server: %s", reply) +} + +type pingHandler struct{} + +func (h *pingHandler) ServeHTTP(writer http.ResponseWriter, request *http.Request) { + log.Println("ping requested, reponding with HTTP 200") + writer.WriteHeader(http.StatusOK) +} + +func main() { + log.Println("Starting server, listening on port " + getServerPort()) + + colorTellerEndpoint, err := getColorTellerEndpoint() + if err != nil { + log.Fatalln(err) + } + tcpEchoEndpoint, err := getTCPEchoEndpoint() + if err != nil { + log.Println(err) + } + + log.Println("Using color-teller at " + colorTellerEndpoint) + log.Println("Using tcp-echo at " + tcpEchoEndpoint) + + xraySegmentNamer := xray.NewFixedSegmentNamer(fmt.Sprintf("%s-gateway", getStage())) + + http.Handle("/color", xray.Handler(xraySegmentNamer, &colorHandler{})) + http.Handle("/color/clear", xray.Handler(xraySegmentNamer, &clearColorStatsHandler{})) + http.Handle("/tcpecho", xray.Handler(xraySegmentNamer, &tcpEchoHandler{})) + http.Handle("/ping", xray.Handler(xraySegmentNamer, &pingHandler{})) + log.Fatal(http.ListenAndServe(":"+getServerPort(), nil)) +} diff --git a/walkthroughs/howto-access-log/colorapp-ecs/route_canary.sh b/walkthroughs/howto-access-log/colorapp-ecs/route_canary.sh new file mode 100755 index 00000000..80af19fc --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/route_canary.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +# vim:syn=sh:ts=4:sw=4:et:ai + +shopt -s nullglob + +# Optional pre-load script +if [ -f meshvars.sh ]; then + source meshvars.sh +fi + +if [ ! -z "$AWS_PROFILE" ]; then + PROFILE_OPT="--profile ${AWS_PROFILE}" +fi + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" +UPDATE_ROUTES_DIR="${DIR}/config/update_routes/" + +source "$DIR/.region-config.sh" + +: ${AWS_DEFAULT_REGION:=$DEFAULT_REGION} + +if [ "$APPMESH_ENDPOINT" = "" ]; then + appmesh_cmd="aws appmesh" +else + appmesh_cmd="aws --endpoint-url "${APPMESH_ENDPOINT}" appmesh" +fi + +print() { + printf "[MESH] [$(date)] : %s\n" "$*" +} + +err() { + msg="Error: $1" + print "${msg}" + code=${2:-"1"} + exit ${code} +} + +contains() { + local e match="$1" + shift + for e; do [[ "$e" == "$match" ]] && return 0; done + return 1 +} + +sanity_check() { + if ! contains "${AWS_DEFAULT_REGION}" "${SUPPORTED_REGIONS[@]}"; then + err "Region ${AWS_DEFAULT_REGION} is not supported at this time (Supported regions: ${SUPPORTED_REGIONS[*]})" + fi + + if [ -z "${MESH_NAME}" ]; then + err "MESH_NAME is not set" + fi +} + +update_route() { + route_spec_file=$1 + cmd=( $appmesh_cmd update-route --mesh-name "${MESH_NAME}" \ + ${PROFILE_OPT} \ + --cli-input-json "file:///${route_spec_file}" \ + --query route.metadata.uid --output text ) + print "${cmd[@]}" + uid=$("${cmd[@]}") || err "Unable to update route" "$?" + print "--> ${uid}" +} + +main() { + sanity_check + files=($(ls "${UPDATE_ROUTES_DIR}")) + while true; do \ + f=${files[$RANDOM % ${#files[@]}]} + print "Using route in file ${f}" + update_route "${UPDATE_ROUTES_DIR}/${f}" + sleep ${SLEEP_TIME:-"600s"} + done +} + +main diff --git a/walkthroughs/howto-access-log/colorapp-ecs/xray-container.json b/walkthroughs/howto-access-log/colorapp-ecs/xray-container.json new file mode 100644 index 00000000..1a736a58 --- /dev/null +++ b/walkthroughs/howto-access-log/colorapp-ecs/xray-container.json @@ -0,0 +1,23 @@ +{ + "name": "xray-daemon", + "image": "public.ecr.aws/xray/aws-xray-daemon", + "user": "1337", + "essential": true, + "cpu": 32, + "memoryReservation": 256, + "portMappings": [ + { + "hostPort": 2000, + "containerPort": 2000, + "protocol": "udp" + } + ], + "logConfiguration": { + "logDriver": "awslogs", + "options": { + "awslogs-group": $ECS_SERVICE_LOG_GROUP, + "awslogs-region": $AWS_REGION, + "awslogs-stream-prefix": $AWS_LOG_STREAM_PREFIX + } + } +} \ No newline at end of file diff --git a/walkthroughs/howto-access-log/fargate.png b/walkthroughs/howto-access-log/fargate.png new file mode 100644 index 00000000..cc9ca971 Binary files /dev/null and b/walkthroughs/howto-access-log/fargate.png differ diff --git a/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.sh b/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.sh new file mode 100755 index 00000000..6658a2e8 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-appmesh-mesh" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/appmesh-mesh.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + AppMeshMeshName="${MESH_NAME}" diff --git a/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.yaml b/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.yaml new file mode 100644 index 00000000..c4660ed5 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/appmesh-mesh.yaml @@ -0,0 +1,25 @@ +--- +Parameters: + + EnvironmentName: + Description: An environment name that will be prefixed to resource names + Type: String + + AppMeshMeshName: + Type: String + Description: Name of mesh + +Resources: + + Mesh: + Type: AWS::AppMesh::Mesh + Properties: + MeshName: !Ref AppMeshMeshName + +Outputs: + + Mesh: + Description: A reference to the AppMesh Mesh + Value: !Ref Mesh + Export: + Name: !Sub "${EnvironmentName}:Mesh" diff --git a/walkthroughs/howto-access-log/infrastructure/ecs-cluster.sh b/walkthroughs/howto-access-log/infrastructure/ecs-cluster.sh new file mode 100755 index 00000000..eb32c89e --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/ecs-cluster.sh @@ -0,0 +1,16 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-ecs-cluster" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/ecs-cluster.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + KeyName="${KEY_PAIR_NAME}" \ + ECSServicesDomain="${SERVICES_DOMAIN}" \ + ClusterSize="${CLUSTER_SIZE:-5}" diff --git a/walkthroughs/howto-access-log/infrastructure/ecs-cluster.yaml b/walkthroughs/howto-access-log/infrastructure/ecs-cluster.yaml new file mode 100644 index 00000000..cb2d299c --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/ecs-cluster.yaml @@ -0,0 +1,445 @@ +Description: > + This template deploys an ECS cluster to the provided VPC and subnets + using an Auto Scaling Group + +Parameters: + + EnvironmentName: + Description: An environment name that will be prefixed to resource names + Type: String + + InstanceType: + Description: Which instance type should we use to build the ECS cluster? + Type: String + Default: c4.large + + ClusterSize: + Description: How many ECS hosts do you want to initially deploy? + Type: Number + Default: 5 + + KeyName: + Description: The EC2 Key Pair to allow SSH access to the instances + Type: AWS::EC2::KeyPair::KeyName + + ECSServiceLogGroupRetentionInDays: + Type: Number + Default: 30 + + ECSServicesDomain: + Type: String + Description: "Domain name registerd under Route-53 that will be used for Service Discovery" + + ECSAmi: + Description: ECS AMI ID + Type: AWS::SSM::Parameter::Value + Default: "/aws/service/ecs/optimized-ami/amazon-linux/recommended/image_id" + + EC2Ami: + Description: EC2 AMI ID + Type: AWS::SSM::Parameter::Value + Default: "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2" + +Resources: + + ECSCluster: + Type: AWS::ECS::Cluster + Properties: + ClusterName: !Ref EnvironmentName + + ECSInstancesSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: "Security group for the instances" + VpcId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + SecurityGroupIngress: + - CidrIp: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VpcCIDR" + IpProtocol: -1 + + ECSAutoScalingGroup: + Type: AWS::AutoScaling::AutoScalingGroup + Properties: + VPCZoneIdentifier: + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet1" + - 'Fn::ImportValue': !Sub "${EnvironmentName}:PrivateSubnet2" + LaunchConfigurationName: !Ref ECSLaunchConfiguration + MinSize: !Ref ClusterSize + MaxSize: !Ref ClusterSize + DesiredCapacity: !Ref ClusterSize + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} ECS host + PropagateAtLaunch: true + CreationPolicy: + ResourceSignal: + Timeout: PT15M + UpdatePolicy: + AutoScalingRollingUpdate: + MinInstancesInService: 1 + MaxBatchSize: 1 + PauseTime: PT15M + SuspendProcesses: + - HealthCheck + - ReplaceUnhealthy + - AZRebalance + - AlarmNotification + - ScheduledActions + WaitOnResourceSignals: true + + ECSLaunchConfiguration: + Type: AWS::AutoScaling::LaunchConfiguration + Properties: + ImageId: !Ref ECSAmi + InstanceType: !Ref InstanceType + KeyName: !Ref KeyName + SecurityGroups: + - !Ref ECSInstancesSecurityGroup + IamInstanceProfile: !Ref ECSInstanceProfile + UserData: + "Fn::Base64": !Sub | + #!/bin/bash + yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm + yum install -y aws-cfn-bootstrap hibagent + /opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration + /opt/aws/bin/cfn-signal -e $? --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSAutoScalingGroup + /usr/bin/enable-ec2-spot-hibernation + + Metadata: + AWS::CloudFormation::Init: + config: + packages: + yum: + awslogs: [] + + commands: + 01_add_instance_to_cluster: + command: !Sub echo ECS_CLUSTER=${ECSCluster} >> /etc/ecs/ecs.config + files: + "/etc/cfn/cfn-hup.conf": + mode: 000400 + owner: root + group: root + content: !Sub | + [main] + stack=${AWS::StackId} + region=${AWS::Region} + + "/etc/cfn/hooks.d/cfn-auto-reloader.conf": + content: !Sub | + [cfn-auto-reloader-hook] + triggers=post.update + path=Resources.ECSLaunchConfiguration.Metadata.AWS::CloudFormation::Init + action=/opt/aws/bin/cfn-init -v --region ${AWS::Region} --stack ${AWS::StackName} --resource ECSLaunchConfiguration + + "/etc/awslogs/awscli.conf": + content: !Sub | + [plugins] + cwlogs = cwlogs + [default] + region = ${AWS::Region} + + "/etc/awslogs/awslogs.conf": + content: !Sub | + [general] + state_file = /var/lib/awslogs/agent-state + + [/var/log/dmesg] + file = /var/log/dmesg + log_group_name = ${ECSCluster}-/var/log/dmesg + log_stream_name = ${ECSCluster} + + [/var/log/messages] + file = /var/log/messages + log_group_name = ${ECSCluster}-/var/log/messages + log_stream_name = ${ECSCluster} + datetime_format = %b %d %H:%M:%S + + [/var/log/docker] + file = /var/log/docker + log_group_name = ${ECSCluster}-/var/log/docker + log_stream_name = ${ECSCluster} + datetime_format = %Y-%m-%dT%H:%M:%S.%f + + [/var/log/ecs/ecs-init.log] + file = /var/log/ecs/ecs-init.log.* + log_group_name = ${ECSCluster}-/var/log/ecs/ecs-init.log + log_stream_name = ${ECSCluster} + datetime_format = %Y-%m-%dT%H:%M:%SZ + + [/var/log/ecs/ecs-agent.log] + file = /var/log/ecs/ecs-agent.log.* + log_group_name = ${ECSCluster}-/var/log/ecs/ecs-agent.log + log_stream_name = ${ECSCluster} + datetime_format = %Y-%m-%dT%H:%M:%SZ + + [/var/log/ecs/audit.log] + file = /var/log/ecs/audit.log.* + log_group_name = ${ECSCluster}-/var/log/ecs/audit.log + log_stream_name = ${ECSCluster} + datetime_format = %Y-%m-%dT%H:%M:%SZ + + services: + sysvinit: + cfn-hup: + enabled: true + ensureRunning: true + files: + - /etc/cfn/cfn-hup.conf + - /etc/cfn/hooks.d/cfn-auto-reloader.conf + awslogs: + enabled: true + ensureRunning: true + files: + - /etc/awslogs/awslogs.conf + - /etc/awslogs/awscli.conf + + # This IAM Role is attached to all of the ECS hosts. It is based on the default role + # published here: + # http://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html + # + # You can add other IAM policy statements here to allow access from your ECS hosts + # to other AWS services. Please note that this role will be used by ALL containers + # running on the ECS host. + + ECSInstanceRole: + Type: AWS::IAM::Role + Properties: + Path: / + AssumeRolePolicyDocument: | + { + "Statement": [{ + "Action": "sts:AssumeRole", + "Effect": "Allow", + "Principal": { + "Service": "ec2.amazonaws.com" + } + }] + } + Policies: + - PolicyName: ecs-service + PolicyDocument: | + { + "Statement": [{ + "Effect": "Allow", + "Action": [ + "ecs:CreateCluster", + "ecs:DeregisterContainerInstance", + "ecs:DiscoverPollEndpoint", + "ecs:Poll", + "ecs:RegisterContainerInstance", + "ecs:StartTelemetrySession", + "ecs:Submit*", + "logs:CreateLogStream", + "logs:PutLogEvents", + "ecr:BatchCheckLayerAvailability", + "ecr:BatchGetImage", + "ecr:GetDownloadUrlForLayer", + "ecr:GetAuthorizationToken", + "ssm:DescribeAssociation", + "ssm:GetDeployablePatchSnapshotForInstance", + "ssm:GetDocument", + "ssm:GetManifest", + "ssm:GetParameters", + "ssm:ListAssociations", + "ssm:ListInstanceAssociations", + "ssm:PutInventory", + "ssm:PutComplianceItems", + "ssm:PutConfigurePackageResult", + "ssm:UpdateAssociationStatus", + "ssm:UpdateInstanceAssociationStatus", + "ssm:UpdateInstanceInformation", + "ec2messages:AcknowledgeMessage", + "ec2messages:DeleteMessage", + "ec2messages:FailMessage", + "ec2messages:GetEndpoint", + "ec2messages:GetMessages", + "ec2messages:SendReply", + "cloudwatch:PutMetricData", + "ec2:DescribeInstanceStatus", + "ds:CreateComputer", + "ds:DescribeDirectories", + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:DescribeLogGroups", + "logs:DescribeLogStreams", + "logs:PutLogEvents", + "s3:PutObject", + "s3:GetObject", + "s3:AbortMultipartUpload", + "s3:ListMultipartUploadParts", + "s3:ListBucket", + "s3:ListBucketMultipartUploads" + ], + "Resource": "*" + }] + } + + ECSInstanceProfile: + Type: AWS::IAM::InstanceProfile + Properties: + Path: / + Roles: + - !Ref ECSInstanceRole + + ECSServiceAutoScalingRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + Action: + - 'sts:AssumeRole' + Effect: Allow + Principal: + Service: + - application-autoscaling.amazonaws.com + Path: / + Policies: + - PolicyName: ecs-service-autoscaling + PolicyDocument: + Statement: + Effect: Allow + Action: + - application-autoscaling:* + - cloudwatch:DescribeAlarms + - cloudwatch:PutMetricAlarm + - ecs:DescribeServices + - ecs:UpdateService + Resource: "*" + + ECSServiceSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: "Security group for the service" + VpcId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + SecurityGroupIngress: + - CidrIp: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VpcCIDR" + IpProtocol: -1 + + TaskIamRole: + Type: AWS::IAM::Role + Properties: + Path: / + AssumeRolePolicyDocument: | + { + "Statement": [{ + "Effect": "Allow", + "Principal": { "Service": [ "ecs-tasks.amazonaws.com" ]}, + "Action": [ "sts:AssumeRole" ] + }] + } + ManagedPolicyArns: + - arn:aws:iam::aws:policy/CloudWatchFullAccess + - arn:aws:iam::aws:policy/AWSXRayDaemonWriteAccess + - arn:aws:iam::aws:policy/AWSAppMeshEnvoyAccess + + TaskExecutionIamRole: + Type: AWS::IAM::Role + Properties: + Path: / + AssumeRolePolicyDocument: | + { + "Statement": [{ + "Effect": "Allow", + "Principal": { "Service": [ "ecs-tasks.amazonaws.com" ]}, + "Action": [ "sts:AssumeRole" ] + }] + } + ManagedPolicyArns: + - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly + - arn:aws:iam::aws:policy/CloudWatchLogsFullAccess + + ECSServiceLogGroup: + Type: 'AWS::Logs::LogGroup' + Properties: + RetentionInDays: + Ref: ECSServiceLogGroupRetentionInDays + + ECSServiceDiscoveryNamespace: + Type: AWS::ServiceDiscovery::PrivateDnsNamespace + Properties: + Vpc: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + Name: { Ref: ECSServicesDomain } + + BastionSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Allow http to client host + VpcId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:VPC" + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: 22 + ToPort: 22 + CidrIp: 0.0.0.0/0 + + BastionHost: + Type: AWS::EC2::Instance + Properties: + ImageId: !Ref EC2Ami + KeyName: !Ref KeyName + InstanceType: t2.micro + SecurityGroupIds: + - !Ref BastionSecurityGroup + SubnetId: + 'Fn::ImportValue': !Sub "${EnvironmentName}:PublicSubnet1" + Tags: + - Key: Name + Value: bastion-host + +Outputs: + + Cluster: + Description: A reference to the ECS cluster + Value: !Ref ECSCluster + Export: + Name: !Sub "${EnvironmentName}:ECSCluster" + + ECSServiceAutoScalingRole: + Description: A reference to ECS service auto scaling role + Value: !GetAtt ECSServiceAutoScalingRole.Arn + + ECSAutoScalingGroupName: + Description: A reference to ECS AutoScaling Group Name + Value: !Ref ECSAutoScalingGroup + + ECSServiceDiscoveryNamespace: + Description: A SDS namespace that will be used by all services in this cluster + Value: !Ref ECSServiceDiscoveryNamespace + Export: + Name: !Sub "${EnvironmentName}:ECSServiceDiscoveryNamespace" + + ECSServiceLogGroup: + Description: Log group for services to publish logs + Value: !Ref ECSServiceLogGroup + Export: + Name: !Sub "${EnvironmentName}:ECSServiceLogGroup" + + ECSServiceSecurityGroup: + Description: Security group to be used by all services in the cluster + Value: !Ref ECSServiceSecurityGroup + Export: + Name: !Sub "${EnvironmentName}:ECSServiceSecurityGroup" + + TaskExecutionIamRoleArn: + Description: Task Executin IAM role used by ECS tasks + Value: { "Fn::GetAtt": TaskExecutionIamRole.Arn } + Export: + Name: !Sub "${EnvironmentName}:TaskExecutionIamRoleArn" + + TaskIamRoleArn: + Description: IAM role to be used by ECS task + Value: { "Fn::GetAtt": TaskIamRole.Arn } + Export: + Name: !Sub "${EnvironmentName}:TaskIamRoleArn" + + BastionIP: + Description: Public IP for ssh access to bastion host + Value: + 'Fn::GetAtt': [ BastionHost, PublicIp ] + diff --git a/walkthroughs/howto-access-log/infrastructure/eks-cluster.sh b/walkthroughs/howto-access-log/infrastructure/eks-cluster.sh new file mode 100755 index 00000000..663f1756 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/eks-cluster.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-eks-cluster" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/eks-cluster.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" \ + KeyName="${KEY_PAIR_NAME}" \ No newline at end of file diff --git a/walkthroughs/howto-access-log/infrastructure/eks-cluster.yaml b/walkthroughs/howto-access-log/infrastructure/eks-cluster.yaml new file mode 100644 index 00000000..562713f9 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/eks-cluster.yaml @@ -0,0 +1,252 @@ +Description: > + This template deploys an EKS cluster to the provided VPC and subnets + using an Auto Scaling Group + +Parameters: + + EnvironmentName: + Description: An environment name that will be prefixed to resource names + Type: String + + InstanceType: + Description: Which instance type should we use to build the EKS cluster? + Type: String + Default: c4.large + + NodeAutoScalingGroupMinSize: + Type: Number + Description: Minimum size of Node Group ASG. + Default: 10 + + NodeAutoScalingGroupMaxSize: + Type: Number + Description: Maximum size of Node Group ASG. + Default: 20 + + NodeVolumeSize: + Type: Number + Description: Node volume size + Default: 20 + + NodeGroupName: + Description: Unique identifier for the Node Group. + Type: String + Default: "ng-1" + + BootstrapArguments: + Description: Arguments to pass to the bootstrap script. See files/bootstrap.sh in https://github.com/awslabs/amazon-eks-ami + Default: "" + Type: String + + KeyName: + Description: The EC2 Key Pair to allow SSH access to the instances + Type: AWS::EC2::KeyPair::KeyName + +Mappings: + + # Source: https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html + AWSRegionToEKSAmi: + eu-west-1: + AMI: ami-01e08d22b9439c15a + us-east-1: + AMI: ami-0c24db5df6badc35a + us-east-2: + AMI: ami-0c2e8d28b1f854c68 + us-west-2: + AMI: ami-0a2abab4107669c1b + +Resources: + + AWSServiceRoleForAmazonEKS: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: + - eks.amazonaws.com + Action: + - sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/AmazonEKSServicePolicy + - arn:aws:iam::aws:policy/AmazonEKSClusterPolicy + + EKSControlPlaneSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: "Security group for EKS control-plane" + VpcId: + 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, VPC ] ] + SecurityGroupIngress: + - CidrIp: + 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, VpcCIDR ] ] + IpProtocol: -1 + + EKSCluster: + Type: AWS::EKS::Cluster + Properties: + Name: !Ref EnvironmentName + RoleArn: !GetAtt AWSServiceRoleForAmazonEKS.Arn + ResourcesVpcConfig: + SecurityGroupIds: + - { Ref: EKSControlPlaneSecurityGroup } + SubnetIds: + - 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, PrivateSubnet1 ] ] + - 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, PrivateSubnet2 ] ] + + NodeInstanceProfile: + Type: AWS::IAM::InstanceProfile + Properties: + Path: "/" + Roles: + - !Ref NodeInstanceRole + + NodeInstanceRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: + - ec2.amazonaws.com + Action: + - sts:AssumeRole + Path: "/" + ManagedPolicyArns: + - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy + - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy + - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly + + NodeSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: Security group for all nodes in the cluster + VpcId: + 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, VPC ] ] + Tags: + - Key: !Sub "kubernetes.io/cluster/${EnvironmentName}" + Value: 'owned' + + NodeSecurityGroupIngress: + Type: AWS::EC2::SecurityGroupIngress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow node to communicate with each other + GroupId: !Ref NodeSecurityGroup + SourceSecurityGroupId: !Ref NodeSecurityGroup + IpProtocol: '-1' + FromPort: 0 + ToPort: 65535 + + NodeSecurityGroupFromControlPlaneIngress: + Type: AWS::EC2::SecurityGroupIngress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow worker Kubelets and pods to receive communication from the cluster control plane + GroupId: !Ref NodeSecurityGroup + SourceSecurityGroupId: !Ref EKSControlPlaneSecurityGroup + IpProtocol: tcp + FromPort: 1025 + ToPort: 65535 + + ControlPlaneEgressToNodeSecurityGroup: + Type: AWS::EC2::SecurityGroupEgress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow the cluster control plane to communicate with worker Kubelet and pods + GroupId: !Ref EKSControlPlaneSecurityGroup + DestinationSecurityGroupId: !Ref NodeSecurityGroup + IpProtocol: tcp + FromPort: 1025 + ToPort: 65535 + + NodeSecurityGroupFromControlPlaneOn443Ingress: + Type: AWS::EC2::SecurityGroupIngress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow pods running extension API servers on port 443 to receive communication from cluster control plane + GroupId: !Ref NodeSecurityGroup + SourceSecurityGroupId: !Ref EKSControlPlaneSecurityGroup + IpProtocol: tcp + FromPort: 443 + ToPort: 443 + + ControlPlaneEgressToNodeSecurityGroupOn443: + Type: AWS::EC2::SecurityGroupEgress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow the cluster control plane to communicate with pods running extension API servers on port 443 + GroupId: !Ref EKSControlPlaneSecurityGroup + DestinationSecurityGroupId: !Ref NodeSecurityGroup + IpProtocol: tcp + FromPort: 443 + ToPort: 443 + + EKSControlPlaneSecurityGroupIngress: + Type: AWS::EC2::SecurityGroupIngress + DependsOn: NodeSecurityGroup + Properties: + Description: Allow pods to communicate with the cluster API Server + GroupId: !Ref EKSControlPlaneSecurityGroup + SourceSecurityGroupId: !Ref NodeSecurityGroup + IpProtocol: tcp + ToPort: 443 + FromPort: 443 + + NodeGroup: + Type: AWS::AutoScaling::AutoScalingGroup + DependsOn: EKSCluster + Properties: + DesiredCapacity: !Ref NodeAutoScalingGroupMaxSize + LaunchConfigurationName: !Ref NodeLaunchConfig + MinSize: !Ref NodeAutoScalingGroupMinSize + MaxSize: !Ref NodeAutoScalingGroupMaxSize + VPCZoneIdentifier: + - 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, PrivateSubnet1 ] ] + - 'Fn::ImportValue': !Join [ ":", [ !Ref EnvironmentName, PrivateSubnet2 ] ] + Tags: + - Key: Name + Value: !Sub "${EnvironmentName}-${NodeGroupName}-Node" + PropagateAtLaunch: 'true' + - Key: !Sub 'kubernetes.io/cluster/${EnvironmentName}' + Value: 'owned' + PropagateAtLaunch: 'true' + UpdatePolicy: + AutoScalingRollingUpdate: + MinInstancesInService: '1' + MaxBatchSize: '1' + + NodeLaunchConfig: + Type: AWS::AutoScaling::LaunchConfiguration + Properties: + AssociatePublicIpAddress: 'true' + IamInstanceProfile: !Ref NodeInstanceProfile + ImageId: !FindInMap [AWSRegionToEKSAmi, !Ref "AWS::Region", AMI] + InstanceType: !Ref InstanceType + KeyName: !Ref KeyName + SecurityGroups: + - !Ref NodeSecurityGroup + BlockDeviceMappings: + - DeviceName: /dev/xvda + Ebs: + VolumeSize: !Ref NodeVolumeSize + VolumeType: gp2 + DeleteOnTermination: true + UserData: + Fn::Base64: + !Sub | + #!/bin/bash + set -o xtrace + /etc/eks/bootstrap.sh ${EnvironmentName} ${BootstrapArguments} + /opt/aws/bin/cfn-signal --exit-code $? \ + --stack ${AWS::StackName} \ + --resource NodeGroup \ + --region ${AWS::Region} + +Outputs: + NodeInstanceRole: + Value: !GetAtt NodeInstanceRole.Arn diff --git a/walkthroughs/howto-access-log/infrastructure/vpc.sh b/walkthroughs/howto-access-log/infrastructure/vpc.sh new file mode 100755 index 00000000..1a01628e --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/vpc.sh @@ -0,0 +1,13 @@ +#!/bin/bash + +set -ex + +DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )" + +aws --profile "${AWS_PROFILE}" --region "${AWS_DEFAULT_REGION}" \ + cloudformation deploy \ + --stack-name "${ENVIRONMENT_NAME}-vpc" \ + --capabilities CAPABILITY_IAM \ + --template-file "${DIR}/vpc.yaml" \ + --parameter-overrides \ + EnvironmentName="${ENVIRONMENT_NAME}" diff --git a/walkthroughs/howto-access-log/infrastructure/vpc.yaml b/walkthroughs/howto-access-log/infrastructure/vpc.yaml new file mode 100644 index 00000000..f5327b76 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/vpc.yaml @@ -0,0 +1,241 @@ +Description: > + This template deploys a VPC, with a pair of public and private subnets spread + across two Availabilty Zones. It deploys an Internet Gateway, with a default + route on the public subnets. It deploys a pair of NAT Gateways (one in each AZ), + and default routes for them in the private subnets. + +Parameters: + + EnvironmentName: + Description: An environment name that will be prefixed to resource names + Type: String + + VpcCIDR: + Description: Please enter the IP range (CIDR notation) for this VPC + Type: String + Default: 10.0.0.0/16 + + PublicSubnet1CIDR: + Description: Please enter the IP range (CIDR notation) for the public subnet in the first Availability Zone + Type: String + Default: 10.0.0.0/19 + + PublicSubnet2CIDR: + Description: Please enter the IP range (CIDR notation) for the public subnet in the second Availability Zone + Type: String + Default: 10.0.32.0/19 + + PrivateSubnet1CIDR: + Description: Please enter the IP range (CIDR notation) for the private subnet in the first Availability Zone + Type: String + Default: 10.0.64.0/19 + + PrivateSubnet2CIDR: + Description: Please enter the IP range (CIDR notation) for the private subnet in the second Availability Zone + Type: String + Default: 10.0.96.0/19 + +Resources: + + VPC: + Type: AWS::EC2::VPC + Properties: + CidrBlock: !Ref VpcCIDR + EnableDnsHostnames: true + Tags: + - Key: Name + Value: !Ref EnvironmentName + + InternetGateway: + Type: AWS::EC2::InternetGateway + Properties: + Tags: + - Key: Name + Value: !Ref EnvironmentName + + InternetGatewayAttachment: + Type: AWS::EC2::VPCGatewayAttachment + Properties: + InternetGatewayId: !Ref InternetGateway + VpcId: !Ref VPC + + PublicSubnet1: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + AvailabilityZone: !Select [ 0, !GetAZs '' ] + CidrBlock: !Ref PublicSubnet1CIDR + MapPublicIpOnLaunch: true + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Public Subnet (AZ1) + + PublicSubnet2: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + AvailabilityZone: !Select [ 1, !GetAZs '' ] + CidrBlock: !Ref PublicSubnet2CIDR + MapPublicIpOnLaunch: true + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Public Subnet (AZ2) + + PrivateSubnet1: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + AvailabilityZone: !Select [ 0, !GetAZs '' ] + CidrBlock: !Ref PrivateSubnet1CIDR + MapPublicIpOnLaunch: false + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Private Subnet (AZ1) + - Key: "kubernetes.io/role/internal-elb" + Value: "1" + + PrivateSubnet2: + Type: AWS::EC2::Subnet + Properties: + VpcId: !Ref VPC + AvailabilityZone: !Select [ 1, !GetAZs '' ] + CidrBlock: !Ref PrivateSubnet2CIDR + MapPublicIpOnLaunch: false + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Private Subnet (AZ2) + - Key: "kubernetes.io/role/internal-elb" + Value: "1" + + NatGateway1EIP: + Type: AWS::EC2::EIP + DependsOn: InternetGatewayAttachment + Properties: + Domain: vpc + + NatGateway2EIP: + Type: AWS::EC2::EIP + DependsOn: InternetGatewayAttachment + Properties: + Domain: vpc + + NatGateway1: + Type: AWS::EC2::NatGateway + Properties: + AllocationId: !GetAtt NatGateway1EIP.AllocationId + SubnetId: !Ref PublicSubnet1 + + NatGateway2: + Type: AWS::EC2::NatGateway + Properties: + AllocationId: !GetAtt NatGateway2EIP.AllocationId + SubnetId: !Ref PublicSubnet2 + + PublicRouteTable: + Type: AWS::EC2::RouteTable + Properties: + VpcId: !Ref VPC + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Public Routes + + DefaultPublicRoute: + Type: AWS::EC2::Route + DependsOn: InternetGatewayAttachment + Properties: + RouteTableId: !Ref PublicRouteTable + DestinationCidrBlock: 0.0.0.0/0 + GatewayId: !Ref InternetGateway + + PublicSubnet1RouteTableAssociation: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + RouteTableId: !Ref PublicRouteTable + SubnetId: !Ref PublicSubnet1 + + PublicSubnet2RouteTableAssociation: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + RouteTableId: !Ref PublicRouteTable + SubnetId: !Ref PublicSubnet2 + + PrivateRouteTable1: + Type: AWS::EC2::RouteTable + Properties: + VpcId: !Ref VPC + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Private Routes (AZ1) + + DefaultPrivateRoute1: + Type: AWS::EC2::Route + Properties: + RouteTableId: !Ref PrivateRouteTable1 + DestinationCidrBlock: 0.0.0.0/0 + NatGatewayId: !Ref NatGateway1 + + PrivateSubnet1RouteTableAssociation: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + RouteTableId: !Ref PrivateRouteTable1 + SubnetId: !Ref PrivateSubnet1 + + PrivateRouteTable2: + Type: AWS::EC2::RouteTable + Properties: + VpcId: !Ref VPC + Tags: + - Key: Name + Value: !Sub ${EnvironmentName} Private Routes (AZ2) + + DefaultPrivateRoute2: + Type: AWS::EC2::Route + Properties: + RouteTableId: !Ref PrivateRouteTable2 + DestinationCidrBlock: 0.0.0.0/0 + NatGatewayId: !Ref NatGateway2 + + PrivateSubnet2RouteTableAssociation: + Type: AWS::EC2::SubnetRouteTableAssociation + Properties: + RouteTableId: !Ref PrivateRouteTable2 + SubnetId: !Ref PrivateSubnet2 + +Outputs: + + VPC: + Description: A reference to the created VPC + Value: !Ref VPC + Export: + Name: !Sub "${EnvironmentName}:VPC" + + PublicSubnet1: + Description: A reference to the public subnet in the 1st Availability Zone + Value: !Ref PublicSubnet1 + Export: + Name: !Sub "${EnvironmentName}:PublicSubnet1" + + PublicSubnet2: + Description: A reference to the public subnet in the 2nd Availability Zone + Value: !Ref PublicSubnet2 + Export: + Name: !Sub "${EnvironmentName}:PublicSubnet2" + + PrivateSubnet1: + Description: A reference to the private subnet in the 1st Availability Zone + Value: !Ref PrivateSubnet1 + Export: + Name: !Sub "${EnvironmentName}:PrivateSubnet1" + + PrivateSubnet2: + Description: A reference to the private subnet in the 2nd Availability Zone + Value: !Ref PrivateSubnet2 + Export: + Name: !Sub "${EnvironmentName}:PrivateSubnet2" + + VpcCIDR: + Description: VPC CIDR + Value: !Ref VpcCIDR + Export: + Name: !Sub "${EnvironmentName}:VpcCIDR" + diff --git a/walkthroughs/howto-access-log/infrastructure/wipe_out_mesh.sh b/walkthroughs/howto-access-log/infrastructure/wipe_out_mesh.sh new file mode 100755 index 00000000..ae4d6df6 --- /dev/null +++ b/walkthroughs/howto-access-log/infrastructure/wipe_out_mesh.sh @@ -0,0 +1,73 @@ +#!/bin/bash + +set -ex + +if [ ! -z "${AWS_PROFILE}" ]; then + PROFILE_OPT="--profile ${AWS_PROFILE}" +fi + +if [ "$APPMESH_ENDPOINT" = "" ]; then + appmesh_cmd="aws appmesh" +else + appmesh_cmd="aws --endpoint-url "${APPMESH_ENDPOINT}" appmesh" +fi + +virtual_services=($($appmesh_cmd list-virtual-services \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + | jq -r ".virtualServices[].virtualServiceName")) + +for virtual_service_name in ${virtual_services[@]} +do + $appmesh_cmd delete-virtual-service \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + --virtual-service-name "${virtual_service_name}" || echo "Unable to delete virtual-service $virtual_service_name $?" +done + +virtual_routers=($($appmesh_cmd list-virtual-routers \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + | jq -r ".virtualRouters[].virtualRouterName")) + +for virtual_router_name in ${virtual_routers[@]} +do + routes=($($appmesh_cmd list-routes \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + --virtual-router-name "${virtual_router_name}" \ + | jq -r ".routes[].routeName")) + + for route_name in ${routes[@]} + do + $appmesh_cmd delete-route \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + --virtual-router-name "${virtual_router_name}" \ + --route-name "${route_name}" || echo "Unable to delete route $route_name $?" + done +done + +for virtual_router_name in ${virtual_routers[@]} +do + $appmesh_cmd delete-virtual-router \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + --virtual-router-name "${virtual_router_name}" || echo "Unable to delete virtual-router $virtual_router_name $?" +done + + +virtual_nodes=($($appmesh_cmd list-virtual-nodes \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + | jq -r ".virtualNodes[].virtualNodeName")) + +for virtual_node_name in ${virtual_nodes[@]} +do + $appmesh_cmd delete-virtual-node \ + ${PROFILE_OPT} \ + --mesh-name "${MESH_NAME}" \ + --virtual-node-name "$virtual_node_name" || echo "Unable to delete virtual-node $virtual_node_name $?" +done + +$appmesh_cmd delete-mesh ${PROFILE_OPT} --mesh-name "${MESH_NAME}" || echo "Unable to delete mesh ${MESH_NAME} $?" diff --git a/walkthroughs/howto-access-log/src/blue-json-format.json b/walkthroughs/howto-access-log/src/blue-json-format.json new file mode 100644 index 00000000..4b3578ea --- /dev/null +++ b/walkthroughs/howto-access-log/src/blue-json-format.json @@ -0,0 +1,48 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout", + "format": { + "json": [ + { + "key": "BlueTestLog", + "value": "%RESPONSE_CODE%" + }, + { + "key": "protocol", + "value": "%PROTOCOL%" + } + ] + } + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-blue.logging.local" + } + } + }, + "virtualNodeName": "colorteller-blue-vn" +} diff --git a/walkthroughs/howto-access-log/src/blue-no-format.json b/walkthroughs/howto-access-log/src/blue-no-format.json new file mode 100644 index 00000000..34f3953f --- /dev/null +++ b/walkthroughs/howto-access-log/src/blue-no-format.json @@ -0,0 +1,36 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout" + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-blue.logging.local" + } + } + }, + "virtualNodeName": "colorteller-blue-vn" +} diff --git a/walkthroughs/howto-access-log/src/blue-text-format.json b/walkthroughs/howto-access-log/src/blue-text-format.json new file mode 100644 index 00000000..2b6fab7c --- /dev/null +++ b/walkthroughs/howto-access-log/src/blue-text-format.json @@ -0,0 +1,39 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout", + "format": { + "text": "BlueTestLog:%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n" + } + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-blue.logging.local" + } + } + }, + "virtualNodeName": "colorteller-blue-vn" +} diff --git a/walkthroughs/howto-access-log/src/green-json-format.json b/walkthroughs/howto-access-log/src/green-json-format.json new file mode 100644 index 00000000..578930b3 --- /dev/null +++ b/walkthroughs/howto-access-log/src/green-json-format.json @@ -0,0 +1,48 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout", + "format": { + "json": [ + { + "key": "GreenTestLog", + "value": "%RESPONSE_CODE%" + }, + { + "key": "protocol", + "value": "%PROTOCOL%" + } + ] + } + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-green.logging.local" + } + } + }, + "virtualNodeName": "colorteller-green-vn" +} diff --git a/walkthroughs/howto-access-log/src/green-no-format.json b/walkthroughs/howto-access-log/src/green-no-format.json new file mode 100644 index 00000000..4517f68a --- /dev/null +++ b/walkthroughs/howto-access-log/src/green-no-format.json @@ -0,0 +1,36 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout" + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-green.logging.local" + } + } + }, + "virtualNodeName": "colorteller-green-vn" +} diff --git a/walkthroughs/howto-access-log/src/green-text-format.json b/walkthroughs/howto-access-log/src/green-text-format.json new file mode 100644 index 00000000..928cbaf6 --- /dev/null +++ b/walkthroughs/howto-access-log/src/green-text-format.json @@ -0,0 +1,39 @@ +{ + "meshName": "appmesh-mesh-logging", + "spec": { + "backends": [], + "listeners": [ + { + "healthCheck": { + "healthyThreshold": 2, + "intervalMillis": 5000, + "path": "/ping", + "port": 9080, + "protocol": "http", + "timeoutMillis": 2000, + "unhealthyThreshold": 2 + }, + "portMapping": { + "port": 9080, + "protocol": "http" + } + } + ], + "logging": { + "accessLog": { + "file": { + "path": "/dev/stdout", + "format": { + "text": "GreenTestLog:%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n" + } + } + } + }, + "serviceDiscovery": { + "dns": { + "hostname": "colorteller-green.logging.local" + } + } + }, + "virtualNodeName": "colorteller-green-vn" +}