Skip to content

0.7 AWS Platform Deployment

Jump to bottom
Piotr Milewski edited this page Jan 23, 2017 · 25 revisions

Prerequisites

Note: TAP install requires Internet connectivity

  1. An Amazon Web Services (AWS) account with Virtual Private Cloud (VPC).

  2. Root account credentials with full access to all resources in the account.

  3. The EC2 Instance Limit increased to 40.

  4. An SMTP server.

    ℹ️ Information
    Amazon Simple Email Service or Mailgun can be used if you don't have an SMTP server.

  5. An Elastic IP address and a configured domain.

    ℹ️ Information
    The xip.io service can be used, only for development or evaluation purposes, if you don't have a domain.

  6. A key pair. The key pair you choose will be used to access the created VMs via SSH during and after the installation completes.

    ℹ️ Information
    Please note that key pairs are region specific, make sure that the key pair you chose is available in the region you are doing the installation!

Create a Stack

Start in AWS

  1. Click Launch Stack button.

  2. Choose Next.

  3. In the Name box, type a name for the stack (for example: TAP).

  4. In Parameters, provide required parameters, and then choose Next.

    ℹ️ Information
    Changing the instance sizes for Cloudera and Cloud Foundry is not recommended.
    This is mainly intended for special use cases like minimizing instance cost when unused Reserved Instances are available in your account.

  5. Select the I acknowledge that this template might cause AWS CloudFormation to create IAM resources box.

  6. To monitor the installation progress, check the Events tab of the AWS Cloud Formation: 

    This provides a real-time view of resource creation, and in case of failure shows the problem that caused it.  

  7. This part of the installation is successful once the stack has entered the CREATE_COMPLETE stage, and unsuccessful otherwise. Please check the "Accessing installation logs" at the end of this document to check how to troubleshoot the process.

    ℹ️ Information
    Please note that in case of failure the environment will not be able to delete itself. This is related to the way BOSH manages virtual machines.
    To delete an environment that went to the CREATE_FAILED or ROLLBACK_FAILED states, please go to the "Removing an environment" section at the end of the page.

Deploy the platform

  1. When the stack is created - log in to a Jump Box instance using SSH: ssh ubuntu@<JumpBoxEIP> -i <key.pem>

  2. Run a shell script to finish the installation:

    1. with Kerberos disabled:

    sudo -i curl -Sso tqd.sh https://s3.amazonaws.com/trustedanalytics/tqd.sh && sudo -i bash tqd.sh

    1. with Kerberos enabled:

    sudo -i curl -Sso tqd.sh https://s3.amazonaws.com/trustedanalytics/tqd.sh && sudo -i KERBEROS_ENABLED=True bash tqd.sh

    ℹ️ Information
    There is an additional flags you can pass to the tqd.sh script:
    KUBERNETES_ENABLED which turns on an experimental feature of TAP, multi-node services on Kubernetes.

The whole deployment process should take from 2 to 5 hours. Once the process is complete (the script finishes without writing about failure), you can access the TAP console via https://console.DOMAIN_NAME_YOU_CHOSE and login with the username admin and the password provided as a parameter to Cloud Formation.

To access individual VMs, please SSH into the Jump Box machine using the procedure from the "Accessing installation logs" section below.

Troubleshooting

Accessing AWS installation logs

  1. Go to the AWS Cloud Formation Resources tab and get the Jump box IP address:
  2. SSH to the instance using the user ubuntu and the key provided during the installation.
  3. Search /var/log/ansible.log for failed steps.

Accessing CDH manager

  1. Log in to a Jump Box instance using SSH with port forwarding set up to the cdh-master-2 machine: ssh ubuntu@<JumpBoxEIP> -L 7180:cdh-master-2:7180 -i <key.pem>
  2. You should be able to access the CDH Manager web UI via http://localhost:7180

Removing an AWS environment

  1. Go to the AWS EC2 Instances tab and search for the stack name you want to delete.
  2. If you selected "Termination protection" during the deployment, you need to turn it off for the Jump Box, Cloudera Manager and NGINX instances by right-clicking them and selecting "Change Termination Protection":
  3. Select all of the machines and right click and terminate them:
  4. Go to the AWS Cloud Formation.
  5. Right click on the stack you wish to remove and select "Delete Stack".
  6. The stack should get deleted without problems, removing the environment.
  7. Delete SSH key "{your stack name}-key" from AWS EC2 Key Pairs.

Platform upgrades

Upgrade from version 0.7.0 to version 0.7.1

  1. Log in to a Jump Box instance using SSH: ssh ubuntu@<JumpBoxEIP> -i <key.pem>

  2. Run a shell script to finish the installation:

    1. with Kerberos disabled:

    sudo -i curl -Sso update_v0.7.0_to_v0.7.1.sh https://s3.amazonaws.com/trustedanalytics/update_v0.7.0_to_v0.7.1.sh && sudo -i bash update_v0.7.0_to_v0.7.1.sh

    1. with Kerberos enabled:

    sudo -i curl -Sso update_v0.7.0_to_v0.7.1.sh https://s3.amazonaws.com/trustedanalytics/update_v0.7.0_to_v0.7.1.sh && sudo -i KERBEROS_ENABLED=True bash update_v0.7.0_to_v0.7.1.sh

For concrete upgrade contents please refer to 0.7.1 Release Notes

Home | FAQs

Clone this wiki locally