Howto get kubernetes anywhere with vsphere and kubernetes 1.11.1 working behind proxy (no direct internet access)

Welcome to the kubernetes-anywhere wiki!

If you need a proxy, the deployment is a little bit harder and need pre planning the cluster infrastructure.

Deployment Network: 10.10.10.0/24

vCenter IP: 10.10.100.6 NodeIPs: 10.10.10.102, 10.10.10.103, 10.10.10.104, 10.10.10.105, 10.10.10.106 Internet Proxy: 10.10.80.98

All IPs you want to use, need to configure in different locations!

First at all, you have to manipulate the PhotonOS Template: Start the VM, login with root/kubernetes Edit /etc/sysconfig/proxy USE_PROXY=yes HTTP_PROXY="http://10.XXX.XXX.XXX:8080/" HTTPS_PROXY="http://10.XXX.XXX.XXX:8080/" NO_PROXY="127.0.0.1, localhost, 10.89.200.6,10.10.10.102, 10.10.10.103, 10.10.10.104, 10.10.10.105, 10.10.10.106"

Next Step: run the docker "deployment image" with env variables for proxy:

docker pull cnastorage/kubernetes-anywhere:latest docker run -it -v /tmp:/tmp --rm --env="PS1=[container]:\w> " \ --env="NO_PROXY=localhost,127.0.0.1,10.10.100.6,10.10.10.102,10.10.10.103,10.10.10.104,10.10.10.105,10.10.10.106" \ --env "HTTPS_PROXY=http://10.10.80.98:8080/" \ --env "HTTP_PROXY=http://10.10.80.98:8080/" \ --net=host cnastorage/kubernetes-anywhere:latest /bin/bash

Now in container shell, make config with your settings:

[container]:/opt/kubernetes-anywhere> make config

Use following Options for kubernetes v1.11.1 : # Phase 2: Node Bootstrapping # .phase2.kubernetes_version="v1.11.1" .phase2.provider="ignition" .phase2.installer_container="docker.io/cnastorage/k8s-ignition:v1.8-dev-release" .phase2.docker_registry="gcr.io/google-containers"

now we have to say the nodes, that docker must need a proxy: edit file: kubernetes-anywhere/phase1/vsphere/configure-vm.sh near line 60 (after echo "${nodes_dns_mappings}" >> /etc/hosts)

add these lines : echo 'HTTP_PROXY=\"http:///10.10.80.98:8080/\"' >> /etc/default/docker echo 'HTTPS_PROXY=\"http:///10.10.80.98:8080/\"' >> /etc/default/docker echo 'NO_PROXY="localhost,127.0.0.1,10.10.100.6,10.10.10.102,10.10.10.103,10.10.10.104,10.10.10.105,10.10.10.106"' >> /etc/default/docker

save the file...

then: make deploy

at my site the cluster created successfully.

dont forget to copy phase1/vsphere/<your_defined_clustername>/kubeconfig.json before closing the deployment container

Now you have to use a seperate machine with kubectl kubectl proxy --address='10.10.10.100' --port=8001 --accept-hosts='.*' since any one version, the access to WebUI is following: http://10.10.10.100:8011/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/#/workload?namespace=default