This website uses cookies to improve its user experience and provide personalized content for you. We use cookies for web analytics and advertising.
You can accept these cookies by clicking "OK" or go to Details in order to manage your cookies preferences more precisely. To learn more, check out our Privacy and Cookies Policy
Essential website cookies are necessary to provide you with services available through the website, autosave your settings and preferences, and to enhance the performance and security of the website - you have the right not to accept them through you web browser's settings, but your access to some functionality and areas of our website may be restricted.
Analytics cookies: (our own and third-party : Google, HotJar) – you can accept these cookies below:
Marketing cookies (third-party cookies: Hubspot, Facebook, LinkedIn) – you can accept these cookies below:
efk_deployment_enabled
parameter added to Opscontrol terraform.tfvars
file. It allows to disable the ELK (OpenSearch) installation in Opscontrol. Default value is true
.velero_snapshot_volumes
parameter in k8s-deployment.json
which can be used to enable PersistentVolume snapshot feautre in Velero backup service.ansible_strategy
parameter added to Opscontrol terraform.tfvars
file. You can check possible values here: https://docs.ansible.com/ansible/latest/user_guide/playbooks_strategies.htmlconcourse_ui_certificate_name
and grafana_certificate_name
from terraform.tfvars
users
parameter in config.json
that can be used to add custom users to all jumpboxes (Opscontrol and all Environments). This is an array of object with two parameters:name
– string with usernamessh_key
– base64 encoded public key. Note: public key should include in a comment valid user email address
# Example of config.json file with users
{
"envs": [
{
"name": "test",
"backend_type": "aws",
"config_repo_url": "...",
"config_repo_branch": "..."
}
],
"users": [
{
"name": "test",
"ssh_key": "<base64_encoded_public_key>"
}
]
}
env.json
:
(...)
{
"name": "update_users",
"file": "ci/pipelines/update-users.yml",
"vars": [
{"name": "timer_interval", "value": "24h"}
]
},
(...)
bosh_
variables from env.json
bosh delete-env -n \
--state /etc/bosh-state/state.json \
--vars-store /etc/bosh-state/creds.yml \
~/configure_jumpbox_bosh_workspace/manifest.yml
ansible_strategy
parameter in k8s-deployment.json
which can be used to modify strategy for kubespray deployment. See above example from terraform.tfvars to see possible values. Note: you can use "opscontrol_var": "ansible_strategy"
to reuse value from Opscontrol.sudo kubeadm init --config /etc/kubernetes/kubeadm-config.yaml phase control-plane all
# Name for DNS virtual machine template
# Default: 13
dns_template_name = "..."
# Variable to specify hardware version
vm_hardware_version = 18
# Filenames for key-pairs in sensitive-data bucket
sensitive_data_k8s_public_key_filename = "k8s.pub"
sensitive_data_k8s_private_key_filename = "k8s.key"
sensitive_data_dns_public_key_filename = "dns.pub"
sensitive_data_dns_pprivate_key_filename = "dns.key"
# Flag that indicates whether OpsControl should use external cloud provider.
# Default: true
use_external_cloud_provider = ((use_external_cloud_provider))
# The name of the AMI
ami_name = "..."
# VM instance type that should be used for DNS
dns_instance_type = "..."
# The ID of the hosted Route53 zone to contain DNS records
hosted_zone_id = "..."
# Private IP address of DNS instance
dns_instance_private_ip = "..."
# Filenames for key-pairs in sensitive-data bucket
sensitive_data_k8s_public_key_filename = "k8s.pub"
sensitive_data_k8s_private_key_filename = "k8s.key"
sensitive_data_dns_public_key_filename = "dns.pub"
sensitive_data_dns_pprivate_key_filename = "dns.key"
# URI for docker image repository
docker_image_repo = "..."
bosh delete-deployment -d dns
kubectl delete -n concourse sts concourse-postgresql
kubectl delete -n concourse service concourse-web
kubectl delete -n concourse service concourse-web-worker-gateway
# common.json:
{"name": "k8s_private_key", "opscontrol_var": "k8s_private_key"},
{"name": "k8s_public_key", "opscontrol_var": "k8s_public_key"},
{"name": "dns_private_key", "opscontrol_var": "dns_private_key"},
{"name": "dns_public_key", "opscontrol_var": "dns_public_key"},
{"name": "vm_hardware_version", "opscontrol_var": "vm_hardware_version"},
# env.json:
{"name": "k8s_lb_enabled", "value": "false"},
{"name": "k8s_lb_cidr", "value": "10.92.1.128/26"},
{"name": "k8s_lb_gateway", "value": "10.92.1.129"},
{"name": "k8s_lb_allocation_start", "value": "10.92.1.170"},
{"name": "k8s_lb_allocation_end", "value": "10.92.1.180"},
{"name": "k8s_node_ports_enabled", "value": "false"},
{"name": "k8s_node_ports_tcp", "value": "[\"30000-32767\"]"},
{"name": "k8s_node_ports_udp", "value": "[\"30000-32767\"]"},
{"name": "k8s_node_ports_whitelist", "value": "[\"100.64.112.0/24\"]"},
# k8s-deployment.json:
{"name": "k8s_vm_hardware_version", "opscontrol_var": "vm_hardware_version"},
{"name": "enable_lb_service", "value": "false"},
# common.json:
{"name": "dns_private_key", "opscontrol_var": "dns_private_key"},
{"name": "dns_public_key", "opscontrol_var": "dns_public_key"},
{"name": "k8s_private_key", "opscontrol_var": "k8s_private_key"},
{"name": "k8s_public_key", "opscontrol_var": "k8s_public_key"},
{"name": "hosted_zone_id", "opscontrol_var": "hosted_zone_id"},
{"name": "ami_name", "opscontrol_var": "ami_name"},
# env.json:
- {"name": "dns_private_master_ip", "value": "10.90.2.141"},
- {"name": "dns_private_slave_ip", "value": "10.90.2.142"},
+ {"name": "dns_instance_private_ip", "value": "10.90.2.141"},
bosh delete-deployment -d dns
sudo kubeadm init --config /etc/kubernetes/kubeadm-config.yaml phase control-plane all
ingress_type
added. Possible values are traefik
, nginx
or none
docker_image_repo
parameter in OpsControl to overwrite default dockerhub.
- {"name": "traefik_certificate_bucket", "value": ""},
- {"name": "traefik_certificate_files", "value": ""},
+ {"name": "ingress_additional_files_bucket", "value": ""},
+ {"name": "ingress_additional_files", "value": ""},
+ {"name": "ingress_type", "value": "traefik"},
(...)
- {"name": "windows_worker_ips", "value": ""},
+ {"name": "k8s_worker_is_windows", "value": "false"},
terraform state mv module.nsxt_policy[0].module.instances.vsphere_virtual_machine.jumpbox module.nsxt_policy[0].module.instances.vsphere_virtual_machine.vm
kubectl delete ds traefik-ingress-controller -n traefik-ingress
bosh delete-deployment -d concourse
docker_image_repo
property affects only OpsControl and can be overwriten for control-plane with extensions.docker_image_repo
added to support custom docker repository
# renamed:
jumpbox_public_ip -> jumpbox_ip
# added
jumpbox_network_cidr" (dmz will be used by default)
kube_version
vsphere_k8s_username
vsphere_k8s_password
control_plane_template_name
control_plane_master_ips
control_plane_master_cpu
control_plane_master_ram
control_plane_master_network_name
control_plane_master_gateway_ip
control_plane_master_network_cidr
control_plane_worker_ips
control_plane_worker_cpu
control_plane_worker_ram
control_plane_worker_disk
# removed:
jumpbox_private_ip
# new parameters
{"name": "ntp_servers", "value": "[ntp.ubuntu.com, ntp.ubuntu.local]"},
# moved form env.json
{"name": "vcenter_datastore", "opscontrol_var": "vcenter_ds"},
{"name": "vcenter_cluster", "opscontrol_var": "vcenter_cluster"},
{"name": "vcenter_allow_unverified_ssl", "opscontrol_var": "vcenter_allow_unverified_ssl"},
{"name": "vcenter_resource_pool", "value": "#####"},
{"name": "nsx_password", "opscontrol_var": "nsx_password"},
{"name": "nsx_user", "opscontrol_var": "nsx_user"},
{"name": "nsx_host", "opscontrol_var": "nsx_address"},
{"name": "nsx_allow_unverified_ssl", "opscontrol_var": "nsx_allow_unverified_ssl"},
{"name": "nsx_ca", "opscontrol_var": "nsx_ca"},
{"name": "nsx_remote_auth", "opscontrol_var": "nsx_remote_auth"},
{"name": "nsx_policy_api", "opscontrol_var": "nsx_policy_api"},
{"name": "tier0_router_name", "opscontrol_var": "tier0_router_name"},
{"name": "translated_snat_ip", "opscontrol_var": "translated_snat_ip"},
{"name": "overlay_tz_name", "opscontrol_var": "overlay_tz_name"},
{"name": "edge_cluster_name", "value": "#####"},
{"name": "public_dns_ip", "value": "#####"},
{"name": "dns_instance_private_ip", "value": "#####"},
{"name": "jumpbox_public_key", "opscontrol_var": "jumpbox_public_key"},
# moved to common.json
{"name": "vcenter_datastore", "opscontrol_var": "vcenter_ds"},
{"name": "vcenter_cluster", "opscontrol_var": "vcenter_cluster"},
{"name": "vcenter_allow_unverified_ssl", "opscontrol_var": "vcenter_allow_unverified_ssl"},
{"name": "vcenter_resource_pool", "value": "#####"},
{"name": "nsx_password", "opscontrol_var": "nsx_password"},
{"name": "nsx_user", "opscontrol_var": "nsx_user"},
{"name": "nsx_host", "opscontrol_var": "nsx_address"},
{"name": "nsx_allow_unverified_ssl", "opscontrol_var": "nsx_allow_unverified_ssl"},
{"name": "nsx_ca", "opscontrol_var": "nsx_ca"},
{"name": "nsx_remote_auth", "opscontrol_var": "nsx_remote_auth"},
{"name": "nsx_policy_api", "opscontrol_var": "nsx_policy_api"},
{"name": "tier0_router_name", "opscontrol_var": "tier0_router_name"},
{"name": "translated_snat_ip", "opscontrol_var": "translated_snat_ip"},
{"name": "overlay_tz_name", "opscontrol_var": "overlay_tz_name"},
{"name": "edge_cluster_name", "value": "#####"},
{"name": "public_dns_ip", "value": "#####"},
{"name": "dns_instance_private_ip", "value": "#####"},
{"name": "jumpbox_public_key", "opscontrol_var": "jumpbox_public_key"},
# added:
{"name": "k8s_version", "value":"v1.19.7"},
{"name": "vcenter_k8s_user", "opscontrol_var": "vcenter_k8s_user"},
{"name": "vcenter_k8s_password", "opscontrol_var": "vcenter_k8s_password"},
{"name": "use_external_cloud_provider", "value":"true"},
{"name": "k8s_template_name", "value": "####"},
{"name": "k8s_master_ips", "value": "####"},
{"name": "k8s_master_cpu", "value": "8"},
{"name": "k8s_master_ram", "value": "8096"},
{"name": "k8s_master_network", "value": "k8s"},
{"name": "k8s_master_network_cidr", "value": "####"},
{"name": "k8s_master_gateway_ip", "value": "####"},
{"name": "k8s_worker_ips", "value": "####"},
{"name": "k8s_worker_cpu", "value": "8"},
{"name": "k8s_worker_ram", "value": "8096"},
{"name": "k8s_worker_disk", "value": "200"},
{"name": "k8s_worker_network", "value": "k8s"},
{"name": "k8s_worker_network_cidr", "value": "####"},
{"name": "k8s_worker_gateway_ip", "value": "####"},
// Configuration requires Windows parameters even if we put empty values
{"name": "windows_worker_ips", "value": ""},
{"name": "windows_template_name", "value": ""},
{"name": "windows_admin_password", "value": ""},
{"name": "windows_netmask", "value": ""},
# removed:
{"name": "k8s_masters", "value": "1"},
{"name": "k8s_workers", "value": "2"},
{"name": "k8s_masters_type", "value": "general_small"},
{"name": "k8s_workers_type", "value": "storage_large"},
{"name": "k8s_network_name", "value": "k8s"},
{"name": "k8s_network_sg", "value": "k8s-sg"},
# renamed:
- {"name": "extensions_provider_directory", "value": "vsphere/env/cb-k8s-provider-deployment"},
- {"name": "extensions_provider_properties", "value": "k8s-provider.properties"}
+ {"name": "extensions_terraform_directory", "value": "vsphere/env/cb-k8s-provider-deployment"},
+ {"name": "extensions_terraform_properties", "value": "k8s.tfvars"}
# renamed:
bosh_ip -> bosh_private_ip
concourse_cert -> concourse_ui_cert
vsphere_bosh_datastore -> vsphere_bosh_datastore_name
# added:
nsxt_remote_auth - bool, indicates whether Terraform should use remote auth with NSX-T
nsxt_policy_api - bool, indicates whether Terraform should use PolicyAPI or ManagerAPI with NSX-T
# removed:
control_plane_certificate_name
{"name": "nsx_remote_auth", "opscontrol_var": "nsx_remote_auth"},
{"name": "nsx_policy_api", "opscontrol_var": "nsx_policy_api"},
{"name": "cf_router_lb_app_profile_name", "value": "((cf_router_lb_app_profile_name))"},
{"name": "cf_router_lb_client_ssl_profile_name", "value": "((cf_router_lb_client_ssl_profile_name))"},
{"name": "cf_ssh_lb_app_profile_name", "value": "((cf_ssh_lb_app_profile_name))"},
{"name": "k8s_lb_app_profile_name", "value": "((k8s_lb_app_profile_name))"},
{"name": "enable_cf", "value": "((enable_cf))"},
{"name": "extensions_terraform_directory", "value": "vsphere/env/cb-env-deployment"},
{"name": "extensions_terraform_properties", "value": "terraform.tfvars"},
terraform state replace-provider -state terraform.tfstate registry.terraform.io/-/vsphere registry.terraform.io/hashicorp/vsphere
terraform state replace-provider -state terraform.tfstate registry.terraform.io/-/nsxt registry.terraform.io/vmware/nsxt
terraform state replace-provider -state terraform.tfstate registry.terraform.io/-/template registry.terraform.io/hashicorp/template
{"name": "extensions_cloud_config_directory", "value": "vsphere/env/cb-cloud-config"},
{"name": "extensions_cloud_config_properties","value": "cloud-config.properties"},
# On the OpenDistro master pod
cd plugins/opendistro_security/tools/
chmod +x securityadmin.sh
./securityadmin.sh -icl -nhnv \
-cacert ../../../config/admin-root-ca.pem \
-cert ../../../config/admin-crt.pem \
-key ../../../config/admin-key.pem \
-cd ../securityconfig/