Skip to content

Create AWS infra and IAM resources separately

The default behavior of the hypershift create cluster aws command is to create cloud infrastructure along with the HostedCluster and apply it. It is possible to create the cloud infrastructure portion separately so that the hypershift create cluster aws command can just be used to create the cluster, or simply render it so it can be modified before applying.

In order to do this, you need to:

  1. Create the AWS infrastructure
  2. Create AWS IAM resources
  3. Create the cluster

Creating the AWS infra

Use the hypershift create infra aws command:

hypershift create infra aws --name CLUSTER_NAME \
    --aws-creds AWS_CREDENTIALS_FILE \
    --base-domain BASEDOMAIN \
    --infra-id INFRA_ID \
    --region REGION \
    --output-file OUTPUT_INFRA_FILE

where

  • CLUSTER_NAME is the name of the hosted cluster you intend to create. This is used for creating the Route53 private hosted zones that belong to the cluster.
  • AWS_CREDENTIALS_FILE points to an AWS credentials file that has permission to create infrastructure resources for your cluster such as VPCs, subnets, NAT gateways, etc. It should correspond to the AWS account for your guest cluster, where workers will live.
  • BASEDOMAIN is the base domain that will be used for your hosted cluster's ingress. It must correspond to an existing Route53 public zone that you have access to create records in.
  • INFRA_ID is a unique name that will be used to identify your infrastructure via tags. It is used by the cloud controller manager in Kubernetes and the CAPI manager to identify infrastructure for your cluster. Typically this is the name of your cluster (CLUSTER_NAME) with a random suffix appended to it.
  • REGION is the region where you want to create the infrastructure for your cluster.
  • OUTPUT_INFRA_FILE is the file where IDs of the infrastructure that has been created will be stored in JSON format. This file can then be used as input to the hypershift create cluster aws command to populate the appropriate fields in the HostedCluster and NodePool resources.

Running this command should result in the following resources getting created:

  • 1 VPC
  • 1 DHCP Options
  • 1 Private Subnet
  • 1 Public Subnet
  • 1 Internet Gateway
  • 1 NAT Gateway
  • 1 Security Group for Worker Nodes
  • 2 Route Tables (1 Private, 1 Public)
  • 2 Private Hosted Zones (1 for Cluster Ingress, 1 for PrivateLink, in case you will be creating a private cluster)

All of these resources will contain the following tag: kubernetes.io/cluster/INFRA_ID=owned where INFRA_ID is what you specified on the command invocation.

Creating the AWS IAM resources

Use the hypershift create iam aws command:

hypershift create iam aws --infra-id INFRA_ID \
    --aws-creds AWS_CREDENTIALS_FILE \
    --oidc-storage-provider-s3-bucket-name OIDC_BUCKET_NAME \
    --oidc-storage-provider-s3-region OIDC_BUCKET_REGION \
    --region REGION \
    --public-zone-id PUBLIC_ZONE_ID \
    --private-zone-id PRIVATE_ZONE_ID \
    --local-zone-id LOCAL_ZONE_ID \
    --output-file OUTPUT_IAM_FILE

where

  • INFRA_ID should be the same id that was specified in the create infra aws command. It is used to identify the IAM resources associated with the hosted cluster.
  • AWS_CREDENTIALS_FILE points to an AWS credentials file that has permission to create IAM resources such as roles. It does not have to be the same credentials specified to create the infrastructure but it does have to correspond to the same AWS account.
  • OIDC_BUCKET_NAME is the name of the bucket used to store OIDC documents. This bucket should have been created as a prerequisite for installing Hypershift (See Prerequisites) The name of the bucket is used to construct URLs for the OIDC provider created by this command.
  • OIDC_BUCKET_REGION is the region where the OIDC bucket lives.
  • REGION is the region where the infrastructure of the cluster will live. This is used to create a worker instance profile for machines that belong to the hosted cluster.
  • PUBLIC_ZONE_ID is the ID of the public zone for the guest cluster. It is used in creating the policy for the ingress operator. It can be found in the OUTPUT_INFRA_FILE generated by the create infra aws command.
  • PRIVATE_ZONE_ID is the ID of the private zone for the guest cluster. It is used in creating the policy for the ingress operator. It can be found in the OUTPUT_INFRA_FILE generated by the create infra aws command.
  • LOCAL_ZONE_ID is the ID of the local zone for the guest cluster (when creating a private cluster). It is used in creating the policy for the control plane operator so it can manage records for the PrivateLink endpoint. It can be found in the OUTPUT_INFRA_FILE generated by the create infra aws command.
  • OUTPUT_IAM_FILE is the file where IDs of the IAM resources that have been created will be stored in JSON format. This file can then be used as input to the hypershift create cluster aws command to populate the appropriate fields in the HostedCluster and NodePool resource.

Running this command should result in the following resources getting created:

  • 1 OIDC Provider (required for enabling STS authentication)
  • 7 Roles (separate roles for every component that interacts with the provider: kube controller manager, capi provider, registry, etc)
  • 1 Instance Profile (the profile that is assigned to all worker instances of the cluster)

Creating the Cluster

Use the hypershift create cluster aws command:

hypershift create cluster aws \
    --infra-id INFRA_ID \
    --name CLUSTER_NAME \
    --aws-creds AWS_CREDENTIALS \
    --infra-json OUTPUT_INFRA_FILE \
    --iam-json OUTPUT_IAM_FILE \
    --pull-secret PULL_SECRET_FILE \
    --generate-ssh \
    --node-pool-replicas 3

where

  • INFRA_ID should be the same id that was specified in the create infra aws command. It is used to identify the IAM resources associated with the hosted cluster.
  • CLUSTER_NAME should be the same name that was specified in the create infra aws command.
  • AWS_CREDENTIALS should be the same that was specified in the create infra aws command.
  • OUTPUT_INFRA_FILE is the file where the output of the create infra aws command was saved.
  • OUTPUT_IAM_FILE is the file where the output of the create iam aws command was saved.
  • PULL_SECRET_FILE is a file that contains a valid OpenShift pull secret.

Note

The --generate-ssh flag is optional but is a good idea to have in case you need to ssh to your workers. An ssh key will have been generated for you and stored as a secret in the same namespace as the hosted cluster.

Running this command should result in the following resources getting applied to your cluster:

  • Namespace
  • Secret with your pull secret
  • HostedCluster
  • NodePool
  • 3 AWS STS secrets for control plane components
  • 1 SSH key secret (if --generate-ssh was specified)

You can also add the --render flag to the command and redirect output to a file where you can do further editing of the resources before applying them to the cluster.