BT Autoscaler Deployment Process

This package is a collection of TF and PS scripts to automate creating a QSEoW environment in GCP.

The original testing Terraform is being operationalized by ASG, and will be executed from within a GitLab runner.

To execute manually, start with Filestore and Cloud SQL.

Cloud SQL

Change into the "psql" directory and run the terraform init, plan and apply.

   /root  cd /root/bt-autoscaler/tf/psql/

Run terraform init to prepare the environment.

   /root/bt-autoscaler/tf/psql  terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google-beta...
- Finding latest version of hashicorp/null...
- Finding latest version of hashicorp/random...
- Installing hashicorp/random v3.0.1...
- Installed hashicorp/random v3.0.1 (signed by HashiCorp)
- Installing hashicorp/google-beta v3.53.0...
- Installed hashicorp/google-beta v3.53.0 (signed by HashiCorp)
- Installing hashicorp/null v3.0.0...
- Installed hashicorp/null v3.0.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run terraform plan and save plan for later execution.

   /root/bt-autoscaler/tf/psql  terraform plan -out=$PWD/"$(date +%Y-%m-%d).plan"

<SNIP OUTPUT>

------------------------------------------------------------------------

This plan was saved to: /root/bt-autoscaler/tf/psql/2021-01-21.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "/root/bt-autoscaler/tf/psql/2021-01-21.plan"

After review, this plan file can then be executed. It is considered best practice to have this be an automated task, or if manually executed, to always execute the saved .plan file.

Filestore

Change into the filestore directory, init and save plan for later execution.

   /root  cd /root/bt-autoscaler/tf/file/

Run terraform init to prepare the environment.

   /root/bt-autoscaler/tf/file  terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google-beta...
- Installing hashicorp/google-beta v3.53.0...
- Installed hashicorp/google-beta v3.53.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run terraform plan and save plan for later execution.

   /root/bt-autoscaler/tf/file  terraform plan -out=$PWD/"$(date +%Y-%m-%d).plan"

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # google_filestore_instance.bt-filestore will be created
  + resource "google_filestore_instance" "bt-filestore" {
      + create_time = (known after apply)
      + etag        = (known after apply)
      + id          = (known after apply)
      + name        = "qseow-files"
      + project     = "esoteric-parsec-243510"
      + tier        = "BASIC_HDD"
      + zone        = "europe-west1-d"

      + file_shares {
          + capacity_gb = 1024
          + name        = "qlikshare"
        }

      + networks {
          + ip_addresses      = (known after apply)
          + modes             = [
              + "MODE_IPV4",
            ]
          + network           = "default"
          + reserved_ip_range = (known after apply)
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + fs_instance_ip   = (known after apply)
  + fs_instance_name = "qseow-files"

------------------------------------------------------------------------

This plan was saved to: /root/bt-autoscaler/tf/file/2021-01-21.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "/root/bt-autoscaler/tf/file/2021-01-21.plan"

After review, this plan file can then be executed. It is considered best practice to have this be an automated task, or if manually executed, to always execute the saved .plan file.

Central Node

Change into the filestore directory, init and save plan for later execution.

   /root  cd /root/bt-autoscaler/tf/central/

Run terraform init to prepare the environment.

   /root/bt-autoscaler/tf/central  terraform init

Initializing the backend...

Initializing provider plugins...
- Finding latest version of hashicorp/google-beta...
- Finding latest version of hashicorp/random...
- Installing hashicorp/google-beta v3.53.0...
- Installed hashicorp/google-beta v3.53.0 (signed by HashiCorp)
- Installing hashicorp/random v3.0.1...
- Installed hashicorp/random v3.0.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run terraform plan and save plan for later execution.

   /root/bt-autoscaler/tf/central  terraform plan -out=$PWD/"$(date +%Y-%m-%d).plan"

<SNIP OUTPUT>

------------------------------------------------------------------------

This plan was saved to: /root/bt-autoscaler/tf/central/2021-01-21.plan

To perform exactly these actions, run the following command to apply:
    terraform apply "/root/bt-autoscaler/tf/central/2021-01-21.plan"

After review, this plan file can then be executed. It is considered best practice to have this be an automated task, or if manually executed, to always execute the saved .plan file.

The last step in this Terraform script is to set the "windows-startup-script-url" GCE metadata tag to the bootstrap.ps1 script. The bootstrap script will format the "E:" or "data" drive, copy tooling from a GCS bucket, open the WinFW for QSEoW and install the Windows NFS client (for Filestore access).

Once the bootstrap script completes, it will remove the metadata tag and create a Registry entry for the next "startup" action. The GCE instance is rebooted, and the qsInstall.ps1 script executes on instance startup. This will install the base Qlik Sense package, license the deployment and the execute the latest patch stored in the binary archive.

Rim Node

Based on hostname, the script will decide if it is installing the "Central" node or a "Rim" node. The "Rim" node process will also join the node to the cluster and update Virtual Proxy settings.