Merge pull request #32 from gruntwork-io/module_dependencies

Standardizing dependencies to a list input

README.md

@@ -2,13 +2,36 @@
 
 # Cloud SQL Modules
 
-This repo contains modules for running relational databases such as MySQL and PostgreSQL on Google's
-[Cloud SQL](https://cloud.google.com/sql/) on [GCP](https://cloud.google.com/).
+This repo contains modules for running relational databases such as MySQL and PostgreSQL on
+[Google Cloud Platform (GCP)](https://cloud.google.com/) using [Cloud SQL](https://cloud.google.com/sql/).
 
-## Code included in this Module
+## Quickstart
 
-* [cloud-sql](/modules/cloud-sql): Deploy a Cloud SQL [MySQL](https://cloud.google.com/sql/docs/mysql/) or [PostgreSQL](https://cloud.google.com/sql/docs/postgres/) cluster.
+If you want to quickly spin up a Cloud SQL database, you can run the example that is in the root of this repo. Check out
+[postgres-private-ip example documentation](https://github.com/gruntwork-io/terraform-google-sql/blob/master/examples/postgres-private-ip)
+for instructions.
 
+## What's in this repo
+
+This repo has the following folder structure:
+
+* [root](https://github.com/gruntwork-io/terraform-google-sql/tree/master): The root folder contains an example of how
+  to deploy a private PostgreSQL instance in Cloud SQL. See [postgres-private-ip](https://github.com/gruntwork-io/terraform-google-sql/blob/master/examples/postgres-private-ip)
+  for the documentation.
+
+* [modules](https://github.com/gruntwork-io/terraform-google-sql/tree/master/modules): This folder contains the
+  main implementation code for this Module, broken down into multiple standalone submodules.
+
+  The primary module is:
+
+    * [cloud-sql](/modules/cloud-sql): Deploy a Cloud SQL [MySQL](https://cloud.google.com/sql/docs/mysql/) or
+    [PostgreSQL](https://cloud.google.com/sql/docs/postgres/) database.
+
+* [examples](https://github.com/gruntwork-io/terraform-google-sql/tree/master/examples): This folder contains
+  examples of how to use the submodules.
+
+* [test](https://github.com/gruntwork-io/terraform-google-sql/tree/master/test): Automated tests for the submodules
+  and examples.
 
 ## What is Cloud SQL?
 
@@ -22,6 +45,18 @@ your relational databases on Google Cloud Platform. Cloud SQL automatically incl
 
 You can learn more about Cloud SQL from [the official documentation](https://cloud.google.com/sql/docs/).
 
+## What's a Module?
+
+A Module is a canonical, reusable, best-practices definition for how to run a single piece of infrastructure, such
+as a database or server cluster. Each Module is written using a combination of [Terraform](https://www.terraform.io/)
+and scripts (mostly bash) and include automated tests, documentation, and examples. It is maintained both by the open
+source community and companies that provide commercial support.
+
+Instead of figuring out the details of how to run a piece of infrastructure from scratch, you can reuse
+existing code that has been proven in production. And instead of maintaining all that infrastructure code yourself,
+you can leverage the work of the Module community to pick up infrastructure improvements through
+a version number bump.
+
 ## Who maintains this Module?
 
 This Module and its Submodules are maintained by [Gruntwork](http://www.gruntwork.io/). Read the [Gruntwork Philosophy](/GRUNTWORK_PHILOSOPHY.md) document to learn more about how Gruntwork builds production grade infrastructure code. If you are looking for help or
@@ -54,4 +89,6 @@ MINOR, and PATCH versions on each release to indicate any incompatibilities.
 
 ## License
 
-Please see [LICENSE.txt](/LICENSE.txt) for details on how the code in this repo is licensed.
+Please see [LICENSE](/LICENSE) for how the code in this repo is licensed.
+
+Copyright © 2019 Gruntwork, Inc.

examples/mysql-private-ip/main.tf

@@ -92,7 +92,7 @@ module "mysql" {
   private_network = "${google_compute_network.private_network.self_link}"
 
   # Wait for the vpc connection to complete
-  wait_for = "${google_service_networking_connection.private_vpc_connection.network}"
+  dependencies = ["${google_service_networking_connection.private_vpc_connection.network}"]
 
   # Set auto-increment flags to test the
   # feature during automated testing

examples/postgres-private-ip/main.tf

@@ -92,7 +92,7 @@ module "postgres" {
   private_network = "${google_compute_network.private_network.self_link}"
 
   # Wait for the vpc connection to complete
-  wait_for = "${google_service_networking_connection.private_vpc_connection.network}"
+  dependencies = ["${google_service_networking_connection.private_vpc_connection.network}"]
 
   custom_labels = {
     test-id = "postgres-private-ip-example"

main.tf

@@ -0,0 +1,100 @@
+# ------------------------------------------------------------------------------
+# LAUNCH A POSTGRES CLOUD SQL PRIVATE IP INSTANCE
+# ------------------------------------------------------------------------------
+
+# ------------------------------------------------------------------------------
+# CONFIGURE OUR GCP CONNECTION
+# ------------------------------------------------------------------------------
+
+provider "google-beta" {
+  version = "~> 2.1.0"
+  region  = "${var.region}"
+  project = "${var.project}"
+}
+
+# Use Terraform 0.10.x so that we can take advantage of Terraform GCP functionality as a separate provider via
+# https://github.com/terraform-providers/terraform-provider-google
+terraform {
+  required_version = ">= 0.10.3"
+}
+
+# ------------------------------------------------------------------------------
+# CREATE A RANDOM SUFFIX AND PREPARE RESOURCE NAMES
+# ------------------------------------------------------------------------------
+
+resource "random_id" "name" {
+  byte_length = 2
+}
+
+locals {
+  # If name_override is specified, use that - otherwise use the name_prefix with a random string
+  instance_name        = "${length(var.name_override) == 0 ? format("%s-%s", var.name_prefix, random_id.name.hex) : var.name_override}"
+  private_network_name = "private-network-${random_id.name.hex}"
+  private_ip_name      = "private-ip-${random_id.name.hex}"
+}
+
+# ------------------------------------------------------------------------------
+# CREATE COMPUTE NETWORKS
+# ------------------------------------------------------------------------------
+
+# Simple network, auto-creates subnetworks
+resource "google_compute_network" "private_network" {
+  provider = "google-beta"
+  name     = "${local.private_network_name}"
+}
+
+# Reserve global internal address range for the peering
+resource "google_compute_global_address" "private_ip_address" {
+  provider      = "google-beta"
+  name          = "${local.private_ip_name}"
+  purpose       = "VPC_PEERING"
+  address_type  = "INTERNAL"
+  prefix_length = 16
+  network       = "${google_compute_network.private_network.self_link}"
+}
+
+# Establish VPC network peering connection using the reserved address range
+resource "google_service_networking_connection" "private_vpc_connection" {
+  provider                = "google-beta"
+  network                 = "${google_compute_network.private_network.self_link}"
+  service                 = "servicenetworking.googleapis.com"
+  reserved_peering_ranges = ["${google_compute_global_address.private_ip_address.name}"]
+}
+
+# ------------------------------------------------------------------------------
+# CREATE DATABASE INSTANCE WITH PRIVATE IP
+# ------------------------------------------------------------------------------
+
+module "postgres" {
+  # When using these modules in your own templates, you will need to use a Git URL with a ref attribute that pins you
+  # to a specific version of the modules, such as the following example:
+  # source = "git::git@github.com:gruntwork-io/terraform-google-sql.git//modules/cloud-sql?ref=v0.1.0"
+  source = "./modules/cloud-sql"
+
+  project = "${var.project}"
+  region  = "${var.region}"
+  name    = "${local.instance_name}"
+  db_name = "${var.db_name}"
+
+  engine       = "${var.postgres_version}"
+  machine_type = "${var.machine_type}"
+
+  # These together will construct the master_user privileges, i.e.
+  # 'master_user_name'@'master_user_host' IDENTIFIED BY 'master_user_password'.
+  # These should typically be set as the environment variable TF_VAR_master_user_password, etc.
+  # so you don't check these into source control."
+  master_user_password = "${var.master_user_password}"
+
+  master_user_name = "${var.master_user_name}"
+  master_user_host = "%"
+
+  # Pass the private network link to the module
+  private_network = "${google_compute_network.private_network.self_link}"
+
+  # Wait for the vpc connection to complete
+  wait_for = "${google_service_networking_connection.private_vpc_connection.network}"
+
+  custom_labels = {
+    test-id = "postgres-private-ip-example"
+  }
+}

modules/cloud-sql/main.tf

@@ -30,7 +30,7 @@ locals {
 # ------------------------------------------------------------------------------
 
 resource "google_sql_database_instance" "master" {
-  depends_on = ["null_resource.wait_for"]
+  depends_on = ["null_resource.dependency_getter"]
 
   provider         = "google-beta"
   name             = "${var.name}"
@@ -111,11 +111,16 @@ resource "google_sql_user" "default" {
 }
 
 # ------------------------------------------------------------------------------
-# CREATE A NULL RESOURCE TO EMULATE DEPENDENCIES
+# SET MODULE DEPENDENCY RESOURCE
+# This works around a terraform limitation where we can not specify module dependencies natively.
+# See https://github.com/hashicorp/terraform/issues/1178 for more discussion.
+# By resolving and computing the dependencies list, we are able to make all the resources in this module depend on the
+# resources backing the values in the dependencies list.
 # ------------------------------------------------------------------------------
-resource "null_resource" "wait_for" {
-  triggers = {
-    instance = "${var.wait_for}"
+
+resource "null_resource" "dependency_getter" {
+  provisioner "local-exec" {
+    command = "echo ${length(var.dependencies)}"
   }
 }
 

modules/cloud-sql/variables.tf

@@ -208,7 +208,17 @@ variable "resource_timeout" {
   default     = "60m"
 }
 
-variable "wait_for" {
-  description = "By passing a value to this variable, you can effectively tell this module to wait to deploy until the given variable's value is resolved, which is a way to require that this module depend on some other module. Note that the actual value of this variable doesn't matter."
-  default     = ""
+# ---------------------------------------------------------------------------------------------------------------------
+# MODULE DEPENDENCIES
+# Workaround Terraform limitation where there is no module depends_on.
+# See https://github.com/hashicorp/terraform/issues/1178 for more details.
+# This can be used to make sure the module resources are created after other bootstrapping resources have been created.
+# For example:
+# dependencies = ["${google_service_networking_connection.private_vpc_connection.network}"]
+# ---------------------------------------------------------------------------------------------------------------------
+
+variable "dependencies" {
+  description = "Create a dependency between the resources in this module to the interpolated values in this list (and thus the source resources). In other words, the resources in this module will now depend on the resources backing the values in this list such that those resources need to be created before the resources in this module, and the resources in this module need to be destroyed before the resources in the list."
+  type        = "list"
+  default     = []
 }

outputs.tf

@@ -0,0 +1,42 @@
+# ------------------------------------------------------------------------------
+# MASTER OUTPUTS
+# ------------------------------------------------------------------------------
+
+output "master_instance_name" {
+  description = "The name of the database instance"
+  value       = "${module.postgres.master_instance_name}"
+}
+
+output "master_ip_addresses" {
+  description = "All IP addresses of the instance as list of maps, see https://www.terraform.io/docs/providers/google/r/sql_database_instance.html#ip_address-0-ip_address"
+  value       = "${module.postgres.master_ip_addresses}"
+}
+
+output "master_private_ip" {
+  description = "The private IPv4 address of the master instance"
+  value       = "${module.postgres.master_private_ip_address}"
+}
+
+output "master_instance" {
+  description = "Self link to the master instance"
+  value       = "${module.postgres.master_instance}"
+}
+
+output "master_proxy_connection" {
+  description = "Instance path for connecting with Cloud SQL Proxy. Read more at https://cloud.google.com/sql/docs/mysql/sql-proxy"
+  value       = "${module.postgres.master_proxy_connection}"
+}
+
+# ------------------------------------------------------------------------------
+# DB OUTPUTS
+# ------------------------------------------------------------------------------
+
+output "db_name" {
+  description = "Name of the default database"
+  value       = "${module.postgres.db_name}"
+}
+
+output "db" {
+  description = "Self link to the default database"
+  value       = "${module.postgres.db}"
+}

variables.tf

@@ -0,0 +1,49 @@
+# ---------------------------------------------------------------------------------------------------------------------
+# REQUIRED PARAMETERS
+# These variables are expected to be passed in by the operator
+# ---------------------------------------------------------------------------------------------------------------------
+
+variable "project" {
+  description = "The project ID to host the database in."
+}
+
+variable "region" {
+  description = "The region to host the database in."
+}
+
+# Note, after a name db instance is used, it cannot be reused for up to one week.
+variable "name_prefix" {
+  description = "The name prefix for the database instance. Will be appended with a random string. Use lowercase letters, numbers, and hyphens. Start with a letter."
+}
+
+variable "master_user_name" {
+  description = "The username part for the default user credentials, i.e. 'master_user_name'@'master_user_host' IDENTIFIED BY 'master_user_password'. This should typically be set as the environment variable TF_VAR_master_user_name so you don't check it into source control."
+}
+
+variable "master_user_password" {
+  description = "The password part for the default user credentials, i.e. 'master_user_name'@'master_user_host' IDENTIFIED BY 'master_user_password'. This should typically be set as the environment variable TF_VAR_master_user_password so you don't check it into source control."
+}
+
+# ---------------------------------------------------------------------------------------------------------------------
+# OPTIONAL PARAMETERS
+# Generally, these values won't need to be changed.
+# ---------------------------------------------------------------------------------------------------------------------
+variable "postgres_version" {
+  description = "The engine version of the database, e.g. `POSTGRES_9_6`. See https://cloud.google.com/sql/docs/db-versions for supported versions."
+  default     = "POSTGRES_9_6"
+}
+
+variable "machine_type" {
+  description = "The machine type to use, see https://cloud.google.com/sql/pricing for more details"
+  default     = "db-f1-micro"
+}
+
+variable "db_name" {
+  description = "Name for the db"
+  default     = "default"
+}
+
+variable "name_override" {
+  description = "You may optionally override the name_prefix + random string by specifying an override"
+  default     = ""
+}