This application pulls Terraform code for deploying cloud infrastructure, security defined policies, and environment configuration (currently from Github). Then it compares the configuration against security policies and runs deploys with python-terraform. Afterwards it will log the changes and report stats to a monitoring system (currently GCP Stackdriver).

These instructions will get you an environment, ready for enterprise-cloud-admin-tool development.

1. This tool has been tested to work with:

• Python version >=3.6,<=3.7.4
• Terraform version >=0.12.0,<=0.12.1

1. You should have terraform in your PATH environment variable. For example, you can download terraform and extract it to /usr/local/bin. Also, you can create make tf_bin directory right where you are, copy terraform into this directory, and run export PATH=$PATH:$PWD/tf_bin. Official Terraform documentation points here for additional instructions.

2. Your platform should have installed pipenv.

3. You must have access token of your Github account, and this account should be admin of some Github organization. You can generate it here: Settings -> Developer settings -> Personal access tokens -> Generate new token. This token needs permissions for 'repo', 'admin:org', and 'delete_repo'.

4. You must have a project on Google Cloud Platform that will be used as the build project and as a monitoring namespace.

   • "Cloud Resource Manager" and "Cloud Billing" APIs need to be enabled on the project.

   As this project will also be used as a monitoring namespace you need to create that namespace. To do so go to 'Monitoring' menu and create a monitoring space in Stackdriver.

5. The build/monitoring project must have a service account. Switch to your project, go to IAM menu, and add a service account. Then assign the following permissions to that service account:

   • "Billing Account User" role set at the organization level or on the specified billing account by an org/billing admin. (Check billing access control documentation here)
   • "Project Creator" role set minimum at the folder level.
   • "Monitoring Metric Writer" role assigned at the project level.

   You can find how to create a service account here. It doesn't matter for which project you will create service account, you will be able to use it for any API activity.

6. You must create, export and save your GCP service account private key in json format.

More details about here.

1. Clone this repo:

   git clone https://github.com/mesoform/enterprise-cloud-admin-tool.git && cd enterprise-cloud-admin-tool
   

2. Update pipfile with current version of python

   vi $(pwd)/Pipfile
   

3. Install all dependencies:

   pipenv install --dev
   

4. Activate virtual environment:

   pipenv shell
   

5. Copy terraform binary in current directory. Path for this binary is configurable, check cat settings/default_settings.py | grep TERRAFORM_BINARY_PATH. Then,

   cp ~/Downloads/terraform $(pwd)/terraform
   

6. Copy GCP service account token in resources directory:

   cp ~/Downloads/gcp_service_account_key.json $(pwd)/resources/gcp_service_account_key.json
   

7. Point gcloud to service account token file via GOOGLE_CREDENTIALS environment variable:

   export GOOGLE_CREDENTIALS=$(pwd)/resources/gcp_service_account_key.json
   

pytest

If you just cloned repo and have docker installed, you can just build the container and run tests without installing anything in your environment.

Build container and run bash shell:

docker build . -t test_eca
docker run -it test_eca bash

After that, you will be attached to container's bash shell. You can run tests:

pytest

You can run pytest in verbose mode:

pytest -v

Then, you will be able to see each test, that generated by parametrize decorator, like:

tests/test_deployer.py::test_deploy_different_states[test_state0-real_state0] PASSED [ 89%]
tests/test_deployer.py::test_deploy_different_states[test_state1-real_state1] PASSED [ 94%]
tests/test_deployer.py::test_deploy_different_states[test_state2-real_state2] PASSED [100%]

Also, if some of parametrized tests is failing, you are able to determine which one by it's name in square brackets, like test_state1-real_state1: here number of parametrize argument is a number after test name.

In order to test a deployment we reqire a github repo which will contain the configuration files and another repo for the deployment code.

We prepared two example repos:

1. example-ecat-project-config — contains terraform variable files.
2. example-ecat-deployment-code — contains terraform infrastructure code.

In order to perform test deployment using these examples, you should fork these repos to your organization, and then customise the configuration as per below:

• example-ecat-project-config/gcp/project_settings.auto.tfvars.json — In this file, you should set unique project_id (project creation docs), set or remove any remaining key value pair according your requirements. Be aware, that project_id unique across whole GCP platform, even six month after deletion. So, if someone already have project with your id, you will receive unclear error.
• Also add a valid billing_id, it's mandatory (billing docs).
• folder_id means folder numeric ID, more information about how it can be obtained.

If you wish to create a config repo manually this command will create the required repo and required config files

./cloudctl -p <project id> \
  -o <github organization name> \
  -O <github organization name> \
  --vcs-token <github token> \
  --key-file resources/gcp_service_account_key.json \
  --monitoring-namespace <monitoring project id> \
  --debug true config create \
  --config-repo <config repo> \
  --force

Where:

• project id — id of project, that will be created.
• github organization name — name of organization, that holds repos with code/config.
• github token — you developer's github token, that you have obtained in prerequisites section.
• monitoring project id — id of existing monitoring project. You should have one if followed prerequisites section.
• config repo — name of repo, that will contain terraform variables files.

In the project settings file created within the config repo you shuld ensure a unique project_id is set (project creation docs), set or remove any remaining key value pair according your requirements. Be aware, that project_id unique across whole GCP platform, even six month after deletion. So, if someone already have project with your id, you will receive unclear error.

• Also add a valid billing_id, it's mandatory (billing docs).
• And update the folder_id (numeric folderID from GCP), more information about how it can be obtained.

If you want to override config files, you can just run again the same command as for creation. If you see this:

{'message': 'Could not update file: At least 1 approving review is required by reviewers with write access.', 'documentation_url': 'https://help.github.com/articles/about-protected-branches'}

then, try to pass --bypass-branch-protection option to config subcommand.

Once the created/example config and code repos have been updated, you can perform test deployment woth the following command:

./cloudctl -p <project id> \
  -o <github organization name> \
  -O <github organization name> \
  --vcs-token <github token> \
  --key-file resources/gcp_service_account_key.json \
  --monitoring-namespace <monitoring project id> deploy --cloud gcp \
  --code-repo <code repo> \
  --config-repo <config repo>

Where:

• project id — id of project, that will be created.
• code repo — name of repo, that contain created infrastructure code.
• config repo — name of repo, that contain created terraform variables files.
• github organization name — name of organization, that holds repos with code/config.
• github token — you developer's github token, that you have obtained in prerequisites section.
• monitoring project id — id of existing monitoring project. You should have one if followed prerequisites section.

After that, you should receive success message in console, and metrics in your GCP monitoring project workspace.

Sometimes it's really hard to interpret immediately what GCP error means, so terraform community members created curated list of common problems: TROUBLESHOOTING.md.

1. If a "testing*" project has already been created but the deployment failed to complete, retrying the deployment will throw the "requested entity already exists" error due to an already existing "testing*" project. To bypass this issue the name of the project needs to be changed. E.g: xyz-eca-test01 to xyz-eca-test02

STDERR:
Error: error creating project testing-123456a-123456b (testing-123456a-123456b): googleapi: Error 409: Requested entity already exists, alreadyExists. If you received a 403 error, make sure you have the `roles/resourcemanager.projectCreator` permission
  on project.tf line 9, in resource "google_project" "project":
   9: resource "google_project" "project" {

2. Check that the service account has the minimum required billing permissions granted. More: https://cloud.google.com/billing/docs/how-to/billing-access

STDERR:
Error: Error setting billing account "01234A-12345B-23456C" for project "projects/testing-123456a-123456b": googleapi: Error 403: The caller does not have permission, forbidden
  on project.tf line 9, in resource "google_project" "project":
   9: resource "google_project" "project" {

3. Check whether Cloud Resource Manager API is enabled on the project owning the service account which is used for the deployment.

STDERR:
Error: error creating project testing-123456a-123456b (testing-123456a-123456b): googleapi: Error 403: Cloud Resource Manager API has not been used in project 123456789101 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=123456789101 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured. If you received a 403 error, make sure you have the `roles/resourcemanager.projectCreator` permission
  on project.tf line 9, in resource "google_project" "project":
   9: resource "google_project" "project" {

4. If the limit of projects associated with the billing account has been reached a "precodition check failed" error will be shown. Try removing some projects from that billing account.

STDERR:
Error: Error setting billing account "01234A-12345B-23456C" for project "projects/xyz-eca-test07": googleapi: Error 400: Precondition check failed., failedPrecondition
  on project.tf line 9, in resource "google_project" "project":
   9: resource "google_project" "project" {

Please read CONTRIBUTING.md for the process for submitting pull requests.

This project is licensed under the MPL 2.0