From f6c5031a9014a29f489866c7ede64ea75b6bfb34 Mon Sep 17 00:00:00 2001 From: alicethorne-ab Date: Tue, 30 Mar 2021 12:40:16 -0400 Subject: [PATCH] AWS terraform changes for 2.0 --- README.md | 6 +-- aws-terraform/README.md | 33 ++++++-------- .../terraform/deploy/example_env/main.tf | 3 +- .../example_env/user_data/02-environment.yml | 3 +- .../terraform/deploy/example_env/variables.tf | 11 ++--- docker/README.md | 36 ++++++++-------- kubernetes/README.md | 43 +++++++++++-------- 7 files changed, 66 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index d485b76..bc1150e 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ -## 1Password SCIM Bridge deployment examples +## 1Password SCIM bridge deployment examples -Here you can find configuration files and best practice instructions for deploying the 1Password SCIM Bridge on various public and private cloud providers. +Here you can find configuration files and best practice instructions for deploying the 1Password SCIM bridge on various public and private cloud providers. ### Before Deploying -Before you begin deploying the 1Password SCIM Bridge, please refer to the [Preparation Guide](https://github.com/1Password/scim-examples/tree/master/PREPARATION.md). You will need to make some decisions about certain key details, and it also contains ways to avoid common pitfalls. +Before you begin deploying the 1Password SCIM bridge, please refer to the [Preparation Guide](https://github.com/1Password/scim-examples/tree/master/PREPARATION.md). You will need to make some decisions about certain key details, and it also contains ways to avoid common pitfalls. ## Automatic (one click) Deployment diff --git a/aws-terraform/README.md b/aws-terraform/README.md index 2ca19e3..b81fde9 100644 --- a/aws-terraform/README.md +++ b/aws-terraform/README.md @@ -1,17 +1,15 @@ -# Deploying the 1Password SCIM Bridge in AWS/Terraform +# Deploying the 1Password SCIM bridge in AWS/Terraform This document describes deploying the 1Password SCIM bridge to your Amazon Web Service cloud services using Terraform. - ## Preparing -This is only an example of how you could deploy the 1Password SCIM Bridge to your existing AWS infrastructure. As an advanced form of deployment, this AWS/Terraform example can be integrated into your existing infrastructure. Feel free to modify the deployment to suit your specific needs. +This is only an example of how you could deploy the 1Password SCIM bridge to your existing AWS infrastructure. As an advanced form of deployment, this AWS/Terraform example can be integrated into your existing infrastructure. Feel free to modify the deployment to suit your specific needs. -If you find that you are not using a majority of the services listed in this documentation within your AWS account - for instance, you have an external DNS provider, or you are providing your own certificates through a third-party service - consider deploying through [Kubernetes](https://github.com/1Password/scim-examples/tree/master/kubernetes/README.md) using AWS Elastic Kubernetes Service (EKS). The 1Password SCIM Bridge is not a resource-intensive service, and a full AWS deployment may include more infrastructure than you require, unless it fits neatly into your existing setup. +If you find that you are not using a majority of the services listed in this documentation within your AWS account - for instance, you have an external DNS provider, or you are providing your own certificates through a third-party service - consider deploying through [Kubernetes](https://github.com/1Password/scim-examples/tree/master/kubernetes/README.md) using AWS Elastic Kubernetes Service (EKS). The 1Password SCIM bridge is not a resource-intensive service, and a full AWS deployment may include more infrastructure than you require, unless it fits neatly into your existing setup. To continue, please ensure you've read through [PREPARATION.md](/PREPARATION.md) before beginning deployment. - ## Deployment overview ### Required Tools @@ -20,20 +18,17 @@ The minimum supported version of Terraform is `0.12.x`. You will also want to ensure the `aws` and `terraform` CLI tools are installed for your operating system. - ### Instance Size For most installations, one [t3.micro](https://github.com/1Password/scim-examples/tree/master/aws-terraform/terraform/deploy/example_env/variables.tf#L96) size instance will be adequate to serve Identity Provider requests. We recommend enabling auto-scaling, but it is not strictly necessary. - ### AWS Components -* **AWS Elastic Compute Cloud** - Elastic Compute Cloud (EC2) provides the compute necessary to deploy the SCIM Bridge. +* **AWS Elastic Compute Cloud** - Elastic Compute Cloud (EC2) provides the compute necessary to deploy the SCIM bridge. * **AWS Key Management Service and Secrets Manager** - Key Management Service (KMS) key is used to encrypt/decrypt Secrets Manager data. You’ll be placing the `scimsession` file (mentioned in [PREPARATION.md](/PREPARATION.md)) in the steps outlined here. * **AWS S3 Bucket** - This can be used to store the terraform state file, load balancer logs, and so on. * **AWS Certificate Manager** - Used to manage SSL certificates for your deployment. - ## Code Structure Below is the overall code structure of the Terraform deployment. @@ -88,14 +83,14 @@ Custom Key Management Service key can be specified by `--kms-key-id `. 5. Use the following `terraform` commands to deploy and verify your installation. From the `terraform/deploy/{testing,production}` directory: ```bash - terraform init - terraform plan -out=./op-scim.plan - terraform apply ./op-scim.plan +terraform init +terraform plan -out=./op-scim.plan +terraform apply ./op-scim.plan ``` 6. Update/Create 1Password SCIM configuration in your Identity Provider using your generated [bearer token](/PREPARATION.md) and newly-created subdomain. -7. Test your SCIM Bridge deployment using the following `curl` command: +7. Test your SCIM bridge deployment using the following `curl` command: ```bash curl --header "Authorization: Bearer TOKEN_GOES_HERE" https:///scim/Users @@ -107,7 +102,6 @@ curl --header "Authorization: Bearer TOKEN_GOES_HERE" https:///scim/User All logs go to `/var/log/syslog` (AWS EC2 instance OS). You can use the AWS System Manager to view logs for your instance. - ### Upgrading and Redeployment You can destroy and redeploy the instance whenever you feel the need to. No permanent data is stored within the SCIM bridge instance itself, as secrets are stored in the AWS Secrets Manager. This is useful for upgrading your SCIM bridge with important bugfix or feature releases. @@ -116,14 +110,13 @@ You can destroy and redeploy the instance whenever you feel the need to. No perm terraform destroy ``` - ### Debian Package -The 1Password SCIM Bridge is distributed as a Debian package and installed automatically during the deployment process. The following commands are run during the deployment: +The 1Password SCIM bridge is distributed as a Debian package and installed automatically during the deployment process. The following commands are run during the deployment: ```bash -- curl -L https://apt.agilebits.com/op-scim/1Password.asc 2> /dev/null | apt-key add - -- echo '${SCIM_REPO}' > /etc/apt/sources.list.d/op-scimrepo.list -- apt-get -y -qq update && apt-get -y -qq install op-scim -- echo '10 * * * * root /usr/local/bin/op-scim-upgrade.sh 2>&1 | logger -t op-scim-deploy-cron' > /etc/cron.d/50_op-scim && chmod 0644 /etc/cron.d/50_op-scim +curl -L https://apt.agilebits.com/op-scim/1Password.asc 2> /dev/null | apt-key add - +echo '${SCIM_REPO}' > /etc/apt/sources.list.d/op-scimrepo.list +apt-get -y -qq update && apt-get -y -qq install op-scim +echo '10 * * * * root /usr/local/bin/op-scim-upgrade.sh 2>&1 | logger -t op-scim-deploy-cron' > /etc/cron.d/50_op-scim && chmod 0644 /etc/cron.d/50_op-scim ``` diff --git a/aws-terraform/terraform/deploy/example_env/main.tf b/aws-terraform/terraform/deploy/example_env/main.tf index c7bcb24..117465d 100644 --- a/aws-terraform/terraform/deploy/example_env/main.tf +++ b/aws-terraform/terraform/deploy/example_env/main.tf @@ -60,8 +60,7 @@ data "template_file" "environment" { template = file("${path.module}/user_data/02-environment.yml") vars = { - REDIS = var.cache_dns_name - REDISPORT = var.cache_port + REDISURL = var.cache_url SCIM_USER = var.scim_user SCIM_GROUP = var.scim_group SCIM_PATH = var.scim_path diff --git a/aws-terraform/terraform/deploy/example_env/user_data/02-environment.yml b/aws-terraform/terraform/deploy/example_env/user_data/02-environment.yml index 47232dd..c47ec3e 100644 --- a/aws-terraform/terraform/deploy/example_env/user_data/02-environment.yml +++ b/aws-terraform/terraform/deploy/example_env/user_data/02-environment.yml @@ -3,8 +3,7 @@ write_files: # configure op-scim service/systemd unit default environment variables - content: | - REDIS="${REDIS}" - REDISPORT="${REDISPORT}" + REDISURL="${REDISURL}" SCIM_USER="${SCIM_USER}" SCIM_GROUP="${SCIM_GROUP}" SCIM_PATH="${SCIM_PATH}" diff --git a/aws-terraform/terraform/deploy/example_env/variables.tf b/aws-terraform/terraform/deploy/example_env/variables.tf index 55b069c..0b95efe 100644 --- a/aws-terraform/terraform/deploy/example_env/variables.tf +++ b/aws-terraform/terraform/deploy/example_env/variables.tf @@ -130,12 +130,12 @@ variable "instance_type" { default = "t3.micro" } -data "aws_ami" "ubuntu18" { +data "aws_ami" "ubuntu20" { most_recent = true filter { name = "name" - values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"] + values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"] } filter { @@ -148,10 +148,7 @@ data "aws_ami" "ubuntu18" { // cache variables: -variable "cache_port" { - default = "6379" +variable "cache_url" { + default = "redis://localhost:6379" } -variable "cache_dns_name" { - default = "localhost" -} diff --git a/docker/README.md b/docker/README.md index 1e14250..7d0d8dd 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1,18 +1,18 @@ -# Deploying the 1Password SCIM Bridge using Docker +# Deploying the 1Password SCIM bridge using Docker -This example describes the methods of deploying the 1Password SCIM Bridge using Docker. The Docker Compose and Docker Swarm managers are available and deployment using each manager is described below. +This example describes the methods of deploying the 1Password SCIM bridge using Docker. The Docker Compose and Docker Swarm managers are available and deployment using each manager is described below. ## Preparing -Please ensure you've read through the [PREPARATION.md](/PREPARATION.md) document before beginning deployment. +Ensure you've read through the [PREPARATION.md](/PREPARATION.md) document before beginning deployment. ## Docker Compose vs Docker Swarm Using Docker, you have two different deployment options: `docker-compose` and Docker Swarm. -Docker Swarm is the recommended option. While setting up Swarm is beyond the scope of this documentation, you can either set one up on your own infrastructure, or on a cloud provider of your choice. +Docker Swarm is the recommended option, but both options can be used successfully depending on your deployment needs. While setting up a Docker host is beyond the scope of this documentation, you can either set one up on your own infrastructure, or on a cloud provider of your choice. -While Docker Compose is useful for testing, it is not recommended for use in a production environment. The `scimsession` file is passed into the docker container via an environment variable, which is less secure than Docker Swarm secrets, Kubernetes secrets, or AWS Secrets Manager, all of which are supported and recommended for production use. +The `scimsession` file is passed into the docker container via an environment variable, which is less secure than Docker Swarm secrets, Kubernetes secrets, or AWS Secrets Manager, all of which are supported and recommended for production use. ## Install Docker tools @@ -30,15 +30,15 @@ For macOS users who use Homebrew, ensure you're using the _cask_ app-based versi For this, you will need to have joined a Docker Swarm with the target deployment node. Please refer to [the official Docker documentation](https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/) on how to do that. -Once set up and you've logged into your Swarm with `docker swarm join` or created a new one with `docker swarm init`, it's recommended to use the provided the bash script [./docker/deploy.sh](deploy.sh) to deploy your SCIM Bridge. +Once set up and you've logged into your Swarm with `docker swarm join` or created a new one with `docker swarm init`, it's recommended to use the provided the bash script [./docker/deploy.sh](deploy.sh) to deploy your SCIM bridge. The script will do the following: 1. Add your `scimsession` file as a Docker Secret within your Swarm cluster. -2. Prompt you for your SCIM Bridge domain name which will configure LetsEncrypt to automatically issue a certificate for your Bridge. This is the domain you selected in [PREPARATION.md](/PREPARATION.md). -3. Deploy a container using `1password/scim`, and a `redis` container. The `redis` container is necessary to store LetsEncrypt certificates. +2. Prompt you for your SCIM bridge domain name which will configure LetsEncrypt to automatically issue a certificate for your Bridge. This is the domain you selected in [PREPARATION.md](/PREPARATION.md). +3. Deploy a container using `1password/scim`, and a `redis` container. The `redis` container is necessary to store LetsEncrypt certificates, as well as act as a cache for Identity Provider data. -The logs from the SCIM Bridge and redis containers will be streamed to your machine. If everything seems to have deployed successfully, press Ctrl+C to exit, and the containers will remain running on the remote machine. +The logs from the SCIM bridge and Redis containers will be streamed to your machine. If everything seems to have deployed successfully, press Ctrl+C to exit, and the containers will remain running on the remote machine. At this point you should set the DNS record for the domain name you prepared to the IP address of the `op-scim` container. You can also continue setting up your Identity Provider at this point. @@ -114,9 +114,9 @@ docker service logs --raw -f op-scim_scim ### Testing -To test if your SCIM Bridge came online, you can browse to the public IP address of your SCIM Bridge’s Docker Host with a web browser, and input your Bearer Token into the provided Bearer Token field. +To test if your SCIM bridge came online, you can browse to the public IP address of your SCIM bridge’s Docker Host with a web browser, and input your Bearer Token into the provided Bearer Token field. -You can also use the following `curl` command to test the SCIM Bridge from the command line: +You can also use the following `curl` command to test the SCIM bridge from the command line: ```bash curl --header "Authorization: Bearer TOKEN_GOES_HERE" https:///scim/Users @@ -124,7 +124,7 @@ curl --header "Authorization: Bearer TOKEN_GOES_HERE" https:///scim/User ### Upgrading -Upgrading the SCIM Bridge should be relatively simple. +Upgrading the SCIM bridge should be relatively simple. First, you `git pull` the latest versions from this repository. Then, you re-apply the `.yml` file. @@ -135,22 +135,24 @@ cd docker/{swarm or compose}/ docker-compose -f docker-compose.yml up --build -d ``` -This should seamlessly upgrade your SCIM Bridge to the latest version. The process takes about 2-3 minutes for the Bridge to come back online. +This should seamlessly upgrade your SCIM bridge to the latest version. The process takes about 2-3 minutes for the Bridge to come back online. #### October 2020 Update As of October 2020, if you’re upgrading from a previous version of the repository, ensure that you’ve reconfigured your environment variables within `scim.env` before upgrading. -#### March 2021 Update (SCIM Bridge 2.0) +#### April 2021 Update (SCIM bridge 2.0) -The environment variables `OP_REDIS_HOST` and `OP_REDIS_PORT` have been deprecated in favour of `OP_REDIS_URL`, which takes a full `redis://` or `rediss://` (for TLS) redis URL. For example: `OP_REDIS_URL=redis://redis:6379` +With the release of SCIM bridge 2.0, the environment variables `OP_REDIS_HOST` and `OP_REDIS_PORT` have been deprecated in favour of `OP_REDIS_URL`, which takes a full `redis://` or `rediss://` (for TLS) Redis URL. For example: `OP_REDIS_URL=redis://redis:6379` + +Unless you have customized your Redis deployment, there shouldn’t be any action you need to take. ### Advanced `scim.env` file options -These should only be used for advanced setups. +The following options are available for advanced or custom deployments. Unless you have a specific need, these options do not need to be modified. * `OP_PORT` - when `OP_LETSENCRYPT_DOMAIN` is set to blank, you can use `OP_PORT` to change the default port from 3002 to one of your choosing. -* `OP_REDIS_URL` - you can specify `redis://` or `rediss://` (for TLS) URL here to point towards an alternative redis host. You can then strip out the sections in `docker-compose.yml` that refer to redis to not deploy that container. Note that redis is still required for the SCIM Bridge to function. +* `OP_REDIS_URL` - you can specify `redis://` or `rediss://` (for TLS) URL here to point towards an alternative Redis host. You can then strip out the sections in `docker-compose.yml` that refer to Redis to not deploy that container. Note that Redis is still required for the SCIM bridge to function. #### Generating `scim.env` file on Windows diff --git a/kubernetes/README.md b/kubernetes/README.md index 8019b1b..024ec50 100644 --- a/kubernetes/README.md +++ b/kubernetes/README.md @@ -1,4 +1,4 @@ -# Deploying the 1Password SCIM Bridge on Kubernetes +# Deploying the 1Password SCIM bridge on Kubernetes This example explains how to deploy the 1Password SCIM bridge on a Kubernetes cluster. It assumes your Kubernetes cluster supports "load balancer" services. @@ -10,20 +10,20 @@ The deployment process consists of these steps: 1. Create a Kubernetes secret with your `scimsession` file 2. Configure the SCIM bridge (through the `op-scim-config.yaml` file) -3. Deploy the SCIM Bridge and Redis services -4. Set up DNS entries for your SCIM Bridge +3. Deploy the SCIM bridge and Redis services +4. Set up DNS entries for your SCIM bridge ## Structure -- `op-scim-deployment.yaml`: The SCIM Bridge deployment. -- `op-scim-service.yaml`: Public load balancer for the SCIM Bridge server. -- `op-scim-config.yaml`: Configuration for the SCIM Bridge server. You’ll be modifying this file during the deployment. +- `op-scim-deployment.yaml`: The SCIM bridge deployment. +- `op-scim-service.yaml`: Public load balancer for the SCIM bridge server. +- `op-scim-config.yaml`: Configuration for the SCIM bridge server. You’ll be modifying this file during the deployment. - `redis-deployment.yaml`: [(optional*)](#external-redis-server) A redis server deployment. - `redis-service.yaml`: [(optional*)](#external-redis-server) Kubernetes service for the redis deployment. ## Preparing -Please ensure you've read through the [PREPARATION.md](/PREPARATION.md) document before beginning deployment. +Ensure you've read through the [PREPARATION.md](/PREPARATION.md) document before beginning deployment. ## Clone `scim-examples` @@ -51,7 +51,7 @@ kubectl create secret generic scimsession --from-file=/path/to/scimsession ## Configuring the SCIM bridge -You'll need to edit the `op-scim-config.yaml` file and change the variable `OP_LETSENCRYPT_DOMAIN` to the domain you've decided on for your SCIM Bridge. This allows LetsEncrypt to issue your deployment an SSL certificate necessary for encrypted traffic. +You'll need to edit the `op-scim-config.yaml` file and change the variable `OP_LETSENCRYPT_DOMAIN` to the domain you've decided on for your SCIM bridge. This allows LetsEncrypt to issue your deployment an SSL certificate necessary for encrypted traffic. ## Deploy to the Kubernetes cluster @@ -64,7 +64,7 @@ kubectl apply -f . ## Configuring the DNS entries -The Kubernetes deployment creates a public load balancer in your environment pointing to the SCIM Bridge. +The Kubernetes deployment creates a public load balancer in your environment pointing to the SCIM bridge. To get its public IP address: @@ -87,11 +87,11 @@ You can do this with `curl`, as an example: curl --header "Authorization: Bearer TOKEN_GOES_HERE" https:///scim/Users ``` -You can now continue with the administration guide to configure your Identity Provider to enable provisioning with your SCIM Bridge. +You can now continue with the administration guide to configure your Identity Provider to enable provisioning with your SCIM bridge. ## Upgrading -To upgrade your SCIM Bridge, follow these steps: +To upgrade your SCIM bridge, follow these steps: ```bash cd scim-examples/ @@ -100,11 +100,11 @@ cd kubernetes/ kubectl apply -f . ``` -This will upgrade your SCIM Bridge to the latest version, which should take about 2-3 minutes for Kubernetes to process. +This will upgrade your SCIM bridge to the latest version, which should take about 2-3 minutes for Kubernetes to process. ### October 2020 Upgrade Changes -As of October 2020, the `scim-examples` Kubernetes deployment now uses `op-scim-config.yaml` to set the configuration needed for your SCIM Bridge, and has changed the deployment names from `op-scim` to `op-scim-bridge`, and `redis` to `op-scim-redis` for clarity and consistency. +As of October 2020, the `scim-examples` Kubernetes deployment now uses `op-scim-config.yaml` to set the configuration needed for your SCIM bridge, and has changed the deployment names from `op-scim` to `op-scim-bridge`, and `redis` to `op-scim-redis` for clarity and consistency. You’ll need to re-configure your options in `op-scim-config.yaml`, particularly `OP_LETSENCRYPT_DOMAIN`. You may also want to delete your previous `op-scim` and `redis` deployments to prevent conflict between the two versions. @@ -113,18 +113,25 @@ kubectl delete deployment/op-scim deployment/redis kubectl apply -f . ``` -You’ll then need to update your SCIM Bridge’s domain name DNS record. You can find the IP for that with: +You’ll then need to update your SCIM bridge’s domain name DNS record. You can find the IP for that with: ```bash kubectl describe service/op-scim-bridge # look for ‘LoadBalancer Ingress’ ``` -This is a one-time operation to change the deployment and service names of the SCIM Bridge so they are more easily identifiable to administrators. +This is a one-time operation to change the deployment and service names of the SCIM bridge so they are more easily identifiable to administrators. -### March 2021 Upgrade Changes (SCIM Bridge 2.0) +### April 2021 Upgrade Changes (SCIM bridge 2.0) -As of March 2021, the environment variables `OP_REDIS_HOST` and `OP_REDIS_PORT` have been deprecated and replaced with `OP_REDIS_URL`. Ensure that your `op-scim-config.yaml` file has changed to reflect this new environment variable, and reapplied to your pods with `kubectl apply -f op-scim-config.yaml`. +With the release of SCIM bridge 2.0, the environment variables `OP_REDIS_HOST` and `OP_REDIS_PORT` have been deprecated and in favour of `OP_REDIS_URL`. Ensure that your `op-scim-config.yaml` file has changed to reflect this new environment variable, and reapplied to your pods with: + +```bash +cd scim-examples/kubernetes +git pull +kubectl apply -f . +kubectl scale deploy op-scim-bridge --replicas=0 && sleep 3 && kubectl scale deploy op-scim-bridge --replicas=1 +``` ## Advanced deployments @@ -132,7 +139,7 @@ The following are helpful tips in case you wish to perform an advanced deploymen ### External load balancer -In `op-scim-config.yaml`, you can set the `OP_LETSENCRYPT_DOMAIN` variable to blank, which will have the SCIM Bridge serve on port 3002. You can then map your pre-existing webserver (Apache, NGINX, etc). You will no longer be issued a certificate by LetsEncrypt. +In `op-scim-config.yaml`, you can set the `OP_LETSENCRYPT_DOMAIN` variable to blank, which will have the SCIM bridge serve on port 3002. You can then map your pre-existing webserver (Apache, NGINX, etc). You will no longer be issued a certificate by LetsEncrypt. ### External redis server