Terraform & GCP : Error 403 when attempting to introduce impersonation on project-level

I am quite lost when it comes to applying principles that enable service account impersonation...

My terraform project structure has a root module per environment, base for basic infrastructure, dev for the dev environment and prod for the production environment.

terraform-infra-genesis
 ┣ base
 ┃ ┣ ...
 ┃ ┣ impersonators_x_users.tf  <- user email me@domain.com is granted iam.serviceAccounts.getAccessToken role on 'super-admin' here (On all the organization)
 ┃ ┣ ...
 ┃ ┣ providers_x_access_tokens.tf 
 ┃ ┣ service_accounts_x_roles.tf   <- 'dev-admin' service account declared here
 ┃ ┣ terraform.tfstate
 ┃ ┗ terraform.tfstate.backup
 ┣ dev <- Everything here belongs to the dev environment
 ┃ ┣ backend.tf
 ┃ ┣ data_products.tf  <- Usage of the module 'marketing-hub' here
 ┃ ┣ ...
 ┃ ┣ impersonators_x_providers_x_access_tokens.tf  <- Declaration of as_dev_admin provider to 'delegate' ressource creation (such as folders) to the dev environment "super" administrator. I also declared "as_<project>_dev_admin" that should in principle, be able to create ressources only within its own <project>
 ┣ modules
 ┃ ┣ data-products
 ┃ ┃ ┣ cmi
 ┃ ┃ ┃ ┗ feedback-hub
 ┃ ┃ ┗ ddm
 ┃ ┃ ┃ ┣ analytics-hub
 ┃ ┃ ┃ ┣ marketing-hub
 ┃ ┃ ┃ ┃ ┣ marketing_hub.tf <- Usage of module 'data-project' here to bundle all logical projects together.
 ┃ ┃ ┃ ┗ media-hub
 ┃ ┗ data-project
 ┃ ┃ ┣ data_project.tf <- Module to create a GCP project and its project-dev-admin service account here
 ┣ prod
 ┃ ┣ ...
 ┣ .gitignore
 ┗ README.md

As described in the annotations on this structure, I generally use a provider = as_dev_admin within the dev/ root-module. I successfully created dev_coupons (A GCP project) using it.

Here is the structure of my dev/impersonators_x_providers_x_access_tokens.tf

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">=3.85.0"
    }
  }
}

locals {
  tier_1_scopes = [
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/userinfo.email",
  ]
  tier_2_scopes = [
    "cloud-platform",
    "userinfo-email",
  ]
}
# Dev Admin impersonation
provider "google" {
  alias  = "impersonation"
  scopes = local.tier_1_scopes
}

data "google_service_account_access_token" "dev-admin" {
  provider               = google.impersonation
  target_service_account = data.terraform_remote_state.base.outputs.service-accounts.dev-admin.email
  scopes                 = local.tier_2_scopes
  lifetime               = "1200s"
}

provider "google" {
  alias        = "as_dev_admin"
  access_token = data.google_service_account_access_token.dev-admin.access_token
  region       = var.region
  zone         = var.zone
}


################################################################################
##################### Impersonation of a service account #######################
############################ as_dev_coupons_admin ##############################
################################################################################
# Copy/paste this block in order to introduce the 
# impersonation of any service account

data "google_service_account_access_token" "dev-coupons-admin" {
  provider               = google.impersonation
  target_service_account = module.marketing-hub-products.projects.coupons.admin_service_account.email
  scopes                 = local.tier_2_scopes
  lifetime               = var.lifetime
}

provider "google" {
  alias        = "as_dev_coupons_admin"
  project      = module.marketing-hub-products.projects.coupons.project_info.project_id
  access_token = data.google_service_account_access_token.dev-coupons-admin.access_token
  region       = var.region
  zone         = var.zone
}

resource "google_service_account_iam_member" "dev-coupons-admin-impersonators" {
  provider = google.as_dev_admin  # Global dev environment admin will grant this permission
  for_each = toset([
    for account in var.user_accs_impersonators_info.as_dev_coupons_admin :
    "${account.acc_type}:${account.acc_details.email}"
  ])

  service_account_id = module.marketing-hub-products.projects.coupons.admin_service_account.name
  role               = "roles/iam.serviceAccountTokenCreator"
  member             = each.value
}


################################################################################
################################### End of #####################################
############################ as_dev_coupons_admin ##############################
################################################################################

My project name is dev-coupons.

When I try to declare the additionnal provider alias as_dev_coupons_admin to a specific project dev-coupons admin, I get this error :

│ Error: googleapi: Error 403: The caller does not have permission, forbidden
│
│   with data.google_service_account_access_token.dev-coupons-admin,
│   on impersonators_x_providers_x_access_tokens.tf line 48, in data "google_service_account_access_token" "dev-coupons-admin":
│   48: data "google_service_account_access_token" "dev-coupons-admin" {
│

I don't understand why creating the "google_service_account_access_token" "dev_coupons_admin" returns a 403... At first, I thought it is because some parent module's provider was interfering, but no, here we are at the base module dev, with the same credentials that created the whole dev environment ressources, with the same associated user email, yet this denial of access is returned.

I then enabled logs export TF_VARS=DEBUG; export TF_LOG_PATH="terraform_log.txt", and I find this line :

---[ REQUEST ]---------------------------------------
POST /v1/projects/-/serviceAccounts/dev-coupons-admin@<redacted_project_id>.iam.gserviceaccount.com:generateAccessToken?alt=json&prettyPrint=false HTTP/1.1
Host: iamcredentials.googleapis.com
User-Agent: google-api-go-client/0.5 Terraform/1.2.9 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/dev
Content-Length: 129
Content-Type: application/json
X-Goog-Api-Client: gl-go/1.18.1 gdcl/0.92.0
Accept-Encoding: gzip

{
 "lifetime": "1200s",
 "scope": [
  "https://www.googleapis.com/auth/cloud-platform",
  "https://www.googleapis.com/auth/userinfo.email"
 ]
}

-----------------------------------------------------: timestamp=2022-09-21T21:25:58.442+0200
2022-09-21T21:25:58.534+0200 [INFO]  provider.terraform-provider-google_v4.36.0_x5: 2022/09/21 21:25:58 [DEBUG] Google API Response Details:
---[ RESPONSE ]--------------------------------------
HTTP/2.0 403 Forbidden
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Wed, 21 Sep 2022 19:26:04 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

{
  "error": {
    "code": 403,
    "message": "The caller does not have permission",
    "errors": [
      {
        "message": "The caller does not have permission",
        "domain": "global",
        "reason": "forbidden"
      }
    ],
    "status": "PERMISSION_DENIED"
  }
}

Perhaps it is trying to access "all" projects via the designated service account? See - in /v1/projects/-/serviceAccounts/?

If you are able to shed some light on where my understanding is lacking, I would greatly appreciate it.

EDIT : dev-coupons-admin is the owner of dev-coupons



Comments

Popular posts from this blog

Today Walkin 14th-Sept

Spring Elasticsearch Operations

Hibernate Search - Elasticsearch with JSON manipulation