Bug Description
In gemini-stage-0, discovery-engine.tf creates a GCS bucket (google_storage_bucket.gemini_enterprise_gcs_bucket) for the document corpus when create_bucket = true, and cmek.tf grants the project's Discovery Engine service agent roles/cloudkms.cryptoKeyEncrypterDecrypter on the CMEK key. But nothing grants that service agent (service-<PROJECT_NUMBER>@gcp-sa-discoveryengine.iam.gserviceaccount.com) roles/storage.objectViewer (or objectUser) on the bucket it just created — iam.tf only binds project-level roles to the admin/user GROUPS, and there is no google_storage_bucket_iam_member for the DE service agent anywhere in the stage. As a result, when the operator later runs gem4gov datastore import --source-type=gcs --gcs-bucket=<that bucket>, the ImportDocuments long-running operation completes done: true / error: null but with failureCount == totalCount: every document fails with code 7 PERMISSION_DENIED — "Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store indexes zero documents. Because gem4gov's import does not poll or surface the per-document result (the counts live only in the operation metadata / response.errorSamples), this looks like a silent success.
Environment and Deployment Context
- Stellar Engine Version/Commit:
main (gemini-stage-0/discovery-engine.tf + iam.tf + cmek.tf; gem4gov-cli/data_stores.py import_gcs_documents)
- Deployment Type:
- Affected Component:
blueprints/fedramp-high/gemini-enterprise/gemini-stage-0/iam.tf (no bucket-scoped grant for the Discovery Engine service agent) + discovery-engine.tf (google_storage_bucket.gemini_enterprise_gcs_bucket).
Steps to Reproduce
- Run
gemini-stage-0 with a GCS data store and create_bucket = true so the stage creates …-<display-name>-data.
- Copy documents into that bucket.
- Run
gem4gov datastore import --project-id <proj> --source-type=gcs --data-store-id <id> --gcs-bucket <that bucket> and press Enter for the root path.
- Poll the import operation:
metadata.failureCount == metadata.totalCount, response.errorSamples[*].code == 7, message "Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store's documents list is empty.
Expected Behavior
gemini-stage-0 should grant the Discovery Engine service agent read on the corpus bucket it creates, e.g. a google_storage_bucket_iam_member binding roles/storage.objectViewer (or roles/storage.objectUser) for serviceAccount:service-<num>@gcp-sa-discoveryengine.iam.gserviceaccount.com on google_storage_bucket.gemini_enterprise_gcs_bucket — mirroring the CMEK grant it already makes for the same agent. Then a corpus import from that bucket succeeds without a manual IAM step.
Actual Behavior
The stage creates the bucket and the CMEK grant but not the storage grant, so every subsequent document import fails PERMISSION_DENIED and the data store indexes nothing — with no error surfaced by the CLI.
Additional Context
The KMS grant for the same agent is already present (google_kms_crypto_key_iam_member.discoveryengine_sa_kms_access), so the missing piece is a single parallel storage binding. Related: gem4gov's import is fire-and-forget with no polling/validation, which is what makes this failure silent — worth its own hardening (surface failureCount/errorSamples, or poll the LRO).
Bug Description
In
gemini-stage-0,discovery-engine.tfcreates a GCS bucket (google_storage_bucket.gemini_enterprise_gcs_bucket) for the document corpus whencreate_bucket = true, andcmek.tfgrants the project's Discovery Engine service agentroles/cloudkms.cryptoKeyEncrypterDecrypteron the CMEK key. But nothing grants that service agent (service-<PROJECT_NUMBER>@gcp-sa-discoveryengine.iam.gserviceaccount.com)roles/storage.objectViewer(orobjectUser) on the bucket it just created —iam.tfonly binds project-level roles to the admin/user GROUPS, and there is nogoogle_storage_bucket_iam_memberfor the DE service agent anywhere in the stage. As a result, when the operator later runsgem4gov datastore import --source-type=gcs --gcs-bucket=<that bucket>, the ImportDocuments long-running operation completesdone: true/error: nullbut withfailureCount == totalCount: every document fails withcode 7 PERMISSION_DENIED—"Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store indexes zero documents. Because gem4gov's import does not poll or surface the per-document result (the counts live only in the operationmetadata/response.errorSamples), this looks like a silent success.Environment and Deployment Context
main(gemini-stage-0/discovery-engine.tf + iam.tf + cmek.tf; gem4gov-cli/data_stores.pyimport_gcs_documents)blueprints/fedramp-high/gemini-enterprise/gemini-stage-0/iam.tf(no bucket-scoped grant for the Discovery Engine service agent) +discovery-engine.tf(google_storage_bucket.gemini_enterprise_gcs_bucket).Steps to Reproduce
gemini-stage-0with a GCS data store andcreate_bucket = trueso the stage creates…-<display-name>-data.gem4gov datastore import --project-id <proj> --source-type=gcs --data-store-id <id> --gcs-bucket <that bucket>and press Enter for the root path.metadata.failureCount == metadata.totalCount,response.errorSamples[*].code == 7, message"Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store'sdocumentslist is empty.Expected Behavior
gemini-stage-0should grant the Discovery Engine service agent read on the corpus bucket it creates, e.g. agoogle_storage_bucket_iam_memberbindingroles/storage.objectViewer(orroles/storage.objectUser) forserviceAccount:service-<num>@gcp-sa-discoveryengine.iam.gserviceaccount.comongoogle_storage_bucket.gemini_enterprise_gcs_bucket— mirroring the CMEK grant it already makes for the same agent. Then a corpus import from that bucket succeeds without a manual IAM step.Actual Behavior
The stage creates the bucket and the CMEK grant but not the storage grant, so every subsequent document import fails
PERMISSION_DENIEDand the data store indexes nothing — with no error surfaced by the CLI.Additional Context
The KMS grant for the same agent is already present (
google_kms_crypto_key_iam_member.discoveryengine_sa_kms_access), so the missing piece is a single parallel storage binding. Related: gem4gov's import is fire-and-forget with no polling/validation, which is what makes this failure silent — worth its own hardening (surfacefailureCount/errorSamples, or poll the LRO).