Skip to content

[Bug] gemini-stage-0 creates the corpus GCS bucket but never grants the Discovery Engine service agent storage.objectViewer on it — gem4gov datastore import then fails every document with PERMISSION_DENIED #145

Description

@JohnHales

Bug Description

In gemini-stage-0, discovery-engine.tf creates a GCS bucket (google_storage_bucket.gemini_enterprise_gcs_bucket) for the document corpus when create_bucket = true, and cmek.tf grants the project's Discovery Engine service agent roles/cloudkms.cryptoKeyEncrypterDecrypter on the CMEK key. But nothing grants that service agent (service-<PROJECT_NUMBER>@gcp-sa-discoveryengine.iam.gserviceaccount.com) roles/storage.objectViewer (or objectUser) on the bucket it just created — iam.tf only binds project-level roles to the admin/user GROUPS, and there is no google_storage_bucket_iam_member for the DE service agent anywhere in the stage. As a result, when the operator later runs gem4gov datastore import --source-type=gcs --gcs-bucket=<that bucket>, the ImportDocuments long-running operation completes done: true / error: null but with failureCount == totalCount: every document fails with code 7 PERMISSION_DENIED"Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store indexes zero documents. Because gem4gov's import does not poll or surface the per-document result (the counts live only in the operation metadata / response.errorSamples), this looks like a silent success.

Environment and Deployment Context

  • Stellar Engine Version/Commit: main (gemini-stage-0/discovery-engine.tf + iam.tf + cmek.tf; gem4gov-cli/data_stores.py import_gcs_documents)
  • Deployment Type:
    • US Region Restricted (e.g., Access Policy constraint)
    • FedRAMP Medium
    • FedRAMP High
    • DoD IL4
    • DoD IL5
    • Stand-alone / Custom
  • Affected Component: blueprints/fedramp-high/gemini-enterprise/gemini-stage-0/iam.tf (no bucket-scoped grant for the Discovery Engine service agent) + discovery-engine.tf (google_storage_bucket.gemini_enterprise_gcs_bucket).

Steps to Reproduce

  1. Run gemini-stage-0 with a GCS data store and create_bucket = true so the stage creates …-<display-name>-data.
  2. Copy documents into that bucket.
  3. Run gem4gov datastore import --project-id <proj> --source-type=gcs --data-store-id <id> --gcs-bucket <that bucket> and press Enter for the root path.
  4. Poll the import operation: metadata.failureCount == metadata.totalCount, response.errorSamples[*].code == 7, message "Permission denied while checking the GCS content of gs://<bucket>/<file>". The data store's documents list is empty.

Expected Behavior

gemini-stage-0 should grant the Discovery Engine service agent read on the corpus bucket it creates, e.g. a google_storage_bucket_iam_member binding roles/storage.objectViewer (or roles/storage.objectUser) for serviceAccount:service-<num>@gcp-sa-discoveryengine.iam.gserviceaccount.com on google_storage_bucket.gemini_enterprise_gcs_bucket — mirroring the CMEK grant it already makes for the same agent. Then a corpus import from that bucket succeeds without a manual IAM step.

Actual Behavior

The stage creates the bucket and the CMEK grant but not the storage grant, so every subsequent document import fails PERMISSION_DENIED and the data store indexes nothing — with no error surfaced by the CLI.

Additional Context

The KMS grant for the same agent is already present (google_kms_crypto_key_iam_member.discoveryengine_sa_kms_access), so the missing piece is a single parallel storage binding. Related: gem4gov's import is fire-and-forget with no polling/validation, which is what makes this failure silent — worth its own hardening (surface failureCount/errorSamples, or poll the LRO).


Metadata

Metadata

Assignees

No one assigned

    Labels

    Level of Effort - LowQuick, well-defined tasks with no unknowns; takes a few hours up to one day to completePriority - MediumStandard features and non-blocking bugs; important for the current milestone but not urgentbugSomething isn't workinggemini for governmentGemini for Government (G4G) related

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions