Skip to content

[gpu] install_gpu_driver.sh is missing YARN group mapping permissions (video/render) for Secure Boot container workloads #1399

Description

@cjac

Context

When launching Apache Spark workloads in containers on Dataproc GPU clusters, the container executor runs under unprivileged user contexts (such as the job submitting user or the system yarn user).

By default, NVIDIA driver installations restrict access to the GPU device nodes in /dev/ (such as /dev/nvidia* and /dev/nvidia-uvm ) to the video and render groups with 0660 permissions.

The Problem

The gpu/install_gpu_driver.sh script in the initialization-actions repository does not configure group mappings to allow YARN Node Manager or job execution users to belong to the video and render groups.

As a result:

  1. The YARN auto-discovery script getGpusResources.sh (which executes nvidia-smi as the yarn user) fails with a permission denial when trying to access /dev/nvidia*.
  2. The YARN Node Manager daemon crashes on startup during node registration, leading to cluster creation timeouts (Error code 13).
  3. Containerized workloads launched by YARN are blocked from accessing GPU hardware because the container processes run without membership in the host's video or render groups.

Proposed Fix

Modify the gpu/install_gpu_driver.sh script to configure these group mappings (for example, executing usermod -aG video,render yarn ) during the YARN configuration phase, or ensure they are captured in the generated first-boot deferred configuration script.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions