nvidia-drm: return -ENOSYS when PRIME sg_table is unavailable - #1317
Open
VaggelisGian wants to merge 1 commit into
Open
nvidia-drm: return -ENOSYS when PRIME sg_table is unavailable#1317VaggelisGian wants to merge 1 commit into
VaggelisGian wants to merge 1 commit into
Conversation
Vidmem-backed NvKmsKapiMemory has no struct pages, so __nv_drm_gem_nvkms_memory_prime_get_sg_table() can never build an sg_table for it. It returned ERR_PTR(-ENOMEM), which propagates unchanged through dma_buf_map_attachment() to the importing driver, so a cross-device PRIME import of a vidmem GEM object reports an out-of-memory condition that never happened. On multi-GPU systems this makes compositors treat an unsupported export as memory exhaustion and abort (NVIDIA#1302). Return -ENOSYS instead, matching the value drm_prime.c uses when a GEM object has no get_sg_table implementation. The errno is the only change: no caller in nvidia-drm switches on the old value, and real allocation failures on this path still return -ENOMEM from their own sites. Test Plan: No build or runtime test possible in the preparation environment (no Linux kernel toolchain, no NVIDIA GPU). Verified statically: git diff --stat : 1 file changed, 1 insertion(+), 1 deletion(-) pages_count == 0 occurs only for vidmem-backed objects; sysmem objects whose page list cannot be built fail at object init. No caller under kernel-open/nvidia-drm branches on the errno value; DRM core forwards PTR_ERR unchanged to the importer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the misleading error reported in #1302.
Problem
__nv_drm_gem_nvkms_memory_prime_get_sg_table()returnsERR_PTR(-ENOMEM)for NvKmsKapiMemory backed by video memory (pages_count == 0, no struct pages). The value propagates verbatim throughdma_buf_map_attachment()to the importing driver: a cross-device PRIME import of such an object reports an out-of-memory condition that never happened. As reported in #1302, kwin_wayland treats this as real memory exhaustion and aborts instead of falling back or reporting an unsupported operation.Change
One line: return
ERR_PTR(-ENOSYS)instead ofERR_PTR(-ENOMEM)in that case.-ENOSYSis what DRM core itself uses when a GEM object has no sg_table implementation:WARN_ON(!obj->funcs->get_sg_table); return ERR_PTR(-ENOSYS);-ENOSYS.The errno is the only change:
PTR_ERRunchanged.-ENOMEMfrom their own sites.pages_count == 0occurs only for vidmem objects: sysmem objects whose page list cannot be built fail at object init (__nv_drm_nvkms_gem_obj_init).Test Plan
Prepared on Windows without Linux kernel toolchain or NVIDIA GPU, so no build and no runtime test were possible; verification was static:
Traced all four constructors of nvkms-memory GEM objects (dumb_create, import_nvkms_memory_ioctl, alloc_nvkms_memory_ioctl, prime_dup) to confirm
pages_count == 0implies vidmem; grepped all callers ofnv_drm_gem_prime_get_sg_tablefor errno-specific branching (none). A hardware test on any multi-GPU system importing a vidmem-backed nvidia-drm buffer from another driver would exercise the changed path.