fix(checkpoint): avoid DCP collectives during initialization

[HuiyingLi]

What does this PR do ?

Load base-model checkpoints independently on each rank instead of invoking
PyTorch DCP's distributed coordinator during initialization.

Initialization has already materialized each rank's local tensor destinations,
so the coordinator's gather/global-plan/scatter collectives are unnecessary.
With an NCCL process group, those object collectives leave persistent non-PyTorch
GPU allocations and disproportionately charge coordinator rank 0.

Training-checkpoint resume remains distributed and continues to use the
checkpointer's process group.

Changelog

• Pass no_dist=True to DCP only for is_init_step base-checkpoint loads.
• Preserve process-group coordination for model and optimizer resume loads.
• Add regression tests covering both branches.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• No documentation update is required; this does not change config or public API.

Validation

• ruff format --check on both changed files: passed.
• ruff check nemo_automodel/components/checkpoint/checkpointing.py: passed.
• pytest -q tests/unit_tests/checkpoint/test_checkpointing.py: 202 passed, 5 skipped.
  One-node, eight-H100 A/B at the first base-checkpoint DCP boundary:

Path	Rank 0 non-PyTorch GPU memory	Ranks 1-7	Rank skew
Distributed DCP coordinator	4.15 GiB	1.90 GiB/rank	2.25 GiB
Independent rank-local DCP	1.47 GiB	1.47 GiB/rank	0 GiB

• At setup completion, the distributed path retained 4.66 GiB on rank 0 and
  2.41 GiB on ranks 1-7. The rank-local path retained 1.98 GiB on every rank.
• The rank-local path completed a 40-step, eight-H100 training run with finite
  loss and gradient norm. Loss decreased from 7.6569 at step 0 to 0.0451 at
  step 39.

The H100 qualification used the same _do_load behavior as current main;
changes to this file between the qualification base and a41906d6 are PEP 604
annotation-only changes.

Additional Information

• Related to perf(checkpoint): load standard HF safetensors with DCP #3616 and perf(checkpoint): bound quantized DCP load memory #3619, but not duplicated by them. Those drafts reduce
  checkpoint tensor/CPU memory and route more initialization loads through DCP;
  their current heads still pass process_group to _do_load and therefore
  retain the coordinator collectives addressed here.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.