Scalarizer Tracker

[ValerianRey]

This issue tracks the candidate methods that could maybe be implemented as Scalarizer in torchjd.scalarization.

Status	Name	Ref	Stateful	Existing implementations	Special Remarks
🟡 In progress	STCH (Smooth TCHebyshev)	Smooth Tchebycheff Scalarization for Multi-Objective Optimization (ICML 2024, 87 citations)	No	official, LibMTL
🔵 To investigate	IMTL-L	Towards Impartial Multi-Task Learning (ICLR 2021, 279 citations)	?	official, LibMTL (maybe this is IMTL-G)	Need more investigation
🔵 To investigate	FAMO	FAMO: Fast Adaptive Multitask Optimization (NeurIPS 2023, 124 citations)	?	official, LibMTL	Not sure this is a Scalarizer, need more investigation.
🔵 To investigate	GradNorm	GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks (ICML 2018, 2334 citations)	Yes, trainable state	unofficial, LibMTL	Not sure this is a Scalarizer, need more investigation.
🔵 To investigate	UW (UncertaintyWeighting)	Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics (CVPR 2018, 5535 citations)	?	unofficial, LibMTL	Not sure this is a Scalarizer, need more investigation.
🔵 To investigate	DWA (Dynamic Weight Averaging)	End-to-End Multi-Task Learning with Attention (CVPR 2019, 1836 citations)	?	official, LibMTL	Not sure this is a Scalarizer, need more investigation.

[ValerianRey]

List of other methods in LibMTL that we don't support yet and that are not listed here or in #665

• MOML (bilevel)
• AutoLambda (bilevel)
• Forum (bilevel)
• ExcessMTL (probably a scalarizer or an aggregator)