Skip to content

Support for NamedTuple in focus macros - #1610

Open
arainko wants to merge 24 commits into
optics-dev:masterfrom
arainko:named-tuples-support
Open

Support for NamedTuple in focus macros#1610
arainko wants to merge 24 commits into
optics-dev:masterfrom
arainko:named-tuples-support

Conversation

@arainko

@arainko arainko commented Jul 23, 2026

Copy link
Copy Markdown

The general design revolves around NamedTuples - a small helper class with a bunch of methods for dissecting/reconstructing/constructing named tuples.

NamedTuples is resolved once per each macro invocation via the NamedTuples.Support val, the value of which is None on Scalas < 3.6.x and Some everywhere else (this is achieved by trying to look up the symbol for the NamedTuple companion and the NamedTuple opaque type itself, which is nested inside that companion).

I've also added the scalaNextTest subproject which is supposed to be pinned to the newest (stable) release of Scala available at the time (or uhh until 3.9 comes out?) which houses a bunch of tests that cannot live in a subproject that doesn't support named tuples.

I'll provide more in-depth rundown through the various methods in the diff itself.

Most of the stuff here is just following on from my experience in implementing various named tuple bits and bobs in:

  • ducktape (1, 2, 3),
  • chanterelle (no specific PRs linked, that project is just named tuples from top to bottom)

Closes #1491

Also, since it doesn't go without saying nowadays, no LLMs were involved in making any of this.


(action.fromDescription.sourceType.asType, action.toType.asType) match {
case ('[f], '[t]) =>
if (action.fromDescription.values.size == 1) {

@arainko arainko Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright so you might be wondering why this weird check - case classes have a whole separate Generator/Parser for when they have a single field and multiple fields so why not just follow suit?

That's because I've already got all of the information that I might need from a single SelectNamedTupleFieldParser call which provides us with a NamedTuples.Description which is more or less a full description of a given named tuple (its field names, value types, all the other goodies) which I then just reuse here (with the one caveat being this size check here since I wanted to follow the case class behavior as close as possible and generate an Iso if we're operating on a single-field named tuple).

term match {
// the compiler expands a call like 'someNamedTuple.someField' to a call on NamedTuple.apply[Names, Values](someNamedTuple)(idx) where idx == index of 'someField' in the Names tuple
case Apply(
Apply(TypeApply(Select(ident, "apply"), _), remainingCode :: Nil),

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ident could be replaced with Ident("NamedTuple") but comparing the symbols directly is waaaaaaay more bulletproof

) if ident.symbol == namedTuples.companion =>
for {
description <- namedTuples.describe(getType(remainingCode))
fieldType <- description.values.lift(fieldIndex)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say that those .lift calls could be direct .apply calls instead since fieldIndex is guaranteed to be defined for desciption.values and description.names unless something goes very very bad with the information extraction from the named tuple itself (in which case I'd be having a pretty embarassing bug to resolve), we might save on 2 Option allocs that way but I decided against it because I'm already in an Option context so might as well do it the proper way 🤷 AND it looks cooler ig


def getType(code: Term): TypeRepr =
code.tpe.widen.dealias
code.tpe.widen.dealias.simplified

@arainko arainko Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .simplified call here (and later on) is needed because otherwise if an unreduced match type (or NamedTuple.From) arrives at this call site we're greeted with a compiler barf, something akin to:

Cannot get type of value [...]

As a reminder, the way the compiler extracts a field from a named tuple like someNamedTuple.someField is to call NamedTuple.apply(someNamedTuple)(<index of someField)>) (which is again, a desugared form of an extension call like someNamedTuple.apply(<index of someField>)) and that method is defined as:

extension [N <: Tuple, V <: Tuple](x: NamedTuple[N, V])
      /** The value (without the name) at index `n` of this tuple. */
    inline def apply(n: Int): Elem[NamedTuple[N, V], n.type] =
      x.toTuple.apply(n).asInstanceOf[Elem[NamedTuple[N, V], n.type]]

emphasis on the return type: Elem[NamedTuple[N, V], n.type] which IS a match type so we need to reduce those (which .simplified does)

object CaseClass {
def unapply(term: Term): Option[Term] =
term.tpe.classSymbol.flatMap { sym =>
term.tpe.simplified.classSymbol.flatMap { sym =>

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same reason for that .simplified addition as in https://github.com/optics-dev/Monocle/pull/1610/changes#r3641484068

fieldName: String,
fromDescription: NamedTuples.Description,
toType: TypeRepr,
namedTuples: NamedTuples

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might feel weird to carry the whole helper class around but IMO it's the cleanest way of saying 'this is only valid if named tuples are supported' (+ it has a bunch of uses in SelectNamedTupleFieldGenerator)

.appliedToTypes(description.namesTpe :: description.valuesTpe :: Nil)
.appliedTo(term)

def accessFieldByName(term: Term, action: FocusAction.SelectNamedTupleField): Term = {

@arainko arainko Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this essentially comes down to something like this:

// let's say the field we want to access is 'field2'
val someNamedTuple = (field1 = 1, field2 = 2)
someNamedTuple.toTuple._2 // but ONLY when the result of .toTuple actually has a ._N accessor, there are cases where it might not have it (see unsafeAccessFieldByIndex doc comment)

And if you're curious why we aren't doing the same thing the compiler is doing underneath (someNamedTuple.apply(1) in this case) - it's the match type return type again (Elem[NamedTuple[N, V], n.type]), those things are a major annoyance when constructing trees by hand and I just couldn't get it to behave.


test("Applied focus returning an Optional in nested named tuples") {
// works when type of address is widened from Some to Option
// the macro cannot find a matching overload otherwise - note that this also happens in a normal .focus invocation on case classes when a type is described as Some instead of Option

@arainko arainko Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left this small piece of commentary here because we get a rather ugly error if an Option field gets narrowed down to Some (which is very easy to do in test code like this because I'm just usually calling Some(blahblah), this also happens if we declare a case class with a Some so I didn't pay it much more attention after that).

import monocle.Focus
import monocle.Iso

final class NamedTupleFocusTest extends munit.FunSuite {

@arainko arainko Jul 23, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftr the tests in this file are repurposed versions of tests for case classes from around the codebase (well, these are the ones I managed to find and identify as focusing on 'focus' and case classes) plus some additions to test the behavior in presence of NamedTuple.From

Comment thread build.sbt
.settings(
crossScalaVersions := Seq(scalaNextVersion),
libraryDependencies ++= Seq(munitDiscipline.value),
scalacOptions --= Seq("-release:8", "-Ykind-projector")

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this gets compiled under 3.8.4 I had to get rid of those two options (tbh only the -release one was a hard error due to 3.8.x shifting to JDK17, the kind projector one was a deprecation warning since it was promoted to an -X option)

@xuwei-k
xuwei-k removed their request for review July 24, 2026 05:41
@julien-truffaut

Copy link
Copy Markdown
Member

Thanks for your proposal.

I am not entirely convinced we should support named tupled. I haven't used this feature but it feels like something you would use for top level data structure where optics aren't extremely useful for.

@arainko

arainko commented Jul 24, 2026

Copy link
Copy Markdown
Author

Thanks for your proposal.

I am not entirely convinced we should support named tupled. I haven't used this feature but it feels like something you would use for top level data structure where optics aren't extremely useful for.

Obviously my views on that are completely subjective (and maybe even biased) but I've been quite the heavy user of named tuples since they came out and I had pretty great success in cases like deserializing JSON bodies into nested named tuples - a quick example from one of my projects:

 private def search(query: String) = {
      val req = Request[IO]()
        .withMethod(Method.GET)
        .withUri((baseUrl / "api" / "get-music").withQueryParam("q", query).withQueryParam("offset", 0))

      client
        .expect[(data: (albums: (items: List[(id: String)])))](req)
        .map(_.data.albums.items)
    }

...and being able to quickly modify such nested structures comes in handy faster than one could imagine (ofc, this could be represented as a stack of one-off case classes, but that just feels rather boilerplatey now that we have actual ad-hoc records).

There are more examples of this from around the ecosystem now, like scala-sql's support for named tuples for deserializing DB rows, jsoniter's support for named tuples and some other projects that I'm forgetting now...

Another argument I have would be the fact that monocle already supports case classes and positional tuples and leaving named tuples out of the equation would feel rather... Weird and incomplete? Especially given that they are the ad-hoc mirror to nominal case classes AND there's no easy way of modifying a named tuple out of the box (there's no .copy or an equivalent that one could call, your best option is .toTuple.copy(_N = newValue).build[(<field names copied over>)] and that's only for the toplevel value).

Even if one goes with an assumption that these are only fit for toplevel structures - those toplevel structures would usually also have case classes as their fields (or Options, collections and combinations of all of the above) and not supporting named tuples immediately disqualifies it from the convenience of .focus(_.namedTupleField.some.caseClassField).replace(blah).

Last but not least, there seems to be buy-in from the users (at least #1491 would suggest so) and before venturing out for my adventure here I tested the waters on discord on whether there's an appetite for such a feature from the side of the maintainers.

All of that being said I also respect the opinion of not allowing this in and I hope my arguments at least make you think about it again :D I really like the library and I've been using it since forever so thank you for your maintenance over all these years.

@cquiroz

cquiroz commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Maybe i jumped the gun, i thought this is a good idea for monocle, maybe because I'm a user of named tuples and it seems like just a field for a case class that we already support.

@julien-truffaut We could put it on an experimental package maybe before committing it to the api?

@arainko

arainko commented Jul 26, 2026

Copy link
Copy Markdown
Author

I've hit a rather annoying technicality with the build - CI builds against JDK11 and JDK25 and Scala 3.8.x is JDK17+ which means that my newly introduced tests subproject fails to run in the JDK11 matrix cells:

java.lang.UnsupportedClassVersionError: dotty/tools/xsbt/CompilerBridge has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0

There's no easy way of exluding a specific subproject from the whole matrix run (for example, with githubWorkflowBuildMatrixExclusions since that only covers the toplevel projects root{JVM,JS/Native}, at least that's what sbt told me after trying :c) so I think the best way of doing that is NOT aggregating scalaNextTest in root and giving it its own workflow step:

// prolly will also need duplicates for JS and Native?
githubWorkflowBuild += WorkflowStep.Sbt(
      List("scalaNextTestJVM/test"),
      name = Some("Run Scala Next Tests"),
      cond = Some(s"matrix.java == 'temurin@25' && matrix.scala == '3' && matrix.project == 'rootJVM'")
    )

But before I do that I'd like to know if you're okay with this kinda build complications. An alternative to this is letting scalaNextTest be built on 3.7.4 and not 3.8.4 since that version still supports JDK11 but that's just kicking the can down the road. Yet another alternative is the nuclear option - building on JDK17 instead of 11 (obviously that prolly wont fly).

Comment thread build.sbt
lazy val catsVersion = "2.13.0"
lazy val scala2Version = "2.13.18"
lazy val scala3Version = "3.3.8"
lazy val scalaNextVersion = "3.8.4"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have two scala versions here, as a library we want to stay on LTS but it seems 3.8 is only used for new test. Is this because named tuples are not present on LTS?

@arainko arainko Jul 26, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, that's exactly it. I needed a test project to test those new macros

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does it mean that the new macros aren't available on a scala LTS version?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are available - there's a branching path that hinges on the availability of Symbol.requiredModule("scala.NamedTuple") - this particular call will resolve to NoSymbol when used from an app that compiles on Scalas lower than 3.6 (where named tuples were an experimental feature) because NamedTuple quite literally doesn't exist on those versions, then if you add monocle as dependency in a Scala 3.6+ project and use focus on a named tuple you'll get proper support for the feature.

Note that a named tuple isn't a proper new language feature, it's just a bunch of sugar on top of opaque type NamedTuple[Names <: Tuple, Values <: Tuple] >: Tuple = Values so supporting it from 3.3.x is fully possible (and implemented in this PR!)

@julien-truffaut

Copy link
Copy Markdown
Member

@cquiroz @arainko fair enough. I haven't used this feature like you did. I am happy to add support for it in Monocle but we need to think about the best ways to do it.

In particular, what are the minimum requirements for it to work (JDK/scala) and how will it behave on older versions.

@arainko

arainko commented Jul 28, 2026

Copy link
Copy Markdown
Author

@cquiroz @arainko fair enough. I haven't used this feature like you did. I am happy to add support for it in Monocle but we need to think about the best ways to do it.

In particular, what are the minimum requirements for it to work (JDK/scala) and how will it behave on older versions.

Thank you for the reconsideration ♥️

To answer your question: there's no need to bump neither Java nor Scala in the published artifacts - this is fully backwards compatible.

Comment thread build.sbt
name = Some("Run documentation"),
cond = Some(s"matrix.scala == '2.13' && matrix.project == 'rootJVM'")
)
) ++ scalaNextTest.projects.map { case (platform, project) =>

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's the what (and the why) on what I did to the build:

  • scalaNextTest is built with Scala 3.8.4 (it could be built with 3.7.x or even 3.6.x but named tuples are experimental in 3.6.x so 3.7.x is the lowest possible version we could really test against) but I didn't find much reasons to not go with the shinest and brightest version,
  • Scala 3.8.x is only built on JDK17+ so I needed to ensure that we're building on JDK25 (because it was already in the build matrix and I didn't want to meddle with the build even further).

Note that this DOES NOT mean the library needs to built against 3.8.4, this is just a small subproject with tests to test my impl.

@cquiroz

cquiroz commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator
// prolly will also need duplicates for JS and Native?
githubWorkflowBuild += WorkflowStep.Sbt(
      List("scalaNextTestJVM/test"),
      name = Some("Run Scala Next Tests"),
      cond = Some(s"matrix.java == 'temurin@25' && matrix.scala == '3' && matrix.project == 'rootJVM'")
    )

But before I do that I'd like to know if you're okay with this kinda build complications.

I think it is ok

import monocle.Focus
import monocle.Iso

final class NamedTupleFocusTest extends munit.FunSuite {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do law testing for named tuples optics?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like

checkAll("Focus named tuple field", LensTests(Focus[(a: Int, b: String)](_.a)))
checkAll("Focus single-field named tuple", IsoTests(Focus[(a: Int)](_.a)))

def construct(description: NamedTuples.Description, values: Seq[Expr[Any]]): Term =
Typed(Expr.ofTupleFromSeq(values).asTerm, TypeTree.of(using description.sourceType.asType))

// there's a chance that we're operating on a non-normalized (non TupleN) tuple (for example when N is > 22 or when using NamedTuple.From)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have tests for tuples with more than 22 named params?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are tests for named tuples derived from NamedTuple.From but not explicit > 22 ones, I'll toss a test or two in.


test("Single field access") {
assertEquals(
Focus[Animal](_.name).get(Animal("Bob")),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need the type aliases for Focus to work?

@cquiroz cquiroz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, in general this seems good, I left a few comments for you to consider, probably the most important thing would be to do law testing for lenses.

I think we may want to consider how to best relase this consider julien's concerns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NamedTuple support

3 participants