Support for NamedTuple in focus macros - #1610
Conversation
…g a narrowed-down type of the lambda param), add additional tests
… tuples), cleanup the NamedTuples api
…ch of random blankspace changes I did across multiple files lol
|
|
||
| (action.fromDescription.sourceType.asType, action.toType.asType) match { | ||
| case ('[f], '[t]) => | ||
| if (action.fromDescription.values.size == 1) { |
There was a problem hiding this comment.
alright so you might be wondering why this weird check - case classes have a whole separate Generator/Parser for when they have a single field and multiple fields so why not just follow suit?
That's because I've already got all of the information that I might need from a single SelectNamedTupleFieldParser call which provides us with a NamedTuples.Description which is more or less a full description of a given named tuple (its field names, value types, all the other goodies) which I then just reuse here (with the one caveat being this size check here since I wanted to follow the case class behavior as close as possible and generate an Iso if we're operating on a single-field named tuple).
| term match { | ||
| // the compiler expands a call like 'someNamedTuple.someField' to a call on NamedTuple.apply[Names, Values](someNamedTuple)(idx) where idx == index of 'someField' in the Names tuple | ||
| case Apply( | ||
| Apply(TypeApply(Select(ident, "apply"), _), remainingCode :: Nil), |
There was a problem hiding this comment.
ident could be replaced with Ident("NamedTuple") but comparing the symbols directly is waaaaaaay more bulletproof
| ) if ident.symbol == namedTuples.companion => | ||
| for { | ||
| description <- namedTuples.describe(getType(remainingCode)) | ||
| fieldType <- description.values.lift(fieldIndex) |
There was a problem hiding this comment.
I'd say that those .lift calls could be direct .apply calls instead since fieldIndex is guaranteed to be defined for desciption.values and description.names unless something goes very very bad with the information extraction from the named tuple itself (in which case I'd be having a pretty embarassing bug to resolve), we might save on 2 Option allocs that way but I decided against it because I'm already in an Option context so might as well do it the proper way 🤷 AND it looks cooler ig
|
|
||
| def getType(code: Term): TypeRepr = | ||
| code.tpe.widen.dealias | ||
| code.tpe.widen.dealias.simplified |
There was a problem hiding this comment.
The .simplified call here (and later on) is needed because otherwise if an unreduced match type (or NamedTuple.From) arrives at this call site we're greeted with a compiler barf, something akin to:
Cannot get type of value [...]
As a reminder, the way the compiler extracts a field from a named tuple like someNamedTuple.someField is to call NamedTuple.apply(someNamedTuple)(<index of someField)>) (which is again, a desugared form of an extension call like someNamedTuple.apply(<index of someField>)) and that method is defined as:
extension [N <: Tuple, V <: Tuple](x: NamedTuple[N, V])
/** The value (without the name) at index `n` of this tuple. */
inline def apply(n: Int): Elem[NamedTuple[N, V], n.type] =
x.toTuple.apply(n).asInstanceOf[Elem[NamedTuple[N, V], n.type]]emphasis on the return type: Elem[NamedTuple[N, V], n.type] which IS a match type so we need to reduce those (which .simplified does)
| object CaseClass { | ||
| def unapply(term: Term): Option[Term] = | ||
| term.tpe.classSymbol.flatMap { sym => | ||
| term.tpe.simplified.classSymbol.flatMap { sym => |
There was a problem hiding this comment.
same reason for that .simplified addition as in https://github.com/optics-dev/Monocle/pull/1610/changes#r3641484068
| fieldName: String, | ||
| fromDescription: NamedTuples.Description, | ||
| toType: TypeRepr, | ||
| namedTuples: NamedTuples |
There was a problem hiding this comment.
it might feel weird to carry the whole helper class around but IMO it's the cleanest way of saying 'this is only valid if named tuples are supported' (+ it has a bunch of uses in SelectNamedTupleFieldGenerator)
| .appliedToTypes(description.namesTpe :: description.valuesTpe :: Nil) | ||
| .appliedTo(term) | ||
|
|
||
| def accessFieldByName(term: Term, action: FocusAction.SelectNamedTupleField): Term = { |
There was a problem hiding this comment.
this essentially comes down to something like this:
// let's say the field we want to access is 'field2'
val someNamedTuple = (field1 = 1, field2 = 2)
someNamedTuple.toTuple._2 // but ONLY when the result of .toTuple actually has a ._N accessor, there are cases where it might not have it (see unsafeAccessFieldByIndex doc comment)And if you're curious why we aren't doing the same thing the compiler is doing underneath (someNamedTuple.apply(1) in this case) - it's the match type return type again (Elem[NamedTuple[N, V], n.type]), those things are a major annoyance when constructing trees by hand and I just couldn't get it to behave.
|
|
||
| test("Applied focus returning an Optional in nested named tuples") { | ||
| // works when type of address is widened from Some to Option | ||
| // the macro cannot find a matching overload otherwise - note that this also happens in a normal .focus invocation on case classes when a type is described as Some instead of Option |
There was a problem hiding this comment.
I left this small piece of commentary here because we get a rather ugly error if an Option field gets narrowed down to Some (which is very easy to do in test code like this because I'm just usually calling Some(blahblah), this also happens if we declare a case class with a Some so I didn't pay it much more attention after that).
| import monocle.Focus | ||
| import monocle.Iso | ||
|
|
||
| final class NamedTupleFocusTest extends munit.FunSuite { |
There was a problem hiding this comment.
ftr the tests in this file are repurposed versions of tests for case classes from around the codebase (well, these are the ones I managed to find and identify as focusing on 'focus' and case classes) plus some additions to test the behavior in presence of NamedTuple.From
| .settings( | ||
| crossScalaVersions := Seq(scalaNextVersion), | ||
| libraryDependencies ++= Seq(munitDiscipline.value), | ||
| scalacOptions --= Seq("-release:8", "-Ykind-projector") |
There was a problem hiding this comment.
since this gets compiled under 3.8.4 I had to get rid of those two options (tbh only the -release one was a hard error due to 3.8.x shifting to JDK17, the kind projector one was a deprecation warning since it was promoted to an -X option)
|
Thanks for your proposal. I am not entirely convinced we should support named tupled. I haven't used this feature but it feels like something you would use for top level data structure where optics aren't extremely useful for. |
Obviously my views on that are completely subjective (and maybe even biased) but I've been quite the heavy user of named tuples since they came out and I had pretty great success in cases like deserializing JSON bodies into nested named tuples - a quick example from one of my projects: private def search(query: String) = {
val req = Request[IO]()
.withMethod(Method.GET)
.withUri((baseUrl / "api" / "get-music").withQueryParam("q", query).withQueryParam("offset", 0))
client
.expect[(data: (albums: (items: List[(id: String)])))](req)
.map(_.data.albums.items)
}...and being able to quickly modify such nested structures comes in handy faster than one could imagine (ofc, this could be represented as a stack of one-off case classes, but that just feels rather boilerplatey now that we have actual ad-hoc records). There are more examples of this from around the ecosystem now, like scala-sql's support for named tuples for deserializing DB rows, jsoniter's support for named tuples and some other projects that I'm forgetting now... Another argument I have would be the fact that Even if one goes with an assumption that these are only fit for toplevel structures - those toplevel structures would usually also have case classes as their fields (or Options, collections and combinations of all of the above) and not supporting named tuples immediately disqualifies it from the convenience of Last but not least, there seems to be buy-in from the users (at least #1491 would suggest so) and before venturing out for my adventure here I tested the waters on discord on whether there's an appetite for such a feature from the side of the maintainers. All of that being said I also respect the opinion of not allowing this in and I hope my arguments at least make you think about it again :D I really like the library and I've been using it since forever so thank you for your maintenance over all these years. |
|
Maybe i jumped the gun, i thought this is a good idea for monocle, maybe because I'm a user of named tuples and it seems like just a field for a case class that we already support. @julien-truffaut We could put it on an experimental package maybe before committing it to the api? |
|
I've hit a rather annoying technicality with the build - CI builds against JDK11 and JDK25 and Scala 3.8.x is JDK17+ which means that my newly introduced tests subproject fails to run in the JDK11 matrix cells: There's no easy way of exluding a specific subproject from the whole matrix run (for example, with // prolly will also need duplicates for JS and Native?
githubWorkflowBuild += WorkflowStep.Sbt(
List("scalaNextTestJVM/test"),
name = Some("Run Scala Next Tests"),
cond = Some(s"matrix.java == 'temurin@25' && matrix.scala == '3' && matrix.project == 'rootJVM'")
)But before I do that I'd like to know if you're okay with this kinda build complications. An alternative to this is letting |
| lazy val catsVersion = "2.13.0" | ||
| lazy val scala2Version = "2.13.18" | ||
| lazy val scala3Version = "3.3.8" | ||
| lazy val scalaNextVersion = "3.8.4" |
There was a problem hiding this comment.
Why do we have two scala versions here, as a library we want to stay on LTS but it seems 3.8 is only used for new test. Is this because named tuples are not present on LTS?
There was a problem hiding this comment.
Yup, that's exactly it. I needed a test project to test those new macros
There was a problem hiding this comment.
does it mean that the new macros aren't available on a scala LTS version?
There was a problem hiding this comment.
They are available - there's a branching path that hinges on the availability of Symbol.requiredModule("scala.NamedTuple") - this particular call will resolve to NoSymbol when used from an app that compiles on Scalas lower than 3.6 (where named tuples were an experimental feature) because NamedTuple quite literally doesn't exist on those versions, then if you add monocle as dependency in a Scala 3.6+ project and use focus on a named tuple you'll get proper support for the feature.
Note that a named tuple isn't a proper new language feature, it's just a bunch of sugar on top of opaque type NamedTuple[Names <: Tuple, Values <: Tuple] >: Tuple = Values so supporting it from 3.3.x is fully possible (and implemented in this PR!)
Thank you for the reconsideration To answer your question: there's no need to bump neither Java nor Scala in the published artifacts - this is fully backwards compatible. |
| name = Some("Run documentation"), | ||
| cond = Some(s"matrix.scala == '2.13' && matrix.project == 'rootJVM'") | ||
| ) | ||
| ) ++ scalaNextTest.projects.map { case (platform, project) => |
There was a problem hiding this comment.
here's the what (and the why) on what I did to the build:
scalaNextTestis built with Scala 3.8.4 (it could be built with 3.7.x or even 3.6.x but named tuples are experimental in 3.6.x so 3.7.x is the lowest possible version we could really test against) but I didn't find much reasons to not go with the shinest and brightest version,- Scala 3.8.x is only built on JDK17+ so I needed to ensure that we're building on JDK25 (because it was already in the build matrix and I didn't want to meddle with the build even further).
Note that this DOES NOT mean the library needs to built against 3.8.4, this is just a small subproject with tests to test my impl.
I think it is ok |
| import monocle.Focus | ||
| import monocle.Iso | ||
|
|
||
| final class NamedTupleFocusTest extends munit.FunSuite { |
There was a problem hiding this comment.
Should we do law testing for named tuples optics?
There was a problem hiding this comment.
Something like
checkAll("Focus named tuple field", LensTests(Focus[(a: Int, b: String)](_.a)))
checkAll("Focus single-field named tuple", IsoTests(Focus[(a: Int)](_.a)))| def construct(description: NamedTuples.Description, values: Seq[Expr[Any]]): Term = | ||
| Typed(Expr.ofTupleFromSeq(values).asTerm, TypeTree.of(using description.sourceType.asType)) | ||
|
|
||
| // there's a chance that we're operating on a non-normalized (non TupleN) tuple (for example when N is > 22 or when using NamedTuple.From) |
There was a problem hiding this comment.
Do we have tests for tuples with more than 22 named params?
There was a problem hiding this comment.
there are tests for named tuples derived from NamedTuple.From but not explicit > 22 ones, I'll toss a test or two in.
|
|
||
| test("Single field access") { | ||
| assertEquals( | ||
| Focus[Animal](_.name).get(Animal("Bob")), |
There was a problem hiding this comment.
Do we need the type aliases for Focus to work?
cquiroz
left a comment
There was a problem hiding this comment.
Thanks a lot, in general this seems good, I left a few comments for you to consider, probably the most important thing would be to do law testing for lenses.
I think we may want to consider how to best relase this consider julien's concerns
The general design revolves around
NamedTuples- a small helper class with a bunch of methods for dissecting/reconstructing/constructing named tuples.NamedTuplesis resolved once per each macro invocation via theNamedTuples.Supportval, the value of which isNoneon Scalas < 3.6.x andSomeeverywhere else (this is achieved by trying to look up the symbol for theNamedTuplecompanion and theNamedTupleopaque type itself, which is nested inside that companion).I've also added the
scalaNextTestsubproject which is supposed to be pinned to the newest (stable) release of Scala available at the time (or uhh until 3.9 comes out?) which houses a bunch of tests that cannot live in a subproject that doesn't support named tuples.I'll provide more in-depth rundown through the various methods in the diff itself.
Most of the stuff here is just following on from my experience in implementing various named tuple bits and bobs in:
Closes #1491
Also, since it doesn't go without saying nowadays, no LLMs were involved in making any of this.