Today work has focused (as per the usual) on two things; Working out some more the details of the ctest extension for modules, and continuing work on the Typst parser (which has seen a refactor to greatly simplify things.)
As I repeatedly looked over more and more instances where there were uses of what once was the ident function but now is the path function, I realized I had to come up with a bit more clever solutions to those one-off call sites.
The one I struggled a bit more to solve was concerned with the function that maps Rust types to C types such that they may be given to the mapper functions we get through TestGenerator’s public API. Basically, stuff works out just fine with new types like records or untagged unions, but no so much with type aliases meant to replicate C enumerations.
The former I can quite easily make a lookup indexed by the symbol’s identifier and not the symbol’s full path. The latter previously made a direct look up into the set of global types (across all modules) that should be mapped to C enumerations in the generated C tests.
The issue is that we carry through a single instance of TestGenerator at a time, and that holds skips and mappings for all modules. This works out just fine everywhere else because those skips use the full paths to the symbols, which each module for which we generate tests keeps (because the types representing those items themselves keep that information.)
I eventually settled for performing first a lookup on the current module for which we are generating tests to see whether there exists some type alias that matches the identifier the overarching mapping function gets passed. If this check is successful, then I instead pass the full path of the found item (in the current module, but relative to the top-level crate root) to the routine that checks if TestGenerator has a mapping for that type alias.
This works out just fine because that last path is unique for each item.
With that, I thought everything that needed changing was covered, so I went back to looking through the test templates for uses of each of the fields in the types used for generating the tests. Some of these need changing the use of ident and path functions, as well as escaping the returned path by replacing the default path separator with _.
That I had already started on yesterday, and I have continued working on today. I also worked on a utility routine that would allow escaping all item paths by getting rid of every segment of the recursively containing paths (e.g. an array of arrays,) but soon realized it was useless. It gets the job done, but I should’ve thought more about my actual need for it before starting to work on it.
In terms of compiler work, I have changed the approach I thought up yesterday to providing context to each newly introduced binding that used pattern destructuring. That was an idea I came up with late yesterday to somehow pack the required context to validate a binding such that I can actually ensure its refutability at eval-time.
Today I thought I could go simpler; Instead of packing context that is copied across each binding introduced in a pattern (which I also have to walk to detect each named binding,) I can simply modify the type used for bindings to contain one of two polymorphic variants; A simple binding that contains a single direct pattern, and a grouped binding that contains a destructuring pattern that introduced multiple bindings.
Then I also (1) worked on some utility functions to make it more ergonomic to handle expressions within blocks, (2) got rid of the parameter type for functions as those can be generalized as patterns with an additional named variant only valid at the top-level layer of a potentially nested pattern, (3) and refactored a bunch of productions across rules of the Menhir grammar to stop doing any form of error detection.
This latter task remains unfinished, but it’s a work in progress. I also got rid of the arg type, which was fairly useless as it could be inlined through polymorphic variants with a closed matching on the call type that needs it.
Finally, I refactored the type infrastructure around the yielder type to stop using a functor and multiple modules that would be fed into the functor to resolve the type constructor into a concrete parameter for each of the yielder’s data constructors. Now the yielder is a parameterized type that passes off its type variable α into the expr parameter of one of its data constructors.
None.
I expect the ctest stuff to progress a lot more on the side of getting the generated tests to point to symbols with the right paths tomorrow. Beyond that, the only thing left is to change the code a few abstraction layers above that, such that we may generate multiple filled-in tests templates for each parsed module.
Compiler work is progressing nicely. I think I’ll also have to refactor the operator type because in my attempts to constrain it throughout this week, I forgot that I could easily cause a compile-time type error on the generated Menhir grammar, as typing there is far less stronger (I can’t really guarantee that a certain expression gets parsed in a specific production without being overly verbose with inlined productions that copy productions from the overarching expression rule.) We’ll see.