Thank you, sponsors for your continued support.
This months update on what I did/tried for Reason/OCaml OSS
1. esy's source cache is now versioned. (unreleased)
This would let esy developers to bust the sources cache should anything change regarding tarballs are installed.
A good example of this was how the `extra-sources` field is handled - it changed the contents of sources that get installed. When I implemented support for `extra-sources` and released it, users would see the fix work since esy would consider the older cached version valid. This was considered okay because fetching and installing sources rarely change.
By prefixing the sources path with a version, we can now invalidate the old cache and ask esy, internally, to fetch the sources as per the latest implementation.
2. De-duplicate opam-repository and esy-opam-override repository sync in the new multiplat solver (WIP)
As a first step towards portable dependency solutions, I had parameterised the exisiting solver function over os and arch and decided to loop over the list of supported (os, arch) tuples.
This got me to a working solution quickly, but also meant it was unnecessarily running some operations. One of which was the opam-repository and esy-opam-override repository syncs.
3. esy-opam override for ocamlfind 1.9.8
The new ocamlfind 1.9.8 package needed an updated patch for Windows. This is unrelated to the earlier issue where ocamlfind incorrectly assumes it does work on Windows - to fix this users could opt into creating opam overlays and use the `opamRepositories` field in package.json.
4. Fix prefix rewriting in postinstall generated by esy/github-action
`postinstall` command in the esy/github-action
The github action, which generates a postinstall script so that CLI apps created with esy can running multiple platforms packaged in the same NPM tarball, needed a fix to correctly figure the ocaml version to make sure prefix rewriting works correctly.
5. Understanding why rtop was broken on OCaml 5 and writing a vanilla rtop (without depending on utop)
rtop was initially broken, but cleverly fixed by anmonteiro (thanks!). However, I couldn't figure out the fix, or why it worked. Turns out, OCaml 5.2 introduced evaluation of multiple phrases in the same line, which wouldn't work with rtop's internal lexbuf state - reason parser, for some reason, resets it. I still couldn't figure this one out. You can find my exploratory patch in the next post.
Big thanks for my sponsors who continue to motivate me to work on Reason/OCaml. Here's the work done last month
esy: Support for platform specific packages
Packages like `libwasmtime` are platform specific and it's evident from their names
libwasmtime.0.21.0+linux-x86_64
libwasmtime.0.21.0+macos-x86_64
libwasmtime.0.22.0+linux-x86_64
libwasmtime.0.22.0+macos-x86_64
Another good example is `eio_main` - this package contains dependencies field that looks like this,
"eio_linux" {= version & os = "linux" & (os-distribution != "centos" | os-version > "7")}
"eio_posix" {= version & os != "win32"}
"eio_windows" {= version & os = "win32"}
Such expressions are evaluated during package installations on the client.
These individual packages, use the `available` field to hide/show themselves to the solver. For example, `eio_linux` contains the following,
available: [os = "linux"]
Thus,
1. On non-Linux machines, solver is supposed to ignore `eio_linux`
2. On Linux machines, `eio_main`'s dependencies field is supposed to look like this
"eio_posix" {= "1.2"}
"eio_windows" {= "1.2"}
But on Linux,
"eio_linux" {= "1.2"}
"eio_posix" {= "1.2"}
"eio_windows" {= "1.2"}
One can see how it can pose problems for projects that are supposed to build on multiple platforms.
To make things worse, some packages make only a certain version unavailable. `ocamlfind` `1.9.6` for example isn't available on Windows, but `1.9.5` is!
To address these things, the package manager fetching OPAM packages must compute different solutions for each platform. However, this can
slow things down significantly for the user - solutions have to be computed for all the expected (expectations set by the project owner) os-arch combinations.
To avoid unnecessary computations, esy does the following: compute the solution like it used to. Scan the dependency graph for packages that wont build everywhere - compute this set of platforms
where all the graph nodes aren't available, and only for those platforms re-compute the solution. A good example is esy itself - in it's dependency graph, only `ocamlfind` isn't the dependency that
doesn't build uniformly (1.9.6 doesn't build on Windows, 1.9.5 does).
This way, the re-computation of the solution is only triggered if necessary.
Note that this work is still being tested and awaiting feedback. Feel free to try out the nightly once it's built on CI and ping me on Reason Discord
esy: windows ci: fix broken inline test runner
Inline test runner, that builds some c++ code failed to build because it wasn't configured to statically link with libstdc++.dll
esy: minor internal documentation and maintenance chores.
You can find all the relevant links in the next post.