So, for now, I've sent a PR which adds a little bit of caching of semver objects to the hot part of pnpm; this gets most of the speedup (~3min -> ~2min, +/- variance).
But, I think it'd be definitely cool to try and optimize semver or rethink its API to help everyone out.
Has anyone looked into optimizing Node's semver library? It turns out that 30%+ of pnpm's install time is spent comparing versions 😅