The biggest problem of all is that derivers are not unique! A separate &quot;build trace&quot; map will solve this.

Presumably this would support a big improvement to both SBOM generation as well as various UX features and workflow improvements.

I think Eelco has in mind a separate thing that would still be a store object field. But IMO we should not do that since derives are unique, and we should instead use the &quot;build trace&quot; instead, which properly handles that.As Martin Schwaighofer has discussed, it is fine and in fact good for build traces entries to have arbitrary meta data, so the &quot;claims&quot; being cryptographically signed are more precise. (This is good for auditing, and if something looks suspicious, having full accountability.)So on that grounds, if eelco would like to include some &quot;this came from this flake&quot; information as informal metadata. (formally the key must still the resolved derivation.) That is fine with me.---As I linked in my other reply, see my fast-growing <a href="https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;14408" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;14408</a> docs PR where I try to formally nail all this stuff down for the first time.

is that the &#x27;build-trace&#x27; feature I saw John write about ?
(I want to explore that more)

The deriver field in Nix has always been a misfeature. It was intended to provide traceability back to the Nix expression used to create the derivation, but it doesn&#x27;t actually do that (since that wasn&#x27;t really possible in the pre-flakes world, without hermetic evaluation). So instead it just causes a lot of confusion when the deriver recorded in the binary cache doesn&#x27;t match the local evaluation result, due to fixed-output derivations changing.In the future, Nix will hopefully gain proper provenance tracking that will tell you exactly where a store path came from: <a href="https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;11749" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;11749</a>

+1 to Farid, great write-up! What you’re seeing is the long-standing “deriver” mismatch: fixed-output derivations can change their .drv without changing the output path. Eelco is calling it out as well in the comment below. I believe the idea behind the path forward is there but happy to hear more!Also. Check out Farid&#x27;s other posts.

The core store layer is quite small, and I am trying to thoroughly document it, with all 3 of:- a more &quot;academic&quot; spec of what it does- nuts-and-bolts JSON schema for many data types- JSON golden tests instead of C++ literals in the unit tests as often as possible.I hope this will make additional store layer easy to churn out.(The &quot;hash derivation modulo&quot; that is so fiddly described in this blog post can be dropped in a world where we no longer have input addressing, and just have content-addressing. Or, in a world where we have a new, simpler type of input-addressing instead.)

I really wish Guix worked on macOS. Nix-Darwin and home-manager have been game changers -- sharing much config and tooling between my Mac, arch, and nixos machines has been a blessing.

Isn&#x27;t there a way to transpile the scripts from Nix to Guix?

AFAIK Guix uses parts of Nix as a backend.

Well, there&#x27;s Guix as an alternative if you want a similar concept but different implementation philosophy. For me the major disadvantage of Guix is lack of package availability compared to Nix.

Pulumi may be what you&#x27;re looking for. Same concept as Terraform, and many of its provider libraries are just wrappers around Terraform provider libraries, but you can use a variety of common programming languages to declare your desired state, rather than HCL.

I feel the same about HCL in Terraform. The tool is perfect, the language is bollocks.

It has been rewritten a few times already. The &quot;fixed output hash&quot; is a dirty optimisation hack borne out of real-world needs and not a research idea.

Eh. This can be applied to so many technologies that run the world..

&gt; The road to Nix enlightenment is no joke and full of dragons.Nix was a great research project. Now is the time to rewrite it from the ground up.

Yes, a hope of mine is that we can stop using &quot;hash derivation modulo&quot; entirely.I&#x27;ve recently started some fancy formal spec-level documentation here <a href="https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;14408" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;NixOS&#x2F;nix&#x2F;pull&#x2F;14408</a> The &quot;resolution&quot; equivalence class is both simpler and better than the &quot;hash derivation modulo ...&quot; one.(The fact that it is a mouthful to say what the derivations are modulo kinda gives the game away! I put &quot;hash quotient derivation&quot; in the docs to side-step the issue.)

To be clear, there is no bug here: derivers are simply not uniquely determined in the presence of fixed-output derivations, which is by design. That&#x27;s even more true with CA derivations.CA derivations also introduce the opposite situation, namely that the same derivation can produce different output paths for different users (if the build is not bitwise reproducible).

That makes sense, thanks for clarifying. Great writeup.

ca-derivations from what i understand, fixed-output derivations but more general.The point of the article to me (author) was that i found it odd that Nix replaces the derivations when calculating the output path but not the derivation path.
(talking about &quot;paths&quot; in Nix is so hard!)

If I understand this correctly, upcoming Ca-derivations will fix this by making these situations expected, properly-handled cases rather than a weird bug? <a href="https:&#x2F;&#x2F;nixos.wiki&#x2F;wiki&#x2F;Ca-derivations" rel="nofollow">https:&#x2F;&#x2F;nixos.wiki&#x2F;wiki&#x2F;Ca-derivations</a>

As a mere mortal I find none of this surprising, mostly because I never understood any of it in the first place ... :)

it&#x27;s primary for every human involved, also, the way you check whether it&#x27;s changed is by automatically comparing that full hash, not its starting symbols, so you don&#x27;t care where in the full string it&#x27;s positioned&gt; The semantic is &quot;what did this configuration generate&quot;, not &quot;what&#x27;s this package&#x27;s version&quot;.Then why have the name&#x2F;version at all like in those nameless cache dirs?

The package name is &quot;secondary&quot; information in this context. The hash is the primary one because it&#x27;s stable unless the input changes.The semantic is &quot;what did this configuration generate&quot;, not &quot;what&#x27;s this package&#x27;s version&quot;.

If I had my way1. store paths would have no names at all2. listing the contents of the store directory would not be allowed3. store paths have more bits of informationThen store paths are halfway decent (but non-revocable) capabilities.

I grep through the store pretty regularly looking for names. The tone of the original comment is annoying but the suggestion is imo quite a good one.

In nix packages (derivations) are so lightweight that your store has tens of thousands of them, many with the same name, or with no meaningful name at all. On the rare occasions that you need to look in the store for a package you’re much more likely to be looking for a particular hash than a particular name. That, and having the hash as a prefix looks nicer in tabular output.

&gt; Pros&#x2F;cons under different search &#x2F; inspection conditions.But what&#x27;s the pro? The tail alignment is worse than the head alignment since you read head to tail, not the other way aground

I can&#x27;t find the video of the talk where either Eelco Dolstra (nix) or Todd Gamblin (spack) talks about this, but IIRC it&#x27;s a design decision; you generally don&#x27;t go spelunking in the nix store, but if you do, and you ls &#x2F;nix&#x2F;store, you&#x27;ll see a huge list of packages, and due to the hash being a constant length, you can visually separate the tails of the package names like<pre><code> 0009flr197p89fz2vg032g556014z7v1-libass-0.17.3.drv
 000ghm78048kh2prsfzkf93xm3803m0r-default.md
 001f6fysrshkq7gaki4lv8qkl38vjr6a-python-runtime-deps-check-hook.sh.drv
 001gp43bjqzx60cg345n2slzg7131za8-nix-nss-open-files.patch
 001im7qm8achbyh0ywil6hif6rqf284z-bootstrap-stage0-binutils-wrapper-boot.drv
 001pc0cpvpqix4hy9z296qnp0yj00f4n-zbmath-review-template.r59693.tar.xz.drv
</code></pre>
Spack, another deterministic builder &#x2F; package manager, IIRC uses the reversed order so the hash is at the tail. Pros&#x2F;cons under different search &#x2F; inspection conditions.

That contradicts the simple fact that the name includes &quot;ruby&quot; and isn&#x27;t just a hash

It&#x27;s done that way on purpose.
Precisely so you don&#x27;t try to use the paths semantically. The names literally mean nothing in this context.

You could just use the last hyphen. It&#x27;s not as simple since you need to scan the whole string, but it certainly doesn&#x27;t seem like a challenge to me.

The reason it&#x27;s like this is because the only way to reliably grab it is to cut the string at the first hyphen - then the rest can be almost free text.It you do it the other way it&#x27;s harder. You can try this with nix commands &#x2F;nix&#x2F;store&#x2F;&lt;hash&gt;-x is a valid way to refer to something in the store most of the time.

It really doesn&#x27;t matter. As a normal user, you don&#x27;t use `drv` files directly, and everything you configure yourself will use attribute paths in nixpkgs. E.g. `pkgs.ruby` or `pkgs.ruby_3_3`.

&gt; nix&#x2F;store&#x2F;24v9wpp393ib1gllip7ic13aycbi704g-ruby-3.3.9.drvA different type of madness, but are ugly names so common, why not start with ruby-3.3.9 so any list of files is semantically sorted&#x2F;readable?

Nix Derivation Madness