Daniel Duan's Articles About Nix

Daniel Duan's Articles About Nix https://duan.ca/tag/nix/ Flake For Non-Nix Projects The ideal outcome of using Nix or home-manager is to have as many things as possible under their control. Among these things are developer tools for coding. You may need <code>cargo</code>, and <code>rust-analyzer</code> for Rust projects; <code>ipython</code>, and <code>python-lsp-server</code> for Python projects, etc. Of course, the simplest solution is to have them all installed and configured in home-manager or the OS level. But, over time, you'll end up with a ton of stuff. Maybe that won't bother you. Fine, but this approach is at odds with some convention of the Nix community. You see, there's this command called <code>nix-shell</code>, or <code>nix develop</code>, which sets up a environment with certain packages, with the expectation that you only need them sometimes, usually within the context of developing a certain software project. So, arguably, a better alternative to installing everything you possibly need is to keep only the essential tools you always need, regardless of what you are working on. Things like <code>neovim</code>, <code>git</code>, <code>ripgrep</code>, etc. When a project demands things such as <code>rust-analyzer</code>, running <code>nix develop</code> should set it up for you. How do we implement that? In a imaginary world where every single software project is built with a flake.nix, the <code>devShell</code> property should provide everything the software project owner expect you to need. And you just <code>cd</code> into the root of the project, run <code>nix develop</code>, and you are off to the races. Several problems: <ol> <li>running <code>nix develop</code> is repetitive, and therefore, annoying.</li> <li>the owner of a flake.nix may have drastically different taste for the best dev tools for their project (bet!).</li> <li>we don't live in a world built with Nix flakes.</li> </ol> Yikes. To solve <code>1</code>, enter <a href="https://github.com/nix-community/nix-direnv">direnv</a>. It's a piece of software that let you create a environment for a directory, and automatically switch to it when you enter said directory. The "environment" could contain envvars, and, you guessed it: a Nix developer shell. If you have a flake.nix, add <code>use flake</code> in your project directory's <code>.envrc</code> file, direnv will automatically call <code>nix develop</code> the next time you <code>cd</code> in. Neat! To make this work in practice, you'll want to add <code>.envrc</code> and <code>.direnv/</code> to your global git ignore list, as they are personal preferences that probably shouldn't end up in git history. Ok, we are so close to solve problem <code>2</code>, and <code>3</code> now. In short, <code>nix develop</code> may set up a environment, but it may not be the environment that suits you the best. <a href="https://github.com/nix-community/nix-direnv">nix-direnv</a> extends direnv to save us: <pre><code>use flake path/to/flake#target </code></pre> ... with this in <code>.envrc</code>, direnv will set up the Nix environment according to <code>path/to/flake#target</code>. That could point to any location on your hard drive! You can have a flake.nix whose <code>devShell</code> has <code>cargo</code>, and <code>rust-analyzer</code>. You can have another with <code>ipython</code>, and <code>python-lsp-server</code>. Mix and match to your liking to the infinite degree... For now, I've decided to give nix-direnv a try. Alongside my home-manager configuration, I've also included extra flakes for generic Python/Rust projects, and specific projects that may require a mix of tools. A lot of the project I work on don't use flake as their package manager. With this approach, I get to customize my setup for them each, and stay in the comfort of Nix and home-manager. direnv and nix-direnv can be configured together with home-manager. To achieve everything mentioned in this article, including direnv's shell integration (Fish, for me), it's as simple as <pre><code class="language-Nix"># In home manager config... programs.direnv = { enable = true; nix-direnv = { enable = true; enableFlakes = true; }; } </code></pre> To recap, having the snippet above in my home-manager setup, I now can enter any project's root directory and add a <code>.envrc</code> file with the content <code>use flake ~/src/dotfiles/direnvs/python</code>. <code>~/src/dotfiles/direnvs/python</code> contains a <code>flake.nix</code> (and a <code>flake.lock</code>) that has the <code>devShell</code> value I like for all Python projects. When I <code>cd</code> into this project, <code>(nix-)direnv</code> will read from that <code>devShell</code> and set every tool listed under there. The tools are cached in a <code>.direnv</code> directory so when I return here, the setup is basically instantaneous. Since I make <code>git</code> to ignore <code>.envrc</code>, and <code>.direnv</code> no matter where they are, this project I'm working on is unaffected by all this. Sat, 19 Mar 2022 00:37:01 -0700 https://duan.ca/2022/03/19/nix-dirnev/ https://duan.ca/2022/03/19/nix-dirnev/ Flake, Home Manager, and Extra Packages So, you use a standalone home-manager, it's set up with flake, tracking a particular nixpkgs channel. How do you use a package from another channel? This seemingly simple task took me, a Nix noob, quite a bit of research to solve. Here's how I did it. A simple flake.nix for home-manager might look like this: <pre><code class="language-nix">{ inputs = { nixpkgs.url = "github:nixos/nixpkgs/nixos-21.11"; home-manager.url = "github:nix-community/home-manager/release-21.11"; home-manager.inputs.nixpkgs.follows = "nixpkgs"; }; outputs = { self, nixpkgs, home-manager }: { "dan@some-mac" = home-manager.lib.homeManagerConfiguration { username = "dan"; system = "x86_64-darwin"; homeDirectory = "/home/dan"; configuration = { config, pkgs ... }: { home.packages = [ pkgs.hello ]; }; }; }; } </code></pre> A few things of note: <ol> <li>home manager is set to follow <code>nixpkgs</code>, which tracks <code>nixos-21.11</code>.</li> <li><code>pkgs.hello</code> refers to the package in <code>nixos-21.11</code> as well.</li> </ol> To put things in concrete terms, our goal is to put a package from a channel other than <code>nixos-21.11</code> alongside <code>pkgs.hello</code>. The key to my solution is the <code>extraModules</code> in <code>home-manager.lib.homeManagerConfiguration</code>'s argument set. We'll leverage it to modify the environment made available to its sibling, <code>configuration</code>. First, add the new channel as input <code>nixpkgs-unstable</code>: <pre><code class="language-nix">{ inputs = { # ... nixpkgs-unstable.url = "github:nixos/nixpkgs"; }; ... } </code></pre> Then, add a small module to <code>extraModules</code>. In it we make <code>nixpkgs-unstable</code> an argument to <code>_module</code>. <pre><code class="language-nix">{ input = { ... }; outputs = { self, nixpkgs, nixpkgs-unstable, home-manager }: { # Note we also pass in nixpkgs-unstable here "dan@some-mac" = home-manager.lib.homeManagerConfiguration { extraModules = [ ({ pkgs, ... }: rec { _module.args.nixpkgs-unstable = import nixpkgs-unstable { inherit system; }; }) ]; # ... }; }; } </code></pre> ... now that we added <code>args.nixpkgs-unstable</code>, it becomes available in the configuration: <pre><code class="language-nix">{ input = { ... }; outputs = { self, nixpkgs, nixpkgs-unstable, home-manager }: { # Note we also pass in nixpkgs-unstable here "dan@some-mac" = home-manager.lib.homeManagerConfiguration { # ... configuration = { config, pkgs, nixpkgs-unstable, ... }: home.packages = [ pkgs.hello nixpkgs-unstable.python39Packages.python-lsp-server ]; }; }; }; } </code></pre> There, we made <code>python39Packages.python-lsp-server</code> from nixpkgs's master branch appear alongside our standard, default channel. And that's how you add packages from a different channel in a flake setup for standalone home-manager. Tue, 15 Mar 2022 18:53:59 -0700 https://duan.ca/2022/03/15/standalone-home-manager-flake-channels/ https://duan.ca/2022/03/15/standalone-home-manager-flake-channels/ Installing Nix on macOS 11 Big Sur Note: as of Feburary, 2022, the steps written below has been incorporated by the official Nix install script for macOS. This article is, therefore, obsolete. Here are the steps for installing nix on macOS 11 Big Sur. <h3>Preparation</h3> Decide on which disk the nix store is to be installed. By end of this process, you'll have a disk name like <code>disk4</code>. Personally, I like looking it up in the Disk Utility app: <img src="/assets/2020/12/disk-utility.png" alt="Disk Utility" /> You can also use other methods, such us the <code>diskutil list</code> command. Have this disk name ready. <h3>Installation</h3> <h4>1. Create the path <code>/nix</code> the macOS way.</h4> Edit or create the file <code>/etc/synthetic.conf</code>, adding this line: <pre><code>nix </code></pre> Reboot the Mac. <h4>2. Create a APFS volume for the nix store.</h4> <pre><code>sudo diskutil apfs addVolume diskX APFS 'Nix Store' -mountpoint /nix </code></pre> Make sure to replace <code>diskX</code> with the disk you identified in the preparation step. <h4>3. Mount the volume.</h4> Edit or create the file <code>/etc/fstab</code>, adding this line: <pre><code>LABEL=Nix\040Store /nix apfs rw,nobrowse </code></pre> Reboot the Mac. <h4>4. Install Nix the "normal" way</h4> <pre><code>sh <(curl -L https://nixos.org/nix/install) </code></pre> <h4>5. You are done! Try it out. I ran this little <a href="https://web.archive.org/web/20201130222947/https://nix.dev/tutorials/dev-environment.html">example</a> as a test and things appears to work as expected.</h4> <h3>Bonus: using Nix in Fish shell</h3> As a fish user, I'm used to fixing things up myself since software from non-fish users usually pretends fish doesn't exist. Anyways, in the final steps of its installation, Nix sets up a few environment variables in a bash script. I've adapted these specifically for fish on macOS. Add these to your config.fish: <pre><code>set -x NIX_PROFILES "/nix/var/nix/profiles/default $HOME/.nix-profile" set -x NIX_SSL_CERT_FILE "$HOME/.nix-profile/etc/ssl/certs/ca-bundle.crt" set -x NIX_PATH /nix $HOME/.nix-defexpr/channels set -x PATH $HOME/.nix-profile/bin $PATH </code></pre> It shouldn't be to hard to do this on other OSes. The relevant information is in <code>~/.nix-profile/etc/profile.d/nix.sh</code>. Sun, 13 Dec 2020 12:00:52 -0800 https://duan.ca/2020/12/13/nix-on-macos-11-big-sur/ https://duan.ca/2020/12/13/nix-on-macos-11-big-sur/ Naive NixOS Rust Development tl;dr: To work on Rust project with nix-shell, rls and extensions such as <code>rust-analysis</code>, <code>rust-src</code>, without caring too much about specific Rust toolchain version (except for it being <code>stable</code>), use the following <code>shell.nix</code>: <pre><code class="language-nix">let moz_overlay = import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz); nixpkgs = import <nixpkgs> { overlays = [ moz_overlay ]; }; ruststable = (nixpkgs.latest.rustChannels.stable.rust.override { extensions = [ "rust-src" "rust-analysis" ];} ); in with nixpkgs; stdenv.mkDerivation { name = "rust"; buildInputs = [ rustup ruststable ]; } </code></pre> When you have a Nix hammer, everything looks like a Nix expression. Having used NixOS on a real PC for a number of days, this is the impression I get from the world of Nix. Unfortunately, so far, it's been a negative for me. One of the most exciting thing I want to use Nix for is to bootstrap development environment with <code>nix-shell</code>. I imagined it to be similar to using <a href="https://pipenv-fork.readthedocs.io/en/latest/">pipenv</a> with Python, except for everything. Well, I've since learned that it's not true (yet?) for many reasons. Modern programming languages come with their own attempt at reproducibility. Some does it better than others. To make it concrete, I'm talking about things like <a href="https://haskellstack.org">Stack</a> for Haskell or <a href="https://rustup.rs/">rustup</a> for Rust: given the source code, how do I make it build in the way the project intended? What's the correct version of the compiler, runtime, and tools that works best with this revision of the source code? The common solution usually follows this pattern: as author of a project, specify as much as you can, the environment best suited for the current state of the project. As a "builder", use a single program that's capable of updating itself, as well as ensuring that the project builds exactly as specified, including managing the compiler/runtime/tooling versions, etc. This single program's role is very much the same as the Nix system, except the latter is independent of programming languages: <code>rustup</code> installs Rust, so does Nix. That's a bad thing. As a package manager, Nix either have to tightly integrate with each of these other package managers, leveraging their evolving behaviors to give its user the build environment; or, it must replace them. The former is impractical; the latter, well, sucks. Back to reality. This is the experience I want to have with NixOS: Some programs I use daily such as Alacritty, NeoVim, Firfox, etc, are installed globally and readily available. They are part of my <code>/etc/nixos/configuration.nix</code>. So far so good. Now, I regularly program in a few languages. For each of the project, I'd like to have a <code>shell.nix</code> that brings in its compilers, libraries, LSP servers, etc. This is what <code>nix-shell</code> is supposed to give me! This is known as the "per project" setup. Let's see: with Rust, that means <code>rustc</code> (compiler), <code>cargo</code> (package manager), <code>rls</code>, <code>rust-src</code> and <code>rust-analysis</code> (LSP). In macOS, I'd install all of these globally with <code>rustup</code>. In NixOS...well, I can ask for <code>rustup</code> for my project: <pre><code class="language-nix">with import <nixpkgs> {}; stdenv.mkDerivation { name = "rust"; nativeBuildInputs = [ rustup ]; } </code></pre> ...which gives me <code>rustup</code> and nothing else. That's right, you don't even get a <code>rustc</code> after running <code>nix-shell</code>. But <code>rustup</code> can get you everything else, all I need to do is ask. Hmm, do I need to run a series of set-up commands with <code>rustup</code> every time I enter the environment? No? I just need to run it the first time? Until the cached tools get deleted by some garbage collection mechanism? That seems unsatisfying, doesn't it? Instead of <code>rustup</code>, I could also ask Nix to use the <code>rustc</code>/<code>cargo</code>/<code>rls</code> it knows about directly. This is marginally better. Except I still need <code>rust-src</code> and <code>rust-analysis</code> for my needs. As far as I can tell, these <a href="https://github.com/rust-lang/rls">RLS</a> components are out of Nix's control (as of today). Everywhere on the internet I looked, for every problem that Nix-the-package-manager doesn't work out-of-the-box, there's someone responding along the line of "you can write some Nix expression yourself". In other words, Nix-the-language is powerful enough to solve it, probably. In the case of Rust, luckily, Mozilla wrote enough Nix expressions for us and provides them via an <a href="https://github.com/mozilla/nixpkgs-mozilla">overlay</a>. These expressions are rich enough to meet my needs. As you can see in the tl;dr at the top, when entering the development environment, nix-shell would: download the overlay's source code (from the internet, or local cache), load the expression it includes, mix in my customization, and execute it. That marks the end of my search. I like the final solution because it's mostly "vanilla" Nix and doesn't require me to mess with a bunch of other tools. For solutions that do, read <a href="https://xeiaso.net/blog/how-i-start-nix-2020-03-08/">this</a>. <hr /> At end of the day, my needs are pretty basic: consistency from rustup and convenience from nix-shell. I didn't need to pin the compiler to a specific Rust release, or checksum the final build output. I'm very new to both technologies so there may be a follow-up post sometime in the future. Thu, 07 May 2020 11:04:58 -0700 https://duan.ca/2020/05/07/nix-rust-development/ https://duan.ca/2020/05/07/nix-rust-development/