Automatic install of neovim plugin dependencies (2)

Posted on June 22, 2022 by Matt
Tags: plugins, neovim

See part 1 for the long motivational speech. The short version is: let’s leverage rockspec and the luarocks infrastructure to specify/autoinstall neovim plugin dependencies.

Part 1 presented a vision of this for the package manager nix. The good news is that I finished adding this to “nixpkgs” (=the collection of packages used by the package manager nix).

I want to detail how I did it with the hope my experience helps other package managers such as packer.nvim.

The first part details the vim/lua nix workflow while the second part presents how packer.nvim could achieve autodiscovery of plugin dependencies and At the end I list a few painpoints that could be improved through community effort. its install. If you don’t know about nix, you should probably just read the 2nd part.

lua plugin packaging in nixpkgs

The nixpkgs manual explains how to update the lua package set. To sum it up, this script looks for packages to update in a .csv file. In theory, we could import the list of packages referenced by luarocks but working with a list of manually selected packages allow to deal with issues more sensibly until our scripts get more robust. The script calls our fork of luarocks which adds a nix command to luarocks to convert a rockspec to a nix derivation:

For instance:

$ luarocks nix plenary.nvim
{ buildLuarocksPackage, luaOlder, luaAtLeast
, fetchgit, lua, luassert
}:
buildLuarocksPackage {
  pname = "plenary.nvim";
  version = "scm-1";
  knownRockspec = (fetchurl {
    url    = "mirror://luarocks/plenary.nvim-scm-1.rockspec";
    sha256 = "08kv1s66zhl9amzy9gx3101854ig992kl1gzzr51sx3szr43bx3x";
  }).outPath;
  src = fetchgit ( removeAttrs (builtins.fromJSON ''{
  "url": "https://github.com/nvim-lua/plenary.nvim",
  "rev": "968a4b9afec0c633bc369662e78f8c5db0eba249",
  "date": "2022-06-12T21:41:00+02:00",
  "path": "/nix/store/z2mzxda6fr97axyjfb8117l7wlb47wwb-plenary.nvim",
  "sha256": "05x9hnz960ljcb2psqycxgdmh99j36y16vbb9l92wjv5szkz37x5",
  "fetchLFS": false,
  "fetchSubmodules": true,
  "deepClone": false,
  "leaveDotGit": false
}
 '') ["date" "path"]) ;

  disabled = with lua; (luaOlder "5.1") || (luaAtLeast "5.4");
  propagatedBuildInputs = [ lua luassert ];

  meta = {
    homepage = "http://github.com/nvim-lua/plenary.nvim";
    description = "lua functions you don't want to write ";
    license.fullName = "MIT/X11";
  };
}

The results of running this command per-package are aggregated in generated-packages.nix. As usual with generated nix packages, some packages need more care than others and need some tweaks to work. These tweaks live in overrides.nix that is stable across the package updates (apart for luv whose buildsystem keep changing).

There is a similar process for vim: vim generated file and its overrides. The vim packaging workflow is simpler but it requires setting plugin dependencies by hand. Metadata retreival is also limited by github’s api. Because nixos is one of a kind, it can also be reassuring to be able to run the package test suite. For these reasons I’ve been pushing to be able to install vim plugins as luarocks packages. When generating the vim packageset, the vim updater now checks whether a homonym of the package lives in the lua package set. If yes, the updater calls build buildNeovimPluginFrom2Nix instead of buildVimPluginFrom2Nix, ie., nix does a flat install (i.e., luarocks installs in a lua folder instead of the nested share/lua/5.X/… folders).

dependency discovery in packer.nvim ?

Packer.nvim, as of this writing, relies on a python library to handle its luarocks interactions. I believe it should require luarocks as an dependency/library instead so that any python dependency can be removed.

Neovim expects a certain folder hierarchy when going through its plugins. By default luarocks installs rockspecs in “share/lua/VERSION” but neovim expects these files in a lua folder:

We can tweak luarocks settings to achieve just that: in a file referenced by the environment variable LUAROCKS_CONFIG, set:

lua_modules_path = "lua"

A second difficulty is for copying the “autoload”, “plugin” folders of many neovim plugins. The copy_directories component of the rockspec was conceived to copy data such as those, luarocks doesn’t allow to configure this path though.

To work around this limitation we can move the files installed via the copy_directories directive to the root of the installation folder in a postprocessing step cp -rfv "$TARGET_DIR/$rocksSubdir/$pname/$version/." "$TARGET_DIR".

And this is it for the packaging aspect !

Future work

Here I have listed a few points that could be worth improving:

  • Specifying the source files can be painful: see plenary rockspec to convince yourself. Turns out everything under the ‘lua’ folder is automatically globbed for the ‘builtin’ module type so in this peculiar case, we could have just written nothing. A luarocks glob function could be useful otherwise.
  • A config entry to specify where to install the copy_directories component of the rockspec would remove our postprocessing oneliner.
  • it is a more generic problem but the lack of support for pkg-config is annoying on systems like nixos
  • we could ask for a “neovim” module type if the rockspec approach really takes off, that