[RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[Josh Marshall]

Hello all,

I have come across a use case that I don't find well addressed.  If I
am not wrong (and I very well might be) it would be a feature which
could be of great use.  First sent to a Nix channel, but I think the
same applies here.

```
I want to use git repos with Nix Flakes in them directly as if they
were package dependencies in another Flake. I can't seem to find good
documentation for this. Is it even a thing? The purpose is to improve,
but not impose perfection or slowdown, on developers or researchers as
they Nix-ify their projects. The application is that I have a root
project that needs to be put into a flake, which in turn has a number
of other projects which are not Nix packages but could feasibly add a
Nix Flake. The end goal is to be able to set up and package a project
such that the end result is as easy to replicate for others as
described here:
https://obscuratur-in-arena.readthedocs.io/en/latest/A-good-way-to-set-up-your-python-project-using-Nix-Direnv-and-Poetry.html
```

I regret that I must again be someone who just throws more work
demands into the ether.  The whole "I am finite" thing.

Re: [RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1826 bytes --]

On Mon, Nov 18, 2024 at 01:25:22PM -0500, Josh Marshall wrote:
> Hello all,
> 
> I have come across a use case that I don't find well addressed.  If I
> am not wrong (and I very well might be) it would be a feature which
> could be of great use.  First sent to a Nix channel, but I think the
> same applies here.
> 
> ```
> I want to use git repos with Nix Flakes in them directly as if they
> were package dependencies in another Flake. I can't seem to find good
> documentation for this. Is it even a thing? The purpose is to improve,
> but not impose perfection or slowdown, on developers or researchers as
> they Nix-ify their projects. The application is that I have a root
> project that needs to be put into a flake, which in turn has a number
> of other projects which are not Nix packages but could feasibly add a
> Nix Flake. The end goal is to be able to set up and package a project
> such that the end result is as easy to replicate for others as
> described here:
> https://obscuratur-in-arena.readthedocs.io/en/latest/A-good-way-to-set-up-your-python-project-using-Nix-Direnv-and-Poetry.html
> ```
> 
> I regret that I must again be someone who just throws more work
> demands into the ether.  The whole "I am finite" thing.

There are a number of projects (like shepherd) which are also setup to
function as a channel. And Guix has the ability for channels to depend
on other channels. So it would be possible to create a package in a
channel which depended on a specific branch (or commit) of shepherd as
it appeared upstream.  Is that the type of thing you mean?


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[Josh Marshall]

This is a technical workaround, not direct support.  If such a feature
is to be aimed at general developers and researchers, it can't be so
circuitous. The Invocation from any form of Guix installation needs to
be `guix shell` and just work.
https://guix.gnu.org/manual/en/guix.html#package_002ddevelopment_002dmanifest
 Would need to be extended to take a URI and git reference for
individual dependencies.

On Tue, Nov 19, 2024 at 5:28 AM Efraim Flashner <efraim@flashner.co.il> wrote:
>
> On Mon, Nov 18, 2024 at 01:25:22PM -0500, Josh Marshall wrote:
> > Hello all,
> >
> > I have come across a use case that I don't find well addressed.  If I
> > am not wrong (and I very well might be) it would be a feature which
> > could be of great use.  First sent to a Nix channel, but I think the
> > same applies here.
> >
> > ```
> > I want to use git repos with Nix Flakes in them directly as if they
> > were package dependencies in another Flake. I can't seem to find good
> > documentation for this. Is it even a thing? The purpose is to improve,
> > but not impose perfection or slowdown, on developers or researchers as
> > they Nix-ify their projects. The application is that I have a root
> > project that needs to be put into a flake, which in turn has a number
> > of other projects which are not Nix packages but could feasibly add a
> > Nix Flake. The end goal is to be able to set up and package a project
> > such that the end result is as easy to replicate for others as
> > described here:
> > https://obscuratur-in-arena.readthedocs.io/en/latest/A-good-way-to-set-up-your-python-project-using-Nix-Direnv-and-Poetry.html
> > ```
> >
> > I regret that I must again be someone who just throws more work
> > demands into the ether.  The whole "I am finite" thing.
>
> There are a number of projects (like shepherd) which are also setup to
> function as a channel. And Guix has the ability for channels to depend
> on other channels. So it would be possible to create a package in a
> channel which depended on a specific branch (or commit) of shepherd as
> it appeared upstream.  Is that the type of thing you mean?
>
>
> --
> Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted

Re: [RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[Ludovic Courtès]

Josh Marshall <joshua.r.marshall.1991@gmail.com> skribis:

> This is a technical workaround, not direct support.  If such a feature
> is to be aimed at general developers and researchers, it can't be so
> circuitous. The Invocation from any form of Guix installation needs to
> be `guix shell` and just work.

To make sure we’re talking about the same thing, Efraim is referring to
the conventions documented at:

  https://guix.gnu.org/cookbook/en/html_node/Software-Development.html

Ludo’.

Re: [RFC] [Feature] In project `configuration.scm` allow dependencies to other repos with `configuration.scm`

[Josh Marshall]

Hello Ludo,

I hadn't found that post, but what I'm trying to get across holds.
Barring distribution of binaries since I don't think that is critical
for improving and incremental adoption of Guix to support
infrastructure. A `manifest.scm` could look like the following,
keeping a close eye on the line containing "(remote-repo-dep URI
GIT-REF)".

```
(use-modules (guix)
             (guix build-system gnu)
             ((guix licenses) #:prefix license:)
             (gnu packages autotools)
             (gnu packages ...)
             (gnu remote-repo-dep-mod)
             (gnu packages ...)
             (gnu packages version-control))

(package
  (name "your-project-name")
  (version "???")
  (source #t)
  (build-system ???)
  (native-inputs
   (append (list autoconf
                 ...
                 (remote-repo-dep URI GIT-REF)
                 ...
                 pkg-config)
  (inputs
   (list libffi bash-minimal))
  (propagated-inputs
   (list libunistring libgc))

  (native-search-paths
   (list (search-path-specification
          (variable "GUILE_LOAD_PATH")
          (files '("share/guile/site/3.0")))
         (search-path-specification
          (variable "GUILE_LOAD_COMPILED_PATH")
          (files '("lib/guile/3.0/site-ccache")))))
  (synopsis "...")
  (description "...")
  (home-page "...")
  (license license:lgpl3+))
```

Now, having that single line as a way to include code as opposed to
what can be accomplished similarly with similar expressiveness for the
purpose of the target user group of researchers (think mechanical
engineers, biologists, electrical labs, chemists) and general
developers would lower the barrier of entry to use Guix and improve
the quality of their projects to the point where it just wouldn't make
sense not to.  Now, since there is such a well developed and
documented workflow for a similar process, would I be wrong in
assuming that such functionality could be relatively easy to
implement?

Let me know if gmail is messing up formatting.  I think something is off.

On Wed, Nov 20, 2024 at 4:45 PM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Josh Marshall <joshua.r.marshall.1991@gmail.com> skribis:
>
> > This is a technical workaround, not direct support.  If such a feature
> > is to be aimed at general developers and researchers, it can't be so
> > circuitous. The Invocation from any form of Guix installation needs to
> > be `guix shell` and just work.
>
> To make sure we’re talking about the same thing, Efraim is referring to
> the conventions documented at:
>
>   https://guix.gnu.org/cookbook/en/html_node/Software-Development.html
>
> Ludo’.