A few questions about channels

A few questions about channels

[ison]

Several questions about channels have been on my mind which I haven't been able to find any documentation on. So I figured I would ask them all in one place since they seem like fairly standard questions that could help others setting up channels in the future as well.

1) Is it possible to make a channel from a local git repository? (That is to say, a directory on the local machine initialized as a git repository)

2) Is it possible to use private remote repositories as channels? The issue here seems to be one of authentication. I'm wonder if there is (or should be) an "authentication" field one would add to the channel definition in the channels.scm file.

3) How can a channel provide patch files which can be used with "search-patches"? I attempted to look through the guix code in gnu/packages.scm and from what I can tell it looks like search-patches only looks in paths included from "%load-path". There is also a comment saying that it picks up the paths from $GUIX_PACKAGE_PATH, but I thought that variable was being deprecated in favor of channels. If so, shouldn't search-patches also look inside the channels repository for patch files?

Re: A few questions about channels

[david.larsson]

On Wed, 24 Apr 2019, ison wrote:

> Several questions about channels have been on my mind which I haven't been able to find any documentation on. So I figured I would ask them all in one place since they seem like fairly standard questions that could help others setting up channels in the future as well.
>
> 1) Is it possible to make a channel from a local git repository? (That is to say, a directory on the local machine initialized as a git repository)
>

Yes, for example, cat ~/.config/guix/channels.scm

(cons (channel
        (name 'my-personal-packages)
        (url "file:///home/myuser/src/my-guix-packages"))
       %default-channels)

where the my-guix-packages directory is a git repo which has committed packageModN.scm files.

> 2) Is it possible to use private remote repositories as channels? The issue here seems to be one of authentication. I'm wonder if there is (or should be) an "authentication" field one would add to the channel definition in the channels.scm file.
>
I agree it would be nice to have. The git-fetch method is defined here: https://github.com/guix-mirror/guix/blob/012bf5c4c03e30633f137960bd0677e204c638a7/guix/build/git.scm and I think it can be modified without too much effort to include an argument for the -key flag to the git command. I am not sure though whether git-fetch will have access to the home user's directory in case you provide let's say ~/.ssh/id_rsa as argument. Another possibility would be to check the GIT_SSH_COMMAND env variable. I suppose you also need to make sure ssh is available for git-fetch.

--
Vänliga hälsningar / Best Regards
David L

Re: A few questions about channels

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 1040 bytes --]


[…]

>> 2) Is it possible to use private remote repositories as channels?
>> The issue here seems to be one of authentication. I'm wonder if
>> there is (or should be) an "authentication" field one would add to
>> the channel definition in the channels.scm file.
>>
> I agree it would be nice to have. The git-fetch method is defined
> here:
> https://github.com/guix-mirror/guix/blob/012bf5c4c03e30633f137960bd0677e204c638a7/guix/build/git.scm
> and I think it can be modified without too much effort to include an
> argument for the -key flag to the git command. I am not sure though
> whether git-fetch will have access to the home user's directory in
> case you provide let's say ~/.ssh/id_rsa as argument. Another
> possibility would be to check the GIT_SSH_COMMAND env variable. I
> suppose you also need to make sure ssh is available for git-fetch.

Another way to use a private repo is oauth authentication.

(channel … (url "https://gitlab-ci-token:xxxxxxxxxxxxx-xxxxxx@gitlab.com/author/repository.git")

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: A few questions about channels

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3424 bytes --]

ison <ison@airmail.cc> writes:

> 3) How can a channel provide patch files which can be used with
> "search-patches"? I attempted to look through the guix code in
> gnu/packages.scm and from what I can tell it looks like search-patches
> only looks in paths included from "%load-path". There is also a
> comment saying that it picks up the paths from $GUIX_PACKAGE_PATH, but
> I thought that variable was being deprecated in favor of channels. If
> so, shouldn't search-patches also look inside the channels repository
> for patch files?

This should work just like it would for GUIX_PACKAGE_PATH.  Are you
saying that the patch files you have created in your channel are not
being found?

david.larsson@selfhosted.xyz writes:

> On Wed, 24 Apr 2019, ison wrote:
>
>> Several questions about channels have been on my mind which I
>> haven't been able to find any documentation on. So I figured I would
>> ask them all in one place since they seem like fairly standard
>> questions that could help others setting up channels in the future
>> as well.
>>
>> 1) Is it possible to make a channel from a local git repository? (That is to say, a directory on the local machine initialized as a git repository)
>>
>
> Yes, for example, cat ~/.config/guix/channels.scm
>
> (cons (channel
>        (name 'my-personal-packages)
>        (url "file:///home/myuser/src/my-guix-packages"))
>       %default-channels)

Actually, you can also just write "/home/myuser/src/my-guix-packages".
The "file://" part is fine to add, but it is not necessary.  I think
(but am not sure) that you can use any protocol understood by Git here.

>> 2) Is it possible to use private remote repositories as channels?
>> The issue here seems to be one of authentication. I'm wonder if
>> there is (or should be) an "authentication" field one would add to
>> the channel definition in the channels.scm file.
>>
> I agree it would be nice to have. The git-fetch method is defined
> here:
> https://github.com/guix-mirror/guix/blob/012bf5c4c03e30633f137960bd0677e204c638a7/guix/build/git.scm
> and I think it can be modified without too much effort to include an
> argument for the -key flag to the git command. I am not sure though
> whether git-fetch will have access to the home user's directory in
> case you provide let's say ~/.ssh/id_rsa as argument. Another
> possibility would be to check the GIT_SSH_COMMAND env variable. I
> suppose you also need to make sure ssh is available for git-fetch.

It may also be possible to jump through hoops similar to how I did it
for package origins in this not-yet-merged patch:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=31285

I have been thinking about this problem for a while, but I have not yet
tried to solve it for channels.  The patch above only works for
individual packages, and it is quite a hack, so I am hesitant to merge
it into master because I think maybe there is a better way.

I think the better way might be to give the guix-daemon an SSH key
specifically for the purpose of accessing Git repositories over
authenticated SSH.  In the same way that the Guix daemon has its own
built in downloader for downloading files, we might be able to give it a
built in "git fetcher" to fetch Git repositories over SSH, using the
provided SSH key.  But I haven't looked into it more at this time.  Help
is welcome!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: A few questions about channels

[ison]

> This should work just like it would for GUIX_PACKAGE_PATH.  Are you
> saying that the patch files you have created in your channel are not
> being found?

Correct, they're not being found. But I do not have the variable
$GUIX_PACKAGE_PATH set, because I thought it was being deprecated. Do I need it
to make patches work? Everything else seems to work without it except patches.

So for example, if I create a repository with directory structure:
* gnu/packages for .scm package files
* gnu/packages/patches for .patch files
then the packages are correctly added and I can install them like any other
package. But if I add a .patch file into the patches directory and reference it
with (search-patches "filename.patch") it can't find it.

I have also tried various other combinations of the patches directory, including
placing patches in with the package files, placing everything at the root of the
repo, etc.
Perhaps I still just haven't found the right directory structure?

Also in case it matters, I think the patch format is correct as well, because it
succeeds if I don't use search-patches and instead just supply the full path to
the patch file.

> Actually, you can also just write "/home/myuser/src/my-guix-packages".
> The "file://" part is fine to add, but it is not necessary.  I think
> (but am not sure) that you can use any protocol understood by Git here.

Oh, I feel silly for not having tried this. Thanks.

As for the authentication issue, as long as I can do a local repository that's
all I need personally. So I have no strong opinions on how best to handle it.

Thanks for the replies

Re: A few questions about channels

[Ricardo Wurmus]

ison <ison@airmail.cc> writes:

>> This should work just like it would for GUIX_PACKAGE_PATH.  Are you
>> saying that the patch files you have created in your channel are not
>> being found?
>
> Correct, they're not being found. But I do not have the variable
> $GUIX_PACKAGE_PATH set, because I thought it was being deprecated. Do I need it
> to make patches work? Everything else seems to work without it except patches.

Here’s a channel I’ve been maintaining long before channels were a
thing: https://github.com/BIMSBbioinfo/guix-bimsb

It includes patch files in the root directory and they are found just
fine.

--
Ricardo

Re: A few questions about channels

[ison]

Ok, I set it up the same as that repo and now it finds the patches.
I thought I tried this already but it's possible I forgot to commit or to change
the module definition.

Thanks for the help.