a GUIX_PACKAGE_PATH / modules puzzle

a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Hi,

I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.

I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
There's an issue that I can't seem to get rid of, I'll try my best to describe it now:

I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).

Repository 1: /home/user/src/infotropique/ports
Repository 2: /home/user/src/infotropique/pkgs

The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.

I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
the reository. Both repos worked this way.

After the transition to the new layout, ports works.
pkgs however is throwing errors at me for days. I've tried moving all but one module
out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt

I can not share the actual repo at this moment on this mailinglist, but maybe the
repl message is something that someone could give me a hint, even if it's just a
tiny bit of a "read this manpage" or something like that would help. I've tried
debugging this for days and any help's welcome.

Thanks,
N.

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2107 bytes --]

Nils Gillmann <gillmann@infotropique.org> writes:

> Hi,
>
> I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
>
> I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
> There's an issue that I can't seem to get rid of, I'll try my best to describe it now:
>
> I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
> the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
>
> Repository 1: /home/user/src/infotropique/ports
> Repository 2: /home/user/src/infotropique/pkgs
>
> The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
>
> I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
> the reository. Both repos worked this way.
>
> After the transition to the new layout, ports works.
> pkgs however is throwing errors at me for days. I've tried moving all but one module
> out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
> The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
>
> I can not share the actual repo at this moment on this mailinglist, but maybe the
> repl message is something that someone could give me a hint, even if it's just a
> tiny bit of a "read this manpage" or something like that would help. I've tried
> debugging this for days and any help's welcome.

Could this error be coming from the scheme-files procedure in (guix
discovery)?  It looks like that's the only place where fold-right is
invoked in that module, and it seems to match up with the stack trace,
unless I'm misreading it (which is possible).  You might find it useful
to insert some pk statements into that code to try to figure out what is
being evaluated as "unspecified".  My guess, which could be wrong, is
that perhaps scandir* is returning an unspecified value unintentionally
at some point?  Someone with more experience debugging Guile code can
probably provide better advice.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Chris Marusich transcribed 3.0K bytes:
> Nils Gillmann <gillmann@infotropique.org> writes:
> 
> > Hi,
> >
> > I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
> >
> > I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
> > There's an issue that I can't seem to get rid of, I'll try my best to describe it now:
> >
> > I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
> > the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
> >
> > Repository 1: /home/user/src/infotropique/ports
> > Repository 2: /home/user/src/infotropique/pkgs
> >
> > The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
> >
> > I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
> > the reository. Both repos worked this way.
> >
> > After the transition to the new layout, ports works.
> > pkgs however is throwing errors at me for days. I've tried moving all but one module
> > out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
> > The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
> >
> > I can not share the actual repo at this moment on this mailinglist, but maybe the
> > repl message is something that someone could give me a hint, even if it's just a
> > tiny bit of a "read this manpage" or something like that would help. I've tried
> > debugging this for days and any help's welcome.
> 
> Could this error be coming from the scheme-files procedure in (guix
> discovery)?  It looks like that's the only place where fold-right is
> invoked in that module, and it seems to match up with the stack trace,
> unless I'm misreading it (which is possible).  You might find it useful
> to insert some pk statements into that code to try to figure out what is
> being evaluated as "unspecified".  My guess, which could be wrong, is
> that perhaps scandir* is returning an unspecified value unintentionally
> at some point?  Someone with more experience debugging Guile code can
> probably provide better advice.
> 
> -- 
> Chris

What I found is that: (pkgs sys-kernel) providing common logics for
(pkgs sys-kernel linux linux) [and planned, (pkgs sys-kernel linux-libre)]
and (ports sysutils firmware some-firmware) as well as (pkgs sys-kernel linux-headers)
created a problem. This, moving the definitions from the old linux module I had out to
a common module, was one of the few breaking changes.
I wasn't able to resolve this as amirouche told me on irc. I want to remove some last
bits of these imports tonight. I was able to update my system with the modules in another
place and imports corrected.
What's weird to me is that even after cleaning the ccache of guile in the respective
locations a package that isn't involved in the new change, like for example numactl,
as the only package definition in this place throws errors. In another location it
works. I didn't come up with the name "pkgs" for the root of the repository yesterday,
I have used this for a couple of months in a working state.

Anyways, in which section of the guile Manual would I find "pk"? I've done surprisingly
little debugging with Guile itself so far.

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

Nils Gillmann <ng0@n0.is> writes:

> Anyways, in which section of the guile Manual would I find "pk"? I've
> done surprisingly little debugging with Guile itself so far.

Unfortunately, pk isn't (yet?) documented.  It apparently stands for
"peek stuff".  It prints and returns the value passed to it.  It's a
useful little shortcut for printing values.  Here are some simple
examples of how to use it:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (pk "hello")

;;; ("hello")
$1 = "hello"
scheme@(guile-user)> (pk (list 1 2 3))

;;; ((1 2 3))
$2 = (1 2 3)
scheme@(guile-user)> (define (1+ n) (+ 1 (pk n)))
scheme@(guile-user)> (1+ 2)

;;; (2)
$3 = 3
scheme@(guile-user)> pk
$4 = #<procedure peek stuff>
--8<---------------cut here---------------end--------------->8---

Note that pk does not print out all values when a procedure returns
multiple values, but it is still quite handy:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (pk (values 1 2))

;;; (1)
$5 = 1
--8<---------------cut here---------------end--------------->8---

I hope that helps!

By the way, if you feel like taking a trip down the rabbit hole, here is
the answer to the somewhat unrelated question of "why did pk only print
the first value?"

https://lists.gnu.org/archive/html/guile-user/2017-06/msg00043.html

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

[-- Attachment #1: Type: text/plain, Size: 1700 bytes --]

Chris Marusich transcribed 2.3K bytes:
> Nils Gillmann <ng0@n0.is> writes:
> 
> > Anyways, in which section of the guile Manual would I find "pk"? I've
> > done surprisingly little debugging with Guile itself so far.
> 
> Unfortunately, pk isn't (yet?) documented.  It apparently stands for
> "peek stuff".  It prints and returns the value passed to it.  It's a
> useful little shortcut for printing values.

Okay, thanks.

> Here are some simple
> examples of how to use it:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (pk "hello")
> 
> ;;; ("hello")
> $1 = "hello"
> scheme@(guile-user)> (pk (list 1 2 3))
> 
> ;;; ((1 2 3))
> $2 = (1 2 3)
> scheme@(guile-user)> (define (1+ n) (+ 1 (pk n)))
> scheme@(guile-user)> (1+ 2)
> 
> ;;; (2)
> $3 = 3
> scheme@(guile-user)> pk
> $4 = #<procedure peek stuff>
> --8<---------------cut here---------------end--------------->8---
> 
> Note that pk does not print out all values when a procedure returns
> multiple values, but it is still quite handy:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (pk (values 1 2))
> 
> ;;; (1)
> $5 = 1
> --8<---------------cut here---------------end--------------->8---
> 
> I hope that helps!

Seems like in the case of using (system) it doesn't do anything, I'll
try guile interaction with Guix tonight.

> By the way, if you feel like taking a trip down the rabbit hole, here is
> the answer to the somewhat unrelated question of "why did pk only print
> the first value?"
> 
> https://lists.gnu.org/archive/html/guile-user/2017-06/msg00043.html
> 
> -- 
> Chris



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

[-- Attachment #1: Type: text/plain, Size: 1118 bytes --]

Okay, here's what I learned:

* except for source code, the modules itself in Guile aren't very
* detailed documented. The documentation is good, but...  ... it's
* nowhere mentioned that you can have a module (foo bar baz) and
* possibly also (foo bar baz kim) but (foo baz bar bar) will lead to
* the error I described.

So until I read into the core of Guile, I have 2 questions:

1. Is there a module name maximum length? -> So far I have encountered
very short module names in the wild, 3 the longest.

2. Is this really a module length problem? -> I have a functional set
of packages elsewhere, my non-core packages, named like (ports editors
foo foo) and so forth and they cause no problems. It's just weird to
me that the modules in ports and elsewhere work but in pkgs I can't
dissect the exact problem so far. I'm okay with any namespace, so
I'll simply remove the last element of the module names.

I'd like to improve documentations or other relevant places if
what I experienced is some kind of corner case in module naming,
be it in Guix or Guile... or just to note what to avoid with
regards to modules.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Nils Gillmann transcribed 2.1K bytes:
> Okay, here's what I learned:
> 
> * except for source code, the modules itself in Guile aren't very
> * detailed documented. The documentation is good, but...  ... it's
> * nowhere mentioned that you can have a module (foo bar baz) and
> * possibly also (foo bar baz kim) but (foo baz bar bar) will lead to
> * the error I described.
> 
> So until I read into the core of Guile, I have 2 questions:
> 
> 1. Is there a module name maximum length? -> So far I have encountered
> very short module names in the wild, 3 the longest.
> 
> 2. Is this really a module length problem? -> I have a functional set
> of packages elsewhere, my non-core packages, named like (ports editors
> foo foo) and so forth and they cause no problems. It's just weird to
> me that the modules in ports and elsewhere work but in pkgs I can't
> dissect the exact problem so far. I'm okay with any namespace, so
> I'll simply remove the last element of the module names.
> 
> I'd like to improve documentations or other relevant places if
> what I experienced is some kind of corner case in module naming,
> be it in Guix or Guile... or just to note what to avoid with
> regards to modules.


Actually the problem is not really module names I just found out.

But I'm still curious about any possible guile module naming related limits
and hints from people working longer with Guile.

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Nils Gillmann transcribed 1.4K bytes:
> Nils Gillmann transcribed 2.1K bytes:
> > Okay, here's what I learned:
> > 
> > * except for source code, the modules itself in Guile aren't very
> > * detailed documented. The documentation is good, but...  ... it's
> > * nowhere mentioned that you can have a module (foo bar baz) and
> > * possibly also (foo bar baz kim) but (foo baz bar bar) will lead to
> > * the error I described.
> > 
> > So until I read into the core of Guile, I have 2 questions:
> > 
> > 1. Is there a module name maximum length? -> So far I have encountered
> > very short module names in the wild, 3 the longest.
> > 
> > 2. Is this really a module length problem? -> I have a functional set
> > of packages elsewhere, my non-core packages, named like (ports editors
> > foo foo) and so forth and they cause no problems. It's just weird to
> > me that the modules in ports and elsewhere work but in pkgs I can't
> > dissect the exact problem so far. I'm okay with any namespace, so
> > I'll simply remove the last element of the module names.
> > 
> > I'd like to improve documentations or other relevant places if
> > what I experienced is some kind of corner case in module naming,
> > be it in Guix or Guile... or just to note what to avoid with
> > regards to modules.
> 
> 
> Actually the problem is not really module names I just found out.

Here's what's happening now:

I took my perl module which is in "pkgs/lang-perl/perl.scm", which contains our basic perl packages,
and moved ( + renamed ) them into (core lang-perl perl) in a new directory.

If I run GUIX_PACKAGE_PATH="/home/user/src/core" package -s mc I get results for query "mc".
When I run GUIX_PACKAGE_PATH="/home/user/src/core" package --show=perl I get the same message
as initially pointed out in the opening post.

> But I'm still curious about any possible guile module naming related limits
> and hints from people working longer with Guile.
> 

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Nils Gillmann transcribed 1.9K bytes:
> Nils Gillmann transcribed 1.4K bytes:
> > Nils Gillmann transcribed 2.1K bytes:
> > > Okay, here's what I learned:
> > > 
> > > * except for source code, the modules itself in Guile aren't very
> > > * detailed documented. The documentation is good, but...  ... it's
> > > * nowhere mentioned that you can have a module (foo bar baz) and
> > > * possibly also (foo bar baz kim) but (foo baz bar bar) will lead to
> > > * the error I described.
> > > 
> > > So until I read into the core of Guile, I have 2 questions:
> > > 
> > > 1. Is there a module name maximum length? -> So far I have encountered
> > > very short module names in the wild, 3 the longest.
> > > 
> > > 2. Is this really a module length problem? -> I have a functional set
> > > of packages elsewhere, my non-core packages, named like (ports editors
> > > foo foo) and so forth and they cause no problems. It's just weird to
> > > me that the modules in ports and elsewhere work but in pkgs I can't
> > > dissect the exact problem so far. I'm okay with any namespace, so
> > > I'll simply remove the last element of the module names.
> > > 
> > > I'd like to improve documentations or other relevant places if
> > > what I experienced is some kind of corner case in module naming,
> > > be it in Guix or Guile... or just to note what to avoid with
> > > regards to modules.
> > 
> > 
> > Actually the problem is not really module names I just found out.
> 
> Here's what's happening now:
> 
> I took my perl module which is in "pkgs/lang-perl/perl.scm", which contains our basic perl packages,
> and moved ( + renamed ) them into (core lang-perl perl) in a new directory.
> 
> If I run GUIX_PACKAGE_PATH="/home/user/src/core" package -s mc I get results for query "mc".
> When I run GUIX_PACKAGE_PATH="/home/user/src/core" package --show=perl I get the same message

Sorry, I was wrong (--show=perl works as expected).
However having the same in the old structure with just this module (and the old
name) does throw the old error. ccach of Guile is cleaned.

> as initially pointed out in the opening post.
> 
> > But I'm still curious about any possible guile module naming related limits
> > and hints from people working longer with Guile.
> > 
> 

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

[-- Attachment #1: Type: text/plain, Size: 2316 bytes --]

Chris Marusich transcribed 3.0K bytes:
> Nils Gillmann <gillmann@infotropique.org> writes:
> 
> > Hi,
> >
> > I'm not sure if guile-users / -dev liste is more appropriate. If it is, let me know.
> >
> > I'm currently still using GUIX_PACKAGE_PATH until I got my layout all set up.
> > There's an issue that I can't seem to get rid of, I'll try my best to describe it now:
> >
> > I have package definitions in 2 repositories (3 to be precise, +1 WIP repo). Both follow
> > the exact same layout that can still be understood by Guix and GPP (GUIX_PACKAGE_PATH).
> >
> > Repository 1: /home/user/src/infotropique/ports
> > Repository 2: /home/user/src/infotropique/pkgs
> >
> > The modules are in similar named subfolders, following the layout $REPONAME $CATEGORY $NAME NAME.
> >
> > I only switched to this recently, previously the "CATEGORY NAME NAME" was located in the root of
> > the reository. Both repos worked this way.
> >
> > After the transition to the new layout, ports works.
> > pkgs however is throwing errors at me for days. I've tried moving all but one module
> > out of the repo, same error. It was a simple easy, no repo-internal dependencies, module.
> > The error is always similar to https://ftp.n0.is/pub/pkgs-error.txt
> >
> > I can not share the actual repo at this moment on this mailinglist, but maybe the
> > repl message is something that someone could give me a hint, even if it's just a
> > tiny bit of a "read this manpage" or something like that would help. I've tried
> > debugging this for days and any help's welcome.
> 
> Could this error be coming from the scheme-files procedure in (guix
> discovery)?  It looks like that's the only place where fold-right is
> invoked in that module, and it seems to match up with the stack trace,

Oh, sorry, I skipped this reply. I'll read that module and see if it helps.

> unless I'm misreading it (which is possible).  You might find it useful
> to insert some pk statements into that code to try to figure out what is
> being evaluated as "unspecified".  My guess, which could be wrong, is
> that perhaps scandir* is returning an unspecified value unintentionally
> at some point?  Someone with more experience debugging Guile code can
> probably provide better advice.
> 
> -- 
> Chris



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Ricardo Wurmus]

Nils Gillmann <ng0@n0.is> writes:

> Here's what's happening now:
>
> I took my perl module which is in "pkgs/lang-perl/perl.scm", which contains our basic perl packages,
> and moved ( + renamed ) them into (core lang-perl perl) in a new directory.
>
> If I run GUIX_PACKAGE_PATH="/home/user/src/core" package -s mc I get results for query "mc".
> When I run GUIX_PACKAGE_PATH="/home/user/src/core" package --show=perl I get the same message
> as initially pointed out in the opening post.

I don’t understand what the problem is.  Your description isn’t very
clear.  If you cannot post a minimal example it is virtually impossible
for us to help.

I’d also like to suggest asking questions on guile-user when you have
questions about Guile.

If you have a file “core/lang-perl/perl.scm” and the module is called
“(core lang-perl perl)” then you should not have “core” itself on
GUIX_PACKAGE_PATH but its parent directory.

--
Ricardo

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Nils Gillmann]

Ricardo Wurmus transcribed 1.0K bytes:
> 
> Nils Gillmann <ng0@n0.is> writes:
> 
> > Here's what's happening now:
> >
> > I took my perl module which is in "pkgs/lang-perl/perl.scm", which contains our basic perl packages,
> > and moved ( + renamed ) them into (core lang-perl perl) in a new directory.
> >
> > If I run GUIX_PACKAGE_PATH="/home/user/src/core" package -s mc I get results for query "mc".
> > When I run GUIX_PACKAGE_PATH="/home/user/src/core" package --show=perl I get the same message
> > as initially pointed out in the opening post.
> 
> I don’t understand what the problem is.  Your description isn’t very
> clear.  If you cannot post a minimal example it is virtually impossible
> for us to help.

I am not sure if I am allowed to point to my git server here. The single repository
includes no code that is against the Guidelines Guix follows, but there are repositories
on it which don't and which (currently) are still usable with Guix.

In any case I've decided to move the core packages for now back into another repository.

If I encounter this problem again, I'll provide more details.

> I’d also like to suggest asking questions on guile-user when you have
> questions about Guile.

Okay, thanks.

> If you have a file “core/lang-perl/perl.scm” and the module is called
> “(core lang-perl perl)” then you should not have “core” itself on
> GUIX_PACKAGE_PATH but its parent directory.

In my example (pkgs lang-perl perl) itself had the parent directory "pkgs".
I admit, code examples would've made it more clear..

Since we can point to github.com regularly, I think my private server would
not be considered problematic?

> --
> Ricardo

Re: a GUIX_PACKAGE_PATH / modules puzzle

[Ricardo Wurmus]

Nils Gillmann <ng0@n0.is> writes:

>> If you have a file “core/lang-perl/perl.scm” and the module is called
>> “(core lang-perl perl)” then you should not have “core” itself on
>> GUIX_PACKAGE_PATH but its parent directory.
>
> In my example (pkgs lang-perl perl) itself had the parent directory "pkgs".
> I admit, code examples would've made it more clear..
>
> Since we can point to github.com regularly, I think my private server would
> not be considered problematic?

It is better to provide a minimal example that demonstrates the
problem.  This is better than providing a full repository and ask other
people to figure out what is relevant and what is not.

-- 
Ricardo