Help with channel build system and package

Help with channel build system and package

[Jesse]

[-- Attachment #1: Type: text/plain, Size: 4910 bytes --]

Hello,

I have a channel where I am trying to develop a few packages that need a 
new build system. I want to try and use guix to build toolchains from 
crosstool-ng[1] for embedded development. In an effort to guix-fiy some 
of my development environments I'd like to be able to use the same 
toolchains that I do normally but specified in a manifest for embedded 
projects. I have attached my channel here.

It seems to be able to build the crosstool-ng package fine with

guix build -v3 -L embedded-dev crosstool-ng
guix build: warning: invalid argument list
guix build: warning: source expression failed to match any pattern
guix build: warning: ambiguous package specification `crosstool-ng'
guix build: warning: choosing crosstool-ng@1.26.0 from 
embedded-dev/embedded-dev/packages/crosstool-ng.scm:26:2
/gnu/store/5adzqwspya97z59pgxfvchrlyddl7v6a-crosstool-ng-1.26.0

However, when I try to build one of my packages that depend on 
crosstool-ng I get this:

guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
guix build: warning: invalid argument list
guix build: warning: source expression failed to match any pattern
Backtrace:
In ice-9/boot-9.scm:
   1747:15 19 (with-exception-handler #<procedure 7fd0988d3ea0 at ic…> …)
   1752:10 18 (with-exception-handler _ _ #:unwind? _ # _)
In guix/ui.scm:
     485:6 17 (_)
In guix/scripts/build.scm:
     711:5 16 (_)
In srfi/srfi-1.scm:
    673:15 15 (append-map #<procedure 7fd09b28b140 at guix/scripts/b…> …)
    586:17 14 (map1 ("x86_64-linux"))
In guix/scripts/build.scm:
    713:21 13 (_ _)
In guix/store.scm:
   1380:11 12 (map/accumulate-builds #<store-connection 256.99 7fd0a…> …)
    1298:8 11 (call-with-build-handler #<procedure 7fd0988d3e40 at g…> …)
In guix/scripts/build.scm:
    667:16 10 (_ #<package ct-ng-riscv64-unknown-elf@1.26.0 embedded-…>)
    656:24  9 (_ #<package ct-ng-riscv64-unknown-elf@1.26.0 embedded…> …)
In guix/packages.scm:
   1372:17  8 (supported-package? #<package ct-ng-riscv64-unknown-el…> …)
In guix/memoization.scm:
     101:0  7 (_ #<hash-table 7fd09923d640 0/31> #<package ct-ng-ris…> …)
In guix/packages.scm:
   1350:39  6 (_)
   1612:16  5 (package->bag _ _ _ #:graft? _)
   1709:22  4 (thunk)
In guix/build-system.scm:
      94:2  3 (make-bag _ _ #:source _ #:inputs _ #:native-inputs _ # …)
In ice-9/boot-9.scm:
   1685:16  2 (raise-exception _ #:continuable? _)
   1685:16  1 (raise-exception _ #:continuable? _)
   1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Throw to key `match-error' with args `("match" "no matching pattern" 
#<package crosstool-ng@1.26.0 
embedded-dev/embedded-dev/packages/crosstool-ng.scm:26 7fd09cb334d0>)'.

I'm a little lost on how to proceed debugging this. It seems like it is 
saying it can't find the crosstool-ng package. It seemed to build fine 
and the package definition is in the same file.

For some background, when building a toolchain from crosstool-ng you 
provide it with a defconfig and it has a number of steps it goes through 
to bootstrap it and pull the versioned dependencies down and everything. 
So my plan was to have some of these packages that use crosstool-ng and 
just provide the defconfig as a plain file:

(define-public ct-ng-riscv64-unknown-elf
   (package
     (name "ct-ng-riscv64-unknown-elf")
     (source (plain-file "defconfig"
                         '("CT_CONFIG_VERSION=\"4\"\n"
                           "CT_EXPERIMENTAL=y\n"
                           "CT_ARCH_RISCV=y\n"
                           "# CT_DEMULTILIB is not set\n"
                           "CT_ARCH_USE_MMU=y\n"
                           "CT_ARCH_64=y\n"
                           "CT_DEBUG_GDB=y\n"
                           "# CT_GDB_CROSS_PYTHON is not set\n")))
     (version "1.26.0")
     (build-system crosstool-ng)
     (arguments `(#:sample "riscv64-unknown-elf"))
     (synopsis "A crosstool-ng based riscv64 unknown elf toolchain")
     (description "A crosstool-ng based riscv64 unknown elf toolchain")
     (home-page "https://crosstool-ng.github.io/docs/")
     (license (list gpl2 gpl2+ gpl3+ lgpl2.1 lgpl3+))
     ))

The actual guix build system that I put together mainly follows others 
that I saw in the guix source. It really just deletes a bunch of phases 
from the gnu standard phases and then just runs the crosstool-ng 
commands, one per phase.

There are some tweaks that I want to make here and there to clean things 
up before I see if this is upstreamable but I'd like to get this build 
debugged first.

Thanks for any tips.


1: https://crosstool-ng.github.io/

[-- Attachment #2: embedded-dev.tar.gz --]
[-- Type: application/gzip, Size: 2824 bytes --]

Re: Help with channel build system and package

[Carlo Zancanaro]

Hi Jesse,

I know very little about what you're trying to do, but there's one thing
that looked wrong to me:

On Wed, Feb 07 2024, Jesse wrote:
> ...
>     (build-system crosstool-ng)
> ...

This isn't a build system, right? It's just the package? I think you may
want crosstool-ng-build-system here.

That might be what's causing the error:

> ...
> In guix/build-system.scm:
>      94:2  3 (make-bag _ _ #:source _ #:inputs _ #:native-inputs _ # …)
> In ice-9/boot-9.scm:
>   1685:16  2 (raise-exception _ #:continuable? _)
>   1685:16  1 (raise-exception _ #:continuable? _)
>   1685:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> Throw to key `match-error' with args `("match" "no matching pattern"
> #<package crosstool-ng@1.26.0
> embedded-dev/embedded-dev/packages/crosstool-ng.scm:26
> 7fd09cb334d0>)'.

Looking in make-bag (in guix/build-system.scm), the first think it does
it pattern match on the build system expecting a record of the
<build-system> type. Your package doesn't match that pattern, so this
error is raised.

Carlo

Re: Help with channel build system and package

[Jesse]

[-- Attachment #1: Type: text/plain, Size: 7002 bytes --]

Thanks for taking a look Carlo!

On 2/7/24 22:37, Carlo Zancanaro wrote:
> Hi Jesse,
>
> I know very little about what you're trying to do, but there's one thing
> that looked wrong to me:
>
> On Wed, Feb 07 2024, Jesse wrote:
>> ...
>>      (build-system crosstool-ng)
>> ...
> This isn't a build system, right? It's just the package? I think you may
> want crosstool-ng-build-system here.

You are totally correct here. I changed this to "(build-system 
crosstool-ng-build-system)" and I don't get the same error now. I also 
then realized I was missing "  #:use-module (embedded-dev build-system 
crosstool-ng)" from the list of modules in the package define-module 
expression in the packages/crosstool-ng.scm file. I also tracked down 
some other missing modules.

However, now I get this:

guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
guix build: warning: invalid argument list
guix build: warning: source expression failed to match any pattern
error: crosstool-ng-build-system: unbound variable
hint: Did you forget `(use-modules (embedded-dev build-system 
crosstool-ng))'?

guix build: error: ct-ng-riscv64-unknown-elf: unknown package

This confuses me a bit because it says that it is an unbound variable 
but it is defined in embedded-dev/build-system/crosstool-ng.scm. I 
believe I should be importing this module with the "#:use-module 
(embedded-dev build-system crosstool-ng)" line in the package file. The 
symbol should be exported with:

(define-module (embedded-dev build-system crosstool-ng)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
   #:export (%crosstool-ng-build-system-modules
             crosstool-ng-build
             crosstool-ng-build-system))

which is in the embedded-dev/build-system/crosstool-ng.scm file of my 
channel. I was mainly trying to follow some build-system setups like 
ruby where there are build/ruby-build-system.scm, which is used by 
build-system/ruby.scm, which defines and exports the ruby-build-system 
symbol, and that is used by the packages in packages/ruby.scm.

If I was to manually trace how the build system guile files are used 
from a package definition, it would look like this:

1. In packages/crosstool-ng.scm I have a package definition that uses
        (build-system crosstool-ng-build-system)

2. In that same file I have a line that should include the entry point 
to the crosstool-ng build system:
         #:use-module (embedded-dev build-system crosstool-ng)

3. That line should use the guile module located at 
embedded-dev/build-system/crosstool-ng.scm, since my build invocation 
looks like: "guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf" 
and my channel is in a directory called "embedded-dev". So the fulll 
path would be embedded-dev/embedded-dev/build-system/crosstool-ng.scm

4. In build-system/crosstool-ng.scm the beginning is defined as:

(define-module (embedded-dev build-system crosstool-ng)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
   #:export (%crosstool-ng-build-system-modules
             crosstool-ng-build
             crosstool-ng-build-system))

(define %crosstool-ng-build-system-modules
   ;; Build-side modules imported by default.
   `((embedded-dev build crosstool-ng-build-system)
     ,@%gnu-build-system-modules))

(define (default-crosstool-ng )
   "Return the default crosstool-ng package"
   (let ((crosstool-ng (resolve-interface '(embedded-dev packages 
crosstool-ng))))
     (module-ref crosstool-ng 'crosstool-ng)))


Here I should be exporting the "crosstool-ng-build-system" symbol to be 
used by a guile script that uses it. The 
crosstool-ng-build-system-modules should be looking at 
embedded-dev/build/crosstool-ng-build-system.scm, again taking 
inspiration from other build system files like the ones associated with 
ruby. The default-crosstool-ng symbol should be grabbing from the 
embedded-dev/packages/crosstool-ng.scm file.

5. Later in that file I pass the crosstool-ng-build-system-modules 
symbol to the crosstool-ng-build function as well as the module located 
at embedded-dev/build/crosstool-ng-build-system.scm

(define* (crosstool-ng-build name inputs
                              #:key (sample "unknown")
                              (test-target "test")
                              (tests? #t)
                              (phases '%standard-phases)
                              (outputs '("out"))
                              (search-paths '())
                              (system (%current-system))
                              (guile #f)
                              (imported-modules 
%crosstool-ng-build-system-modules)
                              (modules '((embedded-dev build 
crosstool-ng-build-system)
                                         (guix build utils)))
                              )

6. Then the actual build system symbol that is exported from the top is 
defined at the bottom of this file:

(define crosstool-ng-build-system
   (build-system
     (name 'crosstool-ng)
     (description "Crosstool-ng build system")
     (lower lower)))

It all seems just as threaded together as the ones in the guix source. I 
must be missing something small here.

I'm also confused about the "guix build: warning: invalid argument 
list". It looks like I'm invoking the build command properly.

Is there something else you have to do when setting up a channel to pull 
in the build and build-system directories? I thought that by adding my 
channel to the load path with "-L", the elements passed to use-module 
were essentially directory paths.

>
> That might be what's causing the error:
>
>> ...
>> In guix/build-system.scm:
>>       94:2  3 (make-bag _ _ #:source _ #:inputs _ #:native-inputs _ # …)
>> In ice-9/boot-9.scm:
>>    1685:16  2 (raise-exception _ #:continuable? _)
>>    1685:16  1 (raise-exception _ #:continuable? _)
>>    1685:16  0 (raise-exception _ #:continuable? _)
>>
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> Throw to key `match-error' with args `("match" "no matching pattern"
>> #<package crosstool-ng@1.26.0
>> embedded-dev/embedded-dev/packages/crosstool-ng.scm:26
>> 7fd09cb334d0>)'.
> Looking in make-bag (in guix/build-system.scm), the first think it does
> it pattern match on the build system expecting a record of the
> <build-system> type. Your package doesn't match that pattern, so this
> error is raised.
Thanks for pointing this out, I didn't dig in enough.
>
> Carlo

I've attached the version with the changes mentioned.

Thanks,

Jesse

[-- Attachment #2: embedded-dev.tar.gz --]
[-- Type: application/gzip, Size: 2874 bytes --]

Re: Help with channel build system and package

[Marek Paśnikowski]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

08.02.2024 15:32:44 CET Jesse:
> Thanks for taking a look Carlo!
> 
> On 2/7/24 22:37, Carlo Zancanaro wrote:
> > Hi Jesse,
> > 
> > I know very little about what you're trying to do, but there's one thing
> > that looked wrong to me:
> > 
> > On Wed, Feb 07 2024, Jesse wrote:
> >> ...
> >> 
> >>      (build-system crosstool-ng)
> >> 
> >> ...
> > 
> > This isn't a build system, right? It's just the package? I think you may
> > want crosstool-ng-build-system here.

> 
> However, now I get this:
> 
> guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
> guix build: warning: invalid argument list
> guix build: warning: source expression failed to match any pattern
> error: crosstool-ng-build-system: unbound variable
> hint: Did you forget `(use-modules (embedded-dev build-system
> crosstool-ng))'?
> 
> guix build: error: ct-ng-riscv64-unknown-elf: unknown package
> 
> This confuses me a bit because it says that it is an unbound variable
> but it is defined in embedded-dev/build-system/crosstool-ng.scm. I
> believe I should be importing this module with the "#:use-module
> (embedded-dev build-system crosstool-ng)" line in the package file. The
> symbol should be exported with:

I am just starting to understand how packaging works, but one thing that 
stands out to me is the #:sample keyword in arguments of the *riscv64* package 
definition. Where is it defined? Is the source correctly imported?

My hypothesis is that guix, upon loading the module in question:
1. fails to parse the #:sample keyword (invalid argument list warning)
2. fails to parse the package definition due to the above (pattern failure)
3. fails to find the package definition due to the above (unbound variable)

This problem is educational to me, so give me the benefit of the doubt.

Sincerely,

Marek Paśnikowski

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Help with channel build system and package

[Jesse Millwood]

Thanks for the response Marek

On 2/8/24 12:00, Marek Paśnikowski wrote:
> 08.02.2024 15:32:44 CET Jesse:
>> Thanks for taking a look Carlo!
>>
>> On 2/7/24 22:37, Carlo Zancanaro wrote:
>>> Hi Jesse,
>>>
>>> I know very little about what you're trying to do, but there's one thing
>>> that looked wrong to me:
>>>
>>> On Wed, Feb 07 2024, Jesse wrote:
>>>> ...
>>>>
>>>>       (build-system crosstool-ng)
>>>>
>>>> ...
>>> This isn't a build system, right? It's just the package? I think you may
>>> want crosstool-ng-build-system here.
>> However, now I get this:
>>
>> guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
>> guix build: warning: invalid argument list
>> guix build: warning: source expression failed to match any pattern
>> error: crosstool-ng-build-system: unbound variable
>> hint: Did you forget `(use-modules (embedded-dev build-system
>> crosstool-ng))'?
>>
>> guix build: error: ct-ng-riscv64-unknown-elf: unknown package
>>
>> This confuses me a bit because it says that it is an unbound variable
>> but it is defined in embedded-dev/build-system/crosstool-ng.scm. I
>> believe I should be importing this module with the "#:use-module
>> (embedded-dev build-system crosstool-ng)" line in the package file. The
>> symbol should be exported with:
> I am just starting to understand how packaging works, but one thing that
> stands out to me is the #:sample keyword in arguments of the *riscv64* package
> definition. Where is it defined? Is the source correctly imported?

Ha, I am just trying to understand how packaging works too (or at least 
I think I am, lol).

That's a good catch and suggestion. I thought that by doing this in 
build-system/crosstool-ng.scm:

(define* (crosstool-ng-build name inputs
                              #:key (sample "unknown")
                              (test-target "test")
                              (tests? #t)
                              (phases '%standard-phases)
                              (outputs '("out"))
                              (search-paths '())
                              (system (%current-system))
                              (guile #f)
                              (imported-modules 
%crosstool-ng-build-system-modules)
                              (modules '((guix build 
crosstool-ng-build-system)
                                         (guix build utils)))
                              )

That the #:key (sample "unknown") was defining that as a keyword 
argument. It was a hold over from an idea I had that I kind of abandoned 
but didn't think it was causing any issues. I just removed that and the 
arguments line from the packages.

Now I get two invalid argument list errors:

guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
guix build: warning: invalid argument list
guix build: warning: invalid argument list
error: crosstool-ng-build-system: unbound variable
hint: Did you forget `(use-modules (embedded-dev build-system 
crosstool-ng))'?

guix build: error: ct-ng-riscv64-unknown-elf: unknown package

I guess I was thinking that it was saying that my invocation of the guix 
build command had an invalid argument given. I hadn't thought that it 
was talking about an argument given to one of the functions. That gives 
me some where to start looking at least.

>
> My hypothesis is that guix, upon loading the module in question:
> 1. fails to parse the #:sample keyword (invalid argument list warning)
> 2. fails to parse the package definition due to the above (pattern failure)
> 3. fails to find the package definition due to the above (unbound variable)
>
> This problem is educational to me, so give me the benefit of the doubt.

That's a sound hypothesis. It gives me a good idea of where to look, thanks!

Do you know of any way to increase the verbosity of error messages or 
logs to help debug where things are going awry?

>
> Sincerely,
>
> Marek Paśnikowski

Thanks,

Jesse

Re: Help with channel build system and package

[Marek Paśnikowski]

[-- Attachment #1: Type: text/plain, Size: 2453 bytes --]

08.02.2024 19:33:09 CET Jesse Millwood:
> Thanks for the response Marek
> 
> On 2/8/24 12:00, Marek Paśnikowski wrote:
> > 08.02.2024 15:32:44 CET Jesse:
> >> Thanks for taking a look Carlo!
> >> 
> >> On 2/7/24 22:37, Carlo Zancanaro wrote:
> >>> Hi Jesse,
> >>> 
> >>> I know very little about what you're trying to do, but there's one thing
> >>> that looked wrong to me:
> >>> 
> >>> On Wed, Feb 07 2024, Jesse wrote:
> >>>> ...
> >>>> 
> >>>>       (build-system crosstool-ng)
> >>>> 
> >>>> ...
> >>> 
> >>> This isn't a build system, right? It's just the package? I think you may
> >>> want crosstool-ng-build-system here.
> >> 
> >> However, now I get this:
> >> 
> >> guix build -v3 -L embedded-dev ct-ng-riscv64-unknown-elf
> >> guix build: warning: invalid argument list
> >> guix build: warning: source expression failed to match any pattern
> >> error: crosstool-ng-build-system: unbound variable
> >> hint: Did you forget `(use-modules (embedded-dev build-system
> >> crosstool-ng))'?
> >> 
> >> guix build: error: ct-ng-riscv64-unknown-elf: unknown package

> > I am just starting to understand how packaging works, but one thing that
> > stands out to me is the #:sample keyword in arguments of the *riscv64*
> > package definition. Where is it defined? Is the source correctly
> > imported?

> 
> > My hypothesis is that guix, upon loading the module in question:
> > 1. fails to parse the #:sample keyword (invalid argument list warning)
> > 2. fails to parse the package definition due to the above (pattern
> > failure)
> > 3. fails to find the package definition due to the above (unbound
> > variable)
> > 
> > This problem is educational to me, so give me the benefit of the doubt.
> 
> That's a sound hypothesis. It gives me a good idea of where to look, thanks!
> 
> Do you know of any way to increase the verbosity of error messages or
> logs to help debug where things are going awry?

I always use -v 4 as argument to guix. I do not remember whether 3 or 4 is the 
biggest effective value. You could try with that.

About the warnings of invalid argument list: comment out the argument lists in 
the toolchain packages (I do not remember fully the names, I do not have acces 
to the big computer right now for easy reference). See how the change effects 
the build.

I learned a lot by intentionally breaking code and reading the errors.

Sincerely,

Marek

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Help with channel build system and package

[Jesse]

> About the warnings of invalid argument list: comment out the argument lists in
> the toolchain packages (I do not remember fully the names, I do not have acces
> to the big computer right now for easy reference). See how the change effects
> the build.
>
> I learned a lot by intentionally breaking code and reading the errors.
>
I ended up trying that, removing the arguments from the package, but I 
did not get any change in the output.

One thing I did try is trying to build with the repl. I made this 
script, named build-repl.scm:

(use-modules
  (guix store)
  (guix derivations)
  (guix packages)
  ;;(embedded-dev packages)
  )

(define (build-packages packages)
   (with-store store
     (let ((builds (map (lambda (package)
                          (package-derivation store package))
                        packages)))
       (build-derivations store builds))))
(define (build-package package)
   (build-packages (list package)))

Which i got from here: 
https://lists.gnu.org/archive/html/help-guix/2020-06/msg00173.html

I don't seem to be able to use my channel here for some reason but I 
think I got around that later.

For context, this is my directory layout:

embedded-dev-chan
├── build-repl.scm
├── .guix-channel
└── embedded-dev
     ├── build
     │   └── crosstool-ng-build-system.scm
     ├── build-system
     │   └── crosstool-ng.scm
     └── packages
         └── crosstool-ng.scm

My understanding is that by adding embedded-dev-chan to the load path I 
should be able to use the guile files inside with something like 
(use-modules embedded-dev packages crosstool-ng).

I invoked the repl: guix repl -L /home/jesse/Code/embedded-dev-chan

Then I loaded the "build-repl.scm" script and tried to build my package:

scheme@(guix-user)> ,load /home/jesse/Code/embedded-dev-chan/build-repl.scm
scheme@(guix-user) [2]> (use-modules (embedded-dev packages crosstool-ng))
While compiling expression:
Syntax error:
unknown location: lambda*: invalid argument list in subform ((phases 
(quote %standard-phases)) (outputs (quote ("out"))) (search-paths (quote 
())) (system (%current-system)) (guile #f) (imported-modules 
%crosstool-ng-build-system-modules) (modules (quote ((embedded-dev build 
crosstool-ng-build-system) (guix build utils))))) of (name inputs 
(phases (quote %standard-phases)) (outputs (quote ("out"))) 
(search-paths (quote ())) (system (%current-system)) (guile #f) 
(imported-modules %crosstool-ng-build-system-modules) (modules (quote 
((embedded-dev build crosstool-ng-build-system) (guix build utils)))))

This seems like it is giving me a little more to go off of. I pretified 
the output a bit:

((phases
   (quote %standard-phases))
  (outputs (quote ("out")))
  (search-paths (quote ()))
  (system (%current-system))
  (guile #f)
  (imported-modules %crosstool-ng-build-system-modules)
  (modules
   (quote ((embedded-dev build crosstool-ng-build-system)
           (guix build utils)))))

of

(name
  inputs
  (phases (quote %standard-phases))
  (outputs (quote ("out")))
  (search-paths (quote ()))
  (system (%current-system))
  (guile #f)
  (imported-modules %crosstool-ng-build-system-modules)
  (modules (quote ((embedded-dev build crosstool-ng-build-system)
                   (guix build utils)))))


I'm not quite sure what this is trying to tell me. It seems like it is 
an issue around here in embedded-dev/build-system/crosstool-ng.scm:

(define* (crosstool-ng-build name inputs
                              (phases '%standard-phases)
                              (outputs '("out"))
                              (search-paths '())
                              (system (%current-system))
                              (guile #f)
                              (imported-modules 
%crosstool-ng-build-system-modules)
                              (modules '((embedded-dev build 
crosstool-ng-build-system)
                                         (guix build utils)))
                              )
   "Build Crosstool-ng toolchain"

   (define build
     #~(begin
         (use-modules #$@(sexp->gexp modules))

         #$(with-build-variables inputs outputs
             #~(crosstool-ng-build #:name #$name
                                   #:source #+source
                                   #:system #$system
                                   #:phases #$(if (pair? phases)
                                                  (sexp->gexp phases)
                                                  phases)
                                   #:outputs %outputs
                                   #:search-paths '#$(sexp->gexp
                                                      (map
search-path-specification->sexp
search-paths))
                                   #:inputs %build-inputs))))

   (mlet %store-monad ((guile (package->derivation (or guile 
(default-guile))
                                                   system #:graft? #f)))
     (gexp->derivation name build
                       #:system system
                       #:target #f
                       #:modules imported-modules
                       #:guile-for-build guile)))


My first thought was that there was a mismatch between the arguments of 
crosstool-ng-build -> build -> gexp->derivation. But the arguments all 
seem to match up. I compared to build-system/ruby.scm and that part 
seems really close to that as well except for some arguments that seem 
like they are ruby specific.

There must be some nuance that I'm missing here.

Jesse

Re: Help with channel build system and package

[Jesse Millwood]

I'm still trying to get this to work. I tracked some things I forgot and 
now have my channel hosted at: 
https://codeberg.org/moonling/guix-embedded-dev

When I try to build my toolchain package I get this:

➜ guix build -v 4 -L guix-embedded-dev ct-ng-riscv64-unknown-elf
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 
100.0%
The following derivation will be built:
/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv
building 
/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv...
Backtrace:
            3 (primitive-load "/gnu/store/kghq1xz3wyb509944y9jd8vhy0l?")
In ice-9/eval.scm:
    191:27  2 (_ #f)
    223:20  1 (proc #<directory (guile-user) 7ffff77f7c80>)
In unknown file:
            0 (%resolve-variable (7 . gexp) #<directory (guile-user) ?>)

ERROR: In procedure %resolve-variable:
Unbound variable: gexp
builder for 
`/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
failed with exit code 1
build of 
/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv 
failed
View build log at 
'/var/log/guix/drvs/wm/1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv.gz'.
guix build: error: build of 
`/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
failed


It seems the issue here is an unbound variable "gexp". I'm a little 
confused about this because I have #:use-module (guix gexp) included in 
my package and build-system/crosstool-ng.scm files. The backtrace is a 
little hard to follow.

When I search for unbound variable: gexp I do find some posts that refer 
to unbound variable: ungexp like here 
https://lists.gnu.org/archive/html/help-guix/2019-06/msg00073.html

It seems to be associated with making a file during the build phase. I 
do have this in my package:

(define-public ct-ng-riscv64-unknown-elf
   (package
     (name "ct-ng-riscv64-unknown-elf")
     (source (plain-file "defconfig" "\
CT_CONFIG_VERSION=\"4\"\n
CT_EXPERIMENTAL=y\n
CT_ARCH_RISCV=y\n
# CT_DEMULTILIB is not set\n
CT_ARCH_USE_MMU=y\n
CT_ARCH_64=y\n
CT_DEBUG_GDB=y\n
# CT_GDB_CROSS_PYTHON is not set\n"
))
     (version "1.26.0")
     (build-system crosstool-ng-build-system)
     (synopsis "A crosstool-ng based riscv64 unknown elf toolchain")
     (description "A crosstool-ng based riscv64 unknown elf toolchain")
     (home-page "https://crosstool-ng.github.io/docs/")
     (license (list gpl2 gpl2+ gpl3+ lgpl2.1 lgpl3+))
     ))

I'm wondering if the "plain-file" part is not used properly. I did read 
in the guix docs that "source"

> An object telling how the source code for the package should be 
> acquired. Most of the time, this is an |origin| object, which denotes 
> a file fetched from the Internet (see |origin| Reference 
> <https://guix.gnu.org/manual/en/guix.html#origin-Reference>). It can 
> also be any other “file-like” object such as a |local-file|, which 
> denotes a file from the local file system (see |local-file| 
> <https://guix.gnu.org/manual/en/guix.html#G_002dExpressions>). 
I couldn't find any examples using a "plain-file" but it should be a 
file-like object I think.

If anything sticks out to anyone please send a message,

Jesse

On 2/15/24 08:23, Jesse wrote:
>> About the warnings of invalid argument list: comment out the argument 
>> lists in
>> the toolchain packages (I do not remember fully the names, I do not 
>> have acces
>> to the big computer right now for easy reference). See how the change 
>> effects
>> the build.
>>
>> I learned a lot by intentionally breaking code and reading the errors.
>>
> I ended up trying that, removing the arguments from the package, but I 
> did not get any change in the output.
>
> One thing I did try is trying to build with the repl. I made this 
> script, named build-repl.scm:
>
> (use-modules
>  (guix store)
>  (guix derivations)
>  (guix packages)
>  ;;(embedded-dev packages)
>  )
>
> (define (build-packages packages)
>   (with-store store
>     (let ((builds (map (lambda (package)
>                          (package-derivation store package))
>                        packages)))
>       (build-derivations store builds))))
> (define (build-package package)
>   (build-packages (list package)))
>
> Which i got from here: 
> https://lists.gnu.org/archive/html/help-guix/2020-06/msg00173.html
>
> I don't seem to be able to use my channel here for some reason but I 
> think I got around that later.
>
> For context, this is my directory layout:
>
> embedded-dev-chan
> ├── build-repl.scm
> ├── .guix-channel
> └── embedded-dev
>     ├── build
>     │   └── crosstool-ng-build-system.scm
>     ├── build-system
>     │   └── crosstool-ng.scm
>     └── packages
>         └── crosstool-ng.scm
>
> My understanding is that by adding embedded-dev-chan to the load path 
> I should be able to use the guile files inside with something like 
> (use-modules embedded-dev packages crosstool-ng).
>
> I invoked the repl: guix repl -L /home/jesse/Code/embedded-dev-chan
>
> Then I loaded the "build-repl.scm" script and tried to build my package:
>
> scheme@(guix-user)> ,load 
> /home/jesse/Code/embedded-dev-chan/build-repl.scm
> scheme@(guix-user) [2]> (use-modules (embedded-dev packages 
> crosstool-ng))
> While compiling expression:
> Syntax error:
> unknown location: lambda*: invalid argument list in subform ((phases 
> (quote %standard-phases)) (outputs (quote ("out"))) (search-paths 
> (quote ())) (system (%current-system)) (guile #f) (imported-modules 
> %crosstool-ng-build-system-modules) (modules (quote ((embedded-dev 
> build crosstool-ng-build-system) (guix build utils))))) of (name 
> inputs (phases (quote %standard-phases)) (outputs (quote ("out"))) 
> (search-paths (quote ())) (system (%current-system)) (guile #f) 
> (imported-modules %crosstool-ng-build-system-modules) (modules (quote 
> ((embedded-dev build crosstool-ng-build-system) (guix build utils)))))
>
> This seems like it is giving me a little more to go off of. I 
> pretified the output a bit:
>
> ((phases
>   (quote %standard-phases))
>  (outputs (quote ("out")))
>  (search-paths (quote ()))
>  (system (%current-system))
>  (guile #f)
>  (imported-modules %crosstool-ng-build-system-modules)
>  (modules
>   (quote ((embedded-dev build crosstool-ng-build-system)
>           (guix build utils)))))
>
> of
>
> (name
>  inputs
>  (phases (quote %standard-phases))
>  (outputs (quote ("out")))
>  (search-paths (quote ()))
>  (system (%current-system))
>  (guile #f)
>  (imported-modules %crosstool-ng-build-system-modules)
>  (modules (quote ((embedded-dev build crosstool-ng-build-system)
>                   (guix build utils)))))
>
>
> I'm not quite sure what this is trying to tell me. It seems like it is 
> an issue around here in embedded-dev/build-system/crosstool-ng.scm:
>
> (define* (crosstool-ng-build name inputs
>                              (phases '%standard-phases)
>                              (outputs '("out"))
>                              (search-paths '())
>                              (system (%current-system))
>                              (guile #f)
>                              (imported-modules 
> %crosstool-ng-build-system-modules)
>                              (modules '((embedded-dev build 
> crosstool-ng-build-system)
>                                         (guix build utils)))
>                              )
>   "Build Crosstool-ng toolchain"
>
>   (define build
>     #~(begin
>         (use-modules #$@(sexp->gexp modules))
>
>         #$(with-build-variables inputs outputs
>             #~(crosstool-ng-build #:name #$name
>                                   #:source #+source
>                                   #:system #$system
>                                   #:phases #$(if (pair? phases)
>                                                  (sexp->gexp phases)
>                                                  phases)
>                                   #:outputs %outputs
>                                   #:search-paths '#$(sexp->gexp
>                                                      (map
> search-path-specification->sexp
> search-paths))
>                                   #:inputs %build-inputs))))
>
>   (mlet %store-monad ((guile (package->derivation (or guile 
> (default-guile))
>                                                   system #:graft? #f)))
>     (gexp->derivation name build
>                       #:system system
>                       #:target #f
>                       #:modules imported-modules
>                       #:guile-for-build guile)))
>
>
> My first thought was that there was a mismatch between the arguments 
> of crosstool-ng-build -> build -> gexp->derivation. But the arguments 
> all seem to match up. I compared to build-system/ruby.scm and that 
> part seems really close to that as well except for some arguments that 
> seem like they are ruby specific.
>
> There must be some nuance that I'm missing here.
>
> Jesse

Re: Help with channel build system and package

[Marek Paśnikowski]

[-- Attachment #1: Type: text/plain, Size: 3625 bytes --]

08.03.2024 03:35:11 CET Jesse Millwood:
> I'm still trying to get this to work. I tracked some things I forgot and
> now have my channel hosted at:
> https://codeberg.org/moonling/guix-embedded-dev
> 
> When I try to build my toolchain package I get this:
> 
> ➜ guix build -v 4 -L guix-embedded-dev ct-ng-riscv64-unknown-elf
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'...
> 100.0%
> The following derivation will be built:
> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
> .drv building
> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
> .drv... Backtrace:
>             3 (primitive-load "/gnu/store/kghq1xz3wyb509944y9jd8vhy0l?")
> In ice-9/eval.scm:
>     191:27  2 (_ #f)
>     223:20  1 (proc #<directory (guile-user) 7ffff77f7c80>)
> In unknown file:
>             0 (%resolve-variable (7 . gexp) #<directory (guile-user) ?>)
> 
> ERROR: In procedure %resolve-variable:
> Unbound variable: gexp
> builder for
> `/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.
> 0.drv' failed with exit code 1
> build of
> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
> .drv failed
> View build log at
> '/var/log/guix/drvs/wm/1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-
> elf-1.26.0.drv.gz'. guix build: error: build of
> `/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.
> 0.drv' failed
> 
> 
> It seems the issue here is an unbound variable "gexp". I'm a little
> confused about this because I have #:use-module (guix gexp) included in
> my package and build-system/crosstool-ng.scm files. The backtrace is a
> little hard to follow.

I am self-learning Scheme and yet confident in my skills, so take what I am 
about to write with a dose of healthy scepticism.

1. From the error message alone, I do not know whether the unresolved variable 
is actually in the crosstool-ng package file, or anywhere else.

2. "gexp" or its alias "#~" do not occur in the body of the package module.  
However, the module does depend, among others that I trust to be correct, on 
the crosstool-ng build system.  There is a potential for the unresolved 
variable to live there.

3. Upon opening the build-system file, I was quickly drawn to the nested gexp 
in the definition of build procedure.  It took me some reading of the 
documentation to somewhat understand what the #$@(sexp-gexp modules) form is.  
My current understanding is that it brings an externally defined list of 
modules.

4. The (crosstool-ng-build name inputs _) definition does contain what appears 
to be a "modules" binding to a list of modules.  That list does NOT contain 
(guix gexp), and this in turn makes #~(crosstool-ng-build _) form unreadable 
to the interpreter.  Do mind that at this point the interpreter is not running 
inside the build-system file — it is working on a staged code of #~(begin _).

5. My proposed solution is to include (guix gexp) in the (define* (crosstool-
ng-build name inputs _ (modules)) form.  I have tested the change at the last 
minute — guix no longer complains about unresolved variables, but there is a 
new error about "no code for module (guix gexp)".  More work needs to be put 
into the code overall.

PS: I would appreciate it if someone who feels confident in their Scheme 
knowledge could grade my line of reasoning.  This is my first ever attempt to 
debug completely foreign code.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Help with channel build system and package

[Jesse Millwood]

Marek,

Thanks for the detailed response! I really appreciate it.

I responded inline. Is your main source of knowledge the official Guix 
documentation?

On 3/8/24 05:43, Marek Paśnikowski wrote:
> 08.03.2024 03:35:11 CET Jesse Millwood:
>> I'm still trying to get this to work. I tracked some things I forgot and
>> now have my channel hosted at:
>> https://codeberg.org/moonling/guix-embedded-dev
>>
>> When I try to build my toolchain package I get this:
>>
>> ➜ guix build -v 4 -L guix-embedded-dev ct-ng-riscv64-unknown-elf
>> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
>> substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'...
>> 100.0%
>> The following derivation will be built:
>> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
>> .drv building
>> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
>> .drv... Backtrace:
>>              3 (primitive-load "/gnu/store/kghq1xz3wyb509944y9jd8vhy0l?")
>> In ice-9/eval.scm:
>>      191:27  2 (_ #f)
>>      223:20  1 (proc #<directory (guile-user) 7ffff77f7c80>)
>> In unknown file:
>>              0 (%resolve-variable (7 . gexp) #<directory (guile-user) ?>)
>>
>> ERROR: In procedure %resolve-variable:
>> Unbound variable: gexp
>> builder for
>> `/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.
>> 0.drv' failed with exit code 1
>> build of
>> /gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.0
>> .drv failed
>> View build log at
>> '/var/log/guix/drvs/wm/1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-
>> elf-1.26.0.drv.gz'. guix build: error: build of
>> `/gnu/store/wm1jba8npfb9jiqsbzd50gz4c89n3gjm-ct-ng-riscv64-unknown-elf-1.26.
>> 0.drv' failed
>>
>>
>> It seems the issue here is an unbound variable "gexp". I'm a little
>> confused about this because I have #:use-module (guix gexp) included in
>> my package and build-system/crosstool-ng.scm files. The backtrace is a
>> little hard to follow.
> I am self-learning Scheme and yet confident in my skills, so take what I am
> about to write with a dose of healthy scepticism.
I am very much self learning as well. I've been comparing build system 
files and package files in my channel with upstream ones and pouring 
over the documentation. Any tips are very welcome!
> 1. From the error message alone, I do not know whether the unresolved variable
> is actually in the crosstool-ng package file, or anywhere else.
>
> 2. "gexp" or its alias "#~" do not occur in the body of the package module.
> However, the module does depend, among others that I trust to be correct, on
> the crosstool-ng build system.  There is a potential for the unresolved
> variable to live there.
Oh interesting, I didn't know "#~" was an alias for "gexp"! I see that 
now in the G-Expressions section of the manual.
> 3. Upon opening the build-system file, I was quickly drawn to the nested gexp
> in the definition of build procedure.  It took me some reading of the
> documentation to somewhat understand what the #$@(sexp-gexp modules) form is.
> My current understanding is that it brings an externally defined list of
> modules.
Interesting, this is a line that I didn't quite understand and mainly 
took from similar build systems in the guix source.
> 4. The (crosstool-ng-build name inputs _) definition does contain what appears
> to be a "modules" binding to a list of modules.  That list does NOT contain
> (guix gexp), and this in turn makes #~(crosstool-ng-build _) form unreadable
> to the interpreter.  Do mind that at this point the interpreter is not running
> inside the build-system file — it is working on a staged code of #~(begin _).
Oh this is very good information, thanks!
> 5. My proposed solution is to include (guix gexp) in the (define* (crosstool-
> ng-build name inputs _ (modules)) form.  I have tested the change at the last
> minute — guix no longer complains about unresolved variables, but there is a
> new error about "no code for module (guix gexp)".  More work needs to be put
> into the code overall.

Very interesting, thanks for trying that out! I'll incorporate it and 
keep digging!I do wonder why this is not included in other build systems 
in Guix? For example in guix/build-system/ruby.scm:

(define %ruby-build-system-modules ;; Build-side modules imported by 
default. `((guix build ruby-build-system) ,@%gnu-build-system-modules))

...

(define* (ruby-build name inputs #:key source (gem-flags ''()) 
(test-target "test") (tests? #t) (phases '%standard-phases) (outputs 
'("out")) (search-paths '()) (system (%current-system)) (guile #f) 
(imported-modules %ruby-build-system-modules) (modules '((guix build 
ruby-build-system) (guix build utils))))

This is one that I have been basing my crosstool-ng build system on. I 
appreciate your analysis!

> PS: I would appreciate it if someone who feels confident in their Scheme
> knowledge could grade my line of reasoning.  This is my first ever attempt to
> debug completely foreign code.

Re: Help with channel build system and package

[Richard Sent]

Jesse,

I expect your issue is you have an extra gexp in
build-system/crosstool-ng.scm, line 71. I determined this by:

1. Examining the derivation Guix is trying to build
2. Examining the -builder file mentioned in the derivation
3. Observe that ct-ng-riscv64-unknown-elf-1.26.0-builder wraps
(crosstool-ng-build ...) in (gexp).

The package still did not build locally on my machine after removing it,
but it seems to have progressed to a further stage.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: Help with channel build system and package

[Jesse Millwood]

Thanks for your response Richard!

On 3/8/24 22:39, Richard Sent wrote:
> Jesse,
>
> I expect your issue is you have an extra gexp in
> build-system/crosstool-ng.scm, line 71. I determined this by:
>
> 1. Examining the derivation Guix is trying to build
I'm still trying to learn about the internals of Guix and writing 
packages. How did you examine the derivation Guix is trying to build?
> 2. Examining the -builder file mentioned in the derivation
Ah! I did not notice that file in the output nor open it! I see now that 
the Derivations section of the manual mentions these drv files. Is this 
where you are examining the derivation that Guix is trying to build?
> 3. Observe that ct-ng-riscv64-unknown-elf-1.26.0-builder wraps
> (crosstool-ng-build ...) in (gexp).

Where is this? Are you talking of this in build-system/crosstool-ng:

   (define build
     (with-imported-modules
      imported-modules
      #~(begin
          (use-modules #$@(sexp->gexp modules))
          #~(crosstool-ng-build #:name #$name

Where "#~", I have learned from Marek (earlier in this thread), is an 
alias for "gexp".

Ah, I see the ct-ng-riscv64-unknown-elf-1.26.0-builder in the drv file 
and wow, I didn't know this existed! I see the gexp wrapping that you 
mention.

Is there a way to use the guix command to tell me where the drv file for 
other packages exist? It is apparent that I need to perform a more in 
depth review of the manual. This really helps contextualize things for me.

> The package still did not build locally on my machine after removing it,
> but it seems to have progressed to a further stage.
>
Thanks, I may investigate how other derivations look now that I know 
where to look. I appreciate the detailed input!

Re: Help with channel build system and package

[Jesse Millwood]

Richard and Marek,

The input from both of you helped me get a little farther. I've pushed 
my changes to https://codeberg.org/moonling/guix-embedded-dev.

I feel like I'm really close now. I seem to be able to get into the 
phases of my build system. I can not invoke the crosstool-ng tool. My 
issue is that it seems to not be able to see the defconfig that I define 
in the package declaration as a "plain-file".

I've determined this through the repl. In my repo there is a scheme file 
called "build-repl.scm" and I invoke it like this:

➜ guix repl -L guix-embedded-dev
GNU Guile 3.0.9
Copyright (C) 1995-2023 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> ,l 
/home/jesse/Code/guix-repos/guix-embedded-dev/build-repl.scm
WARNING: (embedded-dev packages crosstool-ng): `which' imported from 
both (guix build utils) and (gnu packages base)
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 
100.0%
building path(s) 
`/gnu/store/nc892p9kgx0my3gg9glyszg848hpf166-ct-ng-riscv64-unknown-elf-1.26.0'
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to 
`/gnu/store/dv009i1wj865n34dyln2fmxryxqg3z7c-crosstool-ng-1.26.0/bin:/gnu/store/sxx22f98vfbavcqmdksm6as8fvskpxiw-tar-1.34/bin:/gnu/store/x24bm49ag5dvki72mjdz195bfb89nrnb-gzip-1.12/bin:/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/bin:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/bin:/gnu/store/zmcf5kpqiighkbh7wslf91qdjwj06yr1-diffutils-3.8/bin:/gnu/store/210yfax18r2g2inxrml9435ikhfcca6m-patch-2.7.6/bin:/gnu/store/c8jyph2lxw0m9na34fg8h70n4nnnz7is-findutils-4.9.0/bin:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/bin:/gnu/store/xxcfsimvxz7z4dj593gnqbkzc6picwzq-sed-4.8/bin:/gnu/store/yrv5f70mn83a876b78i5s79dd2hsh0zf-grep-3.8/bin:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/bin:/gnu/store/a5i8avx826brw5grn3n4qv40g514505c-coreutils-9.1/bin:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/bin:/gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin:/gnu/store/na1dpbbcxjaa3n8wkwrfpch476f90hlf-ld-wrapper-0/bin:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/bin:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/bin:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/bin:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/sbin'
environment variable `BASH_LOADABLES_PATH' unset
environment variable `C_INCLUDE_PATH' set to 
`/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/include:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/include:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/include:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/include:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/include:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/include:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/include:/gnu/store/5iklcps70c0sfkxvlrhg8jhf3q4h18bj-linux-libre-headers-5.15.49/include'
environment variable `CPLUS_INCLUDE_PATH' set to 
`/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/include:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/include:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/include:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/include:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/include:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/include:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include/c++:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/include:/gnu/store/5iklcps70c0sfkxvlrhg8jhf3q4h18bj-linux-libre-headers-5.15.49/include'
environment variable `LIBRARY_PATH' set to 
`/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/lib:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/lib:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/lib:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/lib:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/lib:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib:/gnu/store/l0yryi5jsa1grnvw01c9nkz9c81cv224-glibc-2.35-static/lib:/gnu/store/visfdda934gvivwihwhlm63fdqhhcc8a-glibc-utf8-locales-2.35/lib'
environment variable `GUIX_LOCPATH' set to 
`/gnu/store/visfdda934gvivwihwhlm63fdqhhcc8a-glibc-utf8-locales-2.35/lib/locale'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
using 'en_US.utf8' locale for category "LC_ALL"
phase `install-locale' succeeded after 0.0 seconds
starting phase `patch-usr-bin-file'
phase `patch-usr-bin-file' succeeded after 0.0 seconds
starting phase `patch-source-shebangs'
phase `patch-source-shebangs' succeeded after 0.0 seconds
starting phase `patch-generated-file-shebangs'
phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
starting phase `defconfig'
   CONF  defconfig
***
*** Can't find default configuration "defconfig"!
***
make: *** 
[/gnu/store/dv009i1wj865n34dyln2fmxryxqg3z7c-crosstool-ng-1.26.0/bin/ct-ng:145: 
defconfig] Error 1
error: in phase 'defconfig': uncaught exception:
%exception #<&invoke-error program: "ct-ng" arguments: ("defconfig") 
exit-status: 2 term-signal: #f stop-signal: #f>
phase `defconfig' failed after 0.1 seconds
command "ct-ng" "defconfig" failed with status 2
builder for 
`/gnu/store/f454kx4v789af03s2rz6nzlznhla9i6x-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
failed with exit code 1
While executing meta-command:
ERROR:
   1. &store-protocol-error:
       message: "build of 
`/gnu/store/f454kx4v789af03s2rz6nzlznhla9i6x-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
failed"
       status: 100


The ct-ng tool that is invoked from the build-system is saying that it 
can't find the defconfig file.

Richard,
 From your debugging tips you mentioned looking at the mentioned ".drv" 
file and the associated "-builder" file. In the builder file I can see:

   (crosstool-ng-build #:name "ct-ng-riscv64-unknown-elf-1.26.0" 
#:source "/gnu/store/651xmndi8wshqc7lrgiw9b59mh0nygxj-defconfig"

Which is interesting, I guess I assumed that the plainfile that I 
specified in the package declaration would be put in a build directory 
or something. I see that it is in the store, which makes sense.

I think my main question here then is how can I get the defconfig file 
from the store into the build directory so that I can invoke commands in 
the build system like other build systems do? The difference between 
other packages is that it seems that other packages have a git repo or 
something for package sources and the associated build system can invoke 
commands on the files there. I want to be able to just provide a partial 
defconfig file and operate on that.

Thanks for any pointers,
Jesse


On 3/9/24 10:02, Jesse Millwood wrote:
> Thanks for your response Richard!
>
> On 3/8/24 22:39, Richard Sent wrote:
>> Jesse,
>>
>> I expect your issue is you have an extra gexp in
>> build-system/crosstool-ng.scm, line 71. I determined this by:
>>
>> 1. Examining the derivation Guix is trying to build
> I'm still trying to learn about the internals of Guix and writing 
> packages. How did you examine the derivation Guix is trying to build?
>> 2. Examining the -builder file mentioned in the derivation
> Ah! I did not notice that file in the output nor open it! I see now 
> that the Derivations section of the manual mentions these drv files. 
> Is this where you are examining the derivation that Guix is trying to 
> build?
>> 3. Observe that ct-ng-riscv64-unknown-elf-1.26.0-builder wraps
>> (crosstool-ng-build ...) in (gexp).
>
> Where is this? Are you talking of this in build-system/crosstool-ng:
>
>   (define build
>     (with-imported-modules
>      imported-modules
>      #~(begin
>          (use-modules #$@(sexp->gexp modules))
>          #~(crosstool-ng-build #:name #$name
>
> Where "#~", I have learned from Marek (earlier in this thread), is an 
> alias for "gexp".
>
> Ah, I see the ct-ng-riscv64-unknown-elf-1.26.0-builder in the drv file 
> and wow, I didn't know this existed! I see the gexp wrapping that you 
> mention.
>
> Is there a way to use the guix command to tell me where the drv file 
> for other packages exist? It is apparent that I need to perform a more 
> in depth review of the manual. This really helps contextualize things 
> for me.
>
>> The package still did not build locally on my machine after removing it,
>> but it seems to have progressed to a further stage.
>>
> Thanks, I may investigate how other derivations look now that I know 
> where to look. I appreciate the detailed input!

Re: Help with channel build system and package

[Jesse Millwood]

I actually think I figured it out!

I ended up adding a build phase that copies the plain-file defined as 
the source in the package object to the build directory. I'm finally at 
a point where I'm just having build failures of the package itself, 
which I'm a little more familiar debugging.

Thanks for all the help!

On 3/16/24 17:47, Jesse Millwood wrote:
> Richard and Marek,
>
> The input from both of you helped me get a little farther. I've pushed 
> my changes to https://codeberg.org/moonling/guix-embedded-dev.
>
> I feel like I'm really close now. I seem to be able to get into the 
> phases of my build system. I can not invoke the crosstool-ng tool. My 
> issue is that it seems to not be able to see the defconfig that I 
> define in the package declaration as a "plain-file".
>
> I've determined this through the repl. In my repo there is a scheme 
> file called "build-repl.scm" and I invoke it like this:
>
> ➜ guix repl -L guix-embedded-dev
> GNU Guile 3.0.9
> Copyright (C) 1995-2023 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guix-user)> ,l 
> /home/jesse/Code/guix-repos/guix-embedded-dev/build-repl.scm
> WARNING: (embedded-dev packages crosstool-ng): `which' imported from 
> both (guix build utils) and (gnu packages base)
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> substitute: updating substitutes from 
> 'https://bordeaux.guix.gnu.org'... 100.0%
> building path(s) 
> `/gnu/store/nc892p9kgx0my3gg9glyszg848hpf166-ct-ng-riscv64-unknown-elf-1.26.0'
> starting phase `set-SOURCE-DATE-EPOCH'
> phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
> starting phase `set-paths'
> environment variable `PATH' set to 
> `/gnu/store/dv009i1wj865n34dyln2fmxryxqg3z7c-crosstool-ng-1.26.0/bin:/gnu/store/sxx22f98vfbavcqmdksm6as8fvskpxiw-tar-1.34/bin:/gnu/store/x24bm49ag5dvki72mjdz195bfb89nrnb-gzip-1.12/bin:/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/bin:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/bin:/gnu/store/zmcf5kpqiighkbh7wslf91qdjwj06yr1-diffutils-3.8/bin:/gnu/store/210yfax18r2g2inxrml9435ikhfcca6m-patch-2.7.6/bin:/gnu/store/c8jyph2lxw0m9na34fg8h70n4nnnz7is-findutils-4.9.0/bin:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/bin:/gnu/store/xxcfsimvxz7z4dj593gnqbkzc6picwzq-sed-4.8/bin:/gnu/store/yrv5f70mn83a876b78i5s79dd2hsh0zf-grep-3.8/bin:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/bin:/gnu/store/a5i8avx826brw5grn3n4qv40g514505c-coreutils-9.1/bin:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/bin:/gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16/bin:/gnu/store/na1dpbbcxjaa3n8wkwrfpch476f90hlf-ld-wrapper-0/bin:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/bin:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/bin:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/bin:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/sbin'
> environment variable `BASH_LOADABLES_PATH' unset
> environment variable `C_INCLUDE_PATH' set to 
> `/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/include:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/include:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/include:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/include:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/include:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/include:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/include:/gnu/store/5iklcps70c0sfkxvlrhg8jhf3q4h18bj-linux-libre-headers-5.15.49/include'
> environment variable `CPLUS_INCLUDE_PATH' set to 
> `/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/include:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/include:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/include:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/include:/gnu/store/wj7casda7rb55rvqjnpm0bm7a2zm6618-make-4.3/include:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/include:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include/c++:/gnu/store/5lqhcv91ijy82p92ac6g5xw48l0lwwz4-gcc-11.3.0/include:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/include:/gnu/store/5iklcps70c0sfkxvlrhg8jhf3q4h18bj-linux-libre-headers-5.15.49/include'
> environment variable `LIBRARY_PATH' set to 
> `/gnu/store/j8wlfmlmfvpbza6is9wv9xsd8psrxn00-bzip2-1.0.8/lib:/gnu/store/gr0sy0m1mv36qv54idm6cn10l3mngshq-file-5.44/lib:/gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1/lib:/gnu/store/6k1yys9wqrfn4y41ic1win8gpnimncwj-xz-5.2.8/lib:/gnu/store/zh4x65snfis7svs6906gj1z8i7dx2j3m-binutils-2.38/lib:/gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35/lib:/gnu/store/l0yryi5jsa1grnvw01c9nkz9c81cv224-glibc-2.35-static/lib:/gnu/store/visfdda934gvivwihwhlm63fdqhhcc8a-glibc-utf8-locales-2.35/lib'
> environment variable `GUIX_LOCPATH' set to 
> `/gnu/store/visfdda934gvivwihwhlm63fdqhhcc8a-glibc-utf8-locales-2.35/lib/locale'
> phase `set-paths' succeeded after 0.0 seconds
> starting phase `install-locale'
> using 'en_US.utf8' locale for category "LC_ALL"
> phase `install-locale' succeeded after 0.0 seconds
> starting phase `patch-usr-bin-file'
> phase `patch-usr-bin-file' succeeded after 0.0 seconds
> starting phase `patch-source-shebangs'
> phase `patch-source-shebangs' succeeded after 0.0 seconds
> starting phase `patch-generated-file-shebangs'
> phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
> starting phase `defconfig'
>   CONF  defconfig
> ***
> *** Can't find default configuration "defconfig"!
> ***
> make: *** 
> [/gnu/store/dv009i1wj865n34dyln2fmxryxqg3z7c-crosstool-ng-1.26.0/bin/ct-ng:145: 
> defconfig] Error 1
> error: in phase 'defconfig': uncaught exception:
> %exception #<&invoke-error program: "ct-ng" arguments: ("defconfig") 
> exit-status: 2 term-signal: #f stop-signal: #f>
> phase `defconfig' failed after 0.1 seconds
> command "ct-ng" "defconfig" failed with status 2
> builder for 
> `/gnu/store/f454kx4v789af03s2rz6nzlznhla9i6x-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
> failed with exit code 1
> While executing meta-command:
> ERROR:
>   1. &store-protocol-error:
>       message: "build of 
> `/gnu/store/f454kx4v789af03s2rz6nzlznhla9i6x-ct-ng-riscv64-unknown-elf-1.26.0.drv' 
> failed"
>       status: 100
>
>
> The ct-ng tool that is invoked from the build-system is saying that it 
> can't find the defconfig file.
>
> Richard,
> From your debugging tips you mentioned looking at the mentioned ".drv" 
> file and the associated "-builder" file. In the builder file I can see:
>
>   (crosstool-ng-build #:name "ct-ng-riscv64-unknown-elf-1.26.0" 
> #:source "/gnu/store/651xmndi8wshqc7lrgiw9b59mh0nygxj-defconfig"
>
> Which is interesting, I guess I assumed that the plainfile that I 
> specified in the package declaration would be put in a build directory 
> or something. I see that it is in the store, which makes sense.
>
> I think my main question here then is how can I get the defconfig file 
> from the store into the build directory so that I can invoke commands 
> in the build system like other build systems do? The difference 
> between other packages is that it seems that other packages have a git 
> repo or something for package sources and the associated build system 
> can invoke commands on the files there. I want to be able to just 
> provide a partial defconfig file and operate on that.
>
> Thanks for any pointers,
> Jesse
>
>
> On 3/9/24 10:02, Jesse Millwood wrote:
>> Thanks for your response Richard!
>>
>> On 3/8/24 22:39, Richard Sent wrote:
>>> Jesse,
>>>
>>> I expect your issue is you have an extra gexp in
>>> build-system/crosstool-ng.scm, line 71. I determined this by:
>>>
>>> 1. Examining the derivation Guix is trying to build
>> I'm still trying to learn about the internals of Guix and writing 
>> packages. How did you examine the derivation Guix is trying to build?
>>> 2. Examining the -builder file mentioned in the derivation
>> Ah! I did not notice that file in the output nor open it! I see now 
>> that the Derivations section of the manual mentions these drv files. 
>> Is this where you are examining the derivation that Guix is trying to 
>> build?
>>> 3. Observe that ct-ng-riscv64-unknown-elf-1.26.0-builder wraps
>>> (crosstool-ng-build ...) in (gexp).
>>
>> Where is this? Are you talking of this in build-system/crosstool-ng:
>>
>>   (define build
>>     (with-imported-modules
>>      imported-modules
>>      #~(begin
>>          (use-modules #$@(sexp->gexp modules))
>>          #~(crosstool-ng-build #:name #$name
>>
>> Where "#~", I have learned from Marek (earlier in this thread), is an 
>> alias for "gexp".
>>
>> Ah, I see the ct-ng-riscv64-unknown-elf-1.26.0-builder in the drv 
>> file and wow, I didn't know this existed! I see the gexp wrapping 
>> that you mention.
>>
>> Is there a way to use the guix command to tell me where the drv file 
>> for other packages exist? It is apparent that I need to perform a 
>> more in depth review of the manual. This really helps contextualize 
>> things for me.
>>
>>> The package still did not build locally on my machine after removing 
>>> it,
>>> but it seems to have progressed to a further stage.
>>>
>> Thanks, I may investigate how other derivations look now that I know 
>> where to look. I appreciate the detailed input!