shepherd GEXP module import mystery

shepherd GEXP module import mystery

[Attila Lendvai]

dear Guix,

context: i'm adding logging to shepherd, and while i was testing it i encountered a conflict with my service code that shouldn't happen.

i'm probably missing something from my mental model around guile modules, and/or how shepherd compiles and loads them.

the symptom is that a start GEXP of my service sees bindings that come from the module called (shepherd support), but i have no idea who and where imports that module.

i added the following two format lines to the beginning of my start GEXP:

(format #t "*** start gexp speaking, current module is ~A, module-uses: ~A~%  ringbuffer: ~A~%"
  (current-module)
  (module-uses (current-module))
  (and=> (module-variable (current-module) 'ring-buffer) variable-ref))

(format #t "*** ringbuffer in packages: ~A~%"
  (map (lambda (module-name)
         (and=> (module-variable (resolve-interface module-name)
                                 'ringbuffer)
                variable-ref))
       '((guile) (oop goops) (shepherd service) (srfi srfi-34)
         (system repl error-handling) (guix build utils) (guix build syscalls)
         (gnu build file-systems))))

and they print:

*** start gexp speaking,
current module is #<module (#{ g108}#) 7f6a3af34be0>
module-uses: (
#<interface (guile) 7f6a3ade6dc0>
#<interface (oop goops) 7f6a399eac80>
#<interface (shepherd service) 7f6a39489a00>
#<interface (srfi srfi-34) 7f6a399ea8c0>
#<interface (system repl error-handling) 7f6a39489dc0>
#<custom-interface (guix build utils) 7f6a3af34140>
#<interface (guix build syscalls) 7f6a38cd6e60>
#<custom-interface (guix build utils) 7f6a38cd6500>
#<custom-interface (gnu build file-systems) 7f6a38d6a460>
#<interface (gnu system file-systems) 7f6a38d24b40>
#<custom-interface (guix build utils) 7f6a38d6a3c0>
#<custom-interface (gnu build file-systems) 7f6a38d6a320>
#<custom-interface (guix build utils) 7f6a38d6a280>
#<custom-interface (gnu build file-systems) 7f6a38d6a1e0>
#<custom-interface (guix build utils) 7f6a38d6a140>
#<custom-interface (gnu build file-systems) 7f6a38d6a0a0>
#<custom-interface (guix build utils) 7f6a38d6a000>
#<custom-interface (gnu build file-systems) 7f6a38d93f0

  ringbuffer: #<procedure ring-buffer (size)> 

*** ringbuffer in packages: (#f #f #f #f #f #f #f #f) 

how else can a definition (i.e. 'ringbuffer) get into this module without it coming from one of the modules in its module-uses list?

i'm pretty sure that it's coming from (shepherd support) because i'm neck deep in debugging the original issue: a macro from (shepherd support) overwrote my function that errored when it was called at runtime as a function.

i'm also seeing this warning (i.e. my root issue):

WARNING: (#{ g108}#): `log.debug' imported from both (guix-crypto utils) and (shepherd support)

i've checked the module list of the gexp, and how guix compiles the .go files that are then given to shepherd, and i see nothing obvious. i even looked at the source file that gets generated and compiled for the service in question, and it doesn't contain any mention of this module.

there are no direct mentions of (shepherd support) in the source, that's why i thought maybe something re-exports the entire module, so i checked the presence of 'ringbuffer in all the used modules... but it's not in any of them.

could it be a bug in how different versions/instances of guile serialize/load a .go file? or could it be due to the module magic in (gnu services shepherd) that compiles the shepherd config into a .go file?

i'm out of ideas, any hint is appreciated!

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“People always find partners [with] exactly the same level of unresolved traumas they are at, without any exception. Now the trauma may look differently, and how it manifests may look differently, but the degree of trauma is always equal, I mean with perfect accuracy. […] And that trauma then shows up in the relationship.”
	— Gábor Máté (1944–), 'On The Spot - Az ellenség gyermekei', 48m50s

Re: shepherd GEXP module import mystery

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Attila,

On Mon, Nov 27 2023, Attila Lendvai wrote:

> a start GEXP of my service sees bindings that come from the module
> called (shepherd support), but i have no idea who and where imports
> that module.

Without code it's hazardous to speculate, but could the Guix service
(gnu service mcron) cause that issue when it is being logged?

In my code tree, which is a month behind, (gnu services mcron) is the
only Guix service that imports (shepherd support).

Kind regards
Felix

Re: shepherd GEXP module import mystery

[Attila Lendvai]

[-- Attachment #1: Type: text/plain, Size: 4870 bytes --]

hi Felix, Ludo,


> > a start GEXP of my service sees bindings that come from the module
> > called (shepherd support), but i have no idea who and where imports
> > that module.
> 
> 
> Without code it's hazardous to speculate, but could the Guix service
> (gnu service mcron) cause that issue when it is being logged?


unfortunately it's a complex web of stuff, but i managed to make a small reproducer that is attaced.

it can be run with:

$(guix system --no-graphic vm reproducer.scm)

and in the VM (must use fold, because it's a dumb terminal):

cat /var/log/messages | fold -150

to my surprise this one does list (shepherd support) in the module-use list. and i realized why: the logging infrastructure somewhere siently truncates the lines, and in my original case that module was chopped off.

not that i understand everything here... e.g. why are there several (guix build utils) modules?

*** reproducer gexp speaking, current module: #<module (#{g107}#) 7f5b49cf61e0>,
module-uses: (
#<interface (guile) 7f5b4b652dc0>
#<interface (oop goops) 7f5b4a256c80>
#<interface (shepherd service) 7f5b49cf3c80>
#<interface (srfi srfi-34)
#<interface (system repl error-handling) 7f5b49d9ab40> #<custom-interface (guix build utils) 7f5b49d9a140>
#<interface (guix build syscalls) 7f5b49495e60> #<custom-interface (guix build utils) 7f5b49495500> #<custom-interface (gnu build file-systems) 7f5b49523460>
#<interface (gnu system file-systems) 7f5b494deb40>
#<custom-interface (guix build utils) 7f5b495233c0>
#<custom-interface (gnu build file-systems) 7f5b49523320>
#<custom-interface (guix build utils) 7f5b49523280>
#<custom-interface (gnu build file-systems) 7f5b495231e0>
#<custom-interface (guix build utils) 7f5b49523140>
#<custom-interface (gnu build file-systems) 7f5b495230a0>
#<custom-interface (guix build utils) 7f5b49523000>
#<custom-interface (guix build utils) 7f5b4954df00>
#<custom-interface (guix build utils) 7f5b4954de60>
#<custom-interface (guix build utils) 7f5b4954ddc0>
#<interface (gnu build activation) 7f5b4954db40>
#<interface (gnu system accounts) 7f5b4954d8c0>
#<custom-interface (guix build utils) 7f5b4b5e6000>
#<custom-interface (guix build utils) 7f5b49453f00>
#<interface (gnu build linux-boot) 7f5b4b5e6e60>
#<custom-interface (guix build utils) 7f5b49453e60>
#<interface (ice-9 popen) 7f5b49453be0>
#<interface (ice-9 rdelim) 7f5b4a210f00>
#<interface (ice-9 match) 7f5b4b652460>
#<custom-interface (guix build utils) 7f5b49453b40>
#<interface (srfi srfi-1) 7f5b4b692320>
#<interface (gnu build shepherd) 7f5b494538c0>
#<interface (rnrs bytevectors) 7f5b4b6def00>
#<interface (rnrs io ports) 7f5b49d18dc0>
#<custom-interface (guix build utils) 7f5b49453140>
#<custom-interface (guix build utils) 7f5b494530a0>
#<custom-interface (guix build utils) 7f5b49453000>
#<custom-interface (guix build utils) 7f5b497f6f00>
#<custom-interface (guix build utils) 7f5b497f6e60>
#<custom-interface (guix build utils) 7f5b497f6dc0>
#<custom-interface (guix build utils) 7f5b497f6d20>
#<custom-interface (guix build utils) 7f5b497f6c80>
#<custom-interface (guix build utils) 7f5b497f6be0>
#<custom-interface (guix build utils) 7f5b497f6b40>
#<custom-interface (guix build utils) 7f5b497f6aa0>
#<custom-interface (guix build utils) 7f5b497f6a00>
#<custom-interface (guix build utils) 7f5b497f6960>
#<custom-interface (guix build utils) 7f5b497f68c0>
#<custom-interface (guix build utils) 7f5b497f6820>
#<custom-interface (guix build utils) 7f5b497f6780>
#<custom-interface (guix build utils) 7f5b497f66e0>
#<interface (srfi srfi-26) 7f5b4a64ff00>
#<custom-interface (shepherd support) 7f5b497f6640>
#<custom-interface (guix build utils) 7f5b497f65a0>
#<interface (ip addr) 7f5b497f6280>
#<interface (ip link) 7f5b497f6000>
#<interface (ip route) 7f5b49772820>
#<interface (ice-9 format) 7f5b4b7b86e0>),
ringbuffer: #<procedure ring-buffer (size)>


so, the only mystery left is that i still don't know where it is imported into the unnamed package in which the GEXPs are compiled/loaded, and whether that is intended.

maybe it's part of the shepherd API that (shepherd support) is made available for the service GEXPs? looking at the public definitions in (shepherd support), it's not obvious that those are meant to be available for the users of shepherd, though.

Ludo?


> In my code tree, which is a month behind, (gnu services mcron) is the
> only Guix service that imports (shepherd support).


it's a good hint, but that could only cause this if all the service GEXPs were loaded into the same module, but that would have already broken things in countless other ways.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“We are products of our past, but we don't have to be prisoners of it.”
	— Rick Warren (1954–)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: reproducer.scm --]
[-- Type: text/x-scheme; name=reproducer.scm, Size: 1528 bytes --]

;; Run with something like this:
;; $(guix system --no-graphic vm reproducer.scm)

(define-module (reproducer)
  #:use-module (gnu system)
  #:use-module (gnu system shadow)
  #:use-module (gnu system nss)
  #:use-module (gnu system vm)
  #:use-module (gnu tests)
  #:use-module (gnu services)
  #:use-module (gnu services base)
  #:use-module (gnu services dbus)
  #:use-module (gnu services shepherd)
  #:use-module (gnu packages admin)
  #:use-module (gnu packages base)
  #:use-module (gnu packages bash)
  #:use-module (gnu packages certs)
  #:use-module (gnu packages package-management)
  #:use-module (gnu packages linux)
  #:use-module (guix gexp)
  #:use-module (guix git)
  #:use-module (guix git-download)
  #:use-module (guix store)
  #:use-module (guix modules)
  #:use-module (guix packages)
  #:use-module (srfi srfi-1)
  #:use-module (ice-9 match))

(operating-system
  (inherit %simple-os)
  (services
   (cons*
    (simple-service
     'reproducer
     shepherd-root-service-type
     (list
      (shepherd-service
       (requirement '(file-systems))
       (provision '(reproducer))
       (documentation "")
       (start
        #~(begin
            (lambda _
              (format #t "*** reproducer gexp speaking, \
current module: ~A, \
module-uses: ~A, \
ringbuffer: ~A~%"
                      (current-module)
                      (module-uses (current-module))
                      (and=> (module-variable (current-module) 'ring-buffer) variable-ref))
              0))))))
    %base-services)))

Re: shepherd GEXP module import mystery

[Attila Lendvai]

> so, the only mystery left is that i still don't know where it is
> imported into the unnamed package in which the GEXPs are
> compiled/loaded, and whether that is intended.

FTR, i've filed it as:

https://issues.guix.gnu.org/67649

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“What is history but the story of how politicians have squandered the blood and treasure of the human race?”
	— Thomas Sowell (1930–)