bug#36878: guix system reconfigure broken

bug#36878: guix system reconfigure broken

[Robert Vollmert]

Hi,

it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012

  guix system: Reimplement 'reconfigure’.

breaks guix system reconfigure. In particular, after reconfiguring,
shepherd doesn’t know about the updated versions of services.

The usual output below is missing; after reverting the commit it’s
fine again.

guix system: loading new services: …
To complete the upgrade, run 'herd restart SERVICE' to stop,
upgrade, and restart each service that was not automatically restarted.
shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?).
shepherd: Service user-homes has been started.
shepherd: Service term-auto could not be started.
bootloader successfully installed on '/dev/sda’

I see that some system tests for “guix system reconfigure” were added
after this change. Might I suggest adding them before the change next
time around, and making sure they pass both before and after?

Cheers
Robert

Re: bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

> Hi,
>
> it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012
>
>   guix system: Reimplement 'reconfigure’.
>
> breaks guix system reconfigure. In particular, after reconfiguring,
> shepherd doesn’t know about the updated versions of services.
>
> The usual output below is missing; after reverting the commit it’s
> fine again.

Could you please elaborate on what you mean by this? I can see that the
ouput has changed, but is Shepherd doing something that indicates that
it doesn't know about the new services? And is the line regarding
bootloader installation also missing from 'guix system reconfigure'
post-5c8c8c455?

> I see that some system tests for “guix system reconfigure” were added
> after this change. Might I suggest adding them before the change next
> time around, and making sure they pass both before and after?

Sure. My apologies for not having the foresight to do that.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#36878: guix system reconfigure broken

[Robert Vollmert]

> On 31. Jul 2019, at 18:45, Jakob L. Kreuze <zerodaysfordays@sdf.lonestar.org> wrote:
> Robert Vollmert <rob@vllmrt.net> writes:
> 
>> Hi,
>> 
>> it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012
>> 
>>  guix system: Reimplement 'reconfigure’.
>> 
>> breaks guix system reconfigure. In particular, after reconfiguring,
>> shepherd doesn’t know about the updated versions of services.
>> 
>> The usual output below is missing; after reverting the commit it’s
>> fine again.
> 
> Could you please elaborate on what you mean by this? I can see that the
> ouput has changed, but is Shepherd doing something that indicates that
> it doesn't know about the new services?

The concrete problem is this:

1. nginx is running with config file A
2. make some change to nginx config
3. run guix system reconfigure (which builds a new nginx config file B)
4. run herd restart nginx
5. nginx is still running with config file A

After reverting this commit, if I perform these steps, I end up with
nginx running with config file B in 5.

> And is the line regarding
> bootloader installation also missing from 'guix system reconfigure'
> post-5c8c8c455?

I still get a bootloader line.

Cheers
Robert

Re: bug#36878: guix system reconfigure broken

[Robert Vollmert]

> On 31. Jul 2019, at 18:45, Jakob L. Kreuze <zerodaysfordays@sdf.lonestar.org> wrote:
> Robert Vollmert <rob@vllmrt.net> writes:
> 
>> Hi,
>> 
>> it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012
>> 
>>  guix system: Reimplement 'reconfigure’.
>> 
>> breaks guix system reconfigure. In particular, after reconfiguring,
>> shepherd doesn’t know about the updated versions of services.
>> 
>> The usual output below is missing; after reverting the commit it’s
>> fine again.
> 
> Could you please elaborate on what you mean by this? I can see that the
> ouput has changed, but is Shepherd doing something that indicates that
> it doesn't know about the new services?

The concrete problem is this:

1. nginx is running with config file A
2. make some change to nginx config
3. run guix system reconfigure (which builds a new nginx config file B)
4. run herd restart nginx
5. nginx is still running with config file A

After reverting this commit, if I perform these steps, I end up with
nginx running with config file B in 5.

> And is the line regarding
> bootloader installation also missing from 'guix system reconfigure'
> post-5c8c8c455?

I still get a bootloader line.

Cheers
Robert

bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

> The concrete problem is this:
>
> 1. nginx is running with config file A
> 2. make some change to nginx config
> 3. run guix system reconfigure (which builds a new nginx config file B)
> 4. run herd restart nginx
> 5. nginx is still running with config file A
>
> After reverting this commit, if I perform these steps, I end up with
> nginx running with config file B in 5.

Okay, I see now. Thank you clarifying. Would you be willing to see if
the patch provided by #36880 fixes this issue?

> I still get a bootloader line.

Great -- thanks for checking that for me!

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

> The concrete problem is this:
>
> 1. nginx is running with config file A
> 2. make some change to nginx config
> 3. run guix system reconfigure (which builds a new nginx config file B)
> 4. run herd restart nginx
> 5. nginx is still running with config file A
>
> After reverting this commit, if I perform these steps, I end up with
> nginx running with config file B in 5.

Okay, I see now. Thank you clarifying. Would you be willing to see if
the patch provided by #36880 fixes this issue?

> I still get a bootloader line.

Great -- thanks for checking that for me!

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[Robert Vollmert]

> On 31. Jul 2019, at 20:16, Jakob L. Kreuze <zerodaysfordays@sdf.lonestar.org> wrote:
> 
> Hi Robert,
> 
> Robert Vollmert <rob@vllmrt.net> writes:
> 
>> The concrete problem is this:
>> 
>> 1. nginx is running with config file A
>> 2. make some change to nginx config
>> 3. run guix system reconfigure (which builds a new nginx config file B)
>> 4. run herd restart nginx
>> 5. nginx is still running with config file A
>> 
>> After reverting this commit, if I perform these steps, I end up with
>> nginx running with config file B in 5.
> 
> Okay, I see now. Thank you clarifying. Would you be willing to see if
> the patch provided by #36880 fixes this issue?

Yes, it seems it does!

bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

Robert Vollmert <rob@vllmrt.net> writes:

> Yes, it seems it does!

Great :)

CC'ing Chris and Dave since I don't have write access -- can we
fast-track #36880 into master? I'd consider this to be a high-priority
bug fix.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

Robert Vollmert <rob@vllmrt.net> writes:

> Yes, it seems it does!

Great :)

CC'ing Chris and Dave since I don't have write access -- can we
fast-track #36880 into master? I'd consider this to be a high-priority
bug fix.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 366 bytes --]

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

> Great :)
>
> CC'ing Chris and Dave since I don't have write access -- can we
> fast-track #36880 into master? I'd consider this to be a high-priority
> bug fix.
>
> Regards,
> Jakob

The patch made it into master as 1db6f137d; thanks to Danny for signing
off on it. Should we close this?

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 366 bytes --]

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:

> Great :)
>
> CC'ing Chris and Dave since I don't have write access -- can we
> fast-track #36880 into master? I'd consider this to be a high-priority
> bug fix.
>
> Regards,
> Jakob

The patch made it into master as 1db6f137d; thanks to Danny for signing
off on it. Should we close this?

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

bug#36878: guix system reconfigure broken

[ison]

On Fri, Aug 02, 2019 at 11:11:31AM -0400, Jakob L. Kreuze wrote:
> The patch made it into master as 1db6f137d; thanks to Danny for signing
> off on it. Should we close this?

I'm still experiencing this issue. "guix describe" shows that I'm using
commit 35600cd which should be newer than the fix.

I made the mistake of rebooting my machine after I first got this error, I
thought perhaps it was benign. But when it came back up I was dropped to a
GRUB rescue shell with some error about "grub_file_filters" being an unknown
symbol.

When running grub commands like "insmod normal" would just repeat the error
or say command not found.
The only way I could think to repair it was to run the Guix install disk,
mount my partitions, and do another "guix init" which allows me to boot into
my machine again. However, when I run "guix pull" and then perform a
reconfigure I still keep seeing the same line:
shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?)

I've tried this on several occasions over the last few days with different
commits following the supposed fix, and it still keeps doing the same thing,
and every time I follow the reconfigure with a reboot GRUB is broken again.

Re: bug#36878: guix system reconfigure broken

[Ricardo Wurmus]

ison <ison@airmail.cc> writes:

> I made the mistake of rebooting my machine after I first got this error, I
> thought perhaps it was benign. But when it came back up I was dropped to a
> GRUB rescue shell with some error about "grub_file_filters" being an unknown
> symbol.

This eems unrelated.  The problem sounds like GRUB and its modules
aren’t found for some reason.

> When running grub commands like "insmod normal" would just repeat the error
> or say command not found.
> The only way I could think to repair it was to run the Guix install disk,
> mount my partitions, and do another "guix init" which allows me to boot into
> my machine again. However, when I run "guix pull" and then perform a
> reconfigure I still keep seeing the same line:
> shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?)

I reconfigured a workstation yesterday and also saw this line.  It does
look like an error, but it seems to be completely harmless.  I could
boot juts fine.  I suppose that’s just the part where new service
definitions are loaded.

The problem you’re seeing is unrelated to this output.

-- 
Ricardo

bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1832 bytes --]

Hi ison and Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> ison <ison@airmail.cc> writes:
>
>> I made the mistake of rebooting my machine after I first got this
>> error, I thought perhaps it was benign. But when it came back up I
>> was dropped to a GRUB rescue shell with some error about
>> "grub_file_filters" being an unknown symbol.
>
> This eems unrelated. The problem sounds like GRUB and its modules
> aren’t found for some reason.

It is unrelated to this specific issue, but it still does sound like my
patch is the source of the problem. Would you be willing to open a
separate ticket for tracking this, ison?

>> When running grub commands like "insmod normal" would just repeat the error
>> or say command not found.
>> The only way I could think to repair it was to run the Guix install disk,
>> mount my partitions, and do another "guix init" which allows me to boot into
>> my machine again. However, when I run "guix pull" and then perform a
>> reconfigure I still keep seeing the same line:
>> shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?)
>
> I reconfigured a workstation yesterday and also saw this line.  It does
> look like an error, but it seems to be completely harmless.  I could
> boot juts fine.  I suppose that’s just the part where new service
> definitions are loaded.
>
> The problem you’re seeing is unrelated to this output.

Yes, this output is normal. Shepherd loads service files by evaluating
Scheme expressions. I could have sworn that 'guix system reconfigure'
showed the same line prior to my patch.

ison -- I'm very sorry that you've had this experience. Would you be
willing to share the 'bootloader' field of your operating system
configuration so I can look into this?

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[ison]

On Mon, Aug 05, 2019 at 10:59:57AM -0400, Jakob L. Kreuze wrote:
> ison -- I'm very sorry that you've had this experience. Would you be
> willing to share the 'bootloader' field of your operating system
> configuration so I can look into this?

Thanks for taking an interest. Here is the new bug report:
https://issues.guix.gnu.org/issue/36942

Please let me know if there's any other information or testing I
can do to help figure it out.

bug#36878: guix system reconfigure broken

[ison]

On Mon, Aug 05, 2019 at 10:59:57AM -0400, Jakob L. Kreuze wrote:
> ison -- I'm very sorry that you've had this experience. Would you be
> willing to share the 'bootloader' field of your operating system
> configuration so I can look into this?

Thanks for taking an interest. Here is the new bug report:
https://issues.guix.gnu.org/issue/36942

Please let me know if there's any other information or testing I
can do to help figure it out.

bug#36878: guix system reconfigure broken

[Ricardo Wurmus]

ison <ison@airmail.cc> writes:

> I made the mistake of rebooting my machine after I first got this error, I
> thought perhaps it was benign. But when it came back up I was dropped to a
> GRUB rescue shell with some error about "grub_file_filters" being an unknown
> symbol.

This eems unrelated.  The problem sounds like GRUB and its modules
aren’t found for some reason.

> When running grub commands like "insmod normal" would just repeat the error
> or say command not found.
> The only way I could think to repair it was to run the Guix install disk,
> mount my partitions, and do another "guix init" which allows me to boot into
> my machine again. However, when I run "guix pull" and then perform a
> reconfigure I still keep seeing the same line:
> shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?)

I reconfigured a workstation yesterday and also saw this line.  It does
look like an error, but it seems to be completely harmless.  I could
boot juts fine.  I suppose that’s just the part where new service
definitions are loaded.

The problem you’re seeing is unrelated to this output.

-- 
Ricardo

bug#36878: guix system reconfigure broken

[Ludovic Courtès]

Hello,

zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) skribis:

> zerodaysfordays@sdf.lonestar.org (Jakob L. Kreuze) writes:
>
>> Great :)
>>
>> CC'ing Chris and Dave since I don't have write access -- can we
>> fast-track #36880 into master? I'd consider this to be a high-priority
>> bug fix.
>>
>> Regards,
>> Jakob
>
> The patch made it into master as 1db6f137d; thanks to Danny for signing
> off on it. Should we close this?

I gather that this particular issue is fixed, so closing the bug now!

Ludo’.

bug#36878: guix system reconfigure broken

[Robert Vollmert]

> On 31. Jul 2019, at 20:16, Jakob L. Kreuze <zerodaysfordays@sdf.lonestar.org> wrote:
> 
> Hi Robert,
> 
> Robert Vollmert <rob@vllmrt.net> writes:
> 
>> The concrete problem is this:
>> 
>> 1. nginx is running with config file A
>> 2. make some change to nginx config
>> 3. run guix system reconfigure (which builds a new nginx config file B)
>> 4. run herd restart nginx
>> 5. nginx is still running with config file A
>> 
>> After reverting this commit, if I perform these steps, I end up with
>> nginx running with config file B in 5.
> 
> Okay, I see now. Thank you clarifying. Would you be willing to see if
> the patch provided by #36880 fixes this issue?

Yes, it seems it does!

bug#36878: guix system reconfigure broken

[Jakob L. Kreuze]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

Hi Robert,

Robert Vollmert <rob@vllmrt.net> writes:

> Hi,
>
> it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012
>
>   guix system: Reimplement 'reconfigure’.
>
> breaks guix system reconfigure. In particular, after reconfiguring,
> shepherd doesn’t know about the updated versions of services.
>
> The usual output below is missing; after reverting the commit it’s
> fine again.

Could you please elaborate on what you mean by this? I can see that the
ouput has changed, but is Shepherd doing something that indicates that
it doesn't know about the new services? And is the line regarding
bootloader installation also missing from 'guix system reconfigure'
post-5c8c8c455?

> I see that some system tests for “guix system reconfigure” were added
> after this change. Might I suggest adding them before the change next
> time around, and making sure they pass both before and after?

Sure. My apologies for not having the foresight to do that.

Regards,
Jakob

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: bug#36878: guix system reconfigure broken

[Alex Sassmannshausen]

Thanks for reporting this, I experienced the same issue but blamed it on
my own infrastructure instead!

Good to hear it's resolved now.

Alex

Robert Vollmert <rob@vllmrt.net> writes:

> Hi,
>
> it appears that commit 5c8c8c455420af27189d6045b3599fe6e27ad012
>
>   guix system: Reimplement 'reconfigure’.
>
> breaks guix system reconfigure. In particular, after reconfiguring,
> shepherd doesn’t know about the updated versions of services.
>
> The usual output below is missing; after reverting the commit it’s
> fine again.
>
> guix system: loading new services: …
> To complete the upgrade, run 'herd restart SERVICE' to stop,
> upgrade, and restart each service that was not automatically restarted.
> shepherd: Evaluating user expression (let* ((services (map primitive-load (?))) # ?) ?).
> shepherd: Service user-homes has been started.
> shepherd: Service term-auto could not be started.
> bootloader successfully installed on '/dev/sda’
>
> I see that some system tests for “guix system reconfigure” were added
> after this change. Might I suggest adding them before the change next
> time around, and making sure they pass both before and after?
>
> Cheers
> Robert