all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / Atom feed
* How to debug failed boot after successful reconfigure?
@ 2021-04-08 15:58 Vladilen Kozin
  2021-04-09 11:53 ` Vladilen Kozin
  0 siblings, 1 reply; 3+ messages in thread
From: Vladilen Kozin @ 2021-04-08 15:58 UTC (permalink / raw)
  To: help-guix

Hi list.

I'm trying to wrap my head around running guix. While question in subj
is general it was prompted by a real failure, so really I am
interested both in "general approach" as well as solving the actual
problem I ran into.

having updated with `guix pull` I run `sudo guix system reconfigure
config.scm` which succeeds. I reboot and the boot fails attempting to
mount the new file-systems I've introduced in that config. This being
guix I reboot into older generation. However, now I am unsure how to
debug what'd happened. E.g. I would like to:
- look at failed dmesg or equivalent log,
- look at files that reconfigure produced in /etc, e.g. /etc/fstab

IIUC when I boot even from an older generation, the failed one will be
marked as "current" (that is the newest reconfigured) but everything
in /etc appears to belong to the older generation - hardly surprising.
Following link shown by `guix system describe` in my case
/var/guix/profiles/system-2-link I don't see e.g. /etc/fstab I'm
interested in. Where are those files?

So, the questions above (maybe others I should've asked?) amount to my
asking, how to debug the failure.

My specific case was adding these files-systems to my os declaration:
(file-system
    (mount-point "/mnt/tempb")
    (device (file-system-label "tempb"))
    (type "ext4")
    #;(flags '(no-atime))
    (options "defaults,noatime,discard,user")
    (create-mount-point? #t))

At boot time I got multiple errors from mount unable to figure the
options supplied. This in itself is puzzling seeing how I did check
/etc/fstab after reconfigure and was able to manually mount as per
that generated /etc/fstab without issue. One note is that I think the
(create-mount-point? #t) had not been honored, but I did create those
mount points manually. Dunno, might be a bug in guix proper. But
that's minor.

Thanks

-- 
Best regards
Vlad Kozin


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: How to debug failed boot after successful reconfigure?
  2021-04-08 15:58 How to debug failed boot after successful reconfigure? Vladilen Kozin
@ 2021-04-09 11:53 ` Vladilen Kozin
  2021-04-09 19:36   ` Julien Lepiller
  0 siblings, 1 reply; 3+ messages in thread
From: Vladilen Kozin @ 2021-04-09 11:53 UTC (permalink / raw)
  To: help-guix

Hm, bit puzzled. Noone ever encountered boot failures with Guix?
Should I've sent it to guix-devel?

I'm going to try my luck in a separate thread with a much narrower question.

FWIW for the issue at hand I ended up leveraging the fact that I'm
running on a server with out-of-band ethernet and IPMI on board.
Essentially, I ended up configuring that server to communicate Serial
over Lan, then reconfigured Guix with Grub and Kernel both doing IO to
serial console, then connected over SSH while running inside a Tmux
session ... so now I can scroll up if boot fails and see exactly what
happened. This would probably only work on an actual enterprise hw
that lets you do IPMI SOL. For anyone interested here're relevant
parts of my config.scm:

;; make sure kernel IO to serial both com1 and com2
(kernel-arguments '("console=tty1 console=ttyS0,115200n8
console=ttyS1,115200n8"))

;; make sure grub IO goes over serial both com1 and com2
(bootloader
  (bootloader-configuration
   (bootloader grub-bootloader)
   ;; TODO well that's nuts given that plugging in USB drive changes
   ;; order of block devices
   (target "/dev/sda")
   (keyboard-layout keyboard-layout)
   (terminal-inputs '(console serial_0 serial_1))
   (terminal-outputs '(console serial_0 serial_1))))

Seriously, noone knows where to find dmesg and contents of /etc that
pertain to the last attempt to boot specific generation?
Pretty please?

On Thu, 8 Apr 2021 at 16:58, Vladilen Kozin <vladilen.kozin@gmail.com> wrote:
>
> Hi list.
>
> I'm trying to wrap my head around running guix. While question in subj
> is general it was prompted by a real failure, so really I am
> interested both in "general approach" as well as solving the actual
> problem I ran into.
>
> having updated with `guix pull` I run `sudo guix system reconfigure
> config.scm` which succeeds. I reboot and the boot fails attempting to
> mount the new file-systems I've introduced in that config. This being
> guix I reboot into older generation. However, now I am unsure how to
> debug what'd happened. E.g. I would like to:
> - look at failed dmesg or equivalent log,
> - look at files that reconfigure produced in /etc, e.g. /etc/fstab
>
> IIUC when I boot even from an older generation, the failed one will be
> marked as "current" (that is the newest reconfigured) but everything
> in /etc appears to belong to the older generation - hardly surprising.
> Following link shown by `guix system describe` in my case
> /var/guix/profiles/system-2-link I don't see e.g. /etc/fstab I'm
> interested in. Where are those files?
>
> So, the questions above (maybe others I should've asked?) amount to my
> asking, how to debug the failure.
>
> My specific case was adding these files-systems to my os declaration:
> (file-system
>     (mount-point "/mnt/tempb")
>     (device (file-system-label "tempb"))
>     (type "ext4")
>     #;(flags '(no-atime))
>     (options "defaults,noatime,discard,user")
>     (create-mount-point? #t))
>
> At boot time I got multiple errors from mount unable to figure the
> options supplied. This in itself is puzzling seeing how I did check
> /etc/fstab after reconfigure and was able to manually mount as per
> that generated /etc/fstab without issue. One note is that I think the
> (create-mount-point? #t) had not been honored, but I did create those
> mount points manually. Dunno, might be a bug in guix proper. But
> that's minor.
>
> Thanks
>
> --
> Best regards
> Vlad Kozin



-- 
Best regards
Vlad Kozin


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: How to debug failed boot after successful reconfigure?
  2021-04-09 11:53 ` Vladilen Kozin
@ 2021-04-09 19:36   ` Julien Lepiller
  0 siblings, 0 replies; 3+ messages in thread
From: Julien Lepiller @ 2021-04-09 19:36 UTC (permalink / raw)
  To: help-guix, Vladilen Kozin

Hi, help-guix is the right mailing, but we're not always very responsive, plus there is a delay for your first mail to pass spam filter.

In /var/guix/profiles/system-<n>-link (where n corresponds to the generation you wart to inspect), you will hind an etc directory.

dmesg accumulates in /var/log/messages, I don't think there's a way to find it for a given generation, but you should find the relevant lines by looking at the time they were registered.

HTH!

Le 9 avril 2021 07:53:16 GMT-04:00, Vladilen Kozin <vladilen.kozin@gmail.com> a écrit :
>Hm, bit puzzled. Noone ever encountered boot failures with Guix?
>Should I've sent it to guix-devel?
>
>I'm going to try my luck in a separate thread with a much narrower
>question.
>
>FWIW for the issue at hand I ended up leveraging the fact that I'm
>running on a server with out-of-band ethernet and IPMI on board.
>Essentially, I ended up configuring that server to communicate Serial
>over Lan, then reconfigured Guix with Grub and Kernel both doing IO to
>serial console, then connected over SSH while running inside a Tmux
>session ... so now I can scroll up if boot fails and see exactly what
>happened. This would probably only work on an actual enterprise hw
>that lets you do IPMI SOL. For anyone interested here're relevant
>parts of my config.scm:
>
>;; make sure kernel IO to serial both com1 and com2
>(kernel-arguments '("console=tty1 console=ttyS0,115200n8
>console=ttyS1,115200n8"))
>
>;; make sure grub IO goes over serial both com1 and com2
>(bootloader
>  (bootloader-configuration
>   (bootloader grub-bootloader)
>   ;; TODO well that's nuts given that plugging in USB drive changes
>   ;; order of block devices
>   (target "/dev/sda")
>   (keyboard-layout keyboard-layout)
>   (terminal-inputs '(console serial_0 serial_1))
>   (terminal-outputs '(console serial_0 serial_1))))
>
>Seriously, noone knows where to find dmesg and contents of /etc that
>pertain to the last attempt to boot specific generation?
>Pretty please?
>
>On Thu, 8 Apr 2021 at 16:58, Vladilen Kozin <vladilen.kozin@gmail.com>
>wrote:
>>
>> Hi list.
>>
>> I'm trying to wrap my head around running guix. While question in
>subj
>> is general it was prompted by a real failure, so really I am
>> interested both in "general approach" as well as solving the actual
>> problem I ran into.
>>
>> having updated with `guix pull` I run `sudo guix system reconfigure
>> config.scm` which succeeds. I reboot and the boot fails attempting to
>> mount the new file-systems I've introduced in that config. This being
>> guix I reboot into older generation. However, now I am unsure how to
>> debug what'd happened. E.g. I would like to:
>> - look at failed dmesg or equivalent log,
>> - look at files that reconfigure produced in /etc, e.g. /etc/fstab
>>
>> IIUC when I boot even from an older generation, the failed one will
>be
>> marked as "current" (that is the newest reconfigured) but everything
>> in /etc appears to belong to the older generation - hardly
>surprising.
>> Following link shown by `guix system describe` in my case
>> /var/guix/profiles/system-2-link I don't see e.g. /etc/fstab I'm
>> interested in. Where are those files?
>>
>> So, the questions above (maybe others I should've asked?) amount to
>my
>> asking, how to debug the failure.
>>
>> My specific case was adding these files-systems to my os declaration:
>> (file-system
>>     (mount-point "/mnt/tempb")
>>     (device (file-system-label "tempb"))
>>     (type "ext4")
>>     #;(flags '(no-atime))
>>     (options "defaults,noatime,discard,user")
>>     (create-mount-point? #t))
>>
>> At boot time I got multiple errors from mount unable to figure the
>> options supplied. This in itself is puzzling seeing how I did check
>> /etc/fstab after reconfigure and was able to manually mount as per
>> that generated /etc/fstab without issue. One note is that I think the
>> (create-mount-point? #t) had not been honored, but I did create those
>> mount points manually. Dunno, might be a bug in guix proper. But
>> that's minor.
>>
>> Thanks
>>
>> --
>> Best regards
>> Vlad Kozin
>
>
>
>-- 
>Best regards
>Vlad Kozin

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-04-09 19:39 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-08 15:58 How to debug failed boot after successful reconfigure? Vladilen Kozin
2021-04-09 11:53 ` Vladilen Kozin
2021-04-09 19:36   ` Julien Lepiller

all messages for Guix-related lists mirrored at yhetil.org

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/guix

Example config snippet for mirrors.


AGPL code for this site: git clone http://ou63pmih66umazou.onion/public-inbox.git