How to repair/reinstall guix on foreign distro

How to repair/reinstall guix on foreign distro

[Thorsten Wilms]

Hi! It seems every few weeks i’m thrown into a initramfs prompt, as
root got mounted read-only. The way out is always running fsck, which
reports errors and offers fixes that I either can’t map to anything, or
that refer to paths that belong to guix. Usually only below /gnu/store,
but this time also var/guix.

Now `guix pull` fails with:
```
;;; WARNING: loading compiled
file /gnu/store/8bsnz1fk330qbn1p8k18i0j11vld4jxd-guix-module-union/lib/guile/3.0/site-ccache/guix/build-system/gnu.go
failed: ;;; In procedure load-thunk-from-memory: not an ELF file
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'... guix pull: error: Git
error: invalid data in index - incorrect header signature
```

At first, I focused on the warning (sorry, leoprikler) ... the same git
error happens when using an older guix.

Now a fix for that would be great, but independent of that, I have to
wonder how one should go about reinstalling guix on a foreign distro. I
cannot umount or remount and remove /gnu/store!

Even after `sudo systemctl stop gnu-store.mount`, /gnu/store is busy.
`fuser -kim /gnu/store/` will throw me out of the session, even though
I would think everything that belongs to it is part of the host (Ubuntu
Unity 20.10). No sudo umount or remount or rm -rf allowed, /gnu/store
remains ro and busy. Short of booting another system (say an USB
stick), is there a way to clear /gnu/store? Or to safely replace its
content?


-- 
Thorsten Wilms <t_w_@freenet.de>

Re: How to repair/reinstall guix on foreign distro

[Joshua Branson]

Thorsten Wilms <t_w_@freenet.de> writes:

> Hi! It seems every few weeks i’m thrown into a initramfs prompt, as
> root got mounted read-only. The way out is always running fsck, which
> reports errors and offers fixes that I either can’t map to anything, or
> that refer to paths that belong to guix. Usually only below /gnu/store,
> but this time also var/guix.

Is ubuntu causing this issue?  Why are you seeing an initramfs prompt
every few weeks?  If you are running guix on ubuntu...that seems like
it's ubuntu's fault.  Though I'm probably way wrong.

>
> Now `guix pull` fails with:
> ```
> ;;; WARNING: loading compiled
> file /gnu/store/8bsnz1fk330qbn1p8k18i0j11vld4jxd-guix-module-union/lib/guile/3.0/site-ccache/guix/build-system/gnu.go
> failed: ;;; In procedure load-thunk-from-memory: not an ELF file
> Updating channel 'guix' from Git repository at
> 'https://git.savannah.gnu.org/git/guix.git'... guix pull: error: Git
> error: invalid data in index - incorrect header signature
> ```

Man...I've no idea.  I'm honestly responding just so you get a
response.  Why do you suppose you are getting into an initramfs...As in
you always have to run fsck?  What hardware are you using?

> At first, I focused on the warning (sorry, leoprikler) ... the same git
> error happens when using an older guix.
>
> Now a fix for that would be great, but independent of that, I have to
> wonder how one should go about reinstalling guix on a foreign distro. I
> cannot umount or remount and remove /gnu/store!
>
> Even after `sudo systemctl stop gnu-store.mount`, /gnu/store is busy.
> `fuser -kim /gnu/store/` will throw me out of the session, even though
> I would think everything that belongs to it is part of the host (Ubuntu
> Unity 20.10). No sudo umount or remount or rm -rf allowed, /gnu/store
> remains ro and busy. Short of booting another system (say an USB
> stick), is there a way to clear /gnu/store? Or to safely replace its
> content?

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: How to repair/reinstall guix on foreign distro

[Thorsten Wilms]

On Mon, 17 May 2021 22:23:03 -0400
Joshua Branson <jbranso@dismail.de> wrote:

> Is ubuntu causing this issue?  Why are you seeing an initramfs prompt
> every few weeks?  If you are running guix on ubuntu...that seems like
> it's ubuntu's fault.  Though I'm probably way wrong.

If there are filesystem errors on boot, you get a initramfs shell that
allows running fsck on root for repairs. Sometimes, some files can’t be
fixed. 

The system resides on a Transcend 256GB MTE110S SSD. Now one would
probably assume that SSD is faulty. What’s curious is that so far, any
and all noticeable corruption has been limited to guix-related files!
smartctl repports "overall-health self-assessment test result: PASSED"
and has "No Errors Logged". Full output attached at the end.

It does however claim that all except one power cylce has been an unsafe
shutdown, even though I had only a few hard resets and usually shutdown
by issuing `poweroff`. Searching the web for this issue led me to
turning off fast boot in the BIOS, but that didn’t help.


I would still like to have a minimal invasive way to get beyond:
```
guix pull: error: Git error: invalid data in index - incorrect header
signature
```
Or a way to to remove/replace /gnu/store from the running system.


```
~: sudo smartctl -x /dev/nvme0n1p2
[sudo] password for thorwil: 
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.8.0-53-generic] (local
build) Copyright (C) 2002-19, Bruce Allen, Christian Franke,
www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       TS256GMTE110S
Serial Number:                      G286710001
Firmware Version:                   S0905C3
PCI Vendor/Subsystem ID:            0x126f
IEEE OUI Identifier:                0x000000
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          256.060.514.304 [256 GB]
Namespace 1 Formatted LBA Size:     512
Local Time is:                      Tue May 18 09:23:26 2021 CEST
Firmware Updates (0x12):            1 Slot, no Reset required
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x001f):     Comp Wr_Unc DS_Mngmt Wr_Zero
Sav/Sel_Feat Maximum Data Transfer Size:         64 Pages
Warning  Comp. Temp. Threshold:     83 Celsius
Critical Comp. Temp. Threshold:     90 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.00W       -        -    0  0  0  0        0       0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        48 Celsius
Available Spare:                    88%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    1.136.204 [581 GB]
Data Units Written:                 1.901.390 [973 GB]
Host Read Commands:                 15.204.795
Host Write Commands:                18.097.210
Controller Busy Time:               1.386
Power Cycles:                       276
Power On Hours:                     1.214
Unsafe Shutdowns:                   275
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
```

-- 
Thorsten Wilms <t_w_@freenet.de>

Re: How to repair/reinstall guix on foreign distro

[Joshua Branson]

Thorsten Wilms <t_w_@freenet.de> writes:

> On Mon, 17 May 2021 22:23:03 -0400
> Joshua Branson <jbranso@dismail.de> wrote:
>
>> Is ubuntu causing this issue?  Why are you seeing an initramfs prompt
>> every few weeks?  If you are running guix on ubuntu...that seems like
>> it's ubuntu's fault.  Though I'm probably way wrong.
>
> If there are filesystem errors on boot, you get a initramfs shell that
> allows running fsck on root for repairs. Sometimes, some files can’t be
> fixed. 
>
> The system resides on a Transcend 256GB MTE110S SSD. Now one would
> probably assume that SSD is faulty. What’s curious is that so far, any
> and all noticeable corruption has been limited to guix-related files!
> smartctl repports "overall-health self-assessment test result: PASSED"
> and has "No Errors Logged". Full output attached at the end.
>
> It does however claim that all except one power cylce has been an unsafe
> shutdown, even though I had only a few hard resets and usually shutdown
> by issuing `poweroff`. Searching the web for this issue led me to
> turning off fast boot in the BIOS, but that didn’t help.
>
> I would still like to have a minimal invasive way to get beyond:
> ```
> guix pull: error: Git error: invalid data in index - incorrect header
> signature
> ```
> Or a way to to remove/replace /gnu/store from the running system.

If you are convinced on using guix on a foreign distro...then you could
create a separate partition for /gnu/store and /var/guix.  Then mount
those at boot time.   That way at least, you could not have to manually
run fsck at an initramfs.  You could do it when you've logged into
ubuntu's gnome (or whatever it is that you use.)

I wish I could be more helpful.

-- 
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: How to repair/reinstall guix on foreign distro

[Ricardo Wurmus]

Thorsten Wilms <t_w_@freenet.de> writes:

> It does however claim that all except one power cylce has been 
> an unsafe
> shutdown, even though I had only a few hard resets and usually 
> shutdown
> by issuing `poweroff`.

That’s worrying.  “unsafe shutdown” leads to file system 
corruption, and Guix doesn’t like that at all.

> I would still like to have a minimal invasive way to get beyond:
> ```
> guix pull: error: Git error: invalid data in index - incorrect 
> header
> signature
> ```

Try erasing the git cache at ~/.cache/guix/checkouts first.

-- 
Ricardo

Re: How to repair/reinstall guix on foreign distro

[Thorsten Wilms]

On Tue, 18 May 2021 11:20:41 +0200
Ricardo Wurmus <rekado@elephly.net> wrote:

> > I would still like to have a minimal invasive way to get beyond:
> > ```
> > guix pull: error: Git error: invalid data in index - incorrect 
> > header
> > signature
> > ```
> 
> Try erasing the git cache at ~/.cache/guix/checkouts first.

Thank you, that did the trick!


-- 
Thorsten Wilms <t_w_@freenet.de>

Re: How to repair/reinstall guix on foreign distro

[c4droid]

Thorsten Wilms writes:

> Hi! It seems every few weeks i’m thrown into a initramfs prompt, as
> root got mounted read-only. The way out is always running fsck, which
> reports errors and offers fixes that I either can’t map to anything, or
> that refer to paths that belong to guix. Usually only below /gnu/store,
> but this time also var/guix.
>
> Now `guix pull` fails with:
> ```
> ;;; WARNING: loading compiled
> file /gnu/store/8bsnz1fk330qbn1p8k18i0j11vld4jxd-guix-module-union/lib/guile/3.0/site-ccache/guix/build-system/gnu.go
> failed: ;;; In procedure load-thunk-from-memory: not an ELF file
> Updating channel 'guix' from Git repository at
> 'https://git.savannah.gnu.org/git/guix.git'... guix pull: error: Git
> error: invalid data in index - incorrect header signature
> ```
>
> At first, I focused on the warning (sorry, leoprikler) ... the same git
> error happens when using an older guix.
>
> Now a fix for that would be great, but independent of that, I have to
> wonder how one should go about reinstalling guix on a foreign distro. I
> cannot umount or remount and remove /gnu/store!
>
> Even after `sudo systemctl stop gnu-store.mount`, /gnu/store is busy.
> `fuser -kim /gnu/store/` will throw me out of the session, even though
> I would think everything that belongs to it is part of the host (Ubuntu
> Unity 20.10). No sudo umount or remount or rm -rf allowed, /gnu/store
> remains ro and busy. Short of booting another system (say an USB
> stick), is there a way to clear /gnu/store? Or to safely replace its
> content?

Hi, Thorsten

I was install guix on a ubuntu server and work right, can you reproduce
you guix installation process? maybe the fault happen in install guix
process.

------

Best reguards,
c4droid

Re: How to repair/reinstall guix on foreign distro

[c4droid]

Thorsten Wilms writes:

> Hi! It seems every few weeks i’m thrown into a initramfs prompt, as
> root got mounted read-only. The way out is always running fsck, which
> reports errors and offers fixes that I either can’t map to anything, or
> that refer to paths that belong to guix. Usually only below /gnu/store,
> but this time also var/guix.
>
> Now `guix pull` fails with:
> ```
> ;;; WARNING: loading compiled
> file /gnu/store/8bsnz1fk330qbn1p8k18i0j11vld4jxd-guix-module-union/lib/guile/3.0/site-ccache/guix/build-system/gnu.go
> failed: ;;; In procedure load-thunk-from-memory: not an ELF file
> Updating channel 'guix' from Git repository at
> 'https://git.savannah.gnu.org/git/guix.git'... guix pull: error: Git
> error: invalid data in index - incorrect header signature
> ```
>
> At first, I focused on the warning (sorry, leoprikler) ... the same git
> error happens when using an older guix.
>
> Now a fix for that would be great, but independent of that, I have to
> wonder how one should go about reinstalling guix on a foreign distro. I
> cannot umount or remount and remove /gnu/store!
>
> Even after `sudo systemctl stop gnu-store.mount`, /gnu/store is busy.
> `fuser -kim /gnu/store/` will throw me out of the session, even though
> I would think everything that belongs to it is part of the host (Ubuntu
> Unity 20.10). No sudo umount or remount or rm -rf allowed, /gnu/store
> remains ro and busy. Short of booting another system (say an USB
> stick), is there a way to clear /gnu/store? Or to safely replace its
> content?

Hi, Thorsten

I was install guix on a ubuntu server and work right, can you reproduce
you guix installation process? maybe the fault happen in install guix
process.

------

Best reguards,
c4droid

Re: How to repair/reinstall guix on foreign distro

[Thorsten Wilms]

On Tue, 18 May 2021 10:30:03 +0800
c4droid <c4droid@foxmail.com> wrote:

> I was install guix on a ubuntu server and work right, can you
> reproduce you guix installation process? maybe the fault happen in
> install guix process.

Thanks, but this has been a working guix installation, so I very much
doubt something went wrong during setup.

-- 
Thorsten Wilms <t_w_@freenet.de>