bug#49985: bash-mesboot0: Inscrutable error in build phase

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Carl Dong]

Hi all,

While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299

The error line is L1299: "make: stat:Makefile: sterror: unknown error”

This failure persists even when ran with --cores=1, across reboots. I’ve even tried building that derivation with --check on my own machine (which successfully built that derivation), and every log line matched (plus or minus some timing log lines) up until L1299.

We’ve also tried rebuilding bash-mesboot0-2.05b.drv’s dependency /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv with:

--8<---------------cut here---------------start------------->8---
guix gc --delete /gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0
guix build --cores=1 /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv
--8<---------------cut here---------------end--------------->8---

Which did not make a difference.

What is interesting is that this is not the first time this problem has cropped up for people. I see 2 past mentions of this exact error:
- Mentioned by kagevf on Dec 16th, 2020: https://logs.guix.gnu.org/guix/2020-12-16.log#043712
- Mentioned by genr8_ (with help from rekado_) on Mar 22nd, 2021: https://logs.guix.gnu.org/guix/2021-03-22.log#182406

I’m happy to spend energy investigating, but would love any pointers on what the most promising place to look is!

My friend's system is an Intel i5 system, running Guix 1.3.0 on Ubuntu. His /tmp is on the same partition as / and is ext4.

Cheers,
Carl Dong
contact@carldong.me
"I fight for the users"

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Bengt Richter]

On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
> Hi all,
> 
> While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
> 
> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
>--

To me, sterror looks like a typo for strerror
so grepping for sterror might find the typo, if that's what it is?

Regards,
Bengt Richter

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Bengt Richter]

On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
> On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
> > Hi all,
> > 
> > While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
> > 
> > The error line is L1299: "make: stat:Makefile: sterror: unknown error”
> >--
> 
> To me, sterror looks like a typo for strerror
> so grepping for sterror might find the typo, if that's what it is?
>
Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
 MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
(in git repo) doesn't like.

Then run something like my error-printing kludge that will work on your system:

I called it errno-grep
--8<---------------cut here---------------start------------->8---
#!/usr/bin/bash
if [ -z "$1" ];then
    echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
    echo "    /usr/include/asm-generic/errno-base.h"
    echo "    /usr/include/asm-generic/errno.h"
    exit 0
fi

if [ "${1:0:1}" == "-" ]; then
    exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
else
    exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
fi
--8<---------------cut here---------------end--------------->8---

HTH.

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Bengt Richter]

Just this disclaimer:
I am just a lurker interested in mes, not a mes developer or team member.
(I am trying to do some minimal stuff too, so I cloned the mes repo and pull once
in a while to see what's going on :)

Sorry for any implication that I really know anything about mes ;/

On +2021-08-11 03:21:18 +0200, Bengt Richter wrote:
> On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
> > On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
> > > Hi all,
> > > 
> > > While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
> > > 
> > > The error line is L1299: "make: stat:Makefile: sterror: unknown error”
> > >--
> > 
> > To me, sterror looks like a typo for strerror
> > so grepping for sterror might find the typo, if that's what it is?
> >
> Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
>  MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
> (in git repo) doesn't like.
> 
> Then run something like my error-printing kludge that will work on your system:
> 
> I called it errno-grep
> --8<---------------cut here---------------start------------->8---
> #!/usr/bin/bash
> if [ -z "$1" ];then
>     echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
>     echo "    /usr/include/asm-generic/errno-base.h"
>     echo "    /usr/include/asm-generic/errno.h"
>     exit 0
> fi
> 
> if [ "${1:0:1}" == "-" ]; then
>     exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
> else
>     exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
> fi
> --8<---------------cut here---------------end--------------->8---
> 
> HTH.
> 
> 
> 

-- 
Regards,
Bengt Richter

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Carl Dong]

No worries! We’re all learning :-)
I ended up asking my colleague to use:

strace --env=MES_DEBUG=1

We’ll see what happens!

> On Aug 11, 2021, at 8:53 AM, Bengt Richter <bokr@bokr.com> wrote:
> 
> Just this disclaimer:
> I am just a lurker interested in mes, not a mes developer or team member.
> (I am trying to do some minimal stuff too, so I cloned the mes repo and pull once
> in a while to see what's going on :)
> 
> Sorry for any implication that I really know anything about mes ;/
> 
> On +2021-08-11 03:21:18 +0200, Bengt Richter wrote:
>> On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
>>> On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
>>>> Hi all,
>>>> 
>>>> While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
>>>> 
>>>> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
>>>> --
>>> 
>>> To me, sterror looks like a typo for strerror
>>> so grepping for sterror might find the typo, if that's what it is?
>>> 
>> Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
>> MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
>> (in git repo) doesn't like.
>> 
>> Then run something like my error-printing kludge that will work on your system:
>> 
>> I called it errno-grep
>> --8<---------------cut here---------------start------------->8---
>> #!/usr/bin/bash
>> if [ -z "$1" ];then
>>    echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
>>    echo "    /usr/include/asm-generic/errno-base.h"
>>    echo "    /usr/include/asm-generic/errno.h"
>>    exit 0
>> fi
>> 
>> if [ "${1:0:1}" == "-" ]; then
>>    exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
>> else
>>    exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
>> fi
>> --8<---------------cut here---------------end--------------->8---
>> 
>> HTH.
>> 
>> 
>> 
> 
> -- 
> Regards,
> Bengt Richter

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Mathieu Othacehe]

Hello Carl,

> The error line is L1299: "make: stat:Makefile: sterror: unknown error”

This reminds me of:
https://lists.gnu.org/archive/html/bug-guix/2020-05/msg00335.html.

I never took the time to fix this issue. Bottom line is that building
the bootstrap toolchain fails on NVME disks because some syscalls
(stat64, lstat64 and fstat64) need to be implemented in GNU Mes.

There's a small demonstration program that you can use to demonstrate
this theory :).

Thanks,

Mathieu

bug#49985: bash-mesboot0: Inscrutable error in build phase

[Carl Dong]

Mathieu,

I think this was exactly the problem, because mounting a tmpfs at /tmp solved it. Thanks for your help!

Cheers,
Carl Dong

> On Aug 19, 2021, at 12:20 PM, Mathieu Othacehe <othacehe@gnu.org> wrote:
> 
> 
> Hello Carl,
> 
>> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
> 
> This reminds me of:
> https://lists.gnu.org/archive/html/bug-guix/2020-05/msg00335.html.
> 
> I never took the time to fix this issue. Bottom line is that building
> the bootstrap toolchain fails on NVME disks because some syscalls
> (stat64, lstat64 and fstat64) need to be implemented in GNU Mes.
> 
> There's a small demonstration program that you can use to demonstrate
> this theory :).
> 
> Thanks,
> 
> Mathieu