ld-wrapper and funny bytes in LIBRARY_PATH

ld-wrapper and funny bytes in LIBRARY_PATH

[Danny Milosavljevic]

Hi,

so I'm trying to enable Rust tests and one of their tests does the following (paraphrased) in order to exercise the non-UTF8 linker case (which they want to succeed):

----------------------------------------------
bad_dir := zzz$$'\xff'

all:
        mkdir $(bad_dir)
	cp ... $(bad_dir)/liblibrary.a
	LIBRARY_PATH=$(bad_dir) rustc exec.rs
----------------------------------------------

This fails for us.

strace yields:

[pid 15059] open("/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a", O_RDONLY) = -1 ENOENT (No such file or directory)

But:

/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8 [env]$ ls zzz\377/liblibrary.a 
'zzz'$'\377''/liblibrary.a'

$ ls /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a
'/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz'$'\377''/liblibrary.a'

How do we best proceed?

Re: ld-wrapper and funny bytes in LIBRARY_PATH

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> so I'm trying to enable Rust tests and one of their tests does the following (paraphrased) in order to exercise the non-UTF8 linker case (which they want to succeed):
>
> ----------------------------------------------
> bad_dir := zzz$$'\xff'
>
> all:
>         mkdir $(bad_dir)
> 	cp ... $(bad_dir)/liblibrary.a
> 	LIBRARY_PATH=$(bad_dir) rustc exec.rs
> ----------------------------------------------
>
> This fails for us.

So they expect ‘ld’ to properly deal with UTF-8 file names, right?

> strace yields:
>
> [pid 15059] open("/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a", O_RDONLY) = -1 ENOENT (No such file or directory)
>
> But:
>
> /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8 [env]$ ls zzz\377/liblibrary.a 
> 'zzz'$'\377''/liblibrary.a'
>
> $ ls /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a
> '/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz'$'\377''/liblibrary.a'
>
> How do we best proceed?

This is another instance of the file name decoding problem with Guile.
As you know, Guile 2.0/2.2 decodes file names according to the current
locale, and ‘ld-wrapper’ runs Guile.

However, on Guile 2.0 (current ‘master’) ‘ld-wrapper’ runs in the C
locale because it does not explicitly call ‘setlocale’, hence the
question mark instead of \377 in the file name shown by strace.

Fortunately, Guile 2.2 programs (current ‘core-updates’) always start in
the current locale.  So I think the problem won’t exist in
‘core-updates’, as long as the build runs in a UTF-8 locale (which is
the case.)

HTH!

Ludo’.

Re: ld-wrapper and funny bytes in LIBRARY_PATH

[Danny Milosavljevic]

Hi Ludo,

On Sun, 18 Jun 2017 00:21:28 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Danny Milosavljevic <dannym@scratchpost.org> skribis:
> 
> > so I'm trying to enable Rust tests and one of their tests does the following (paraphrased) in order to exercise the non-UTF8 linker case (which they want to succeed):
> >
> > ----------------------------------------------
> > bad_dir := zzz$$'\xff'
> >
> > all:
> >         mkdir $(bad_dir)
> > 	cp ... $(bad_dir)/liblibrary.a
> > 	LIBRARY_PATH=$(bad_dir) rustc exec.rs
> > ----------------------------------------------
> >
> > This fails for us.  
> 
> So they expect ‘ld’ to properly deal with UTF-8 file names, right?

I think they want ld to properly deal with random file names.  0xFF is not even valid UTF-8.

See also:
- https://github.com/rust-lang/rust/issues/29122 (initial bug report)
- https://github.com/rust-lang/rust/pull/29134 (pull request)
- https://github.com/wthrowe/rust/commit/19664fdf8d1b8fc5fd3b359c2440625b2b4d8cec (test)

> This is another instance of the file name decoding problem with Guile.
> As you know, Guile 2.0/2.2 decodes file names according to the current
> locale, and ‘ld-wrapper’ runs Guile.

Sigh...

Well, I've posted a patch to guix-patches that disables the Rust test in question.  Can't do much else than that.

The other 15444 Rust tests work now :)

Re: ld-wrapper and funny bytes in LIBRARY_PATH

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Sun, 18 Jun 2017 00:21:28 +0200
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>> 
>> > so I'm trying to enable Rust tests and one of their tests does the following (paraphrased) in order to exercise the non-UTF8 linker case (which they want to succeed):
>> >
>> > ----------------------------------------------
>> > bad_dir := zzz$$'\xff'
>> >
>> > all:
>> >         mkdir $(bad_dir)
>> > 	cp ... $(bad_dir)/liblibrary.a
>> > 	LIBRARY_PATH=$(bad_dir) rustc exec.rs
>> > ----------------------------------------------
>> >
>> > This fails for us.  
>> 
>> So they expect ‘ld’ to properly deal with UTF-8 file names, right?
>
> I think they want ld to properly deal with random file names.  0xFF is not even valid UTF-8.
>
> See also:
> - https://github.com/rust-lang/rust/issues/29122 (initial bug report)
> - https://github.com/rust-lang/rust/pull/29134 (pull request)
> - https://github.com/wthrowe/rust/commit/19664fdf8d1b8fc5fd3b359c2440625b2b4d8cec (test)

OK.

>> This is another instance of the file name decoding problem with Guile.
>> As you know, Guile 2.0/2.2 decodes file names according to the current
>> locale, and ‘ld-wrapper’ runs Guile.
>
> Sigh...

Yeah.  :-/

> Well, I've posted a patch to guix-patches that disables the Rust test in question.  Can't do much else than that.
>
> The other 15444 Rust tests work now :)

Awesome, thank you!

Ludo’.

PS: Your email address has been bouncing lately.