bug#48591: ugrep: illegal instruction

bug#48591: ugrep: illegal instruction

[Christopher Howard]

[-- Attachment #1: Type: text/plain, Size: 2192 bytes --]

Hi, I found another package, ugrep, which is evidently using native
instructions that do not work on my processor. When I run ugrep (e.g.,
`ugrep -Q') it immediately dies with error 'illegal instruction'.
However, if I pull the source with `guix build -S' and build and
install it to a local directory, it runs without trouble.

I'm sure this will also be a reproducibility bug, and plan on
submitting guix challenge results. However, the command I am running to
rebuild ugrep without grafts or substitutes is rebuilding everything
all the way back to bootstrap, so it probably will not be completed for
a day or two:

guix gc -D $(guix build --no-grafts ugrep); guix build --no-substitutes 
ugrep

Please compare similar bug 48373. It didn't seem worth the trouble of
running an object dump and figuring out which instruction is illegal,
but I could do that if required.

My system information:


christopher@nightshade ~$ grep 'flags' /proc/cpuinfo | head -n 1
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext
fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl
nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm
cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch
osvw ibs skinit wdt hw_pstate vmmcall npt lbrv svm_lock nrip_save

christopher@nightshade ~$ neofetch --stdout
christopher@nightshade 
---------------------- 
OS: Guix System d4ffa9630277fa8699c783c08381d688626d4bc3 x86_64 
Host: GA-880GM-UD2H 
Kernel: 5.12.5-gnu 
Uptime: 1 hour, 30 mins 
Packages: 107 (guix-system), 93 (guix-user) 
Shell: bash 5.0.16 
Resolution: 1920x1200 
DE: GNOME 3.34.5 
Theme: Adwaita [GTK2/3] 
Icons: Adwaita [GTK2/3] 
Terminal: kitty 
CPU: AMD Athlon II X3 455 (3) @ 3.300GHz 
GPU: NVIDIA GeForce 8400 GS Rev. 3 
Memory: 1573MiB / 7957MiB 

christopher@nightshade ~$ guix describe
Generation 5	May 22 2021 09:43:30	(current)
  guix d4ffa96
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: d4ffa9630277fa8699c783c08381d688626d4bc3


-- 
Christopher Howard
blog: https://librehacker.com
social: https://gnusocial.club/librehacker

[-- Attachment #2: Type: text/html, Size: 3235 bytes --]

bug#48591: ugrep: illegal instruction

[Christopher Howard]

[-- Attachment #1: Type: text/plain, Size: 568 bytes --]

christopher@nightshade ~$ guix challenge ugrep
/gnu/store/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12 contents
differ:
  local hash: 1yf9752hprihmhwnqlhaffkmwmkd3scwzr8dqy7d78vcy70qjkn6
  
https://ci.guix.gnu.org/nar/lzip/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12
: 1ifdjmscb07pacxwcs2408zi12yrxxf0zdwzz800xlccp5x53m7r
  differing file:
    /bin/ugrep

1 store items were analyzed:
  - 0 (0.0%) were identical
  - 1 (100.0%) differed
  - 0 (0.0%) were inconclusive

-- 
Christopher Howard
blog: https://librehacker.com
social: https://gnusocial.club/librehacker

[-- Attachment #2: Type: text/html, Size: 1127 bytes --]

bug#48591: ugrep: illegal instruction

[Leo Famulari]

On Sat, May 22, 2021 at 12:19:30PM -0800, Christopher Howard wrote:
> However, the command I am running to
> rebuild ugrep without grafts or substitutes is rebuilding everything
> all the way back to bootstrap, so it probably will not be completed for
> a day or two:
> 
> guix gc -D $(guix build --no-grafts ugrep); guix build --no-substitutes 
> ugrep

Do this instead:

$ guix environment --no-grafts ugrep -- guix build --no-grafts --no-substitutes ugrep

That way, you can use substitutes for ugrep's dependencies, but then
build ugrep from source.

I hope it helps!

bug#48591: ugrep: illegal instruction

[raingloom]

On Mon, 24 May 2021 13:22:23 -0400
Leo Famulari <leo@famulari.name> wrote:

> On Sat, May 22, 2021 at 12:19:30PM -0800, Christopher Howard wrote:
> > However, the command I am running to
> > rebuild ugrep without grafts or substitutes is rebuilding everything
> > all the way back to bootstrap, so it probably will not be completed
> > for a day or two:
> > 
> > guix gc -D $(guix build --no-grafts ugrep); guix build
> > --no-substitutes ugrep  
> 
> Do this instead:
> 
> $ guix environment --no-grafts ugrep -- guix build --no-grafts
> --no-substitutes ugrep
> 
> That way, you can use substitutes for ugrep's dependencies, but then
> build ugrep from source.
> 
> I hope it helps!
> 
> 
> 

TIL about that. There really ought to be a more obvious way to do that.

bug#48591: ugrep: illegal instruction

[Leo Famulari]

On Wed, Jun 30, 2021 at 11:41:11PM +0200, raingloom wrote:
> TIL about that. There really ought to be a more obvious way to do that.

Agreed! It's a common use case.

bug#48591: ugrep: illegal instruction

[zimoun]

Hi,

On Sat, 22 May 2021 at 12:19, Christopher Howard <christopher@librehacker.com> wrote:

> I'm sure this will also be a reproducibility bug,

Indeed, the package ’ugrep’ is not reproducible:

 guix build ugrep --no-grafts -q
 guix build ugrep --no-grafts -q --check -K
 diff --no-dereference -r /gnu/store/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12{,-check}
 Binary files /gnu/store/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12/bin/ugrep and /gnu/store/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12-check/bin/ugrep differ

The next step is to investigate with diffoscope.


Well, another data point – maybe Berlin has an issue – because I get:

  guix challenge ugrep --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
  /gnu/store/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12 contents differ:
    local hash: 1ifdjmscb07pacxwcs2408zi12yrxxf0zdwzz800xlccp5x53m7r
    https://ci.guix.gnu.org/nar/lzip/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12: 1ifdjmscb07pacxwcs2408zi12yrxxf0zdwzz800xlccp5x53m7r
    https://bordeaux.guix.gnu.org/nar/lzip/a6x7rv7xi3d2pw205znmnha9qfjrvs7k-ugrep-3.1.12: 1yf9752hprihmhwnqlhaffkmwmkd3scwzr8dqy7d78vcy70qjkn6

  guix hash -r $(guix build --no-grafts ugrep)-check
  1yf9752hprihmhwnqlhaffkmwmkd3scwzr8dqy7d78vcy70qjkn6

I fetch by default the substitutes from ci that why “local hash“ and ci
are the same; but my local build with ‘guix build --check’ gives the
same hash as bordeaux.


All the best,
simon

bug#48591: ugrep: reproducibility issues

[kiasoc5 via Bug reports for GNU Guix]

Is this bug fixed yet?

If not, seeing that this issue was motivated by an illegal instruction, 
we can use compile flags to disable SSE and other platform-dependent 
optimizations and investigate further.

WDYT?

https://github.com/Genivia/ugrep#other-platforms-step-3-build

bug#48591: ugrep: reproducibility issues

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1052 bytes --]

On 2023-01-16, kiasoc5 wrote:
> Is this bug fixed yet?
>
> If not, seeing that this issue was motivated by an illegal instruction, 
> we can use compile flags to disable SSE and other platform-dependent 
> optimizations and investigate further.
>
> WDYT?
>
> https://github.com/Genivia/ugrep#other-platforms-step-3-build

--disable-sse2 disable SSE2 and AVX optimizations
--disable-avx2 disable AVX2 and AVX512BW optimizations, but compile with SSE2 when supported
--disable-neon disable ARM NEON/AArch64 optimizations

Seems like --disable-sse2 on x86_64 and x86 (32-bit) and --disable-neon
(armhf, maybe aarch64) would be needed to make the build independent of
which cpu it runs on.


FWIW, debian disables several of these (--disable-sse2 and
--disable-neon), although also passes --disable-avx which is not mentioned in
the upstream documentation, although is present in configure.ac...

  https://salsa.debian.org/debian/ugrep/-/blob/7e2383dfaec23d58587f53c31ce46ef884513975/debian/rules#L5-13


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]