Erroneous uses of regex in the invokation of FIND-FILES

Erroneous uses of regex in the invokation of FIND-FILES

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

Hello guix,

I find out that there are a lof of erroneous uses of regex in the
invokation of FIND-FILES. The correct usage should be:

  (find-files "." "\\.c$")

Instead people write:

  (find-files "." ".*\\.c")

which match unwanted files.

For examples, in the procedure CUSTOM-GCC, the correct regex should be:

  "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"

instead of:

  ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"

Please correct me if I am wrong.

Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Erroneous uses of regex in the invokation of FIND-FILES

[Mark H Weaver]

Hi Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> I find out that there are a lof of erroneous uses of regex in the
> invokation of FIND-FILES. The correct usage should be:
>
>   (find-files "." "\\.c$")
>
> Instead people write:
>
>   (find-files "." ".*\\.c")
>
> which match unwanted files.
>
> For examples, in the procedure CUSTOM-GCC, the correct regex should be:
>
>   "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"
>
> instead of:
>
>   ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"
>
> Please correct me if I am wrong.

You're right.  It would be good to fix these problems incrementally, as
long as the changes don't cause too many rebuilds.

Changes to core packages will need to wait for now, since 'core-updates'
is frozen, and 'core-updates-next' should also be considered frozen,
since it will become 'core-updates' as soon as Berlin has built it out a
bit more.  (The only change in 'core-updates-next' relative to
'core-updates' is that the new bootstrap tarballs have been fixed to be
deterministic.)

For some of these fixes, it might be best to apply them to 'staging'.

> Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
> directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.

I'm uncertain how many rebuilds it would trigger to change 'custom-gcc',
and I don't have confidence that "guix refresh -l" is capable of giving
us a reliable answer.  In the meantime, would you like to file a bug
report for this, so it's not forgotten?

Thanks for looking into it.

     Best,
      Mark

bug#37150: Erroneous uses of regex in the invokation of FIND-FILES

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 2020 bytes --]

Hello guix,

Currently, there are a lot of erroneous uses of regex in the invokation
of FIND-FILES. Please see the conversations below (taken from
https://lists.gnu.org/archive/html/guix-devel/2019-08/msg00149.html).

Mark H Weaver writes:

> Hi Alex,
>
> Alex Vong <alexvong1995@gmail.com> writes:
>
>> I find out that there are a lof of erroneous uses of regex in the
>> invokation of FIND-FILES. The correct usage should be:
>>
>>   (find-files "." "\\.c$")
>>
>> Instead people write:
>>
>>   (find-files "." ".*\\.c")
>>
>> which match unwanted files.
>>
>> For examples, in the procedure CUSTOM-GCC, the correct regex should be:
>>
>>   "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"
>>
>> instead of:
>>
>>   ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"
>>
>> Please correct me if I am wrong.
>
> You're right.  It would be good to fix these problems incrementally, as
> long as the changes don't cause too many rebuilds.
>
> Changes to core packages will need to wait for now, since 'core-updates'
> is frozen, and 'core-updates-next' should also be considered frozen,
> since it will become 'core-updates' as soon as Berlin has built it out a
> bit more.  (The only change in 'core-updates-next' relative to
> 'core-updates' is that the new bootstrap tarballs have been fixed to be
> deterministic.)
>
> For some of these fixes, it might be best to apply them to 'staging'.
>
>> Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
>> directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.
>
> I'm uncertain how many rebuilds it would trigger to change 'custom-gcc',
> and I don't have confidence that "guix refresh -l" is capable of giving
> us a reliable answer.  In the meantime, would you like to file a bug
> report for this, so it's not forgotten?
>
> Thanks for looking into it.
>
>      Best,
>       Mark


-- 
Stand with Hong Kong!
#Eye4HK
Alex

https://twitter.com/freedomhkg
https://twitter.com/stand_with_hk

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#37150: Erroneous uses of regex in the invokation of FIND-FILES

[Alex Vong]

[-- Attachment #1.1: Type: text/plain, Size: 360 bytes --]

Hello Mark,

>> You're right.  It would be good to fix these problems incrementally, as
>> long as the changes don't cause too many rebuilds.
>>
I agree we should fix it incrementally, like how the 'invoke' transition
was handled. The patch below does exactly that (it fixes the problem for
delta which causes 6 rebuilds only according to "guix refresh -l").


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-gnu-delta-Fix-regex-in-the-invokation-of-find-files.patch --]
[-- Type: text/x-diff, Size: 1066 bytes --]

From cb5f2febd564a2bcb550de537156db59588098c4 Mon Sep 17 00:00:00 2001
From: Alex Vong <alexvong1995@gmail.com>
Date: Fri, 23 Aug 2019 11:06:49 +0800
Subject: [PATCH] gnu: delta: Fix regex in the invokation of 'find-files'.

See <https://bugs.gnu.org/37150> for more information.

* gnu/packages/debug.scm (delta)[arguments]: Fix regex.
---
 gnu/packages/debug.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/debug.scm b/gnu/packages/debug.scm
index 82631deef6..e1cba5d3fe 100644
--- a/gnu/packages/debug.scm
+++ b/gnu/packages/debug.scm
@@ -75,7 +75,7 @@
                (begin
                  (for-each (lambda (h)
                              (install-file h doc))
-                           `("License.txt" ,@(find-files "www" ".*\\.html")))
+                           `("License.txt" ,@(find-files "www" "\\.html$")))
                  (for-each (lambda (b)
                              (install-file b bin))
                            `("delta" "multidelta" "topformflat"))))
-- 
2.23.0


[-- Attachment #1.3: Type: text/plain, Size: 1219 bytes --]


>> Changes to core packages will need to wait for now, since 'core-updates'
>> is frozen, and 'core-updates-next' should also be considered frozen,
>> since it will become 'core-updates' as soon as Berlin has built it out a
>> bit more.  (The only change in 'core-updates-next' relative to
>> 'core-updates' is that the new bootstrap tarballs have been fixed to be
>> deterministic.)
>>
>> For some of these fixes, it might be best to apply them to 'staging'.
>>
>>> Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
>>> directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.
>>
>> I'm uncertain how many rebuilds it would trigger to change 'custom-gcc',
>> and I don't have confidence that "guix refresh -l" is capable of giving
>> us a reliable answer.  In the meantime, would you like to file a bug
>> report for this, so it's not forgotten?
>>
It would probably cause a world rebuild... I will wait for the next
cycle for patch submittion.

>> Thanks for looking into it.
>>
>>      Best,
>>       Mark

You're welcomed!


-- 
Stand with Hong Kong!
#Eye4HK #BoycottMulan
Alex

https://twitter.com/freedomhkg
https://twitter.com/stand_with_hk

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Erroneous uses of regex in the invokation of FIND-FILES

[Maxim Cournoyer]

Hi again,

Alex Vong <alexvong1995@gmail.com> writes:

> Hello guix,
>
> I find out that there are a lof of erroneous uses of regex in the
> invokation of FIND-FILES. The correct usage should be:
>
>   (find-files "." "\\.c$")
>
> Instead people write:
>
>   (find-files "." ".*\\.c")
>
> which match unwanted files.
>
> For examples, in the procedure CUSTOM-GCC, the correct regex should be:
>
>   "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"
>
> instead of:
>
>   ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"
>
> Please correct me if I am wrong.
>
> Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
> directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.

Looking at this some more, I'm not sure what we expect to see in the
bin/ directory of gcc-objc, As I found out earlier, the files present
before any deletion are those:

--8<---------------cut here---------------start------------->8---
c++
cpp
g++
gcc
gcc-ar
gcc-nm
gcc-ranlib
gcov
gcov-dump
gcov-tool
x86_64-unknown-linux-gnu-c++
x86_64-unknown-linux-gnu-g++
x86_64-unknown-linux-gnu-gcc
x86_64-unknown-linux-gnu-gcc-8.3.0
x86_64-unknown-linux-gnu-gcc-ar
x86_64-unknown-linux-gnu-gcc-nm
x86_64-unknown-linux-gnu-gcc-ranlib
--8<---------------cut here---------------end--------------->8---

All of those files don't appear to be specific to gcc-objc (they are
also part of regular gcc) and thus it appear that removing them fits the
bill (the patch name says it exists to remove broken or conflicting
files).

The way that these GCC packages affect the GCC installation is by mean
of extending the search path of the GCC libraries, e.g.:

--8<---------------cut here---------------start------------->8---
(define-public gcc-objc-5
  (custom-gcc gcc-5 "gcc-objc" '("objc")
              (list (search-path-specification
                     (variable "OBJC_INCLUDE_PATH")
                     (files '("include")))
                    (search-path-specification
                     (variable "LIBRARY_PATH")
                     (files '("lib" "lib64"))))))
--8<---------------cut here---------------end--------------->8---

not by providing custom binaries.

Here's the list of files of the regular gcc package:

--8<---------------cut here---------------end--------------->8---
ls -l /gnu/store/ginrh3x6qi4w2i005gics37wzz5b78s7-gcc-5.5.0/bin
total 9308
c++
cpp
g++
gcc
gcc-ar
gcc-nm
gcc-ranlib
gcov
gcov-dump
gcov-tool
x86_64-unknown-linux-gnu-c++
x86_64-unknown-linux-gnu-g++
x86_64-unknown-linux-gnu-gcc
x86_64-unknown-linux-gnu-gcc-5.5.0
x86_64-unknown-linux-gnu-gcc-ar
x86_64-unknown-linux-gnu-gcc-nm
x86_64-unknown-linux-gnu-gcc-ranlib
--8<---------------cut here---------------end--------------->8---

It's confusing though that an empty bin directory is left behind.  We
could fix this.

Maxim

Erroneous uses of regex in the invokation of FIND-FILES

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 716 bytes --]

[Resending...]

Hello guix,

I find out that there are a lof of erroneous uses of regex in the
invokation of FIND-FILES. The correct usage should be:

  (find-files "." "\\.c$")

Instead people write:

  (find-files "." ".*\\.c")

which match unwanted files.

For examples, in the procedure CUSTOM-GCC, the correct regex should be:

  "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"

instead of:

  ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"

Please correct me if I am wrong.

Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.

-- 
Stand with Hong Kong!
#Eye4HK
Alex

https://twitter.com/freedomhkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Erroneous uses of regex in the invokation of FIND-FILES

[Maxim Cournoyer]

Hello Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> [Resending...]
>
> Hello guix,
>
> I find out that there are a lof of erroneous uses of regex in the
> invokation of FIND-FILES. The correct usage should be:
>
>   (find-files "." "\\.c$")
>
> Instead people write:
>
>   (find-files "." ".*\\.c")
>
> which match unwanted files.
>
> For examples, in the procedure CUSTOM-GCC, the correct regex should be:
>
>   "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$"
>
> instead of:
>
>   ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"
>
> Please correct me if I am wrong.
>
> Right now, the erroneous use of regex in CUSTOM-GCC casues the 'bin/'
> directory of the output of gccgo, gcc-objc and gcc-objc++ to be empty.

I've done the following experiment to validate your findings:

Modified gcc.scm as follows:

--8<---------------cut here---------------start------------->8---
1 file changed, 4 insertions(+), 4 deletions(-)
gnu/packages/gcc.scm | 8 ++++----

modified   gnu/packages/gcc.scm
@@ -631,10 +631,10 @@ as the 'native-search-paths' field."
         `(modify-phases ,phases
            (add-after 'install 'remove-broken-or-conflicting-files
              (lambda* (#:key outputs #:allow-other-keys)
-               (for-each delete-file
-                         (find-files (string-append (assoc-ref outputs "out") "/bin")
-                                     ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"))
-               #t))))))))
+               ;; (for-each delete-file
+               ;;           (find-files (string-append (assoc-ref outputs "out") "/bin")
+               ;;                       ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"))
+               #f))))))))
 
 (define %generic-search-paths
   ;; This is the language-neutral search path for GCC.  Entries in $CPATH are

--8<---------------cut here---------------end--------------->8---

Then built it with

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix build -K gcc-objc
--8<---------------cut here---------------start------------->8---

The copied the resulting store item under a writable location; e.g.:

--8<---------------cut here---------------start------------->8---
cp -r /gnu/store/7pmfv7bs9krrjdmfnky2q0i1kdhcvmzj-gcc-objc-8.3.0 /tmp/gccgcc-objc-8.3.0
--8<---------------cut here---------------end--------------->8---

And initialize a git repo in that dir:

--8<---------------cut here---------------start------------->8---
cd /tmp/gccgcc-objc-8.3.0; git init && git add . && git commit -m init
--8<---------------cut here---------------end--------------->8---

Then fired up Guile, and ran the following:

$ guile
> ,use (guix build utils)
> (define old-regexp ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)")
> (define new-regexp "(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)$")
> (for-each delete-file (find-files "./bin" old-regexp))

This resulted in the following files being deleted:

--8<---------------cut here---------------start------------->8---
 git status
On branch master
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    bin/c++
        deleted:    bin/cpp
        deleted:    bin/g++
        deleted:    bin/gcc
        deleted:    bin/gcc-ar
        deleted:    bin/gcc-nm
        deleted:    bin/gcc-ranlib
        deleted:    bin/gcov
        deleted:    bin/gcov-dump
        deleted:    bin/gcov-tool
        deleted:    bin/x86_64-unknown-linux-gnu-c++
        deleted:    bin/x86_64-unknown-linux-gnu-g++
        deleted:    bin/x86_64-unknown-linux-gnu-gcc
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-8.3.0
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-ar
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-nm
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-ranlib

no changes added to commit (use "git add" and/or "git commit -a")
--8<---------------cut here---------------end--------------->8---

And the bin directory is empty, as you found:

--8<---------------cut here---------------start------------->8---
ls -al bin/
total 0
drwxr-xr-x 1 mcournoyer users  0 Sep 21 04:31 ./
drwxr-xr-x 1 mcournoyer users 52 Sep 21 04:27 ../
--8<---------------cut here---------------end--------------->8---

Resetting the git repo (git reset --hard), and using the new regexp
instead:

--8<---------------cut here---------------start------------->8---
(for-each delete-file (find-files "./bin" new-regexp))
--8<---------------cut here---------------end--------------->8---

We now obtain:

--8<---------------cut here---------------start------------->8---
git status
On branch master
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    bin/c++
        deleted:    bin/cpp
        deleted:    bin/g++
        deleted:    bin/gcc
        deleted:    bin/gcc-ar
        deleted:    bin/gcc-nm
        deleted:    bin/gcc-ranlib
        deleted:    bin/gcov
        deleted:    bin/x86_64-unknown-linux-gnu-c++
        deleted:    bin/x86_64-unknown-linux-gnu-g++
        deleted:    bin/x86_64-unknown-linux-gnu-gcc
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-8.3.0
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-ar
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-nm
        deleted:    bin/x86_64-unknown-linux-gnu-gcc-ranlib

no changes added to commit (use "git add" and/or "git commit -a")
--8<---------------cut here---------------end--------------->8---

Which is mostly the same except that gcov-dump and gcov-tool are
preserved:

--8<---------------cut here---------------start------------->8---
ls -al ./bin
total 2292
drwxr-xr-x 1 mcournoyer users      36 Sep 21 04:49 ./
drwxr-xr-x 1 mcournoyer users      52 Sep 21 04:27 ../
-rwxr-xr-x 1 mcournoyer users 1155104 Sep 21 04:46 gcov-dump
-rwxr-xr-x 1 mcournoyer users 1185632 Sep 21 04:46 gcov-tool
--8<---------------cut here---------------end--------------->8---

That doesn't seem right either.

Maxim