ELPA Archive Build Fails

ELPA Archive Build Fails

[Ian Dunn]

With the recent changes to uni-confusables, it looks like check_copyrights is broken:

$ admin/update-archive.sh
...
+ make -f /home/skolar/programming/elpa/build/GNUmakefile check_copyrights
Compute exceptions >copyright_exceptions~
diff -u "copyright_exceptions" "copyright_exceptions~"
--- copyright_exceptions	2017-12-09 14:28:44.843470983 -0500
+++ copyright_exceptions~	2017-12-10 16:39:12.466687835 -0500
@@ -70,7 +70,8 @@
 ./muse/htmlize-hack.el
 ./rudel/rudel-loaddefs.el
 ./uni-confusables/gen-confusables.el:;; Copyright (C) 1991-2009, 2010 Unicode, Inc.
-./uni-confusables/gen-confusables.el:;; for the copyright and permission notice.
+./uni-confusables/gen-confusables.el:;; gen-confusables.el.  The copyright and permissions are the same
+./uni-confusables/gen-confusables.el:;; list (covered by the Unicode copyright terms in copyright.html)
 ./uni-confusables/uni-confusables.el
 ./uni-confusables/uni-confusables.el:;; Copyright (C) 1991-2009, 2010 Unicode, Inc.
 ./uni-confusables/uni-confusables.el:;; for the copyright and permission notice.
make: *** [/home/skolar/programming/elpa/build/GNUmakefile:18: check_copyrights] Error 1
+ signal_error 'check_copyright failed'
+ title='check_copyright failed'
+ '[' '' = '' ']'
+ echo 'Error: check_copyright failed'
Error: check_copyright failed
+ exit 1

-- 
Ian Dunn

Re: ELPA Archive Build Fails

[Ted Zlatanov]

On Sun, 10 Dec 2017 17:15:20 -0500 Ian Dunn <dunni@gnu.org> wrote: 

ID> With the recent changes to uni-confusables, it looks like check_copyrights is broken:

ID> $ admin/update-archive.sh
ID> ...
ID> + make -f /home/skolar/programming/elpa/build/GNUmakefile check_copyrights
ID> Compute exceptions >copyright_exceptions~
ID> diff -u "copyright_exceptions" "copyright_exceptions~"
ID> --- copyright_exceptions	2017-12-09 14:28:44.843470983 -0500
ID> +++ copyright_exceptions~	2017-12-10 16:39:12.466687835 -0500
ID> @@ -70,7 +70,8 @@
ID>  ./muse/htmlize-hack.el
ID>  ./rudel/rudel-loaddefs.el
ID>  ./uni-confusables/gen-confusables.el:;; Copyright (C) 1991-2009, 2010 Unicode, Inc.
ID> -./uni-confusables/gen-confusables.el:;; for the copyright and permission notice.
ID> +./uni-confusables/gen-confusables.el:;; gen-confusables.el.  The copyright and permissions are the same
ID> +./uni-confusables/gen-confusables.el:;; list (covered by the Unicode copyright terms in copyright.html)
ID>  ./uni-confusables/uni-confusables.el
ID>  ./uni-confusables/uni-confusables.el:;; Copyright (C) 1991-2009, 2010 Unicode, Inc.
ID>  ./uni-confusables/uni-confusables.el:;; for the copyright and permission notice.
ID> make: *** [/home/skolar/programming/elpa/build/GNUmakefile:18: check_copyrights] Error 1
ID> + signal_error 'check_copyright failed'
ID> + title='check_copyright failed'
ID> + '[' '' = '' ']'
ID> + echo 'Error: check_copyright failed'
ID> Error: check_copyright failed
ID> + exit 1

Looking at that Make target:

check_copyrights:
        @echo "Compute exceptions >$(CR_EXCEPTIONS)~"
        @export LC_ALL=C;                                           \
        (cd packages &&                                             \
        find . -name '.git' -prune -o                               \
               -name 'test' -prune -o                               \
               -name '*.el' -print0 |                               \
            xargs -0 grep -L 'Free Software Foundation, Inc' |      \
            grep -v '\(\.dir-locals\|.-\(pkg\|autoloads\)\)\.el$$'; \
        find . -name '.git' -prune -o -name '*.el' -type f -print | \
            while read f; do                                        \
                fquoted="$$(echo $$f|tr '|' '_')";                  \
                sed -n -e '/[Cc]opyright.*, *[1-9][-0-9]*,\?$$/N'   \
                    -e '/Free Software Foundation/d'                \
                    -e "s|^\\(.*[Cc]opyright\\)|$$fquoted:\\1|p"    \
                   "$$f";                                           \
            done) | sort >$(CR_EXCEPTIONS)~
        diff -u "$(CR_EXCEPTIONS)" "$(CR_EXCEPTIONS)~"

I can sort of understand the shell logic, but not why it's there.

Could someone (Stefan?) please either add comments to the GNUMakefile
explaining what it does and why, or explain here and I'll add the
comments?

Ideally the output of the check would also be improved to explain what's
wrong. I can try to do that as well.

Thanks
Ted

Re: ELPA Archive Build Fails

[Stefan Monnier]

> Looking at that Make target:

> check_copyrights:
>         @echo "Compute exceptions >$(CR_EXCEPTIONS)~"
>         @export LC_ALL=C;                                           \
>         (cd packages &&                                             \
>         find . -name '.git' -prune -o                               \
>                -name 'test' -prune -o                               \
>                -name '*.el' -print0 |                               \
>             xargs -0 grep -L 'Free Software Foundation, Inc' |      \
>             grep -v '\(\.dir-locals\|.-\(pkg\|autoloads\)\)\.el$$'; \
>         find . -name '.git' -prune -o -name '*.el' -type f -print | \
>             while read f; do                                        \
>                 fquoted="$$(echo $$f|tr '|' '_')";                  \
>                 sed -n -e '/[Cc]opyright.*, *[1-9][-0-9]*,\?$$/N'   \
>                     -e '/Free Software Foundation/d'                \
>                     -e "s|^\\(.*[Cc]opyright\\)|$$fquoted:\\1|p"    \
>                    "$$f";                                           \
>             done) | sort >$(CR_EXCEPTIONS)~
>         diff -u "$(CR_EXCEPTIONS)" "$(CR_EXCEPTIONS)~"
>
> I can sort of understand the shell logic, but not why it's there.

> Could someone (Stefan?) please either add comments to the GNUMakefile
> explaining what it does and why, or explain here and I'll add the
> comments?

It does a very crude check to try and catch the files which lack the
"copyright FSF" line (that's the role of the first `find`), and to catch
the files which still have a "copyright <not-the-fsf>" somewhere.

I think in the present case the "right fix" is to update the
$(CR_EXCEPTIONS) file.

> Ideally the output of the check would also be improved to explain what's
> wrong.  I can try to do that as well.

Yes.  I think it's OK that the test is crude: the purpose is not to
perform a thorough check but to catch the obvious mistakes which are
very frequent (i.e. the rule rather than the exception) in new
packages/files).  But the output would benefit from being more
user-friendly.


        Stefan

Re: ELPA Archive Build Fails

[Ted Zlatanov]

On Mon, 11 Dec 2017 10:41:18 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Could someone (Stefan?) please either add comments to the GNUMakefile
>> explaining what it does and why, or explain here and I'll add the
>> comments?

SM> It does a very crude check to try and catch the files which lack the
SM> "copyright FSF" line (that's the role of the first `find`), and to catch
SM> the files which still have a "copyright <not-the-fsf>" somewhere.

SM> I think in the present case the "right fix" is to update the
SM> $(CR_EXCEPTIONS) file.

Thanks for the explanation. I added it to the elpa.git README as part of
the package addition process. I hope that's helpful.

Ted