Fwd: GNU gettext 0.20.1 released

Fwd: GNU gettext 0.20.1 released

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 5124 bytes --]

Bruno,

Wow.  Thank you for this great summary!  Would that all projects 
published such clear (and custom) release notes…  <3

I see that gnu/packages/gettext.scm has a nice chronological list 
of copyright lines, which does make it appear as if I'm the 
current packager of gettext in Guix.  However, the Guix project 
doesn't have this notion of (package) maintainer: everyone 
packages, fixes, and updates what they can whenever they can. 
This might change in future but it works rather well now.

For that reason, I'm CC'ing the guix-devel@gnu.org list.  I 
encourage you to add it to your own for future releases.

I'm having some trouble with the actual upgrade but I'll save that 
for a reply.

Thanks again,

T G-R

[Original message:]

Available at
  https://ftp.gnu.org/gnu/gettext/gettext-0.20.1.tar.xz

New in 0.20.1:

* Important bug fix:
  - Fixed a wrong shared library versioning of libintl.so.

New in 0.20:

* Libtextstyle:
  - This package installs a new library 'libtextstyle', together 
  with a new
    header file <textstyle.h>.  It is a library for styling text 
    output sent
    to a console or terminal emulator.
    Packagers: please see the suggested packaging hints in the 
    file PACKAGING.

* Support for reproducible builds:
  - msgfmt now eliminates the POT-Creation-Date header field from 
  .mo files.

* Improvements for translators:
  - update-po target in Makefile.in.in now uses msgmerge 
  --previous.

* Improvements for maintainers:
  - msgmerge now has an option --for-msgfmt, that produces a PO 
  file meant
    for use by msgfmt only.  This option saves processing time, in 
    particular
    by omitting fuzzy matching that is not useful in this 
    situation.
  - The .pot file in a 'po' directory is now erased by "make 
  maintainer-clean".
  - It is now possible to override xgettext options from the 
  po/Makefile.in.in
    through options in XGETTEXT_OPTIONS (declared in po/Makevars).
  - The --intl option of the gettextize program (deprecated since 
  2010) is
    no longer available. Instead of including the intl sources in 
    your package,
    we suggest making the libintl library an optional prerequisite 
    of your
    package. This will simplify the build system of your package.
  - Accordingly, the Autoconf macro AM_GNU_GETTEXT_INTL_SUBDIR is 
  gone as well.

* Programming languages support:
  - C, C++:
    xgettext now supports strings in u8"..." syntax, as specified 
    in C11
    and C++11.
  - C, C++:
    xgettext now supports 'p'/'P' exponent markers in number 
    tokens, as
    specified in C99 and C++17.
  - C++:
    xgettext now supports underscores in number tokens.
  - C++:
    xgettext now supports single-quotes in number tokens, as 
    specified in
    C++14.
  - Shell:
    o The programs 'gettext', 'ngettext' now support a --context 
    argument.
    o gettext.sh contains new function eval_pgettext and 
    eval_npgettext
      for producing translations of messages with context.
  - Java:
    o xgettext now supports UTF-8 encoded .properties files (a new 
    feature
      of Java 9).
    o The build system and tools now support Java 9, 10, and 
    11. On the
      other hand, support for old versions of Java (Java 5 and 
      older,
      GCJ 4.2.x and older) has been dropped.
  - Perl:
    o Native support for context functions (pgettext, dpgettext, 
    dcpgettext,
      npgettext, dnpgettext, dcnpgettext).
    o better detection of question mark and slash as operators (as 
    opposed 
      to regular expression delimiters).
  - Scheme:
    xgettext now parses the syntax for specialized byte vectors 
    (#u8(...),
    #vu8(...), etc.) correctly.
  - Pascal:
    xgettext can now extract strings from .rsj files, produced by 
    the
    Free Pascal compiler version 3.0.0 or newer.
  - Vala:
    xgettext now parses escape sequences in strings more 
    accurately.
  - JavaScript:
    xgettext now parses template literals correctly.

* Runtime behaviour:
  - The interpretation of the language preferences on macOS has 
  been fixed.
  - Per-thread locales are now also supported on Solaris 11.4.
  - The replacements for the printf()/fprintf()/... functions that 
  are
    provided through <libintl.h> on native Windows and NetBSD are 
    now POSIX
    compliant.  There is no conflict any more between these 
    replacements
    and other possible replacements provided by gnulib or mingw.


According to [1], you seem to be packaging gettext for 
Guix/GuixSD.

I invite you to upgrade to version 0.20.1.

The packaging will need changes, regarding the new libtextstyle; I 
recommend
to package it as a separate binary package (see file PACKAGING).

When doing this upgrade, you can drop the following workarounds:
- test-lock now never hangs any more.
- expat is not longer used (already since 0.19.7).

Best regards,

           Bruno

[1] 
https://git.savannah.gnu.org/gitweb/?p=guix.git;a=blob;f=gnu/packages/gettext.scm

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: GNU gettext 0.20.1 released

[Miguel]

[-- Attachment #1: Type: text/plain, Size: 4261 bytes --]

Hi Tobias,

Tobias Geerinckx-Rice <me@tobias.gr>:
> Bruno,
> 
> Wow.  Thank you for this great summary!  Would that all projects 
> published such clear (and custom) release notes…  <3
> 
> I see that gnu/packages/gettext.scm has a nice chronological list 
> of copyright lines, which does make it appear as if I'm the 
> current packager of gettext in Guix.  However, the Guix project 
> doesn't have this notion of (package) maintainer: everyone 
> packages, fixes, and updates what they can whenever they can. 
> This might change in future but it works rather well now.
> 
> For that reason, I'm CC'ing the guix-devel@gnu.org list.  I 
> encourage you to add it to your own for future releases.
> 
> I'm having some trouble with the actual upgrade but I'll save that 
> for a reply.

What are the issues you have?  I could help with that. :-)

I see a big point that need special care: gettext-tools now depends on
libtextstyle, so gettext-boot0 will definitely fail if only
gettext-minimal is updated to the next version.

This is going to be a big big update in any case.  We can avoid
the new bootstrap, keeping 0.19.8.1 for it, but we should update
gettext-minimal so it may not worth the effort as probably almost all
the packages will be rebuilt.

My snippet for building it is this one, with some code from
gettext-minimal and it need some work:
-------------------->8--------------------
(define-public gettext-next
  (package (inherit gettext-minimal)
    (name "gettext-next")
    (version "0.20.1")
    (source (origin
             (method url-fetch)
             (uri (string-append "mirror://gnu/gettext/gettext-"
                                 version ".tar.gz"))
             (sha256
              (base32
               "0p3zwkk27wm2m2ccfqm57nj7vqkmfpn7ja1nf65zmhz8qqs5chb6"))))
    (inputs
     `(("xml2" ,libxml2)
       ;; Avoid dependency cycles
       ("unistring" ,(@ (gnu packages libunistring) libunistring))
       ("ncurses" ,(@ (gnu packages ncurses) ncurses))))
    (arguments
     `(#:configure-flags
       (list "--with-included-libunistring=no"
             "--with-included-libxml=no"
             (string-append "--with-libxml2-prefix="
                            (assoc-ref %build-inputs "xml2"))
             (string-append "--with-libncurses-prefix="
                            (assoc-ref %build-inputs "ncurses"))
             (string-append "--with-libtermcap-prefix="
                            (assoc-ref %build-inputs "ncurses"))
             (string-append "--with-libunistring-prefix="
                            (assoc-ref %build-inputs "unistring")))
       #:phases
       (modify-phases %standard-phases
        (add-before 'configure 'patch-fixed-paths
         (lambda* (#:key inputs #:allow-other-keys)
           (let* ((bash (which "sh")))
             (substitute* '("gettext-tools/config.h.in"
                            "gettext-tools/gnulib-tests/init.sh"
                            "gettext-tools/tests/init.sh"
                            "gettext-tools/system-tests/run-test")
               (("/bin/sh")
                bash))
             (substitute* '("gettext-tools/src/project-id"
                            "gettext-tools/projects/KDE/trigger"
                            "gettext-tools/projects/GNOME/trigger")
                 (("/bin/pwd")
                  "pwd"))
             #t)))

        (add-before 'check 'patch-tests
         (lambda* (#:key inputs #:allow-other-keys)
           (let* ((bash (which "sh")))
             ;; Some of the files we're patching are
             ;; ISO-8859-1-encoded, so choose it as the default
             ;; encoding so the byte encoding is preserved.
             (with-fluids ((%default-port-encoding #f))
               (substitute*
                   (find-files "gettext-tools/tests"
                               "^(lang-sh|msg(exec|filter)-[0-9])")
                 (("#![[:blank:]]/bin/sh")
                  (format #f "#!~a" bash)))

               #t)))))

       ;; When tests fail, we want to know the details.
       #:make-flags '("VERBOSE=yes")))))
--------------------8<--------------------

Happy hacking,
Miguel

[-- Attachment #2: Firma digital OpenPGP --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: GNU gettext 0.20.1 released

[Marius Bakke]

[-- Attachment #1.1: Type: text/plain, Size: 4563 bytes --]

Hi Miguel,

Miguel <rosen644835@gmail.com> writes:

> Hi Tobias,
>
> Tobias Geerinckx-Rice <me@tobias.gr>:
>> Bruno,
>> 
>> Wow.  Thank you for this great summary!  Would that all projects 
>> published such clear (and custom) release notes…  <3
>> 
>> I see that gnu/packages/gettext.scm has a nice chronological list 
>> of copyright lines, which does make it appear as if I'm the 
>> current packager of gettext in Guix.  However, the Guix project 
>> doesn't have this notion of (package) maintainer: everyone 
>> packages, fixes, and updates what they can whenever they can. 
>> This might change in future but it works rather well now.
>> 
>> For that reason, I'm CC'ing the guix-devel@gnu.org list.  I 
>> encourage you to add it to your own for future releases.
>> 
>> I'm having some trouble with the actual upgrade but I'll save that 
>> for a reply.
>
> What are the issues you have?  I could help with that. :-)
>
> I see a big point that need special care: gettext-tools now depends on
> libtextstyle, so gettext-boot0 will definitely fail if only
> gettext-minimal is updated to the next version.
>
> This is going to be a big big update in any case.  We can avoid
> the new bootstrap, keeping 0.19.8.1 for it, but we should update
> gettext-minimal so it may not worth the effort as probably almost all
> the packages will be rebuilt.
>
> My snippet for building it is this one, with some code from
> gettext-minimal and it need some work:
> -------------------->8--------------------
> (define-public gettext-next
>   (package (inherit gettext-minimal)
>     (name "gettext-next")
>     (version "0.20.1")
>     (source (origin
>              (method url-fetch)
>              (uri (string-append "mirror://gnu/gettext/gettext-"
>                                  version ".tar.gz"))
>              (sha256
>               (base32
>                "0p3zwkk27wm2m2ccfqm57nj7vqkmfpn7ja1nf65zmhz8qqs5chb6"))))
>     (inputs
>      `(("xml2" ,libxml2)
>        ;; Avoid dependency cycles
>        ("unistring" ,(@ (gnu packages libunistring) libunistring))
>        ("ncurses" ,(@ (gnu packages ncurses) ncurses))))
>     (arguments
>      `(#:configure-flags
>        (list "--with-included-libunistring=no"
>              "--with-included-libxml=no"
>              (string-append "--with-libxml2-prefix="
>                             (assoc-ref %build-inputs "xml2"))
>              (string-append "--with-libncurses-prefix="
>                             (assoc-ref %build-inputs "ncurses"))
>              (string-append "--with-libtermcap-prefix="
>                             (assoc-ref %build-inputs "ncurses"))
>              (string-append "--with-libunistring-prefix="
>                             (assoc-ref %build-inputs "unistring")))
>        #:phases
>        (modify-phases %standard-phases
>         (add-before 'configure 'patch-fixed-paths
>          (lambda* (#:key inputs #:allow-other-keys)
>            (let* ((bash (which "sh")))
>              (substitute* '("gettext-tools/config.h.in"
>                             "gettext-tools/gnulib-tests/init.sh"
>                             "gettext-tools/tests/init.sh"
>                             "gettext-tools/system-tests/run-test")
>                (("/bin/sh")
>                 bash))
>              (substitute* '("gettext-tools/src/project-id"
>                             "gettext-tools/projects/KDE/trigger"
>                             "gettext-tools/projects/GNOME/trigger")
>                  (("/bin/pwd")
>                   "pwd"))
>              #t)))
>
>         (add-before 'check 'patch-tests
>          (lambda* (#:key inputs #:allow-other-keys)
>            (let* ((bash (which "sh")))
>              ;; Some of the files we're patching are
>              ;; ISO-8859-1-encoded, so choose it as the default
>              ;; encoding so the byte encoding is preserved.
>              (with-fluids ((%default-port-encoding #f))
>                (substitute*
>                    (find-files "gettext-tools/tests"
>                                "^(lang-sh|msg(exec|filter)-[0-9])")
>                  (("#![[:blank:]]/bin/sh")
>                   (format #f "#!~a" bash)))
>
>                #t)))))
>
>        ;; When tests fail, we want to know the details.
>        #:make-flags '("VERBOSE=yes")))))
> --------------------8<--------------------

Thanks for this!  I came up with the following patch to keep the
bootstrap mostly unchanged:


[-- Attachment #1.2: 0001-gnu-gettext-Update-to-0.20.1.patch --]
[-- Type: text/x-patch, Size: 8724 bytes --]

From a53560b0ffed27cf99ae7bc7aa2d1fc19816c5a6 Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Sun, 30 Jun 2019 16:46:37 +0200
Subject: [PATCH] gnu: gettext: Update to 0.20.1.

* gnu/packages/gettext.scm (gettext-minimal): Update to 0.20.1.
[source](snippet): Remove.
[inputs]: Remove EXPAT.  Add LIBUNISTRING, LIBXML2, and NCURSES.
[arguments]: Add #:configure-flags and a "patch-fixed-paths" phase.  Delete
"link-expat" phase.
* gnu/packages/commencement.scm (gettext-boot0): Stay on version 0.19.8.1.
[arguments]: Adjust accordingly.

Co-authored-by: Miguel <rosen644835@gmail.com>
---
 gnu/packages/commencement.scm | 47 ++++++++++++-----------
 gnu/packages/gettext.scm      | 72 +++++++++++++++++++----------------
 2 files changed, 65 insertions(+), 54 deletions(-)

diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm
index 10b0bd67f5..2dcdb175a8 100644
--- a/gnu/packages/commencement.scm
+++ b/gnu/packages/commencement.scm
@@ -2015,30 +2015,33 @@ exec ~a/bin/~a-~a -B~a/lib -Wl,-dynamic-linker -Wl,~a/~a \"$@\"~%"
   (let ((gettext-minimal
          (package (inherit gettext-minimal)
            (name "gettext-boot0")
+           ;; GNU Gettext 0.20 and later depends on LibXML and ncurses.  To
+           ;; simplify bootstrapping, we stick to this older version here.
+           (version "0.19.8.1")
+           (source (origin
+                     (method url-fetch)
+                     (uri (string-append "mirror://gnu/gettext/gettext-"
+                                         version ".tar.gz"))
+                     (sha256
+                      (base32
+                       "0hsw28f9q9xaggjlsdp2qmbp2rbd1mp0njzan2ld9kiqwkq2m57z"))))
            (inputs '())                           ;zero dependencies
            (arguments
-            (substitute-keyword-arguments
-                `(#:tests? #f
-                  ,@(package-arguments gettext-minimal))
-              ((#:phases phases)
-               `(modify-phases ,phases
-                  ;; Build only the tools.
-                  (add-after 'unpack 'chdir
-                             (lambda _
-                               (chdir "gettext-tools")
-                               #t))
-
-                  ;; Some test programs require pthreads, which we don't have.
-                  (add-before 'configure 'no-test-programs
-                              (lambda _
-                                (substitute* "tests/Makefile.in"
-                                  (("^PROGRAMS =.*$")
-                                   "PROGRAMS =\n"))
-                                #t))
-
-                  ;; Don't try to link against libexpat.
-                  (delete 'link-expat)
-                  (delete 'patch-tests))))))))
+            `(#:tests? #f
+              #:phases (modify-phases %standard-phases
+                         ;; Build only the tools.
+                         (add-after 'unpack 'chdir
+                           (lambda _
+                             (chdir "gettext-tools")
+                             #t))
+
+                         ;; Some test programs require pthreads, which we don't have.
+                         (add-before 'configure 'no-test-programs
+                           (lambda _
+                             (substitute* "tests/Makefile.in"
+                               (("^PROGRAMS =.*$")
+                                "PROGRAMS =\n"))
+                             #t))))))))
     (package-with-bootstrap-guile
      (package-with-explicit-inputs gettext-minimal
                                    %boot1-inputs
diff --git a/gnu/packages/gettext.scm b/gnu/packages/gettext.scm
index ef80af42ea..17593ee1cc 100644
--- a/gnu/packages/gettext.scm
+++ b/gnu/packages/gettext.scm
@@ -4,10 +4,11 @@
 ;;; Copyright © 2015, 2017 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2016, 2019 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2016 Alex Kost <alezost@gmail.com>
-;;; Copyright © 2017 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2017, 2019 Marius Bakke <mbakke@fastmail.com>
 ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2017 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2019 Miguel <rosen644835@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,8 @@
   #:use-module (guix build-system perl)
   #:use-module (gnu packages docbook)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages libunistring)
+  #:use-module (gnu packages ncurses)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages tex)
   #:use-module (gnu packages xml)
@@ -41,34 +44,47 @@
 (define-public gettext-minimal
   (package
     (name "gettext-minimal")
-    (version "0.19.8.1")
+    (version "0.20.1")
     (source (origin
-             (method url-fetch)
-             (uri (string-append "mirror://gnu/gettext/gettext-"
-                                 version ".tar.gz"))
-             (sha256
-              (base32
-               "0hsw28f9q9xaggjlsdp2qmbp2rbd1mp0njzan2ld9kiqwkq2m57z"))
-             (modules '((guix build utils)))
-             (snippet
-              '(begin
-                ;; The gnulib test-lock test is prone to writer starvation
-                ;; with our glibc@2.25, which prefers readers, so disable it.
-                ;; The gnulib commit b20e8afb0b2 should fix this once
-                ;; incorporated here.
-                 (substitute* "gettext-runtime/tests/Makefile.in"
-                   (("TESTS = test-lock\\$\\(EXEEXT\\)") "TESTS ="))
-                 (substitute* "gettext-tools/gnulib-tests/Makefile.in"
-                  (("test-lock\\$\\(EXEEXT\\) ") ""))
-                 #t))))
+              (method url-fetch)
+              (uri (string-append "mirror://gnu/gettext/gettext-"
+                                  version ".tar.gz"))
+              (sha256
+               (base32
+                "0p3zwkk27wm2m2ccfqm57nj7vqkmfpn7ja1nf65zmhz8qqs5chb6"))))
     (build-system gnu-build-system)
     (outputs '("out"
-               "doc"))                            ;8 MiB of HTML
+               "doc"))                            ;9 MiB of HTML
     (inputs
-     `(("expat" ,expat)))
+     `(("libunistring" ,libunistring)
+       ("libxml2" ,libxml2)
+       ("ncurses" ,ncurses)))
     (arguments
-     `(#:phases
+     `(#:configure-flags
+       (list "--with-included-libunistring=no"
+             "--with-included-libxml=no"
+             (string-append "--with-libxml2-prefix="
+                            (assoc-ref %build-inputs "libxml2"))
+             (string-append "--with-libncurses-prefix="
+                            (assoc-ref %build-inputs "ncurses"))
+             (string-append "--with-libtermcap-prefix="
+                            (assoc-ref %build-inputs "ncurses"))
+             (string-append "--with-libunistring-prefix="
+                            (assoc-ref %build-inputs "libunistring")))
+       #:phases
        (modify-phases %standard-phases
+         (add-before 'patch-source-shebangs 'patch-fixed-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* '("gettext-tools/config.h.in"
+                            "gettext-tools/gnulib-tests/init.sh"
+                            "gettext-tools/tests/init.sh"
+                            "gettext-tools/system-tests/run-test")
+               (("/bin/sh") "sh"))
+             (substitute* '("gettext-tools/src/project-id"
+                            "gettext-tools/projects/KDE/trigger"
+                            "gettext-tools/projects/GNOME/trigger")
+               (("/bin/pwd") "pwd"))
+             #t))
         (add-before 'check 'patch-tests
          (lambda* (#:key inputs #:allow-other-keys)
            (let* ((bash (which "sh")))
@@ -92,15 +108,7 @@
                  (("/bin/pwd")
                   "pwd"))
 
-               #t))))
-        (add-before 'configure 'link-expat
-         (lambda _
-           ;; Gettext defaults to opening expat via dlopen on
-           ;; "Linux".  Change to link directly.
-           (substitute* "gettext-tools/configure"
-             (("LIBEXPAT=\"-ldl\"") "LIBEXPAT=\"-ldl -lexpat\"")
-             (("LTLIBEXPAT=\"-ldl\"") "LTLIBEXPAT=\"-ldl -lexpat\""))
-           #t)))
+               #t)))))
 
        ;; When tests fail, we want to know the details.
        #:make-flags '("VERBOSE=yes")))
-- 
2.22.0


[-- Attachment #1.3: Type: text/plain, Size: 1817 bytes --]


Does it look correct to you?

Do you know if it's possible to delete the bundled copies of libxml and
libunistring entirely?  I got build errors with this source snippet:

--8<---------------cut here---------------start------------->8---
              (modules '((guix build utils)))
              (snippet
               '(begin
                  ;; Delete the bundled copies of libxml2 and libunistring.
                  (for-each delete-file-recursively
                            (find-files "." "^(unistr|libxml)$"
                                        #:directories? #t))
                  #t))
--8<---------------cut here---------------end--------------->8---

Anyway the new gettext does seem to work fine.  It has grown from 78.9
to 97.9 MiB (this is on the 'core-updates' branch):

$ ./pre-inst-env guix size gettext-minimal
store item                                                       total    self
/gnu/store/ycnva13hfplvdc0wp9ckfa909qlhyra6-glibc-2.28              37.5    36.0  36.7%
/gnu/store/67z1imya4k73rv8iys59lnslxz3k78z4-gcc-7.4.0-lib           70.2    32.6  33.3%
/gnu/store/0nv03as1mvmw4ii72nirp14857fd4x0w-gettext-minimal-0.20.1    97.9     9.4   9.6%
/gnu/store/v0gk70nylmzwadwsavvdwxm59i1c77w8-libxml2-2.9.9           80.3     7.9   8.1%
/gnu/store/094gw83fgil2scrwzbsgw6dxb7zg3g6m-ncurses-6.1-20190609    76.0     5.9   6.0%
/gnu/store/2xx5pv1zb7v0cw42p25hdgc9yjgpazdn-libunistring-0.9.10     72.5     2.4   2.4%
/gnu/store/ld746xji0id9gcflm1v645d71y55pnbw-bash-static-5.0.7        1.6     1.6   1.6%
/gnu/store/argczg2hxrzh5m11dbcz0gghg1cpgzhd-bash-minimal-5.0.7      38.6     1.0   1.1%
/gnu/store/3pm8plhp2zrybnj59vdzq0r0vjvvnim8-xz-5.2.4                72.1     0.9   0.9%
/gnu/store/nl92qmy1b61biqzm0cjg2hh9f47qf3w1-zlib-1.2.11             70.4     0.2   0.2%
total: 97.9 MiB

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]