bug#26247: Gettext introduces timestamps in .mo files

bug#26247: Gettext introduces timestamps in .mo files

[Ludovic Courtès]

Gettext 0.19.8.1 (current core-updates,
77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
file, without honoring SOURCE_DATE_EPOCH, which leads to
non-reproducible builds (for example ‘guix’).

Ludo’.

bug#26247: Gettext introduces timestamps in .mo files

[zimoun]

Hi Ludo,

This old bug #26247 about timestamp in .mo files from Gettext is still
open:

   <http://issues.guix.gnu.org/issue/26247>

On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:

> Gettext 0.19.8.1 (current core-updates,
> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
> file, without honoring SOURCE_DATE_EPOCH, which leads to
> non-reproducible builds (for example ‘guix’).

Is it still relevant?  Since Gettext is now at 0.20.1.  How can I
reproduce the issue?  Because the usual:

   guix build gettext --no-grafts --check

works fine.


Cheers,
simon

bug#26247: Gettext introduces timestamps in .mo files

[Ludovic Courtès]

Hi Simon,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Gettext 0.19.8.1 (current core-updates,
>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>> non-reproducible builds (for example ‘guix’).
>
> Is it still relevant?  Since Gettext is now at 0.20.1.  How can I
> reproduce the issue?

I still see this:

--8<---------------cut here---------------start------------->8---
$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org https://bayfront.guix.gnu.org"
updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe contents differ:
  no local build for '/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
  https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
  https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
  differing files:
    /share/locale/en@quot/LC_MESSAGES/guix.mo
    /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
    /share/locale/en@boldquot/LC_MESSAGES/guix.mo
    /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
    /share/info/guix-cookbook.de.info.gz
    /lib/guile/3.0/site-ccache/guix/workers.go
    /lib/guile/3.0/site-ccache/guix/ui.go
    /lib/guile/3.0/site-ccache/guix/swh.go
    /lib/guile/3.0/site-ccache/guix/svn-download.go
[…]
--8<---------------cut here---------------end--------------->8---

‘--diff=diffoscope’ is not an option here because it takes too long
looking at all the .go files…

A focused diff shows this:

--8<---------------cut here---------------start------------->8---
$ diffoscope --exclude-directory-metadata=yes /tmp/{t1,t2}/share/locale/en@quot/
--- /tmp/t1/share/locale/en@quot/
+++ /tmp/t2/share/locale/en@quot/
│   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
│ │   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
│ │ ├── msgunfmt {}
│ │ │ @@ -1,12 +1,12 @@
│ │ │  msgid ""
│ │ │  msgstr ""
│ │ │  "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
│ │ │  "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
│ │ │  "Last-Translator: Automatically generated\n"
│ │ │  "Language-Team: none\n"
│ │ │  "Language: en@quot\n"
│ │ │  "MIME-Version: 1.0\n"
│ │ │  "Content-Type: text/plain; charset=UTF-8\n"
│ │ │  "Content-Transfer-Encoding: 8bit\n"
│ │ │  "Plural-Forms: nplurals=2; plural=(n != 1);\n"
│ │   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
│ │ ├── msgunfmt {}
│ │ │ @@ -1,12 +1,12 @@
│ │ │  msgid ""
│ │ │  msgstr ""
│ │ │  "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
│ │ │  "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
│ │ │  "Last-Translator: Automatically generated\n"
│ │ │  "Language-Team: none\n"
│ │ │  "Language: en@quot\n"
│ │ │  "MIME-Version: 1.0\n"
│ │ │  "Content-Type: text/plain; charset=UTF-8\n"
│ │ │  "Content-Transfer-Encoding: 8bit\n"
│ │ │  "Plural-Forms: nplurals=2; plural=(n != 1);\n"
--8<---------------cut here---------------end--------------->8---

Ludo’.

bug#26247: Gettext introduces timestamps in .mo files

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 4275 bytes --]

So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

Le 3 décembre 2020 05:32:54 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hi Simon,
>
>zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>>
>>> Gettext 0.19.8.1 (current core-updates,
>>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in
>.mo
>>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>>> non-reproducible builds (for example ‘guix’).
>>
>> Is it still relevant?  Since Gettext is now at 0.20.1.  How can I
>> reproduce the issue?
>
>I still see this:
>
>--8<---------------cut here---------------start------------->8---
>$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org
>https://bayfront.guix.gnu.org"
>updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
>/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe
>contents differ:
>no local build for
>'/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
>https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
>https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
>  differing files:
>    /share/locale/en@quot/LC_MESSAGES/guix.mo
>    /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>    /share/locale/en@boldquot/LC_MESSAGES/guix.mo
>    /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
>    /share/info/guix-cookbook.de.info.gz
>    /lib/guile/3.0/site-ccache/guix/workers.go
>    /lib/guile/3.0/site-ccache/guix/ui.go
>    /lib/guile/3.0/site-ccache/guix/swh.go
>    /lib/guile/3.0/site-ccache/guix/svn-download.go
>[…]
>--8<---------------cut here---------------end--------------->8---
>
>‘--diff=diffoscope’ is not an option here because it takes too long
>looking at all the .go files…
>
>A focused diff shows this:
>
>--8<---------------cut here---------------start------------->8---
>$ diffoscope --exclude-directory-metadata=yes
>/tmp/{t1,t2}/share/locale/en@quot/
>--- /tmp/t1/share/locale/en@quot/
>+++ /tmp/t2/share/locale/en@quot/
>│   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
>├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
>│ │   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>│ │ ├── msgunfmt {}
>│ │ │ @@ -1,12 +1,12 @@
>│ │ │  msgid ""
>│ │ │  msgstr ""
>│ │ │  "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>│ │ │  "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>│ │ │  "Last-Translator: Automatically generated\n"
>│ │ │  "Language-Team: none\n"
>│ │ │  "Language: en@quot\n"
>│ │ │  "MIME-Version: 1.0\n"
>│ │ │  "Content-Type: text/plain; charset=UTF-8\n"
>│ │ │  "Content-Transfer-Encoding: 8bit\n"
>│ │ │  "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>│ │   --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
>│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
>│ │ ├── msgunfmt {}
>│ │ │ @@ -1,12 +1,12 @@
>│ │ │  msgid ""
>│ │ │  msgstr ""
>│ │ │  "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>│ │ │  "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>│ │ │  "Last-Translator: Automatically generated\n"
>│ │ │  "Language-Team: none\n"
>│ │ │  "Language: en@quot\n"
>│ │ │  "MIME-Version: 1.0\n"
>│ │ │  "Content-Type: text/plain; charset=UTF-8\n"
>│ │ │  "Content-Transfer-Encoding: 8bit\n"
>│ │ │  "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>--8<---------------cut here---------------end--------------->8---
>
>Ludo’.

[-- Attachment #2: Type: text/html, Size: 5140 bytes --]

bug#26247: Gettext introduces timestamps in .mo files

[Ludovic Courtès]

Julien Lepiller <julien@lepiller.eu> skribis:

> So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

They’re generated automatically by the gettext Makefilery that we use,
so I don’t think there’s much we can do?

We could remove them from ‘LINGUAS’, but we’d be losing something,
wouldn’t we?

Ludo’.

bug#26247: Gettext introduces timestamps in .mo files

[Miguel Ángel Arruga Vivas]

Hi!

Ludovic Courtès <ludo@gnu.org> writes:

> Julien Lepiller <julien@lepiller.eu> skribis:
>
>> So it's not gettext itself, but our build system that generates the
>> en@quote and en@boldquote files. Do we really need them? If so, we
>> should find a way to generate them reproducibly.
>
> They’re generated automatically by the gettext Makefilery that we use,
> so I don’t think there’s much we can do?

The issue isn't on those files but on POT generation.  Currently
xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
which is used to fill PO-Revision-Date for auto-generated po files.
These files are included into tarballs generated by make dist, therefore
its date is already fixed, but they aren't present on our git tree---for
obvious reasons.

This bug has already been reported upstream[1] and probably I'll push it
as soon as I have more test cases prepared and receive a brief review,
but I can prepare a patch for previous versions if it's needed too.

Best regards,
Miguel

[1] https://savannah.gnu.org/bugs/?59658

bug#26247: Gettext introduces timestamps in .mo files

[Ludovic Courtès]

Hi Miguel,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Julien Lepiller <julien@lepiller.eu> skribis:
>>
>>> So it's not gettext itself, but our build system that generates the
>>> en@quote and en@boldquote files. Do we really need them? If so, we
>>> should find a way to generate them reproducibly.
>>
>> They’re generated automatically by the gettext Makefilery that we use,
>> so I don’t think there’s much we can do?
>
> The issue isn't on those files but on POT generation.  Currently
> xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
> which is used to fill PO-Revision-Date for auto-generated po files.
> These files are included into tarballs generated by make dist, therefore
> its date is already fixed, but they aren't present on our git tree---for
> obvious reasons.

OK.

> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.

[...]

> [1] https://savannah.gnu.org/bugs/?59658

Thanks for getting to the bottom of this and for preparing a patch
upstream!

In ‘core-updates’, we could either wait for the next Gettext release or
apply your patch in the meantime.

Ludo’.

bug#26247: Gettext introduces timestamps in .mo files

[zimoun]

Hi Miguel,

It is about an old patch [2] and on old message from Dec. 2020.

On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:

> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.
>
> Best regards,
> Miguel
>
> [1] https://savannah.gnu.org/bugs/?59658

Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
ones mentioned here?

Well, I am not sure to understand if it had been fixed upstream [1].


Cheers,
simon

2: <http://issues.guix.gnu.org/issue/26247>

bug#26247: Gettext introduces timestamps in .mo files

[Simon Tournier]

Hi,

It is about this old bug#26247:

    https://issues.guix.gnu.org/issue/26247

On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>
>> This bug has already been reported upstream[1] and probably I'll push it
>> as soon as I have more test cases prepared and receive a brief review,
>> but I can prepare a patch for previous versions if it's needed too.
>>
>> Best regards,
>> Miguel
>>
>> [1] https://savannah.gnu.org/bugs/?59658
>
> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
> ones mentioned here?
>
> Well, I am not sure to understand if it had been fixed upstream [1].

The issue is still not fixed in Guix.

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
  no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
  https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
  https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
  differing files:

[...]

    /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
    /share/locale/en@boldquot/LC_MESSAGES/guix.mo
    /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
    /share/locale/en@quot/LC_MESSAGES/guix.mo

1 store items were analyzed:
  - 0 (0.0%) were identical
  - 1 (100.0%) differed
  - 0 (0.0%) were inconclusive
--8<---------------cut here---------------end--------------->8---

Cheers,
simon

bug#26247: Gettext introduces timestamps in .mo files

[Julien Lepiller]

It's interesting to see that only the generate po files are compiled to non-reproducible mo files. All other translations seem fine, right? Could it be related to the way these po files are created? It might introduce a timestamp at that point, but I can't check right now.

Le 17 octobre 2023 01:48:40 GMT+02:00, Simon Tournier <zimon.toutoune@gmail.com> a écrit :
>Hi,
>
>It is about this old bug#26247:
>
>    https://issues.guix.gnu.org/issue/26247
>
>On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
>> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>>
>>> This bug has already been reported upstream[1] and probably I'll push it
>>> as soon as I have more test cases prepared and receive a brief review,
>>> but I can prepare a patch for previous versions if it's needed too.
>>>
>>> Best regards,
>>> Miguel
>>>
>>> [1] https://savannah.gnu.org/bugs/?59658
>>
>> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
>> ones mentioned here?
>>
>> Well, I am not sure to understand if it had been fixed upstream [1].
>
>The issue is still not fixed in Guix.
>
>--8<---------------cut here---------------start------------->8---
>$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
>/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
>  no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
>  https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
>  https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
>  differing files:
>
>[...]
>
>    /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
>    /share/locale/en@boldquot/LC_MESSAGES/guix.mo
>    /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>    /share/locale/en@quot/LC_MESSAGES/guix.mo
>
>1 store items were analyzed:
>  - 0 (0.0%) were identical
>  - 1 (100.0%) differed
>  - 0 (0.0%) were inconclusive
>--8<---------------cut here---------------end--------------->8---
>
>Cheers,
>simon