bug#65241: 30.0.50; Cannot clone NonGNU ELPA

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Manuel Uberti via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

I am trying to clone the NonGNU ELPA repository in order to contribute
with a patch, but this is what happens:

$ git clone git://git.savannah.gnu.org/emacs/nongnu.git
Cloning into 'nongnu'...
remote: Counting objects: 425227, done.
remote: Compressing objects: 100% (145570/145570), done.
error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c: zeroPaddedFilemode: contains zero-padded file modes
fatal: fsck error in packed object
fatal: fetch-pack: invalid index-pack output


All the best
Manuel


In GNU Emacs 30.0.50 (build 2, x86_64-pc-linux-gnu, cairo version
 1.16.0) of 2023-08-12 built on baruch
Repository revision: be3cae9f559015219af07dd63170dded2fb64056
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201001
System Description: Ubuntu 22.04.3 LTS

Configured using:
 'configure --with-native-compilation --with-x-toolkit=no'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND
SQLITE3 THREADS TIFF TREE_SITTER X11 XDBE XIM XINPUT2 XPM ZLIB

Important settings:
  value of $LC_MONETARY: it_IT.UTF-8
  value of $LC_NUMERIC: it_IT.UTF-8
  value of $LC_TIME: it_IT.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
  goto-address-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  windmove-mode: t
  comint-fontify-input-mode: t
  savehist-mode: t
  winner-mode: t
  vertico-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  window-divider-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/manuel/.emacs.d/elpa/transient-0.4.1.0.20230602.212120/transient hides /usr/local/share/emacs/30.0.50/lisp/transient

Features:
(shadow sort buffer-env jinx mail-extr emacsbug message yank-media puny
dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068
epg-config gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils vertico-directory orderless consult
bookmark pp pcmpl-unix sh-script smie treesit executable goto-addr
files-x shell pcomplete server modus-operandi-theme modus-themes
windmove delight comp comp-cstr rx elec-pair time-date checkdoc lisp-mnt
flymake-proc flymake project compile text-property-search comint
ansi-osc ansi-color warnings icons thingatpt savehist winner ring
vertico compat pdf-loader pcase ibuf-macs delsel cl-extra help-mode
autorevert filenotify info buffer-env-autoloads cape-autoloads
cider-autoloads clojure-mode-autoloads clojure-ts-mode-autoloads
consult-autoloads delight-autoloads dockerfile-mode-autoloads
haskell-mode-autoloads jinx-autoloads latex-table-wizard-autoloads
auctex-autoloads tex-site markdown-mode-autoloads nginx-mode-autoloads
orderless-autoloads parseedn-autoloads parseclj-autoloads
pdf-tools-autoloads queue-autoloads rainbow-mode-autoloads
relint-autoloads sesman-autoloads spinner-autoloads tablist-autoloads
transient-autoloads vertico-autoloads compat-autoloads wgrep-autoloads
xr-autoloads yaml-mode-autoloads package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd touch-screen tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo xinput2 x multi-tty move-toolbar
make-network-process native-compile emacs)

Memory information:
((conses 16 244821 140294) (symbols 48 15446 1) (strings 32 50089 3014)
 (string-bytes 1 1571638) (vectors 16 27452) (vector-slots 8 529327 18903)
 (floats 8 101 319) (intervals 56 2061 71) (buffers 992 15))

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Corwin Brust]

tags: notabug
thanks

On Sat, Aug 12, 2023 at 1:01 AM Manuel Uberti via Bug reports for GNU
Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org>
wrote:
>
>
> Hi,

Hi Manuel

>
> I am trying to clone the NonGNU ELPA repository in order to contribute
> with a patch, but this is what happens:
>
> $ git clone git://git.savannah.gnu.org/emacs/nongnu.git

Unless you have commit for Emacs you will want to use the anonymous
clone option:

  git clone https://git.savannah.gnu.org/git/emacs/nongnu.git

If you do have commit then you are missing providing your username:

This is documented on the Emacs project page on savannah:

  https://savannah.gnu.org/git/?group=emacs

> All the best
> Manuel

Corwin

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Manuel Uberti via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 12/08/23 08:13, Corwin Brust wrote:
> Hi Manuel

Hi Corwin, thanks for the quick reply.

> Unless you have commit for Emacs you will want to use the anonymous
> clone option:
> 
>    git clone https://git.savannah.gnu.org/git/emacs/nongnu.git
> 
> If you do have commit then you are missing providing your username:
> 
> This is documented on the Emacs project page on savannah:
> 
>    https://savannah.gnu.org/git/?group=emacs

Following the anonymous clone option, I am still getting the error though:

$ git clone https://git.savannah.gnu.org/git/emacs/nongnu.git
Cloning into 'nongnu'...
remote: Counting objects: 425227, done.
remote: Compressing objects: 100% (145570/145570), done.
error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c: 
zeroPaddedFilemode: contains zero-padded file modes
fatal: fsck error in packed object
fatal: fetch-pack: invalid index-pack output

-- 
Manuel Uberti
https://manueluberti.eu

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Andreas Schwab]

On Aug 12 2023, Corwin Brust wrote:

> tags: notabug
> thanks
>
> On Sat, Aug 12, 2023 at 1:01 AM Manuel Uberti via Bug reports for GNU
> Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org>
> wrote:
>>
>>
>> Hi,
>
> Hi Manuel
>
>>
>> I am trying to clone the NonGNU ELPA repository in order to contribute
>> with a patch, but this is what happens:
>>
>> $ git clone git://git.savannah.gnu.org/emacs/nongnu.git
>
> Unless you have commit for Emacs you will want to use the anonymous
> clone option:

git:// _is_ anonymous.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Michael Albinus]

Manuel Uberti via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Hi,

Hi Manuel,

> I am trying to clone the NonGNU ELPA repository in order to contribute
> with a patch, but this is what happens:
>
> $ git clone git://git.savannah.gnu.org/emacs/nongnu.git
> Cloning into 'nongnu'...
> remote: Counting objects: 425227, done.
> remote: Compressing objects: 100% (145570/145570), done.
> error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c: zeroPaddedFilemode: contains zero-padded file modes
> fatal: fsck error in packed object
> fatal: fetch-pack: invalid index-pack output

Try

--8<---------------cut here---------------start------------->8---
# git config --global transfer.fsckObjects false
--8<---------------cut here---------------end--------------->8---

> All the best
> Manuel

Best regards, Michael.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Manuel Uberti via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 12/08/23 08:54, Michael Albinus wrote:
> Hi Manuel,

Hi Michael

> Try
> 
> --8<---------------cut here---------------start------------->8---
> # git config --global transfer.fsckObjects false
> --8<---------------cut here---------------end--------------->8---

Thanks, changing transfer.fsckObjects makes it work. Is it really a 
problem with my Git configuration, though?

-- 
Manuel Uberti
https://manueluberti.eu

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Michael Albinus]

Manuel Uberti <manuel.uberti@inventati.org> writes:

> Hi Michael

Hi Manuel,

>> Try
>> --8<---------------cut here---------------start------------->8---
>> # git config --global transfer.fsckObjects false
>> --8<---------------cut here---------------end--------------->8---
>
> Thanks, changing transfer.fsckObjects makes it work. Is it really a
> problem with my Git configuration, though?

No. IIRC, there are commits in ELPA which have zero-padded file
modes. It doesn't seem to be trivial to fix it in the git repo, and so
we need this change. This was discussed somewhere in the mailing lists,
but I have no reference.

I don't know git sufficiently to know, whether this setting could be
restricted just to the elpa repos. But at least it should be added to
the respective README and README.org of the repositories.

Best regards, Michael.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Manuel Uberti via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 12/08/23 09:03, Michael Albinus wrote:
> No. IIRC, there are commits in ELPA which have zero-padded file
> modes. It doesn't seem to be trivial to fix it in the git repo, and so
> we need this change. This was discussed somewhere in the mailing lists,
> but I have no reference.
> 
> I don't know git sufficiently to know, whether this setting could be
> restricted just to the elpa repos. But at least it should be added to
> the respective README and README.org of the repositories.

Fantastic, thanks for the clarification.

> Best regards, Michael.

All the best.

-- 
Manuel Uberti
https://manueluberti.eu

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Manuel Uberti writes:

> I am trying to clone the NonGNU ELPA repository in order to contribute
> with a patch, but this is what happens:
>
> $ git clone git://git.savannah.gnu.org/emacs/nongnu.git
> Cloning into 'nongnu'...
> remote: Counting objects: 425227, done.
> remote: Compressing objects: 100% (145570/145570), done.
> error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c: zeroPaddedFilemode: contains zero-padded file modes
> fatal: fsck error in packed object
> fatal: fetch-pack: invalid index-pack output

I thought we fixed that, but maybe that was only for GNU ELPA?

Stefan M, do you have any comments?  Is there anything we could do here?

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> $ git clone git://git.savannah.gnu.org/emacs/nongnu.git
>> Cloning into 'nongnu'...
>> remote: Counting objects: 425227, done.
>> remote: Compressing objects: 100% (145570/145570), done.
>> error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c:
>> zeroPaddedFilemode: contains zero-padded file modes
>> fatal: fsck error in packed object
>> fatal: fetch-pack: invalid index-pack output
>
> I thought we fixed that, but maybe that was only for GNU ELPA?

Indeed we had such a problem over in `elpa.git` and we solved it, but
that problem can re-appear any time we pull from some (old(enough))
upstream repository.

> Stefan M, do you have any comments?  Is there anything we could do here?

`git show 2699d230e3b592ae42506d7b5c969a7ac6a4593c` suggests this is
part of `coffee-mode`, and indeed:

    git -c 'Transfer.fsckObjects=true' clone https://github.com/defunkt/coffee-mode

gives me the same warning (tho it's a warning rather than an error).

Just use

    git -c 'Transfer.fsckObjects=false' clone ...

to workaround the problem.  I think the problem is fundamentally in
Git's camp: the format that `fsckObjects` enforces is incompatible with
the data it used to generate, and since rewriting history is a big
problem, it's not something that can be fixed.


        Stefan

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> `git show 2699d230e3b592ae42506d7b5c969a7ac6a4593c` suggests this is
> part of `coffee-mode`, and indeed:
>
>     git -c 'Transfer.fsckObjects=true' clone https://github.com/defunkt/coffee-mode
>
> gives me the same warning (tho it's a warning rather than an error).
>
> Just use
>
>     git -c 'Transfer.fsckObjects=false' clone ...
>
> to workaround the problem.  I think the problem is fundamentally in
> Git's camp: the format that `fsckObjects` enforces is incompatible with
> the data it used to generate, and since rewriting history is a big
> problem, it's not something that can be fixed.

I don't think users will realize that, though.  They'll blame us for
having a broken repository.

So I'd personally very much prefer rewriting the git history to living
with the ugly workaround until the end of days.  I guess that was the
position of the Git developers as well (or they would have fixed it).

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas [2023-09-05 16:25:05] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> `git show 2699d230e3b592ae42506d7b5c969a7ac6a4593c` suggests this is
>> part of `coffee-mode`, and indeed:
>>
>>     git -c 'Transfer.fsckObjects=true' clone https://github.com/defunkt/coffee-mode
>>
>> gives me the same warning (tho it's a warning rather than an error).
>>
>> Just use
>>
>>     git -c 'Transfer.fsckObjects=false' clone ...
>>
>> to workaround the problem.  I think the problem is fundamentally in
>> Git's camp: the format that `fsckObjects` enforces is incompatible with
>> the data it used to generate, and since rewriting history is a big
>> problem, it's not something that can be fixed.
>
> I don't think users will realize that, though.  They'll blame us for
> having a broken repository.

Could be, but we (and many other repositories) have lived with it for
several years already.  I think the problem is on Git side: it should be
a warning rather than an error (AFAICT, `git fsck` flagged the problem
as a mere warning).

And indeed, when I try it now, I don't get an error any more, so maybe
it's been fixed in more recent versions of Git?
When you try

    git -c 'Transfer.fsckObjects=true' clone https://github.com/defunkt/coffee-mode

do you get an error or a warning?  I get a warning here with Git-2.39.2
from Debian stable.


        Stefan

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> And indeed, when I try it now, I don't get an error any more, so maybe
> it's been fixed in more recent versions of Git?
> When you try
>
>     git -c 'Transfer.fsckObjects=true' clone https://github.com/defunkt/coffee-mode
>
> do you get an error or a warning?  I get a warning here with Git-2.39.2
> from Debian stable.

I get an error using the most recent Git version from Homebrew on macOS:

$ git --version
git version 2.42.0
$ git -c 'Transfer.fsckObjects=true' clone
https://github.com/defunkt/coffee-mode
Cloning into 'coffee-mode'...
remote: Enumerating objects: 1847, done.
error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c:
zeroPaddedFilemode: contains zero-padded file modes
fatal: fsck error in packed object
fatal: fetch-pack: invalid index-pack output
$

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I get an error using the most recent Git version from Homebrew on macOS:
>
> $ git --version
> git version 2.42.0
> $ git -c 'Transfer.fsckObjects=true' clone
> https://github.com/defunkt/coffee-mode
> Cloning into 'coffee-mode'...
> remote: Enumerating objects: 1847, done.
> error: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c:
> zeroPaddedFilemode: contains zero-padded file modes
> fatal: fsck error in packed object
> fatal: fetch-pack: invalid index-pack output
> $

Duh!  I now see that it's because I have

    [fetch "fsck"]
            zeroPaddedFilemode = warn

in my `~/.gitconfig` (added by gremlins, presumably).

As for rewriting coffee-mode's history to avoid the problem: you need to
convince coffee-mode's upstream to do that first.
[ And you might want to check if coffee-mode is really the only branch
  that bumps into this problem.  ]


        Stefan


PS: Now that I think about it, maybe there's another way, which is to
use a *shallow* branch so the old offending commit is not stored in
`nongnu.git`.
It would hide all the coffee-mode history starting with (and
including) 20-May-2010 commit 2a59ce05bd, i.e. most of Chris Wanstrath's
contributions :-(
It would also take some help from a Savannah admin, but other than that
it seems like it would do the trick: basically just add a file
`.../nongnu.git/shallow` with a single line that says
`b1c16e50e93c662a12c49ac8f18b607656238361`.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

[Copyi

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> As for rewriting coffee-mode's history to avoid the problem: you need to
> convince coffee-mode's upstream to do that first.

This ticket has been open since 2018:

    https://github.com/defunkt/coffee-mode/issues/367

I pinged upstream again now on GitHub.  I couldn't find an email
address, so I couldn't Cc them here.

> [ And you might want to check if coffee-mode is really the only branch
>   that bumps into this problem.  ]

Checking with

    git -c 'fetch.fsck.zeroPaddedFilemode=warn' ${NONGNU_ELPA}

yields only the one warning

    warning: object 2699d230e3b592ae42506d7b5c969a7ac6a4593c: \
    zeroPaddedFilemode: contains zero-padded file modes

> PS: Now that I think about it, maybe there's another way, which is to
> use a *shallow* branch so the old offending commit is not stored in
> `nongnu.git`.
> It would hide all the coffee-mode history starting with (and
> including) 20-May-2010 commit 2a59ce05bd, i.e. most of Chris Wanstrath's
> contributions :-(
> It would also take some help from a Savannah admin, but other than that
> it seems like it would do the trick: basically just add a file
> `.../nongnu.git/shallow` with a single line that says
> `b1c16e50e93c662a12c49ac8f18b607656238361`.

A shallow branch sounds like the best option to me.  It's unfortunate
with the attribution of course, but the full history is preserved on
GitHub, so people can still find it.  We can go back to a regular branch
later, if it gets fixed upstream.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> A shallow branch sounds like the best option to me.  It's unfortunate
> with the attribution of course, but the full history is preserved on
> GitHub, so people can still find it.

We can still preserve the rest of the history (tho on another branch).

I opened ticket #110931 on Savannah.


        Stefan

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I opened ticket #110931 on Savannah.

Thanks.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Corwin Brust]

On Wed, Sep 6, 2023 at 4:10 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> > I opened ticket #110931 on Savannah.
>

Now marked ready to test.

> Thanks.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Corwin Brust <corwin@bru.st> writes:

> On Wed, Sep 6, 2023 at 4:10 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>>
>> > I opened ticket #110931 on Savannah.
>>
>
> Now marked ready to test.

Great, thank you.  I propose that we move ahead with the shallow branch.

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Kangas [2023-09-08 03:59:19] wrote:
> Corwin Brust <corwin@bru.st> writes:
>> On Wed, Sep 6, 2023 at 4:10 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>>> > I opened ticket #110931 on Savannah.
>> Now marked ready to test.
> Great, thank you.  I propose that we move ahead with the shallow branch.

It's done already.


        Stefan

bug#65241: 30.0.50; Cannot clone NonGNU ELPA

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Stefan Kangas [2023-09-08 03:59:19] wrote:
>> Corwin Brust <corwin@bru.st> writes:
>>> On Wed, Sep 6, 2023 at 4:10 PM Stefan Kangas <stefankangas@gmail.com> wrote:
>>>> > I opened ticket #110931 on Savannah.
>>> Now marked ready to test.
>> Great, thank you.  I propose that we move ahead with the shallow branch.
>
> It's done already.

Great, thanks.  I'm closing this bug then.