bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick]

I realize this is vague, but I'm seeing it consistently:

I build emacs 30 with native compilation on my M1, and I open my work clojure project, and I see rainbow parens marking a bunch of unmatched parens.
The file will not compile, and the corruption happens simply when I open the file.

I do the exact same thing with emacs 29 built the same way, and I have no issues.

How can I narrow this down to a useful bug report?

Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Eli Zaretskii]

> Date: Thu, 16 Mar 2023 00:01:12 +0000
> From: Jonathon McKitrick <jcm@SDF.ORG>
> 
> I realize this is vague, but I'm seeing it consistently:
> 
> I build emacs 30 with native compilation on my M1, and I open my work clojure project, and I see rainbow parens marking a bunch of unmatched parens.
> The file will not compile, and the corruption happens simply when I open the file.
> 
> I do the exact same thing with emacs 29 built the same way, and I have no issues.
> 
> How can I narrow this down to a useful bug report?

Thanks for asking, and for the bug report in the first place.

The way to narrow this down is start with "emacs -Q", add only those
of your customizations necessary for reproducing the problem, then
post the results in the form of step by step description of the
reproduction recipe.  It will help a lot if you also post the file
which gets corrupted, both before and after the corruption.  If that
file can be small, that's preferable.

I'm guessing that you use some third-party packages, at least for
editing clojure sources.  In that case, include in your description
the packages one should download to reproduce the issue.

A description of the recipe should, therefore, look something like the
following:

  . download and install the following packages:
    - foo
    - bar
    - ...
  . start "emacs -Q"
  . type the following commands:
    - M-x load-library RET foo RET
    - M-x load-library RET bar RET
    - M-x set-variable RET foo-some-var RET t RET
    -...
  . visit the attached file foobar.clj
  . do <something> to trigger the corrption
  . observe the changes in the file's contents:
    - <description of the corruptions>

TIA

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick]

On Thu, Mar 16, 2023 at 08:50:06AM +0200, Eli Zaretskii wrote:
: > Date: Thu, 16 Mar 2023 00:01:12 +0000
: > From: Jonathon McKitrick <jcm@SDF.ORG>
: > 
: > I realize this is vague, but I'm seeing it consistently:
: > 
: > I build emacs 30 with native compilation on my M1, and I open my work clojure project, and I see rainbow parens marking a bunch of unmatched parens.
: > The file will not compile, and the corruption happens simply when I open the file.
: > 
: > I do the exact same thing with emacs 29 built the same way, and I have no issues.
: > 
: > How can I narrow this down to a useful bug report?
: 
: Thanks for asking, and for the bug report in the first place.
: 
: The way to narrow this down is start with "emacs -Q", add only those
: of your customizations necessary for reproducing the problem, then
: post the results in the form of step by step description of the
: reproduction recipe.  It will help a lot if you also post the file
: which gets corrupted, both before and after the corruption.  If that
: file can be small, that's preferable.
: 
: I'm guessing that you use some third-party packages, at least for
: editing clojure sources.  In that case, include in your description
: the packages one should download to reproduce the issue.
: 
: A description of the recipe should, therefore, look something like the
: following:
: 
:   . download and install the following packages:
:     - foo
:     - bar
:     - ...
:   . start "emacs -Q"
:   . type the following commands:
:     - M-x load-library RET foo RET
:     - M-x load-library RET bar RET
:     - M-x set-variable RET foo-some-var RET t RET
:     -...
:   . visit the attached file foobar.clj
:   . do <something> to trigger the corrption
:   . observe the changes in the file's contents:
:     - <description of the corruptions>
: 
: TIA

I'm frustrated that I cannot reproduce the bug reliably, but it does happen constantly under emacs 30. What I have narrowed it down to
is what seems to be a random injection of code from the same file or the kill ring. That of course trips up paredit and any compilation.
I check the messages, and cannot tell if there's a random command or key binding that triggers a random command responsible for the issue.
I happens with and without native compilation. Since I cannot predict *when* it's going to happen, it's difficult to get work done with
packages disabled, not knowing when I'll see the bug.

Has anything like this been reported anywhere else?



Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Eli Zaretskii]

> Date: Tue, 16 May 2023 20:22:24 +0000
> From: Jonathon McKitrick <jcm@SDF.ORG>
> Cc: 62216@debbugs.gnu.org
> 
> I'm frustrated that I cannot reproduce the bug reliably, but it does happen constantly under emacs 30. What I have narrowed it down to
> is what seems to be a random injection of code from the same file or the kill ring. That of course trips up paredit and any compilation.
> I check the messages, and cannot tell if there's a random command or key binding that triggers a random command responsible for the issue.
> I happens with and without native compilation. Since I cannot predict *when* it's going to happen, it's difficult to get work done with
> packages disabled, not knowing when I'll see the bug.
> 
> Has anything like this been reported anywhere else?

There's bug#63187, which is perhaps similar?  It is also on macOS.

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Dmitry Gutov]

On 16/05/2023 23:22, Jonathon McKitrick wrote:
> I'm frustrated that I cannot reproduce the bug reliably, but it does happen constantly under emacs 30. What I have narrowed it down to
> is what seems to be a random injection of code from the same file or the kill ring. That of course trips up paredit and any compilation.

Something vaguely similar: I had a problem, in one session, that 
reindenting any line or region would, instead of simple whitespace for 
indentation, insert some repeated chunks of code from I don't know where.

I didn't investigate this, though. This only happened once and a restart 
fixed it.

I don't use desktop-mode. If you do, though, it might be carrying over 
some variable(s) from the previous sessions which can trigger the 
problem. The desktop save file, in that case, could help reproduce it 
(either you would post it as a part of recipe, or you would just read it 
through and try applying the values in there one by one until the 
problem resurfaces).

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Gerd Möllmann]

> On 16/05/2023 23:22, Jonathon McKitrick wrote:
>> I'm frustrated that I cannot reproduce the bug reliably, but it does happen constantly under emacs 30. What I have narrowed it down to
>> is what seems to be a random injection of code from the same file or the kill ring. That of course trips up paredit and any compilation.
> 
> Something vaguely similar: I had a problem, in one session, that 
> reindenting any line or region would, instead of simple whitespace for 
> indentation, insert some repeated chunks of code from I don't know where.
> 
> I didn't investigate this, though. This only happened once and a restart 
> fixed it.

Found this bug-report just now.  I'm like Jonathan on an M1 mac, with 
Ventura 13.5.1, and I have seen buffer corruptions happening.

The most recent example was edebug.el, which got auto-reverted because I 
discarded a change in Magit.  When I tried to eval-buffer edebug.el, I 
found that a sequence of closing parentheses had been inserted in the 
buffer which were not present in the file on disk.

Alas, I accidentally hit C-c in the LLDB window, and that was it.

Is this perhaps a macOS thing?  Has somone seen that on other platforms?

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Dmitry Gutov]

On 30/08/2023 10:50, Gerd Möllmann wrote:
>> On 16/05/2023 23:22, Jonathon McKitrick wrote:
>>> I'm frustrated that I cannot reproduce the bug reliably, but it does 
>>> happen constantly under emacs 30. What I have narrowed it down to
>>> is what seems to be a random injection of code from the same file or 
>>> the kill ring. That of course trips up paredit and any compilation.
>>
>> Something vaguely similar: I had a problem, in one session, that 
>> reindenting any line or region would, instead of simple whitespace for 
>> indentation, insert some repeated chunks of code from I don't know where.
>>
>> I didn't investigate this, though. This only happened once and a 
>> restart fixed it.
> 
> Found this bug-report just now.  I'm like Jonathan on an M1 mac, with 
> Ventura 13.5.1, and I have seen buffer corruptions happening.
> 
> The most recent example was edebug.el, which got auto-reverted because I 
> discarded a change in Magit.  When I tried to eval-buffer edebug.el, I 
> found that a sequence of closing parentheses had been inserted in the 
> buffer which were not present in the file on disk.
> 
> Alas, I accidentally hit C-c in the LLDB window, and that was it.
> 
> Is this perhaps a macOS thing?  Has somone seen that on other platforms?

I'm on GNU/Linux, but the behavior I described above (and which hasn't 
repeated since then, so far) was when indenting code, not on 
eval-buffer. So it could be an entirely different thing. Or a similar 
memory corruption, no idea.

bug#62216: Narrowing down the issue

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

After some exhaustive git bisecting and init file tweaking, I've narrowed it down to a surprising culprit: desktop mode

I've been unable to reproduce the issue in my other projects, but I'm still working on it.

The commit that introduces the bug is b950b46f514989442fdd9937a0e96d53a3affa88.

I don't think it's a bug per-se, simply a problematic interaction with desktop mode.
I've been able to build a non-graphic version from source that reverts this commit and does not have the issue.
I'm currently working on a local homebrew tap that will build a macos version, applying the same patch to my local build.

Here's the latest:

Emacs 30 HEAD on MacOS, built weekly, sometimes nightly.

I open a large clojure project to a file with many changes.
Then I checkout another branch, either in a terminal or using magit.
When the file I'm viewing reverts, there seems to be a large chunk of code that was erroneously inserted at the end of the buffer.
This code is part of the diff, appended in the wrong place.

I don't have any other projects that present the same issue so far.


Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Eli Zaretskii]

> Date: Sun, 23 Jul 2023 13:58:34 +0000
> From:  Jonathon McKitrick via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> After some exhaustive git bisecting and init file tweaking, I've narrowed it down to a surprising culprit: desktop mode

What do you mean by "desktop mode"?

P.S. Please don't modify the Subject line when you respond: doing that
makes it harder to track the discussion thread.

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Jul 23, 2023 at 05:35:46PM +0300, Eli Zaretskii wrote:
: > Date: Sun, 23 Jul 2023 13:58:34 +0000
: > From:  Jonathon McKitrick via "Bug reports for GNU Emacs,
: >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
: > 
: > After some exhaustive git bisecting and init file tweaking, I've narrowed it down to a surprising culprit: desktop mode
: 
: What do you mean by "desktop mode"?

I mean 'desktop save mode'.

: P.S. Please don't modify the Subject line when you respond: doing that
: makes it harder to track the discussion thread.

I lost the original thread and had to improvise lol.


Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Jul 23, 2023 at 05:35:46PM +0300, Eli Zaretskii wrote:
: > Date: Sun, 23 Jul 2023 13:58:34 +0000
: > From:  Jonathon McKitrick via "Bug reports for GNU Emacs,
: >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
: > 
: > After some exhaustive git bisecting and init file tweaking, I've narrowed it down to a surprising culprit: desktop mode

So apparently desktop save mode isn't the culprit, just the main agitator. It seems the issue involves reverting buffers, and any mode that does that.
Disabling desktop save mode reduces the issue frequency, but I'm still seeing it in my daily workflow, which involves CIDER and clojure.

I'll keep researching....


Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Narrowed the issue to one line

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Progress!

This line is the culprit, around 4807:

ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);

When I replace it with this, from an older version:

ptrdiff_t trytry = min (total - inserted, READ_BUF_SIZE);

The problem disappears.

This change was made to address bug #9800, when inserting large files from /proc.

I don't understand the code well enough to understand why this change is causing the issue I'm seeing,
or if my 'fix' will break something else. I just know it's working for me and doesn't seem to cause issues otherwise.
However, it ignores 'gap_size', so I'm pretty sure that's not the correct way to fix this code.
Maybe it should be used 2 lines below? Again, I don't have the context to do anything but guess.

I'll keep testing locally, but I'd be interested in feedback on this change.

Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Narrowed the issue to one line

[Eli Zaretskii]

> Date: Sun, 20 Aug 2023 13:38:26 +0000
> From:  Jonathon McKitrick via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Progress!
> 
> This line is the culprit, around 4807:
> 
> ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
> 
> When I replace it with this, from an older version:
> 
> ptrdiff_t trytry = min (total - inserted, READ_BUF_SIZE);
> 
> The problem disappears.
> 
> This change was made to address bug #9800, when inserting large files from /proc.
> 
> I don't understand the code well enough to understand why this change is causing the issue I'm seeing,
> or if my 'fix' will break something else. I just know it's working for me and doesn't seem to cause issues otherwise.
> However, it ignores 'gap_size', so I'm pretty sure that's not the correct way to fix this code.
> Maybe it should be used 2 lines below? Again, I don't have the context to do anything but guess.
> 
> I'll keep testing locally, but I'd be interested in feedback on this change.

Po Lu, Paul: any idea why that could be the case in the OP's scenario?
The only thing that comes to my mind is some weirdness of the OP's
filesystem.

bug#62216: Narrowed the issue to one line

[Paul Eggert]

On 8/20/23 10:26, Eli Zaretskii wrote:

> Po Lu, Paul: any idea why that could be the case in the OP's scenario?
> The only thing that comes to my mind is some weirdness of the OP's
> filesystem.

I don't see even that. What sort of weirdness did you have in mind?

Jonathon: when gap_size != total - inserted in those two lines, what are 
the different values and why does the difference matter?

bug#62216: Narrowed the issue to one line

[Eli Zaretskii]

> Date: Sun, 20 Aug 2023 14:13:25 -0500
> Cc: 62216@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> On 8/20/23 10:26, Eli Zaretskii wrote:
> 
> > Po Lu, Paul: any idea why that could be the case in the OP's scenario?
> > The only thing that comes to my mind is some weirdness of the OP's
> > filesystem.
> 
> I don't see even that. What sort of weirdness did you have in mind?

Something that affects 'seekable', perhaps?  It's far-fetched, I know.
But what else could be at work here?

bug#62216: Narrowed the issue to one line

[Paul Eggert]

On 8/20/23 14:19, Eli Zaretskii wrote:
> Something that affects 'seekable', perhaps?  It's far-fetched, I know.

Oh, a regular file that isn't seekable? Yes, I suppose that could cause 
trouble. Jonathon, is that possible on your platform?

bug#62216: Narrowed the issue to one line

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Aug 20, 2023 at 03:11:46PM -0500, Paul Eggert wrote:
: On 8/20/23 14:19, Eli Zaretskii wrote:
: > Something that affects 'seekable', perhaps?  It's far-fetched, I know.
: 
: Oh, a regular file that isn't seekable? Yes, I suppose that could cause
: trouble. Jonathon, is that possible on your platform?

I'm running macOS Ventura. I know mac has diverged in many ways from 'true'
*nix under the hood, but I'm by no means an expert, and the differences
don't usually cause me an issue.

That said, I can't imagine a file that isn't seekable in my situation,
just editing source code with magit for VC.

Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Narrowed the issue to one line

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Aug 20, 2023 at 03:11:46PM -0500, Paul Eggert wrote:
: On 8/20/23 14:19, Eli Zaretskii wrote:
: > Something that affects 'seekable', perhaps?  It's far-fetched, I know.
: 
: Oh, a regular file that isn't seekable? Yes, I suppose that could cause
: trouble. Jonathon, is that possible on your platform?

I'm happy to research this question, but I'm not sure where I'd start.
Apple developer docs? I'd be surprised if something as fundamental
as file API behavior diverged this far from expectations, but of course,
anything can happen.

Other than comparing the different lengths being calculated and reporting
them, is there any other logging I should run locally that could shed
light on this mystery?

Jonathon McKitrick
--
'My other computer is your Windows box.'

bug#62216: Narrowed the issue to one line

[Paul Eggert]

On 8/22/23 08:08, Jonathon McKitrick wrote:
> Other than comparing the different lengths being calculated and reporting
> them, is there any other logging I should run locally that could shed
> light on this mystery?

Possibly the problem has to do with markers pointing into the buffer, so 
it might be helpful to log their contents before and after the insert in 
question. (Offhand I don't know how to do this.)

bug#62216: Narrowed the issue to one line

[Eli Zaretskii]

> Date: Tue, 22 Aug 2023 09:10:10 -0500
> Cc: Eli Zaretskii <eliz@gnu.org>, luangruo@yahoo.com, 62216@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> On 8/22/23 08:08, Jonathon McKitrick wrote:
> > Other than comparing the different lengths being calculated and reporting
> > them, is there any other logging I should run locally that could shed
> > light on this mystery?
> 
> Possibly the problem has to do with markers pointing into the buffer, so 
> it might be helpful to log their contents before and after the insert in 
> question.

Can you elaborate?  What kind of problems could the markers cause in
this case? disrupt the character-to-byte position conversion?
something else?

bug#62216: Narrowed the issue to one line

[Paul Eggert]

On 8/22/23 10:41, Eli Zaretskii wrote:
>> Possibly the problem has to do with markers pointing into the buffer, so
>> it might be helpful to log their contents before and after the insert in
>> question.
> Can you elaborate?  What kind of problems could the markers cause in
> this case? disrupt the character-to-byte position conversion?
> something else?

Not really. It's just a wild guess. I haven't looked at the code in detail.

bug#62216: Narrowed the issue to one line

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Aug 20, 2023 at 02:13:25PM -0500, Paul Eggert wrote:
: On 8/20/23 10:26, Eli Zaretskii wrote:
: 
: > Po Lu, Paul: any idea why that could be the case in the OP's scenario?
: > The only thing that comes to my mind is some weirdness of the OP's
: > filesystem.
: 
: I don't see even that. What sort of weirdness did you have in mind?
: 
: Jonathon: when gap_size != total - inserted in those two lines, what are the
: different values and why does the difference matter?

I'll see if I can log those values and report back.

Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Narrowed the issue to one line

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Sun, Aug 20, 2023 at 02:13:25PM -0500, Paul Eggert wrote:
: On 8/20/23 10:26, Eli Zaretskii wrote:
: 
: > Po Lu, Paul: any idea why that could be the case in the OP's scenario?
: > The only thing that comes to my mind is some weirdness of the OP's
: > filesystem.
: 
: I don't see even that. What sort of weirdness did you have in mind?
: 
: Jonathon: when gap_size != total - inserted in those two lines, what are the
: different values and why does the difference matter?

What's the best mechanism to log these values in emacs?

Jonathon McKitrick
--
'My other computer is your Linux box.'

bug#62216: Narrowed the issue to one line

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jonathon McKitrick <jcm@SDF.ORG> writes:

> What's the best mechanism to log these values in emacs?

In this case, printf.

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Gerd Möllmann]

Po Lu <luangruo@yahoo.com> writes:

> Jonathon McKitrick <jcm@SDF.ORG> writes:
>
>> What's the best mechanism to log these values in emacs?
>
> In this case, printf.

When I add this assumption:

modified   src/fileio.c
@@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
 
 	/* 'try' is reserved in some compilers (Microsoft C).  */
 	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
+	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
 	if (!NILP (end))
 	  trytry = min (trytry, total - inserted);
+	if (trytry != trytry_old)
+	  abort ();
 
 	if (!seekable && NILP (end))
 	  {

and make bootstrap, temacs aborts.

Can someone tell if this assumption should hold?

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: Jonathon McKitrick <jcm@SDF.ORG>,  Eli Zaretskii <eliz@gnu.org>,  Paul
>  Eggert <eggert@cs.ucla.edu>,  62216@debbugs.gnu.org
> Date: Wed, 30 Aug 2023 10:59:08 +0200
> 
> Po Lu <luangruo@yahoo.com> writes:
> 
> > Jonathon McKitrick <jcm@SDF.ORG> writes:
> >
> >> What's the best mechanism to log these values in emacs?
> >
> > In this case, printf.
> 
> When I add this assumption:
> 
> modified   src/fileio.c
> @@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
>  
>  	/* 'try' is reserved in some compilers (Microsoft C).  */
>  	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
> +	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
>  	if (!NILP (end))
>  	  trytry = min (trytry, total - inserted);
> +	if (trytry != trytry_old)
> +	  abort ();
>  
>  	if (!seekable && NILP (end))
>  	  {
> 
> and make bootstrap, temacs aborts.
> 
> Can someone tell if this assumption should hold?

You expect READ_BUF_SIZE to never be greater than gap_size?

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Gerd Möllmann]

On 30.08.23 14:00, Eli Zaretskii wrote:
>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: Jonathon McKitrick <jcm@SDF.ORG>,  Eli Zaretskii <eliz@gnu.org>,  Paul
>>   Eggert <eggert@cs.ucla.edu>,  62216@debbugs.gnu.org
>> Date: Wed, 30 Aug 2023 10:59:08 +0200
>>
>> Po Lu <luangruo@yahoo.com> writes:
>>
>>> Jonathon McKitrick <jcm@SDF.ORG> writes:
>>>
>>>> What's the best mechanism to log these values in emacs?
>>>
>>> In this case, printf.
>>
>> When I add this assumption:
>>
>> modified   src/fileio.c
>> @@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
>>   
>>   	/* 'try' is reserved in some compilers (Microsoft C).  */
>>   	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
>> +	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
>>   	if (!NILP (end))
>>   	  trytry = min (trytry, total - inserted);
>> +	if (trytry != trytry_old)
>> +	  abort ();
>>   
>>   	if (!seekable && NILP (end))
>>   	  {
>>
>> and make bootstrap, temacs aborts.
>>
>> Can someone tell if this assumption should hold?
> 
> You expect READ_BUF_SIZE to never be greater than gap_size?

Don't know.  This is how I understood Jonathan's nail saying works with 
this old code, doesn't work with this new code.

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Wed, Aug 30, 2023 at 02:07:35PM +0200, Gerd Mllmann wrote:
: On 30.08.23 14:00, Eli Zaretskii wrote:
: > > From: Gerd Möllmann <gerd.moellmann@gmail.com>
: > > Cc: Jonathon McKitrick <jcm@SDF.ORG>,  Eli Zaretskii <eliz@gnu.org>,  Paul
: > >   Eggert <eggert@cs.ucla.edu>,  62216@debbugs.gnu.org
: > > Date: Wed, 30 Aug 2023 10:59:08 +0200
: > > 
: > > Po Lu <luangruo@yahoo.com> writes:
: > > 
: > > > Jonathon McKitrick <jcm@SDF.ORG> writes:
: > > > 
: > > > > What's the best mechanism to log these values in emacs?
: > > > 
: > > > In this case, printf.
: > > 
: > > When I add this assumption:
: > > 
: > > modified   src/fileio.c
: > > @@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
: > >   	/* 'try' is reserved in some compilers (Microsoft C).  */
: > >   	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
: > > +	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
: > >   	if (!NILP (end))
: > >   	  trytry = min (trytry, total - inserted);
: > > +	if (trytry != trytry_old)
: > > +	  abort ();
: > >   	if (!seekable && NILP (end))
: > >   	  {
: > > 
: > > and make bootstrap, temacs aborts.
: > > 
: > > Can someone tell if this assumption should hold?
: > 
: > You expect READ_BUF_SIZE to never be greater than gap_size?
: 
: Don't know.  This is how I understood Jonathan's nail saying works with this
: old code, doesn't work with this new code.

I'm willing to log some meaningful stats if that would help. Last time I logged these values, I was inundated with info, but if I could
narrow it down to the most useful cases, I could log the relevant bits and report back.

Jonathon McKitrick
--
'My other computer is your Windows box.'

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Eli Zaretskii]

> Date: Thu, 31 Aug 2023 14:46:01 +0000
> From: Jonathon McKitrick <jcm@SDF.ORG>
> Cc: Eli Zaretskii <eliz@gnu.org>, luangruo@yahoo.com, eggert@cs.ucla.edu,
>         62216@debbugs.gnu.org
> 
> On Wed, Aug 30, 2023 at 02:07:35PM +0200, Gerd Mllmann wrote:
> : On 30.08.23 14:00, Eli Zaretskii wrote:
> : > > From: Gerd Möllmann <gerd.moellmann@gmail.com>
> : > > Cc: Jonathon McKitrick <jcm@SDF.ORG>,  Eli Zaretskii <eliz@gnu.org>,  Paul
> : > >   Eggert <eggert@cs.ucla.edu>,  62216@debbugs.gnu.org
> : > > Date: Wed, 30 Aug 2023 10:59:08 +0200
> : > > 
> : > > Po Lu <luangruo@yahoo.com> writes:
> : > > 
> : > > > Jonathon McKitrick <jcm@SDF.ORG> writes:
> : > > > 
> : > > > > What's the best mechanism to log these values in emacs?
> : > > > 
> : > > > In this case, printf.
> : > > 
> : > > When I add this assumption:
> : > > 
> : > > modified   src/fileio.c
> : > > @@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
> : > >   	/* 'try' is reserved in some compilers (Microsoft C).  */
> : > >   	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
> : > > +	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
> : > >   	if (!NILP (end))
> : > >   	  trytry = min (trytry, total - inserted);
> : > > +	if (trytry != trytry_old)
> : > > +	  abort ();
> : > >   	if (!seekable && NILP (end))
> : > >   	  {
> : > > 
> : > > and make bootstrap, temacs aborts.
> : > > 
> : > > Can someone tell if this assumption should hold?
> : > 
> : > You expect READ_BUF_SIZE to never be greater than gap_size?
> : 
> : Don't know.  This is how I understood Jonathan's nail saying works with this
> : old code, doesn't work with this new code.
> 
> I'm willing to log some meaningful stats if that would help. Last time I logged these values, I was inundated with info, but if I could
> narrow it down to the most useful cases, I could log the relevant bits and report back.

you mean, the fix I installed on master yesterday didn't fix your
problem?  It did fix Gerd's problem.

bug#62216: Odd file corruption in clojure mode and emacs 30 HEAD

[Jonathon McKitrick via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Thu, Aug 31, 2023 at 06:54:23PM +0300, Eli Zaretskii wrote:
: > Date: Thu, 31 Aug 2023 14:46:01 +0000
: > From: Jonathon McKitrick <jcm@SDF.ORG>
: > Cc: Eli Zaretskii <eliz@gnu.org>, luangruo@yahoo.com, eggert@cs.ucla.edu,
: >         62216@debbugs.gnu.org
: > 
: > On Wed, Aug 30, 2023 at 02:07:35PM +0200, Gerd Mllmann wrote:
: > : On 30.08.23 14:00, Eli Zaretskii wrote:
: > : > > From: Gerd Möllmann <gerd.moellmann@gmail.com>
: > : > > Cc: Jonathon McKitrick <jcm@SDF.ORG>,  Eli Zaretskii <eliz@gnu.org>,  Paul
: > : > >   Eggert <eggert@cs.ucla.edu>,  62216@debbugs.gnu.org
: > : > > Date: Wed, 30 Aug 2023 10:59:08 +0200
: > : > > 
: > : > > Po Lu <luangruo@yahoo.com> writes:
: > : > > 
: > : > > > Jonathon McKitrick <jcm@SDF.ORG> writes:
: > : > > > 
: > : > > > > What's the best mechanism to log these values in emacs?
: > : > > > 
: > : > > > In this case, printf.
: > : > > 
: > : > > When I add this assumption:
: > : > > 
: > : > > modified   src/fileio.c
: > : > > @@ -4805,8 +4805,11 @@ DEFUN ("insert-file-contents", Finsert_file_contents, Sinsert_file_contents,
: > : > >   	/* 'try' is reserved in some compilers (Microsoft C).  */
: > : > >   	ptrdiff_t trytry = min (gap_size, READ_BUF_SIZE);
: > : > > +	ptrdiff_t trytry_old = min (total - inserted, READ_BUF_SIZE);
: > : > >   	if (!NILP (end))
: > : > >   	  trytry = min (trytry, total - inserted);
: > : > > +	if (trytry != trytry_old)
: > : > > +	  abort ();
: > : > >   	if (!seekable && NILP (end))
: > : > >   	  {
: > : > > 
: > : > > and make bootstrap, temacs aborts.
: > : > > 
: > : > > Can someone tell if this assumption should hold?
: > : > 
: > : > You expect READ_BUF_SIZE to never be greater than gap_size?
: > : 
: > : Don't know.  This is how I understood Jonathan's nail saying works with this
: > : old code, doesn't work with this new code.
: > 
: > I'm willing to log some meaningful stats if that would help. Last time I logged these values, I was inundated with info, but if I could
: > narrow it down to the most useful cases, I could log the relevant bits and report back.
: 
: you mean, the fix I installed on master yesterday didn't fix your
: problem?  It did fix Gerd's problem.

Ahhh! I didn't realize that's what that was, lol. I'll try a new build without my patch.

Jonathon McKitrick
--
'My other computer is your Windows box.'