bug#5852: 23.1; Incorrect references in ses-mode

bug#5852: 23.1; Incorrect references in ses-mode

[Göran Uddeborg]

I discovered this when emacs sometimes complains that I have circular
references in ses-mode files where there shouldn't be any.  When
investigating the ses file I've noticed some cells contain references
that are incorrect.  When I remove these incorrect references, the
complaints about circular references go away, so I strongly suspect
the problems are related.  And even if I'm wrong, the incorrect
references is a bug.

While I haven't been able to create a simple test case where I get a
circular reference complaint, I can easily get to a state where there
are incorrect references.  Create a ses file with these keystrokes:

    C-x C-f s . s e s <return> <tab> 1 <return> C-o 2 <return>
    <tab> <left> ( + SPC B 1 SPC A 2 <return>
    C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-a
    C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y C-n C-y
    C-x C-s

In the resulting file, cell B2 has cell B4 incorrectly listed among
its references:

    (ses-cell B2 3 (+ B1 A2) nil (B3 B4))

Only B3 does actually reference B2.


In GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.18.3)
 of 2009-12-02 on x86-3.fedora.phx.redhat.com
Windowing system distributor `Fedora Project', version 11.0.10705000
configured using `configure  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--with-dbus' '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff' '--with-xft' '--with-xpm' '--with-x-toolkit=gtk' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector 
 --param=ssp-buffer-size=4 -m64 -mtune=generic''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: sv_SE.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: SES

Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<tab> 1 <tab> <backspace> <return> C-o 2 <return> <tab> 
<left> ( + SPC B 1 SPC A 2 <return> <return> <return> 
C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-n 
C-o C-p C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y 
C-n C-y C-x C-s <help-echo> M-x r e p o r t - e m a 
c s - b u g <return>

Recent messages:
  [2 times]
Yanking...done
Mark set
Yanking...done
  [2 times]
Yanking...done
  [2 times]
Yanking...done
Saving file /home/g=f6ran/x.ses...
Wrote /home/g=f6ran/x.ses

bug#5852: 23.1; Incorrect references in ses-mode

[Stefan Kangas]

Hi Vincent,

Could you please take a look at the below bug report?  It was reported
in 2010, before you were listed as the maintainer of ses.el.

Thanks in advance.

Best regards,
Stefan Kangas


Göran Uddeborg <goeran@uddeborg.se> writes:

> I discovered this when emacs sometimes complains that I have circular
> references in ses-mode files where there shouldn't be any.  When
> investigating the ses file I've noticed some cells contain references
> that are incorrect.  When I remove these incorrect references, the
> complaints about circular references go away, so I strongly suspect
> the problems are related.  And even if I'm wrong, the incorrect
> references is a bug.
>
> While I haven't been able to create a simple test case where I get a
> circular reference complaint, I can easily get to a state where there
> are incorrect references.  Create a ses file with these keystrokes:
>
>     C-x C-f s . s e s <return> <tab> 1 <return> C-o 2 <return>
>     <tab> <left> ( + SPC B 1 SPC A 2 <return>
>     C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-a
>     C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y C-n C-y
>     C-x C-s
>
> In the resulting file, cell B2 has cell B4 incorrectly listed among
> its references:
>
>     (ses-cell B2 3 (+ B1 A2) nil (B3 B4))
>
> Only B3 does actually reference B2.
>
>
> In GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.18.3)
>  of 2009-12-02 on x86-3.fedora.phx.redhat.com
> Windowing system distributor `Fedora Project', version 11.0.10705000
> configured using `configure  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--with-dbus' '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff' '--with-xft' '--with-xpm' '--with-x-toolkit=gtk' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic''
>
> Important settings:
>   value of $LC_ALL: nil
>   value of $LC_COLLATE: nil
>   value of $LC_CTYPE: nil
>   value of $LC_MESSAGES: nil
>   value of $LC_MONETARY: nil
>   value of $LC_NUMERIC: nil
>   value of $LC_TIME: nil
>   value of $LANG: sv_SE.utf8
>   value of $XMODIFIERS: nil
>   locale-coding-system: utf-8-unix
>   default-enable-multibyte-characters: t
>
> Major mode: SES
>
> Minor modes in effect:
>   tooltip-mode: t
>   tool-bar-mode: t
>   mouse-wheel-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   global-auto-composition-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>
> Recent input:
> <tab> 1 <tab> <backspace> <return> C-o 2 <return> <tab> 
> <left> ( + SPC B 1 SPC A 2 <return> <return> <return> 
> C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-n 
> C-o C-p C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y 
> C-n C-y C-x C-s <help-echo> M-x r e p o r t - e m a 
> c s - b u g <return>
>
> Recent messages:
>   [2 times]
> Yanking...done
> Mark set
> Yanking...done
>   [2 times]
> Yanking...done
>   [2 times]
> Yanking...done
> Saving file /home/g=f6ran/x.ses...
> Wrote /home/g=f6ran/x.ses

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Dear Stefan and Güran,

I confirm that the bug still exists, I followed the instructions by
Güran and I reproduced the same result.

Actually, after doing all these and typing:

M-x ses-repair-cell-reference-all

I get the following answer :

--8<----8<----8<----8<----8<-- begin -->8---->8---->8---->8---->8----
Warning (emacs): ----------------------------------------------------------------
Some references were corrupted.

The following is a list where each element ELT is such
that (car ELT) is the reference of cell CELL with corruption,
and (cdr ELT) is a property list where

* property ‘:corrupt-property’ means that
  property ‘:ses-repair-reference’ of cell CELL was initially non
  nil,

* property ‘:missing’ is a list of missing references

* property ‘:irrelevant’ is a list of non needed references

((B2 :irrelevant (B4)))
--8<----8<----8<----8<----8<--  end  -->8---->8---->8---->8---->8----


This shows that the cut and paste can actually breaks the reference list
in some cases.

Please note that 'ses-repair-cell-reference-all' is the command that I
added to SES after I became the maintainer in order to fix broken
reference lists, as this can still happen if you type C-g during some
cell editing post-commands. However I hadn't realized that even by
normal editing (no C-g) you can get broken references.

I will investigate why and feed back once I have a better idea of what
is happening.

VBR,
	Vincent.


Le 29/02/2020 à 04:14, Stefan Kangas a écrit :
> Hi Vincent,
>
> Could you please take a look at the below bug report?  It was reported
> in 2010, before you were listed as the maintainer of ses.el.
>
> Thanks in advance.
>
> Best regards,
> Stefan Kangas
>
>
> Göran Uddeborg <goeran@uddeborg.se> writes:
>
>> I discovered this when emacs sometimes complains that I have circular
>> references in ses-mode files where there shouldn't be any.  When
>> investigating the ses file I've noticed some cells contain references
>> that are incorrect.  When I remove these incorrect references, the
>> complaints about circular references go away, so I strongly suspect
>> the problems are related.  And even if I'm wrong, the incorrect
>> references is a bug.
>>
>> While I haven't been able to create a simple test case where I get a
>> circular reference complaint, I can easily get to a state where there
>> are incorrect references.  Create a ses file with these keystrokes:
>>
>>     C-x C-f s . s e s <return> <tab> 1 <return> C-o 2 <return>
>>     <tab> <left> ( + SPC B 1 SPC A 2 <return>
>>     C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-a
>>     C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y C-n C-y
>>     C-x C-s
>>
>> In the resulting file, cell B2 has cell B4 incorrectly listed among
>> its references:
>>
>>     (ses-cell B2 3 (+ B1 A2) nil (B3 B4))
>>
>> Only B3 does actually reference B2.
>>
>>
>> In GNU Emacs 23.1.1 (x86_64-redhat-linux-gnu, GTK+ Version 2.18.3)
>>  of 2009-12-02 on x86-3.fedora.phx.redhat.com
>> Windowing system distributor `Fedora Project', version 11.0.10705000
>> configured using `configure  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--with-dbus' '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff' '--with-xft' '--with-xpm' '--with-x-toolkit=gtk' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-DMAIL_USE_LOCKF -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic''
>>
>> Important settings:
>>   value of $LC_ALL: nil
>>   value of $LC_COLLATE: nil
>>   value of $LC_CTYPE: nil
>>   value of $LC_MESSAGES: nil
>>   value of $LC_MONETARY: nil
>>   value of $LC_NUMERIC: nil
>>   value of $LC_TIME: nil
>>   value of $LANG: sv_SE.utf8
>>   value of $XMODIFIERS: nil
>>   locale-coding-system: utf-8-unix
>>   default-enable-multibyte-characters: t
>>
>> Major mode: SES
>>
>> Minor modes in effect:
>>   tooltip-mode: t
>>   tool-bar-mode: t
>>   mouse-wheel-mode: t
>>   menu-bar-mode: t
>>   file-name-shadow-mode: t
>>   global-font-lock-mode: t
>>   font-lock-mode: t
>>   blink-cursor-mode: t
>>   global-auto-composition-mode: t
>>   auto-composition-mode: t
>>   auto-encryption-mode: t
>>   auto-compression-mode: t
>>   line-number-mode: t
>>   transient-mark-mode: t
>>
>> Recent input:
>> <tab> 1 <tab> <backspace> <return> C-o 2 <return> <tab> 
>> <left> ( + SPC B 1 SPC A 2 <return> <return> <return> 
>> C-n C-o 3 <return> ( + SPC B 2 SPC A 3 <return> C-n 
>> C-o C-p C-o 4 <return> C-p C-SPC C-f M-w C-n C-b C-y 
>> C-n C-y C-x C-s <help-echo> M-x r e p o r t - e m a 
>> c s - b u g <return>
>>
>> Recent messages:
>>   [2 times]
>> Yanking...done
>> Mark set
>> Yanking...done
>>   [2 times]
>> Yanking...done
>>   [2 times]
>> Yanking...done
>> Saving file /home/g=f6ran/x.ses...
>> Wrote /home/g=f6ran/x.ses

bug#5852: 23.1; Incorrect references in ses-mode

[Lars Ingebrigtsen]

Vincent Belaïche <vincentbelaiche@orange.fr> writes:

> I will investigate why and feed back once I have a better idea of what
> is happening.

Hi,

this was a year ago.  Did you make any progress on this issue?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Hello Lars,

Sorry for not making any feedback for so long, actually I was just
distracted by some other business, and I somewhat had forgotten about
it.

I suspect a race condition between the post command hook and the next
command. That is to say when one yanks the fomula (the first yank),
one relocates the reference but for this one uses current references
that have not yet been relocated because the previous post-command
hook execution is not yet over.

The bug does not show every time I do the experiment. I tried this
morning with my office PC which is a fast machine, and it proved out
quite hard to reproduce, even when I launched some big C program
compilation in parallel to eat the mips.

Next week I will do some experiment with my home PC which is quite
quite slower, and I make some feedback. Unfortunateless both machine
do not have the same emacs version, but anyway all changes in SES
since the version on my home PC are not related to this bug, so
nevertheless that will be a good indication whether my guess is
correct.

  V.

Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
>
> Vincent Belaïche <vincentbelaiche@orange.fr> writes:
>
> > I will investigate why and feed back once I have a better idea of what
> > is happening.
>
> Hi,
>
> this was a year ago.  Did you make any progress on this issue?
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Just to add something : reading again SES code, and (info "(elisp)
Command Overview") I am a bit skeptical about the explanation in my
previous email.
It seems that only key strokes are queue, but command execution and
post command hook are in sequence, so not done concurrently.
I wrote this function to try to reproduce the bug, and on my office PC
it does not reproduce it:

(defun bug-5852 ();;---
  "Check no irrelevant reference is created when a row is inserted and
a formula is yanked"
  (interactive);;---
  (let ((ses-initial-size '(3 . 3)))
  (with-temp-buffer
    (ses-mode)
    (dolist (val '(
                   (ses-edit-cell 0 1 1)
                   (ses-edit-cell 1 0 2)
                   (ses-edit-cell 1 1 (+ B1 A2))
                   (ses-edit-cell 2 0 3)
                   (ses-edit-cell 2 1 (+ B2 A3))
                   (ses-jump A3)
                   (ses-insert-row 1)
                   (ses-edit-cell 2 0 4)
                   (ses-jump B2)
                   ))
      (apply 'funcall-interactively  val)
      (ses-command-hook))
    (let ((beg (point-marker)))
      (ses-jump 'C2) ; C-n
      (copy-region-as-kill beg (point) nil)
      (set-marker beg nil))
    (dolist (val '(
                   (ses-jump B3)
                   (yank)
                   (ses-jump B4)
                   (yank)
                   ))
      (apply 'funcall-interactively  val)
      (ses-command-hook))
    (ses-repair-cell-reference-all)
    )))

Please note that the main difference with the provided example is that
motion is done with ses-jump, and not with motion commands. Also the
save to keyring is not done interatively.
   V.

Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> Hello Lars,
>
> Sorry for not making any feedback for so long, actually I was just
> distracted by some other business, and I somewhat had forgotten about
> it.
>
> I suspect a race condition between the post command hook and the next
> command. That is to say when one yanks the fomula (the first yank),
> one relocates the reference but for this one uses current references
> that have not yet been relocated because the previous post-command
> hook execution is not yet over.
>
> The bug does not show every time I do the experiment. I tried this
> morning with my office PC which is a fast machine, and it proved out
> quite hard to reproduce, even when I launched some big C program
> compilation in parallel to eat the mips.
>
> Next week I will do some experiment with my home PC which is quite
> quite slower, and I make some feedback. Unfortunateless both machine
> do not have the same emacs version, but anyway all changes in SES
> since the version on my home PC are not related to this bug, so
> nevertheless that will be a good indication whether my guess is
> correct.
>
>   V.
>
> Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> >
> > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> >
> > > I will investigate why and feed back once I have a better idea of what
> > > is happening.
> >
> > Hi,
> >
> > this was a year ago.  Did you make any progress on this issue?
> >
> > --
> > (domestic pets only, the antidote for overdose, milk.)
> >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

On more significant difference between the function bug-5852 and the
example is the the initial size is (3 . 3) instead of (1. 1). This is
because I speculated that only the last row insertion (C-o) plays a
role.

  V.

Le dim. 25 juil. 2021 à 07:48, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> Just to add something : reading again SES code, and (info "(elisp)
> Command Overview") I am a bit skeptical about the explanation in my
> previous email.
> It seems that only key strokes are queue, but command execution and
> post command hook are in sequence, so not done concurrently.
> I wrote this function to try to reproduce the bug, and on my office PC
> it does not reproduce it:
>
> (defun bug-5852 ();;---
>   "Check no irrelevant reference is created when a row is inserted and
> a formula is yanked"
>   (interactive);;---
>   (let ((ses-initial-size '(3 . 3)))
>   (with-temp-buffer
>     (ses-mode)
>     (dolist (val '(
>                    (ses-edit-cell 0 1 1)
>                    (ses-edit-cell 1 0 2)
>                    (ses-edit-cell 1 1 (+ B1 A2))
>                    (ses-edit-cell 2 0 3)
>                    (ses-edit-cell 2 1 (+ B2 A3))
>                    (ses-jump A3)
>                    (ses-insert-row 1)
>                    (ses-edit-cell 2 0 4)
>                    (ses-jump B2)
>                    ))
>       (apply 'funcall-interactively  val)
>       (ses-command-hook))
>     (let ((beg (point-marker)))
>       (ses-jump 'C2) ; C-n
>       (copy-region-as-kill beg (point) nil)
>       (set-marker beg nil))
>     (dolist (val '(
>                    (ses-jump B3)
>                    (yank)
>                    (ses-jump B4)
>                    (yank)
>                    ))
>       (apply 'funcall-interactively  val)
>       (ses-command-hook))
>     (ses-repair-cell-reference-all)
>     )))
>
> Please note that the main difference with the provided example is that
> motion is done with ses-jump, and not with motion commands. Also the
> save to keyring is not done interatively.
>    V.
>
> Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
> <vincent.belaiche@gmail.com> a écrit :
> >
> > Hello Lars,
> >
> > Sorry for not making any feedback for so long, actually I was just
> > distracted by some other business, and I somewhat had forgotten about
> > it.
> >
> > I suspect a race condition between the post command hook and the next
> > command. That is to say when one yanks the fomula (the first yank),
> > one relocates the reference but for this one uses current references
> > that have not yet been relocated because the previous post-command
> > hook execution is not yet over.
> >
> > The bug does not show every time I do the experiment. I tried this
> > morning with my office PC which is a fast machine, and it proved out
> > quite hard to reproduce, even when I launched some big C program
> > compilation in parallel to eat the mips.
> >
> > Next week I will do some experiment with my home PC which is quite
> > quite slower, and I make some feedback. Unfortunateless both machine
> > do not have the same emacs version, but anyway all changes in SES
> > since the version on my home PC are not related to this bug, so
> > nevertheless that will be a good indication whether my guess is
> > correct.
> >
> >   V.
> >
> > Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> > >
> > > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> > >
> > > > I will investigate why and feed back once I have a better idea of what
> > > > is happening.
> > >
> > > Hi,
> > >
> > > this was a year ago.  Did you make any progress on this issue?
> > >
> > > --
> > > (domestic pets only, the antidote for overdose, milk.)
> > >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

The same test function, but with comments :


(defun bug-5852 ();;---
  "Check no irrelevant reference is created when a row is inserted and
a formula is yanked"
  (interactive);;---
  (let ((ses-initial-size '(3 . 3))); size before last row insertion,
due to all preceding TAB & C-o
  (with-temp-buffer
    (ses-mode)
    (dolist (val '(
                   (ses-edit-cell 0 1 1) ; 1 RET
                   (ses-edit-cell 1 0 2) ; 2 RET
                   (ses-edit-cell 1 1 (+ B1 A2)) ; ( + SPC B1 SPC A2 RET
                   (ses-edit-cell 2 0 3) ; 3 RET
                   (ses-edit-cell 2 1 (+ B2 A3)) ; ( + SPC B2 SPC A3 RET
                   (ses-jump A3) ; C-n
                   (ses-insert-row 1) ; C-o
                   (ses-edit-cell 2 0 4) ; 4 RET
                   (ses-jump B2) ; C-p
                   ))
      (apply 'funcall-interactively  val)
      (ses-command-hook))
  (let ((beg (point-marker))); C-SPC
    (ses-jump 'C2) ; C-f
    (copy-region-as-kill beg (point) nil); M-w
    (set-marker beg nil))
  (dolist (val '(
                 (ses-jump B3); C-n C-b
                 (yank) ; C-y
                 (ses-jump B4); C-n
                 (yank); C-y
                 ))
    (apply 'funcall-interactively  val)
    (ses-command-hook))
  (ses-repair-cell-reference-all)
  )))

Le dim. 25 juil. 2021 à 07:56, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> On more significant difference between the function bug-5852 and the
> example is the the initial size is (3 . 3) instead of (1. 1). This is
> because I speculated that only the last row insertion (C-o) plays a
> role.
>
>   V.
>
> Le dim. 25 juil. 2021 à 07:48, Vincent Belaïche
> <vincent.belaiche@gmail.com> a écrit :
> >
> > Just to add something : reading again SES code, and (info "(elisp)
> > Command Overview") I am a bit skeptical about the explanation in my
> > previous email.
> > It seems that only key strokes are queue, but command execution and
> > post command hook are in sequence, so not done concurrently.
> > I wrote this function to try to reproduce the bug, and on my office PC
> > it does not reproduce it:
> >
> > (defun bug-5852 ();;---
> >   "Check no irrelevant reference is created when a row is inserted and
> > a formula is yanked"
> >   (interactive);;---
> >   (let ((ses-initial-size '(3 . 3)))
> >   (with-temp-buffer
> >     (ses-mode)
> >     (dolist (val '(
> >                    (ses-edit-cell 0 1 1)
> >                    (ses-edit-cell 1 0 2)
> >                    (ses-edit-cell 1 1 (+ B1 A2))
> >                    (ses-edit-cell 2 0 3)
> >                    (ses-edit-cell 2 1 (+ B2 A3))
> >                    (ses-jump A3)
> >                    (ses-insert-row 1)
> >                    (ses-edit-cell 2 0 4)
> >                    (ses-jump B2)
> >                    ))
> >       (apply 'funcall-interactively  val)
> >       (ses-command-hook))
> >     (let ((beg (point-marker)))
> >       (ses-jump 'C2) ; C-n
> >       (copy-region-as-kill beg (point) nil)
> >       (set-marker beg nil))
> >     (dolist (val '(
> >                    (ses-jump B3)
> >                    (yank)
> >                    (ses-jump B4)
> >                    (yank)
> >                    ))
> >       (apply 'funcall-interactively  val)
> >       (ses-command-hook))
> >     (ses-repair-cell-reference-all)
> >     )))
> >
> > Please note that the main difference with the provided example is that
> > motion is done with ses-jump, and not with motion commands. Also the
> > save to keyring is not done interatively.
> >    V.
> >
> > Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
> > <vincent.belaiche@gmail.com> a écrit :
> > >
> > > Hello Lars,
> > >
> > > Sorry for not making any feedback for so long, actually I was just
> > > distracted by some other business, and I somewhat had forgotten about
> > > it.
> > >
> > > I suspect a race condition between the post command hook and the next
> > > command. That is to say when one yanks the fomula (the first yank),
> > > one relocates the reference but for this one uses current references
> > > that have not yet been relocated because the previous post-command
> > > hook execution is not yet over.
> > >
> > > The bug does not show every time I do the experiment. I tried this
> > > morning with my office PC which is a fast machine, and it proved out
> > > quite hard to reproduce, even when I launched some big C program
> > > compilation in parallel to eat the mips.
> > >
> > > Next week I will do some experiment with my home PC which is quite
> > > quite slower, and I make some feedback. Unfortunateless both machine
> > > do not have the same emacs version, but anyway all changes in SES
> > > since the version on my home PC are not related to this bug, so
> > > nevertheless that will be a good indication whether my guess is
> > > correct.
> > >
> > >   V.
> > >
> > > Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> > > >
> > > > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> > > >
> > > > > I will investigate why and feed back once I have a better idea of what
> > > > > is happening.
> > > >
> > > > Hi,
> > > >
> > > > this was a year ago.  Did you make any progress on this issue?
> > > >
> > > > --
> > > > (domestic pets only, the antidote for overdose, milk.)
> > > >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

The bug-5852 test function does not reproduce the bug on my home PC either …
  V.

Le dim. 25 juil. 2021 à 08:17, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> The same test function, but with comments :
>
>
> (defun bug-5852 ();;---
>   "Check no irrelevant reference is created when a row is inserted and
> a formula is yanked"
>   (interactive);;---
>   (let ((ses-initial-size '(3 . 3))); size before last row insertion,
> due to all preceding TAB & C-o
>   (with-temp-buffer
>     (ses-mode)
>     (dolist (val '(
>                    (ses-edit-cell 0 1 1) ; 1 RET
>                    (ses-edit-cell 1 0 2) ; 2 RET
>                    (ses-edit-cell 1 1 (+ B1 A2)) ; ( + SPC B1 SPC A2 RET
>                    (ses-edit-cell 2 0 3) ; 3 RET
>                    (ses-edit-cell 2 1 (+ B2 A3)) ; ( + SPC B2 SPC A3 RET
>                    (ses-jump A3) ; C-n
>                    (ses-insert-row 1) ; C-o
>                    (ses-edit-cell 2 0 4) ; 4 RET
>                    (ses-jump B2) ; C-p
>                    ))
>       (apply 'funcall-interactively  val)
>       (ses-command-hook))
>   (let ((beg (point-marker))); C-SPC
>     (ses-jump 'C2) ; C-f
>     (copy-region-as-kill beg (point) nil); M-w
>     (set-marker beg nil))
>   (dolist (val '(
>                  (ses-jump B3); C-n C-b
>                  (yank) ; C-y
>                  (ses-jump B4); C-n
>                  (yank); C-y
>                  ))
>     (apply 'funcall-interactively  val)
>     (ses-command-hook))
>   (ses-repair-cell-reference-all)
>   )))
>
> Le dim. 25 juil. 2021 à 07:56, Vincent Belaïche
> <vincent.belaiche@gmail.com> a écrit :
> >
> > On more significant difference between the function bug-5852 and the
> > example is the the initial size is (3 . 3) instead of (1. 1). This is
> > because I speculated that only the last row insertion (C-o) plays a
> > role.
> >
> >   V.
> >
> > Le dim. 25 juil. 2021 à 07:48, Vincent Belaïche
> > <vincent.belaiche@gmail.com> a écrit :
> > >
> > > Just to add something : reading again SES code, and (info "(elisp)
> > > Command Overview") I am a bit skeptical about the explanation in my
> > > previous email.
> > > It seems that only key strokes are queue, but command execution and
> > > post command hook are in sequence, so not done concurrently.
> > > I wrote this function to try to reproduce the bug, and on my office PC
> > > it does not reproduce it:
> > >
> > > (defun bug-5852 ();;---
> > >   "Check no irrelevant reference is created when a row is inserted and
> > > a formula is yanked"
> > >   (interactive);;---
> > >   (let ((ses-initial-size '(3 . 3)))
> > >   (with-temp-buffer
> > >     (ses-mode)
> > >     (dolist (val '(
> > >                    (ses-edit-cell 0 1 1)
> > >                    (ses-edit-cell 1 0 2)
> > >                    (ses-edit-cell 1 1 (+ B1 A2))
> > >                    (ses-edit-cell 2 0 3)
> > >                    (ses-edit-cell 2 1 (+ B2 A3))
> > >                    (ses-jump A3)
> > >                    (ses-insert-row 1)
> > >                    (ses-edit-cell 2 0 4)
> > >                    (ses-jump B2)
> > >                    ))
> > >       (apply 'funcall-interactively  val)
> > >       (ses-command-hook))
> > >     (let ((beg (point-marker)))
> > >       (ses-jump 'C2) ; C-n
> > >       (copy-region-as-kill beg (point) nil)
> > >       (set-marker beg nil))
> > >     (dolist (val '(
> > >                    (ses-jump B3)
> > >                    (yank)
> > >                    (ses-jump B4)
> > >                    (yank)
> > >                    ))
> > >       (apply 'funcall-interactively  val)
> > >       (ses-command-hook))
> > >     (ses-repair-cell-reference-all)
> > >     )))
> > >
> > > Please note that the main difference with the provided example is that
> > > motion is done with ses-jump, and not with motion commands. Also the
> > > save to keyring is not done interatively.
> > >    V.
> > >
> > > Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
> > > <vincent.belaiche@gmail.com> a écrit :
> > > >
> > > > Hello Lars,
> > > >
> > > > Sorry for not making any feedback for so long, actually I was just
> > > > distracted by some other business, and I somewhat had forgotten about
> > > > it.
> > > >
> > > > I suspect a race condition between the post command hook and the next
> > > > command. That is to say when one yanks the fomula (the first yank),
> > > > one relocates the reference but for this one uses current references
> > > > that have not yet been relocated because the previous post-command
> > > > hook execution is not yet over.
> > > >
> > > > The bug does not show every time I do the experiment. I tried this
> > > > morning with my office PC which is a fast machine, and it proved out
> > > > quite hard to reproduce, even when I launched some big C program
> > > > compilation in parallel to eat the mips.
> > > >
> > > > Next week I will do some experiment with my home PC which is quite
> > > > quite slower, and I make some feedback. Unfortunateless both machine
> > > > do not have the same emacs version, but anyway all changes in SES
> > > > since the version on my home PC are not related to this bug, so
> > > > nevertheless that will be a good indication whether my guess is
> > > > correct.
> > > >
> > > >   V.
> > > >
> > > > Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> > > > >
> > > > > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> > > > >
> > > > > > I will investigate why and feed back once I have a better idea of what
> > > > > > is happening.
> > > > >
> > > > > Hi,
> > > > >
> > > > > this was a year ago.  Did you make any progress on this issue?
> > > > >
> > > > > --
> > > > > (domestic pets only, the antidote for overdose, milk.)
> > > > >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

By doing the test manually on my home PC I found some interesting
difference. After

2 RET

on my home PC the cursor goes to the end of buffer narrowed region,
while on my office PC it goes to B2. This means that the TAB that
follows won't the same effect. On my home PC it creates a 3rd row,
while on my office PC it creates the 3rd column.
I found the reason why so : my home PC had some customization for the
ses-after-entry-functions. After erasing customization I had the same
behaviour …
Anyway, that does not affect the bug-5852 test function as it only
uses ses-jump for motion.
  V.

Le dim. 25 juil. 2021 à 08:18, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> The bug-5852 test function does not reproduce the bug on my home PC either …
>   V.
>
> Le dim. 25 juil. 2021 à 08:17, Vincent Belaïche
> <vincent.belaiche@gmail.com> a écrit :
> >
> > The same test function, but with comments :
> >
> >
> > (defun bug-5852 ();;---
> >   "Check no irrelevant reference is created when a row is inserted and
> > a formula is yanked"
> >   (interactive);;---
> >   (let ((ses-initial-size '(3 . 3))); size before last row insertion,
> > due to all preceding TAB & C-o
> >   (with-temp-buffer
> >     (ses-mode)
> >     (dolist (val '(
> >                    (ses-edit-cell 0 1 1) ; 1 RET
> >                    (ses-edit-cell 1 0 2) ; 2 RET
> >                    (ses-edit-cell 1 1 (+ B1 A2)) ; ( + SPC B1 SPC A2 RET
> >                    (ses-edit-cell 2 0 3) ; 3 RET
> >                    (ses-edit-cell 2 1 (+ B2 A3)) ; ( + SPC B2 SPC A3 RET
> >                    (ses-jump A3) ; C-n
> >                    (ses-insert-row 1) ; C-o
> >                    (ses-edit-cell 2 0 4) ; 4 RET
> >                    (ses-jump B2) ; C-p
> >                    ))
> >       (apply 'funcall-interactively  val)
> >       (ses-command-hook))
> >   (let ((beg (point-marker))); C-SPC
> >     (ses-jump 'C2) ; C-f
> >     (copy-region-as-kill beg (point) nil); M-w
> >     (set-marker beg nil))
> >   (dolist (val '(
> >                  (ses-jump B3); C-n C-b
> >                  (yank) ; C-y
> >                  (ses-jump B4); C-n
> >                  (yank); C-y
> >                  ))
> >     (apply 'funcall-interactively  val)
> >     (ses-command-hook))
> >   (ses-repair-cell-reference-all)
> >   )))
> >
> > Le dim. 25 juil. 2021 à 07:56, Vincent Belaïche
> > <vincent.belaiche@gmail.com> a écrit :
> > >
> > > On more significant difference between the function bug-5852 and the
> > > example is the the initial size is (3 . 3) instead of (1. 1). This is
> > > because I speculated that only the last row insertion (C-o) plays a
> > > role.
> > >
> > >   V.
> > >
> > > Le dim. 25 juil. 2021 à 07:48, Vincent Belaïche
> > > <vincent.belaiche@gmail.com> a écrit :
> > > >
> > > > Just to add something : reading again SES code, and (info "(elisp)
> > > > Command Overview") I am a bit skeptical about the explanation in my
> > > > previous email.
> > > > It seems that only key strokes are queue, but command execution and
> > > > post command hook are in sequence, so not done concurrently.
> > > > I wrote this function to try to reproduce the bug, and on my office PC
> > > > it does not reproduce it:
> > > >
> > > > (defun bug-5852 ();;---
> > > >   "Check no irrelevant reference is created when a row is inserted and
> > > > a formula is yanked"
> > > >   (interactive);;---
> > > >   (let ((ses-initial-size '(3 . 3)))
> > > >   (with-temp-buffer
> > > >     (ses-mode)
> > > >     (dolist (val '(
> > > >                    (ses-edit-cell 0 1 1)
> > > >                    (ses-edit-cell 1 0 2)
> > > >                    (ses-edit-cell 1 1 (+ B1 A2))
> > > >                    (ses-edit-cell 2 0 3)
> > > >                    (ses-edit-cell 2 1 (+ B2 A3))
> > > >                    (ses-jump A3)
> > > >                    (ses-insert-row 1)
> > > >                    (ses-edit-cell 2 0 4)
> > > >                    (ses-jump B2)
> > > >                    ))
> > > >       (apply 'funcall-interactively  val)
> > > >       (ses-command-hook))
> > > >     (let ((beg (point-marker)))
> > > >       (ses-jump 'C2) ; C-n
> > > >       (copy-region-as-kill beg (point) nil)
> > > >       (set-marker beg nil))
> > > >     (dolist (val '(
> > > >                    (ses-jump B3)
> > > >                    (yank)
> > > >                    (ses-jump B4)
> > > >                    (yank)
> > > >                    ))
> > > >       (apply 'funcall-interactively  val)
> > > >       (ses-command-hook))
> > > >     (ses-repair-cell-reference-all)
> > > >     )))
> > > >
> > > > Please note that the main difference with the provided example is that
> > > > motion is done with ses-jump, and not with motion commands. Also the
> > > > save to keyring is not done interatively.
> > > >    V.
> > > >
> > > > Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
> > > > <vincent.belaiche@gmail.com> a écrit :
> > > > >
> > > > > Hello Lars,
> > > > >
> > > > > Sorry for not making any feedback for so long, actually I was just
> > > > > distracted by some other business, and I somewhat had forgotten about
> > > > > it.
> > > > >
> > > > > I suspect a race condition between the post command hook and the next
> > > > > command. That is to say when one yanks the fomula (the first yank),
> > > > > one relocates the reference but for this one uses current references
> > > > > that have not yet been relocated because the previous post-command
> > > > > hook execution is not yet over.
> > > > >
> > > > > The bug does not show every time I do the experiment. I tried this
> > > > > morning with my office PC which is a fast machine, and it proved out
> > > > > quite hard to reproduce, even when I launched some big C program
> > > > > compilation in parallel to eat the mips.
> > > > >
> > > > > Next week I will do some experiment with my home PC which is quite
> > > > > quite slower, and I make some feedback. Unfortunateless both machine
> > > > > do not have the same emacs version, but anyway all changes in SES
> > > > > since the version on my home PC are not related to this bug, so
> > > > > nevertheless that will be a good indication whether my guess is
> > > > > correct.
> > > > >
> > > > >   V.
> > > > >
> > > > > Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> > > > > >
> > > > > > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> > > > > >
> > > > > > > I will investigate why and feed back once I have a better idea of what
> > > > > > > is happening.
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > this was a year ago.  Did you make any progress on this issue?
> > > > > >
> > > > > > --
> > > > > > (domestic pets only, the antidote for overdose, milk.)
> > > > > >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Just to say that I did the test manually on my home PC a couple of
time, and I could not reproduce the problem.
That is quite annoying, yesterday I reproduced it on my office PC once.
  V.

Le dim. 25 juil. 2021 à 08:32, Vincent Belaïche
<vincent.belaiche@gmail.com> a écrit :
>
> By doing the test manually on my home PC I found some interesting
> difference. After
>
> 2 RET
>
> on my home PC the cursor goes to the end of buffer narrowed region,
> while on my office PC it goes to B2. This means that the TAB that
> follows won't the same effect. On my home PC it creates a 3rd row,
> while on my office PC it creates the 3rd column.
> I found the reason why so : my home PC had some customization for the
> ses-after-entry-functions. After erasing customization I had the same
> behaviour …
> Anyway, that does not affect the bug-5852 test function as it only
> uses ses-jump for motion.
>   V.
>
> Le dim. 25 juil. 2021 à 08:18, Vincent Belaïche
> <vincent.belaiche@gmail.com> a écrit :
> >
> > The bug-5852 test function does not reproduce the bug on my home PC either …
> >   V.
> >
> > Le dim. 25 juil. 2021 à 08:17, Vincent Belaïche
> > <vincent.belaiche@gmail.com> a écrit :
> > >
> > > The same test function, but with comments :
> > >
> > >
> > > (defun bug-5852 ();;---
> > >   "Check no irrelevant reference is created when a row is inserted and
> > > a formula is yanked"
> > >   (interactive);;---
> > >   (let ((ses-initial-size '(3 . 3))); size before last row insertion,
> > > due to all preceding TAB & C-o
> > >   (with-temp-buffer
> > >     (ses-mode)
> > >     (dolist (val '(
> > >                    (ses-edit-cell 0 1 1) ; 1 RET
> > >                    (ses-edit-cell 1 0 2) ; 2 RET
> > >                    (ses-edit-cell 1 1 (+ B1 A2)) ; ( + SPC B1 SPC A2 RET
> > >                    (ses-edit-cell 2 0 3) ; 3 RET
> > >                    (ses-edit-cell 2 1 (+ B2 A3)) ; ( + SPC B2 SPC A3 RET
> > >                    (ses-jump A3) ; C-n
> > >                    (ses-insert-row 1) ; C-o
> > >                    (ses-edit-cell 2 0 4) ; 4 RET
> > >                    (ses-jump B2) ; C-p
> > >                    ))
> > >       (apply 'funcall-interactively  val)
> > >       (ses-command-hook))
> > >   (let ((beg (point-marker))); C-SPC
> > >     (ses-jump 'C2) ; C-f
> > >     (copy-region-as-kill beg (point) nil); M-w
> > >     (set-marker beg nil))
> > >   (dolist (val '(
> > >                  (ses-jump B3); C-n C-b
> > >                  (yank) ; C-y
> > >                  (ses-jump B4); C-n
> > >                  (yank); C-y
> > >                  ))
> > >     (apply 'funcall-interactively  val)
> > >     (ses-command-hook))
> > >   (ses-repair-cell-reference-all)
> > >   )))
> > >
> > > Le dim. 25 juil. 2021 à 07:56, Vincent Belaïche
> > > <vincent.belaiche@gmail.com> a écrit :
> > > >
> > > > On more significant difference between the function bug-5852 and the
> > > > example is the the initial size is (3 . 3) instead of (1. 1). This is
> > > > because I speculated that only the last row insertion (C-o) plays a
> > > > role.
> > > >
> > > >   V.
> > > >
> > > > Le dim. 25 juil. 2021 à 07:48, Vincent Belaïche
> > > > <vincent.belaiche@gmail.com> a écrit :
> > > > >
> > > > > Just to add something : reading again SES code, and (info "(elisp)
> > > > > Command Overview") I am a bit skeptical about the explanation in my
> > > > > previous email.
> > > > > It seems that only key strokes are queue, but command execution and
> > > > > post command hook are in sequence, so not done concurrently.
> > > > > I wrote this function to try to reproduce the bug, and on my office PC
> > > > > it does not reproduce it:
> > > > >
> > > > > (defun bug-5852 ();;---
> > > > >   "Check no irrelevant reference is created when a row is inserted and
> > > > > a formula is yanked"
> > > > >   (interactive);;---
> > > > >   (let ((ses-initial-size '(3 . 3)))
> > > > >   (with-temp-buffer
> > > > >     (ses-mode)
> > > > >     (dolist (val '(
> > > > >                    (ses-edit-cell 0 1 1)
> > > > >                    (ses-edit-cell 1 0 2)
> > > > >                    (ses-edit-cell 1 1 (+ B1 A2))
> > > > >                    (ses-edit-cell 2 0 3)
> > > > >                    (ses-edit-cell 2 1 (+ B2 A3))
> > > > >                    (ses-jump A3)
> > > > >                    (ses-insert-row 1)
> > > > >                    (ses-edit-cell 2 0 4)
> > > > >                    (ses-jump B2)
> > > > >                    ))
> > > > >       (apply 'funcall-interactively  val)
> > > > >       (ses-command-hook))
> > > > >     (let ((beg (point-marker)))
> > > > >       (ses-jump 'C2) ; C-n
> > > > >       (copy-region-as-kill beg (point) nil)
> > > > >       (set-marker beg nil))
> > > > >     (dolist (val '(
> > > > >                    (ses-jump B3)
> > > > >                    (yank)
> > > > >                    (ses-jump B4)
> > > > >                    (yank)
> > > > >                    ))
> > > > >       (apply 'funcall-interactively  val)
> > > > >       (ses-command-hook))
> > > > >     (ses-repair-cell-reference-all)
> > > > >     )))
> > > > >
> > > > > Please note that the main difference with the provided example is that
> > > > > motion is done with ses-jump, and not with motion commands. Also the
> > > > > save to keyring is not done interatively.
> > > > >    V.
> > > > >
> > > > > Le sam. 24 juil. 2021 à 23:46, Vincent Belaïche
> > > > > <vincent.belaiche@gmail.com> a écrit :
> > > > > >
> > > > > > Hello Lars,
> > > > > >
> > > > > > Sorry for not making any feedback for so long, actually I was just
> > > > > > distracted by some other business, and I somewhat had forgotten about
> > > > > > it.
> > > > > >
> > > > > > I suspect a race condition between the post command hook and the next
> > > > > > command. That is to say when one yanks the fomula (the first yank),
> > > > > > one relocates the reference but for this one uses current references
> > > > > > that have not yet been relocated because the previous post-command
> > > > > > hook execution is not yet over.
> > > > > >
> > > > > > The bug does not show every time I do the experiment. I tried this
> > > > > > morning with my office PC which is a fast machine, and it proved out
> > > > > > quite hard to reproduce, even when I launched some big C program
> > > > > > compilation in parallel to eat the mips.
> > > > > >
> > > > > > Next week I will do some experiment with my home PC which is quite
> > > > > > quite slower, and I make some feedback. Unfortunateless both machine
> > > > > > do not have the same emacs version, but anyway all changes in SES
> > > > > > since the version on my home PC are not related to this bug, so
> > > > > > nevertheless that will be a good indication whether my guess is
> > > > > > correct.
> > > > > >
> > > > > >   V.
> > > > > >
> > > > > > Le dim. 18 juil. 2021 à 15:37, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
> > > > > > >
> > > > > > > Vincent Belaïche <vincentbelaiche@orange.fr> writes:
> > > > > > >
> > > > > > > > I will investigate why and feed back once I have a better idea of what
> > > > > > > > is happening.
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > this was a year ago.  Did you make any progress on this issue?
> > > > > > >
> > > > > > > --
> > > > > > > (domestic pets only, the antidote for overdose, milk.)
> > > > > > >    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Lars Ingebrigtsen]

Vincent Belaïche <vincent.belaiche@gmail.com> writes:

> Just to say that I did the test manually on my home PC a couple of
> time, and I could not reproduce the problem.
> That is quite annoying, yesterday I reproduced it on my office PC once.

Yeah, timing dependent bugs are really annoying to try to debug..

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Dear Stefan (Monnier),

I am calling to your deep knowledge of Emacs internals.

We have a bug which seems to be real time dependant, which makes me
think that there is some race condition somewhere. I think it can't be
any race between the command and the post-command hook, because they
are run sequentially in the command loop : post-command hook of
command N does not overlap command N+1, please confirm …

Now, I have another speculation : SES gets the current cell from the
cursor position by reading some buffer text property
'cursor-intangible. I speculate that the radix of the bug is that when
a command sets this cursor property, then the buffer is actually
modified in the backgroun, and the change may not be yet in effect
when the next command runs. Could you confirm / infirm this
speculation.

One more thing is the following : in SES the cursor-intangile property
is the symbol corresponding to the cell object (which under the hood
is a vector), not the cell object itself. This means that the property
has to change when there are row/column insertion / deletion, which
also inherently change the buffer, because of symbol relocation. I
think this might be some bad design choice, and pointing directly at
the cell object would have saved some troubles (and probably would
also create a bunch of other problems, so do not take my statement for
sure). Of course the idea of using symbols makes sense as they make it
easy for the formulas, and as such some sort of relocation is
unavoidable. On the other hand, the cell index (say (0 0) for A1, (0
1) for B1, (1 0) for A2, etc…), is retrieved from a property of the
cell symbol, so things are a bit intricate …

  Vincent.



Le dim. 25 juil. 2021 à 08:44, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
>
> Vincent Belaïche <vincent.belaiche@gmail.com> writes:
>
> > Just to say that I did the test manually on my home PC a couple of
> > time, and I could not reproduce the problem.
> > That is quite annoying, yesterday I reproduced it on my office PC once.
>
> Yeah, timing dependent bugs are really annoying to try to debug..
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> are run sequentially in the command loop : post-command hook of
> command N does not overlap command N+1, please confirm …

That's usually true, yes.  There can be exceptions when command N itself
can run commands during execution, via a recursive command-loop
(e.g. query&replace uses that, or minibuffer interactions).

> Now, I have another speculation : SES gets the current cell from the
> cursor position by reading some buffer text property
> 'cursor-intangible.  I speculate that the radix of the bug is that when
> a command sets this cursor property, then the buffer is actually
> modified in the backgroun, and the change may not be yet in effect
> when the next command runs.  Could you confirm / infirm this
> speculation.

Hmm... no that doesn't sound right.  Emacs does try to do a few things
in the background, but it's quite limited (things like `font-lock`,
basically).  When a property like `cursor-intangible` is set, it
happens immediately.

> One more thing is the following : in SES the cursor-intangile property
> is the symbol corresponding to the cell object (which under the hood
> is a vector), not the cell object itself. This means that the property
> has to change when there are row/column insertion / deletion, which
> also inherently change the buffer, because of symbol relocation. I
> think this might be some bad design choice, and pointing directly at
> the cell object would have saved some troubles (and probably would
> also create a bunch of other problems, so do not take my statement for
> sure).

Indeed, pointing directly at the cell's vector would probably be better
(at least from the point of view of insertions/deletions but), but if
so, each cell would probably need to contain its own name (or its
coordinates).


        Stefan

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Thank you for the feedback, I am afraid I will need to work again at
it, in order to create a systematic bug reporduction.

[...]

> > One more thing is the following : in SES the cursor-intangile property
> > is the symbol corresponding to the cell object (which under the hood
> > is a vector), not the cell object itself. This means that the property
> > has to change when there are row/column insertion / deletion, which
> > also inherently change the buffer, because of symbol relocation. I
> > think this might be some bad design choice, and pointing directly at
> > the cell object would have saved some troubles (and probably would
> > also create a bunch of other problems, so do not take my statement for
> > sure).
>
> Indeed, pointing directly at the cell's vector would probably be better
> (at least from the point of view of insertions/deletions but), but if
> so, each cell would probably need to contain its own name (or its
> coordinates).
>
>
>         Stefan
>
What you wrote confirms my gut feeling. I came to this conclusion as I
was thinking how to implement functions such as row transposition.

FYI, AFAIK, the cell already contains the corresponding symbol, and the
(row, col) coordinates can be derived from the symbol through the
ses-cell symbol property.
  V.

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

Dear all,

I am _very very_ glad to a announce that I have fixed bug#5852  ! I
just pushed the fix to master now.

Thank you again to Göran for the report, without which this would not
have been impossible.

The bug was only loosely time dependent. What happened is that the
data area update is deferred on the command hook, if I try to
reproduce the bug by doing some kdb macro, it seems that the command
hook is not called in between the two yanks, and this hides the bug.
That was my mistake during my first investigation. Translating the
manual to French helped me be more aware of all the arcanes of SES.

The root cause of the bug has nothing to do with copy/cut/paste, it
was far deeper, the formula setting function was bugged, as it did not
registered cells the reference list of which is changed by the formula
setting as needing rewrite to the data area !

It was hard to realized where the bug was, because the data structures
in memory was OK, but there was a discrepancy between the data areay
and the data structures, so the bugs only gets manifest when you
reload a sheet --- or as in the new test if you call ses-mode again.
So as long as you do not reload a sheet everything is OK.

@Stefan Kangas is there any record book for longest lived bugs, maybe
I can get a prize for this 13 years old bug, unless Richard has left
some bug form the 1970 or so version of Emacs :-P ?

Happy end of year to you all !

   Vincent.

Le dim. 25 juil. 2021 à 08:44, Lars Ingebrigtsen <larsi@gnus.org> a écrit :
>
> Vincent Belaïche <vincent.belaiche@gmail.com> writes:
>
> > Just to say that I did the test manually on my home PC a couple of
> > time, and I could not reproduce the problem.
> > That is quite annoying, yesterday I reproduced it on my office PC once.
>
> Yeah, timing dependent bugs are really annoying to try to debug..
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

bug#5852: 23.1; Incorrect references in ses-mode

[Stefan Kangas]

Vincent Belaïche <vincent.belaiche@gmail.com> writes:

> I am _very very_ glad to a announce that I have fixed bug#5852  ! I
> just pushed the fix to master now.
>
> Thank you again to Göran for the report, without which this would not
> have been impossible.

Thanks for fixing this old bug, and congratulations!

> @Stefan Kangas is there any record book for longest lived bugs, maybe
> I can get a prize for this 13 years old bug, unless Richard has left
> some bug form the 1970 or so version of Emacs :-P ?

No one is keeping track really, but it's not too hard to dig out some
information from git log (grep for "Bug#[0-9][0-9][0-9][0-9]?[^0-9])".

AFAICT, the current record holder is Mauro Aranda with his fix for
Bug#2807 back in October (or possibly his fix for Bug#997).

Our bug tracker only goes back to 2008 or so though, while some bugs
have been around for 30 or 40 years without being recorded there.  It's
harder to search for those fixes in the git logs, though.

bug#5852: 23.1; Incorrect references in ses-mode

[Eli Zaretskii]

> Cc: Göran Uddeborg <goeran@uddeborg.se>,
>  Vincent Belaïche <vincent.b.1@hotmail.fr>,
>  5852@debbugs.gnu.org
> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 29 Dec 2023 04:36:03 -0800
> 
> Vincent Belaïche <vincent.belaiche@gmail.com> writes:
> 
> > @Stefan Kangas is there any record book for longest lived bugs, maybe
> > I can get a prize for this 13 years old bug, unless Richard has left
> > some bug form the 1970 or so version of Emacs :-P ?
> 
> No one is keeping track really, but it's not too hard to dig out some
> information from git log (grep for "Bug#[0-9][0-9][0-9][0-9]?[^0-9])".
> 
> AFAICT, the current record holder is Mauro Aranda with his fix for
> Bug#2807 back in October (or possibly his fix for Bug#997).

My personal record is bug#24064: a 22-year old bug.  Followed closely
by bug#64253 (a 21-year old bug), bug#31829 (a 16-year old bug), and
bug#56219 (a 15-year old bug).

bug#5852: 23.1; Incorrect references in ses-mode

[Vincent Belaïche]

[-- Attachment #1: Type: text/plain, Size: 1879 bytes --]

Hello,

Thank you for the feedback and congratulations for your own records !

I forgot to mention this : to repair any existing spreadsheet made prior to the bugfix, the procedure is as follows :


  *
M-x ses-repair-cell-reference-all will fix all the reference lists in the in-memory data structures
  *
C-c C-l will recompute the whole spreadsheet, which will cause the data area to be completely rewritten

After these two commands, your spreadsheet is bug#5852-neat.

   V.
________________________________
De : Eli Zaretskii <eliz@gnu.org>
Envoyé : vendredi 29 décembre 2023 15:19
À : Stefan Kangas <stefankangas@gmail.com>
Cc : vincent.belaiche@gmail.com <vincent.belaiche@gmail.com>; larsi@gnus.org <larsi@gnus.org>; goeran@uddeborg.se <goeran@uddeborg.se>; vincent.b.1@hotmail.fr <vincent.b.1@hotmail.fr>; 5852@debbugs.gnu.org <5852@debbugs.gnu.org>
Objet : Re: bug#5852: 23.1; Incorrect references in ses-mode

> Cc: Göran Uddeborg <goeran@uddeborg.se>,
>  Vincent Belaïche <vincent.b.1@hotmail.fr>,
>  5852@debbugs.gnu.org
> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 29 Dec 2023 04:36:03 -0800
>
> Vincent Belaïche <vincent.belaiche@gmail.com> writes:
>
> > @Stefan Kangas is there any record book for longest lived bugs, maybe
> > I can get a prize for this 13 years old bug, unless Richard has left
> > some bug form the 1970 or so version of Emacs :-P ?
>
> No one is keeping track really, but it's not too hard to dig out some
> information from git log (grep for "Bug#[0-9][0-9][0-9][0-9]?[^0-9])".
>
> AFAICT, the current record holder is Mauro Aranda with his fix for
> Bug#2807 back in October (or possibly his fix for Bug#997).

My personal record is bug#24064: a 22-year old bug.  Followed closely
by bug#64253 (a 21-year old bug), bug#31829 (a 16-year old bug), and
bug#56219 (a 15-year old bug).

[-- Attachment #2: Type: text/html, Size: 4570 bytes --]

bug#5852: 23.1; Incorrect references in ses-mode

[Göran Uddeborg]

Thanks a lot Vincent and everyone else for putting all those efforts
into finding this bug! Now I'm looking forward to Emacs 30!

bug#5852: 23.1; Incorrect references in ses-mode

[Mattias Engdegård]

Some test problems fixed in 139037f8; please verify.

bug#5852: 23.1; Incorrect references in ses-mode

[Mattias Engdegård]

> Some test problems fixed in 139037f8; please verify.

That should be a83e60ec, sorry.