bug#64652: 28.2; Adding a property bug on Windows 11

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/html, Size: 10058 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 699 bytes --]

On Jul 16, 2023, at 00:45, Senki Jun <senkijun@gmail.com> wrote:
> 
> ***
>  
>  
> Adding a property with C-c C-x P triggers an error.
> Actually any command that adds Property drawer triggers the error.
>  
> https://i.redd.it/n13a7x8yq3cb1.png
>  
> Everything works fine on linux with same files.
> Tried with clean emacs installation and get the same error.
>  
> Happens on version 26 and 27 as well. Version 25 works fine.
>  
> Thank you.

Sounds like an Org bug report (from the keybind C-c C-x P and the term "property drawer"), so I'm taking the liberty of CC'ing Ihor. 

Senki, please note that there is a dedicated email address (and bug-report command) for Org :)

[-- Attachment #2: Type: text/html, Size: 1960 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Ruijie Yu <ruijie@netyu.xyz> writes:

> Sounds like an Org bug report (from the keybind C-c C-x P and the term "property drawer"), so I'm taking the liberty of CC'ing Ihor. 

Nope. This was originally a question on reddit and I specifically asked
to report as Emacs bug.

Because the problem is only triggered on Windows, and not on Linux.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> Cc: Senki Jun <senkijun@gmail.com>, 64652@debbugs.gnu.org
> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Sat, 15 Jul 2023 17:18:24 +0000
> 
> Ruijie Yu <ruijie@netyu.xyz> writes:
> 
> > Sounds like an Org bug report (from the keybind C-c C-x P and the term "property drawer"), so I'm taking the liberty of CC'ing Ihor. 
> 
> Nope. This was originally a question on reddit and I specifically asked
> to report as Emacs bug.
> 
> Because the problem is only triggered on Windows, and not on Linux.

Please show a minimal recipe for reproducing the problem starting from
"emacs -Q".  Otherwise, I don't see how we can look into this.

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 882 bytes --]

Hey, just recorded a video of minimal recipe starting with emacs -Q.. Hope
it helps.

https://youtu.be/m8xk9sHyBbk



On Sat, Jul 15, 2023 at 9:53 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: Senki Jun <senkijun@gmail.com>, 64652@debbugs.gnu.org
> > From: Ihor Radchenko <yantar92@posteo.net>
> > Date: Sat, 15 Jul 2023 17:18:24 +0000
> >
> > Ruijie Yu <ruijie@netyu.xyz> writes:
> >
> > > Sounds like an Org bug report (from the keybind C-c C-x P and the term
> "property drawer"), so I'm taking the liberty of CC'ing Ihor.
> >
> > Nope. This was originally a question on reddit and I specifically asked
> > to report as Emacs bug.
> >
> > Because the problem is only triggered on Windows, and not on Linux.
>
> Please show a minimal recipe for reproducing the problem starting from
> "emacs -Q".  Otherwise, I don't see how we can look into this.
>

[-- Attachment #2: Type: text/html, Size: 1580 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1398 bytes --]

The versions after 25.3 have 2 bugs that is Windows 11 specific. (no
problems on windows 10 or linux)

1) Adding a property
2) Unable to create a clock report

The latest version that doesn't give these errors is Emacs version 25.3
Wanted to tell clock report bug as well in case they might have the same
cause.

clock report bug : https://youtu.be/qU4zdZkUuXY

Thank you for your time




On Sat, Jul 15, 2023 at 10:24 PM Senki Jun <senkijun@gmail.com> wrote:

> Hey, just recorded a video of minimal recipe starting with emacs -Q.. Hope
> it helps.
>
> https://youtu.be/m8xk9sHyBbk
>
>
>
> On Sat, Jul 15, 2023 at 9:53 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > Cc: Senki Jun <senkijun@gmail.com>, 64652@debbugs.gnu.org
>> > From: Ihor Radchenko <yantar92@posteo.net>
>> > Date: Sat, 15 Jul 2023 17:18:24 +0000
>> >
>> > Ruijie Yu <ruijie@netyu.xyz> writes:
>> >
>> > > Sounds like an Org bug report (from the keybind C-c C-x P and the
>> term "property drawer"), so I'm taking the liberty of CC'ing Ihor.
>> >
>> > Nope. This was originally a question on reddit and I specifically asked
>> > to report as Emacs bug.
>> >
>> > Because the problem is only triggered on Windows, and not on Linux.
>>
>> Please show a minimal recipe for reproducing the problem starting from
>> "emacs -Q".  Otherwise, I don't see how we can look into this.
>>
>

[-- Attachment #2: Type: text/html, Size: 2569 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Senki Jun <senkijun@gmail.com>
> Date: Sat, 15 Jul 2023 22:24:36 +0300
> Cc: Ihor Radchenko <yantar92@posteo.net>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> 
> Hey, just recorded a video of minimal recipe starting with emacs -Q.. Hope it helps.

Thanks, but I still cannot follow it.  I don't understand what you did
after typing "* TODO Test".

Could you please describe the recipe in words, including all the test
and the key sequences you type?  I assure you the investigation will
be much faster after that.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Senki Jun <senkijun@gmail.com>
> Date: Sat, 15 Jul 2023 23:09:54 +0300
> Cc: Ihor Radchenko <yantar92@posteo.net>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> 
> The versions after 25.3 have 2 bugs that is Windows 11 specific. (no problems on windows 10 or
> linux)
> 
> 1) Adding a property
> 2) Unable to create a clock report

Please show recipes for each of these.  I use Org very little, so
"create a clock report" doesn't help me reproduce the issue.

(Frankly, I have hard time believing there could be Windows-specific
bugs in handling properties and regexp search, which is why I thought
the Org developers should look at this first, and explain what core
feature seemingly fails in these cases.  But if Ihor thinks it's
definitely a core bug,, I cannot do anything with these reports
without a complete recipe starting from "emacs -Q".)

> The latest version that doesn't give these errors is Emacs version 25.3  
> Wanted to tell clock report bug as well in case they might have the same cause.
> 
> clock report bug : https://youtu.be/qU4zdZkUuXY

Thanks, but posting a recording in these case is not an efficient
method of describing the recipe.  It is hard to keep track of what you
do to reproduce the problem, because the keys you press are not
visible.

So please describe the recipes in words.  It will take you slightly
more work, but will cut significantly on the amount of time and
messages we already exchanged, which still didn't bring me close
enough to reproducing the issues.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> (Frankly, I have hard time believing there could be Windows-specific
> bugs in handling properties and regexp search, which is why I thought
> the Org developers should look at this first, and explain what core
> feature seemingly fails in these cases.  But if Ihor thinks it's
> definitely a core bug,, I cannot do anything with these reports
> without a complete recipe starting from "emacs -Q".)

I suspect some problem with point mark position when inserting at point
inside `save-excursion'. At least, that's what I can imagine being
failing without doing testing myself (I have no access to Windows).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1876 bytes --]

Ah okay sorry, here is what I do.


Property Bug:

1) Emacs -Q.

2) I create an org file   (C-x C-f   test.org)

3) Write anything. (I create a heading with todo in the example but it's
optional)

4) Add a Property to the text by  (C-c C-x p)

5) Select any property (effort, category, location etc. anything)

6) Enter any value to any property.(I enter 1h to the Effort property in
the example)


Pressing enter gives the error.



Clock Report Bug:

1) Emacs -Q

2) C-x C-f  test2.org

3) Create a TODO heading  * TODO task

4) Clock in with  C-c C-x i

5) Clock out with C-c C-x o

6) M-x org-clock-report gives an error "Not at a dynamic block" instead of
creating the clock report table.


Same steps doesn't give any errors on linux or windows 10 (or wsl2) and it
just adds the property and clock report table.




On Sun, Jul 16, 2023 at 8:59 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > (Frankly, I have hard time believing there could be Windows-specific
> > bugs in handling properties and regexp search, which is why I thought
> > the Org developers should look at this first, and explain what core
> > feature seemingly fails in these cases.  But if Ihor thinks it's
> > definitely a core bug,, I cannot do anything with these reports
> > without a complete recipe starting from "emacs -Q".)
>
> I suspect some problem with point mark position when inserting at point
> inside `save-excursion'. At least, that's what I can imagine being
> failing without doing testing myself (I have no access to Windows).
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #2: Type: text/html, Size: 2698 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Senki Jun <senkijun@gmail.com> writes:

> Clock Report Bug:
> ...
> Same steps doesn't give any errors on linux or windows 10 (or wsl2) and it
> just adds the property and clock report table.

And I also cannot reproduce on Linux.

Just to be sure, may you check and post M-x emacs-version and
M-x org-version in Emacs where you do not have error and where you do
have it?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 930 bytes --]

No errors :   GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27

Errors :  GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13

(all versions after 25.3.1 gives the same errors)

On Sun, Jul 16, 2023 at 9:34 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Senki Jun <senkijun@gmail.com> writes:
>
> > Clock Report Bug:
> > ...
> > Same steps doesn't give any errors on linux or windows 10 (or wsl2) and
> it
> > just adds the property and clock report table.
>
> And I also cannot reproduce on Linux.
>
> Just to be sure, may you check and post M-x emacs-version and
> M-x org-version in Emacs where you do not have error and where you do
> have it?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #2: Type: text/html, Size: 1578 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Senki Jun <senkijun@gmail.com> writes:

> No errors :   GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27
>
> Errors :  GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13
>
> (all versions after 25.3.1 gives the same errors)

I mean on different Windows/Linux version where you do not get an error.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

Sorry forgot to send org versions.

Errors :

GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13
Org-Mode version 9.5.5


No Errors:

GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27
Org-Mode version 8.2.10


I don't have access to windows 10 or linux (endavour os) anymore. But 28.2
worked without problems on both of them.










On Sun, Jul 16, 2023 at 9:43 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Senki Jun <senkijun@gmail.com> writes:
>
> > No errors :   GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27
> >
> > Errors :  GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13
> >
> > (all versions after 25.3.1 gives the same errors)
>
> I mean on different Windows/Linux version where you do not get an error.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #2: Type: text/html, Size: 1974 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1323 bytes --]

Omg It was a conflict in language settings.
I switched the regional settings match with Os display language.
Both of the problems are solved.



On Sun, Jul 16, 2023 at 9:56 AM Senki Jun <senkijun@gmail.com> wrote:

>
> Sorry forgot to send org versions.
>
> Errors :
>
> GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13
> Org-Mode version 9.5.5
>
>
> No Errors:
>
> GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27
> Org-Mode version 8.2.10
>
>
> I don't have access to windows 10 or linux (endavour os) anymore. But 28.2
> worked without problems on both of them.
>
>
>
>
>
>
>
>
>
>
> On Sun, Jul 16, 2023 at 9:43 AM Ihor Radchenko <yantar92@posteo.net>
> wrote:
>
>> Senki Jun <senkijun@gmail.com> writes:
>>
>> > No errors :   GNU Emacs 25.3.1 (X86_64-w64-mingw32) of 2017-09-27
>> >
>> > Errors :  GNU Emacs 28.2 (build 2, x86_64-w64-mingw32) of 2022-09-13
>> >
>> > (all versions after 25.3.1 gives the same errors)
>>
>> I mean on different Windows/Linux version where you do not get an error.
>>
>> --
>> Ihor Radchenko // yantar92,
>> Org mode contributor,
>> Learn more about Org mode at <https://orgmode.org/>.
>> Support Org development at <https://liberapay.com/org-mode>,
>> or support my work at <https://liberapay.com/yantar92>
>>
>

[-- Attachment #2: Type: text/html, Size: 2556 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Senki Jun <senkijun@gmail.com> writes:

> Omg It was a conflict in language settings.
> I switched the regional settings match with Os display language.
> Both of the problems are solved.

Curious.
May you please provide more details?
Does the error re-appear again if you switch the language settings back?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1.1: Type: text/plain, Size: 849 bytes --]

Yes this might be happening because our country's name changed from Turkey
to Türkiye internationally.

[image: image.png]

When I select Türkiye, error re-appear again.





On Sun, Jul 16, 2023 at 10:15 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Senki Jun <senkijun@gmail.com> writes:
>
> > Omg It was a conflict in language settings.
> > I switched the regional settings match with Os display language.
> > Both of the problems are solved.
>
> Curious.
> May you please provide more details?
> Does the error re-appear again if you switch the language settings back?
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #1.2: Type: text/html, Size: 1594 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 9425 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Senki Jun <senkijun@gmail.com>
> Date: Sun, 16 Jul 2023 10:12:06 +0300
> Cc: Eli Zaretskii <eliz@gnu.org>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> 
> Omg It was a conflict in language settings.
> I switched the regional settings match with Os display language.
> Both of the problems are solved.

Thanks, please tell exactly what regional and language settings caused
the problem.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Sun, 16 Jul 2023 05:57:07 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks, but I still cannot follow it.  I don't understand what you did
> > after typing "* TODO Test".
> 
> C-c C-x p <RET> Effort <RET> 1h <RET>

Doesn't reproduce on my Windows system

> According to the backtrace image in the OP, the error is triggered
> inside
> 
> org-get-property-block in
> 
> (force
> 	    (goto-char beg)
> 	    (org-insert-property-drawer)
> 	    (let ((pos (save-excursion (re-search-forward org-property-drawer-re); <-- errs
> 				       (line-beginning-position))))
> 	      (cons pos pos)))
> 
> Normally, BEG should be on a heading and `org-insert-property-drawer'
> should insert the property drawer below current heading and
> `re-search-forward' must not fail (deliberately asserted by absence of
> NOERROR flag)

Yes, I know.  But this doesn't happen on my system.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Senki Jun <senkijun@gmail.com>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 05:59:25 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > (Frankly, I have hard time believing there could be Windows-specific
> > bugs in handling properties and regexp search, which is why I thought
> > the Org developers should look at this first, and explain what core
> > feature seemingly fails in these cases.  But if Ihor thinks it's
> > definitely a core bug,, I cannot do anything with these reports
> > without a complete recipe starting from "emacs -Q".)
> 
> I suspect some problem with point mark position when inserting at point
> inside `save-excursion'.

This is done by generic Emacs code that doesn't involve any
OS-specific APIs.  So it is highly unlikely that the problem is
Windows-specific.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Senki Jun <senkijun@gmail.com>
> Date: Sun, 16 Jul 2023 09:16:35 +0300
> Cc: Eli Zaretskii <eliz@gnu.org>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> 
> Property Bug:
> 
> 1) Emacs -Q.
> 
> 2) I create an org file   (C-x C-f   test.org)
> 
> 3) Write anything. (I create a heading with todo in the example but it's optional)
> 
> 4) Add a Property to the text by  (C-c C-x p)
> 
> 5) Select any property (effort, category, location etc. anything)
> 
> 6) Enter any value to any property.(I enter 1h to the Effort property in the example)
> 
> Pressing enter gives the error.

Doesn't happen here.

> Clock Report Bug:
> 
> 1) Emacs -Q
> 
> 2) C-x C-f  test2.org
> 
> 3) Create a TODO heading  * TODO task
> 
> 4) Clock in with  C-c C-x i
> 
> 5) Clock out with C-c C-x o
> 
> 6) M-x org-clock-report gives an error "Not at a dynamic block" instead of creating the clock report
> table.

Doesn't happen here.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Senki Jun <senkijun@gmail.com>
> Date: Sun, 16 Jul 2023 10:20:08 +0300
> Cc: Eli Zaretskii <eliz@gnu.org>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> 
> Yes this might be happening because our country's name changed from Turkey to Türkiye
> internationally.

That's what I thought.

In Turkish locales, if search of pure ASCII text uses case-fold, it
should do that with ASCII case-table.

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 695 bytes --]

 [image: image.png]

This is the setting causing the error to be precise. When I select Türkiye
error re-appears.
When I select recommended both of the problems getting solved.

On Sun, Jul 16, 2023 at 10:28 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Senki Jun <senkijun@gmail.com>
> > Date: Sun, 16 Jul 2023 10:20:08 +0300
> > Cc: Eli Zaretskii <eliz@gnu.org>, ruijie@netyu.xyz,
> 64652@debbugs.gnu.org
> >
> > Yes this might be happening because our country's name changed from
> Turkey to Türkiye
> > internationally.
>
> That's what I thought.
>
> In Turkish locales, if search of pure ASCII text uses case-fold, it
> should do that with ASCII case-table.
>

[-- Attachment #2: Type: text/html, Size: 1321 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1.1.1: Type: text/plain, Size: 825 bytes --]

[image: image.png]


On Sun, Jul 16, 2023 at 10:31 AM Senki Jun <senkijun@gmail.com> wrote:

> [image: image.png]
>
> This is the setting causing the error to be precise. When I select
> Türkiye  error re-appears.
> When I select recommended both of the problems getting solved.
>
> On Sun, Jul 16, 2023 at 10:28 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: Senki Jun <senkijun@gmail.com>
>> > Date: Sun, 16 Jul 2023 10:20:08 +0300
>> > Cc: Eli Zaretskii <eliz@gnu.org>, ruijie@netyu.xyz,
>> 64652@debbugs.gnu.org
>> >
>> > Yes this might be happening because our country's name changed from
>> Turkey to Türkiye
>> > internationally.
>>
>> That's what I thought.
>>
>> In Turkish locales, if search of pure ASCII text uses case-fold, it
>> should do that with ASCII case-table.
>>
>

[-- Attachment #1.1.2: Type: text/html, Size: 1744 bytes --]

[-- Attachment #1.2: image.png --]
[-- Type: image/png, Size: 32775 bytes --]

[-- Attachment #2: abc.png --]
[-- Type: image/png, Size: 30382 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:
>> Yes this might be happening because our country's name changed from Turkey to Türkiye
>> internationally.
>
> That's what I thought.
>
> In Turkish locales, if search of pure ASCII text uses case-fold, it
> should do that with ASCII case-table.

Org inherits outline-mode-syntax-table, which inherits
standard-syntax-table, AFAIU. Is there anything wrong with this
approach?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Senki Jun <senkijun@gmail.com>, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 07:37:52 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> Yes this might be happening because our country's name changed from Turkey to Türkiye
> >> internationally.
> >
> > That's what I thought.
> >
> > In Turkish locales, if search of pure ASCII text uses case-fold, it
> > should do that with ASCII case-table.
> 
> Org inherits outline-mode-syntax-table, which inherits
> standard-syntax-table, AFAIU. Is there anything wrong with this
> approach?

I'm talking about case-table, not syntax-table.

It sounds like Org searches for "PROPERTIES" with case-fold-search
non-nil, and in Turkish locales that downcases PROPERTIES to
propertıes (not the dotless ı letter instead of i), which then fails
to find the string.  Something like that.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Org inherits outline-mode-syntax-table, which inherits
>> standard-syntax-table, AFAIU. Is there anything wrong with this
>> approach?
>
> I'm talking about case-table, not syntax-table.

Org does nothing about case-table. So, it should be the default one.

> It sounds like Org searches for "PROPERTIES" with case-fold-search
> non-nil, and in Turkish locales that downcases PROPERTIES to
> propertıes (not the dotless ı letter instead of i), which then fails
> to find the string.  Something like that.

Does it also mean that any user with Turkish locale will experience the
same failure when searching for "i"? Like M-x isearch <RET> i

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1193 bytes --]

Not sure if I understand correctly but I can isearch

"i, ı" and "I, İ"

without errors whether Türkiye selected or not.



On Sun, Jul 16, 2023 at 11:06 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> Org inherits outline-mode-syntax-table, which inherits
> >> standard-syntax-table, AFAIU. Is there anything wrong with this
> >> approach?
> >
> > I'm talking about case-table, not syntax-table.
>
> Org does nothing about case-table. So, it should be the default one.
>
> > It sounds like Org searches for "PROPERTIES" with case-fold-search
> > non-nil, and in Turkish locales that downcases PROPERTIES to
> > propertıes (not the dotless ı letter instead of i), which then fails
> > to find the string.  Something like that.
>
> Does it also mean that any user with Turkish locale will experience the
> same failure when searching for "i"? Like M-x isearch <RET> i
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #2: Type: text/html, Size: 1951 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: senkijun@gmail.com, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 08:07:01 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Org inherits outline-mode-syntax-table, which inherits
> >> standard-syntax-table, AFAIU. Is there anything wrong with this
> >> approach?
> >
> > I'm talking about case-table, not syntax-table.
> 
> Org does nothing about case-table. So, it should be the default one.

That's the problem: the default case-table in Turkish locales will
fail case-insensitive search for ASCII strings.  Try this:

  emacs -Q
  M-x set-language-environment RET Turkish RET

Then type "PROPERTIES" into *scratch*, go to its beginning, and type
M-l.  You will see "propertıes".

> > It sounds like Org searches for "PROPERTIES" with case-fold-search
> > non-nil, and in Turkish locales that downcases PROPERTIES to
> > propertıes (not the dotless ı letter instead of i), which then fails
> > to find the string.  Something like that.
> 
> Does it also mean that any user with Turkish locale will experience the
> same failure when searching for "i"? Like M-x isearch <RET> i

Yes (except that Isearch has some heuristics regarding letter-case
when invoked interactively).  After set-language-environment to
Turkish, type "PROPERTIES", go to the beginning of the word you typed,
and do

   M-: (search-forward "i") RET

You will see that it fails.  Now reset language-environment to English
and repeat the same search: it will now succeed.

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1.1.1: Type: text/plain, Size: 1747 bytes --]

I can confirm

[image: 54.png]


On Sun, Jul 16, 2023 at 11:40 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Ihor Radchenko <yantar92@posteo.net>
> > Cc: senkijun@gmail.com, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> > Date: Sun, 16 Jul 2023 08:07:01 +0000
> >
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> > >> Org inherits outline-mode-syntax-table, which inherits
> > >> standard-syntax-table, AFAIU. Is there anything wrong with this
> > >> approach?
> > >
> > > I'm talking about case-table, not syntax-table.
> >
> > Org does nothing about case-table. So, it should be the default one.
>
> That's the problem: the default case-table in Turkish locales will
> fail case-insensitive search for ASCII strings.  Try this:
>
>   emacs -Q
>   M-x set-language-environment RET Turkish RET
>
> Then type "PROPERTIES" into *scratch*, go to its beginning, and type
> M-l.  You will see "propertıes".
>
> > > It sounds like Org searches for "PROPERTIES" with case-fold-search
> > > non-nil, and in Turkish locales that downcases PROPERTIES to
> > > propertıes (not the dotless ı letter instead of i), which then fails
> > > to find the string.  Something like that.
> >
> > Does it also mean that any user with Turkish locale will experience the
> > same failure when searching for "i"? Like M-x isearch <RET> i
>
> Yes (except that Isearch has some heuristics regarding letter-case
> when invoked interactively).  After set-language-environment to
> Turkish, type "PROPERTIES", go to the beginning of the word you typed,
> and do
>
>    M-: (search-forward "i") RET
>
> You will see that it fails.  Now reset language-environment to English
> and repeat the same search: it will now succeed.
>

[-- Attachment #1.1.2: Type: text/html, Size: 2642 bytes --]

[-- Attachment #1.2: 54.png --]
[-- Type: image/png, Size: 17048 bytes --]

[-- Attachment #2: 54.png --]
[-- Type: image/png, Size: 17048 bytes --]

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Org does nothing about case-table. So, it should be the default one.
>
> That's the problem: the default case-table in Turkish locales will
> fail case-insensitive search for ASCII strings.  Try this:
>
>   emacs -Q
>   M-x set-language-environment RET Turkish RET
> ... 
>> Does it also mean that any user with Turkish locale will experience the
>> same failure when searching for "i"? Like M-x isearch <RET> i
>
> Yes (except that Isearch has some heuristics regarding letter-case
> when invoked interactively).  After set-language-environment to
> Turkish, type "PROPERTIES", go to the beginning of the word you typed,
> and do

AFAIU, it then means that any parser/font-lock keywords may fail to
fontify case-insensitive keywords that contain "i" in Turkish language
environment.

On the Org side, we may probably use ascii-case-table. However, this
will break legitimate user edits involving other locales.
Alternatively, Org may wrap every regexp search into `with-case-table',
but that will cause performance degradation, AFAIU.

I feel that the problem may need to be addressed somehow on Emacs side.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: senkijun@gmail.com, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 08:56:47 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Yes (except that Isearch has some heuristics regarding letter-case
> > when invoked interactively).  After set-language-environment to
> > Turkish, type "PROPERTIES", go to the beginning of the word you typed,
> > and do
> 
> AFAIU, it then means that any parser/font-lock keywords may fail to
> fontify case-insensitive keywords that contain "i" in Turkish language
> environment.

In general, yes.  And not only in Turkish, there are a couple more
"problematic" locales we know about.

> On the Org side, we may probably use ascii-case-table. However, this
> will break legitimate user edits involving other locales.
> Alternatively, Org may wrap every regexp search into `with-case-table',
> but that will cause performance degradation, AFAIU.
> 
> I feel that the problem may need to be addressed somehow on Emacs side.

We don't know how to do that, feel free to suggest ideas.

The basic difficulty here is that low-level search code has no idea
about the purpose of the search.  Only the Lisp program and its direct
subroutines know.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

>> Yes (except that Isearch has some heuristics regarding letter-case
>> when invoked interactively).  After set-language-environment to
>> Turkish, type "PROPERTIES", go to the beginning of the word you typed,
>> and do
>
> AFAIU, it then means that any parser/font-lock keywords may fail to
> fontify case-insensitive keywords that contain "i" in Turkish language
> environment.

To demonstrate the problem, let me show a small ECM with
outline-minor-mode:

1. emacs -Q
2. M-x outline-minor-mode <RET>
3. Insert

this
asd
THIS
asd

4. M-: (setq outline-regexp "^this")
5. M-x outline-cycle-buffer <RET>
6. Observe

this...
THIS...

7. M-x outline-show-all <RET>
8. M-x set-language-environment <RET> Turkish <RET>
9. M-x outline-cycle-buffer <RET>
10. Observe

this...

"THIS" is hidden, which is unexpected.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> Cc: senkijun@gmail.com, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 12:03:05 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > AFAIU, it then means that any parser/font-lock keywords may fail to
> > fontify case-insensitive keywords that contain "i" in Turkish language
> > environment.
> 
> In general, yes.  And not only in Turkish, there are a couple more
> "problematic" locales we know about.

Btw, where the pattern includes only upper-case letters, a possibly
cheaper solution is to bind case-fold-search to nil instead.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> Btw, where the pattern includes only upper-case letters, a possibly
> cheaper solution is to bind case-fold-search to nil instead.

Sure, but it is not possible, unfortunately. Org uses case-insensitive
keywords.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Btw, where the pattern includes only upper-case letters, a possibly
>> cheaper solution is to bind case-fold-search to nil instead.
>
> Sure, but it is not possible, unfortunately. Org uses case-insensitive
> keywords.

Hmm... One way could be modifying regexps to something like
[Pp][Rr][Oo][Pp][Ee][Rr][Tt][Ii][Ee][Ss]. May Emacs provide something
like (regexp-case-quote REGEXP &optional CASE-TABLE)?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: senkijun@gmail.com, ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 09:21:50 +0000
> 
> Ihor Radchenko <yantar92@posteo.net> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >> Btw, where the pattern includes only upper-case letters, a possibly
> >> cheaper solution is to bind case-fold-search to nil instead.
> >
> > Sure, but it is not possible, unfortunately. Org uses case-insensitive
> > keywords.
> 
> Hmm... One way could be modifying regexps to something like
> [Pp][Rr][Oo][Pp][Ee][Rr][Tt][Ii][Ee][Ss]. May Emacs provide something
> like (regexp-case-quote REGEXP &optional CASE-TABLE)?

What would regexp-case-quote do with REGEXP?

Regarding [Pp][Rr] etc.: does this have any significant effect on
performance, due to use of character classes?

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Hmm... One way could be modifying regexps to something like
>> [Pp][Rr][Oo][Pp][Ee][Rr][Tt][Ii][Ee][Ss]. May Emacs provide something
>> like (regexp-case-quote REGEXP &optional CASE-TABLE)?
>
> What would regexp-case-quote do with REGEXP?
>
> Regarding [Pp][Rr] etc.: does this have any significant effect on
> performance, due to use of character classes?

Not sure.
CCing Mattias Engdegård.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Mattias Engdegård]

16 juli 2023 kl. 12.25 skrev Ihor Radchenko <yantar92@posteo.net>:

> Eli Zaretskii <eliz@gnu.org> writes:
> 
>>> Hmm... One way could be modifying regexps to something like
>>> [Pp][Rr][Oo][Pp][Ee][Rr][Tt][Ii][Ee][Ss]. May Emacs provide something
>>> like (regexp-case-quote REGEXP &optional CASE-TABLE)?
>> 
>> What would regexp-case-quote do with REGEXP?
>> 
>> Regarding [Pp][Rr] etc.: does this have any significant effect on
>> performance, due to use of character classes?

Yes, that would be substantially slower. The regexp engine combines sequences of literal characters into literal strings that are matched atomically, without either the overhead of individual NFA ops or pushing backtracking points on the NFA stack for each character.

Indeed a quick micro-benchmark bears that out.

How much that affects performance overall is a different matter and can only be answered by measuring.

In general we should make an effort to stop the user's locale leaking into elisp code logic. Ideally, case-folding should be property of a regexp or part of it, not controlled by a user-customisable global variable.

I'm just waiting for the horde of irate Turkish Pascal programmers to complain about broken font-lock.

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Sun, 16 Jul 2023 13:07:01 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  senkijun@gmail.com,
>  ruijie@netyu.xyz,
>  64652@debbugs.gnu.org
> 
> In general we should make an effort to stop the user's locale leaking into elisp code logic. Ideally, case-folding should be property of a regexp or part of it, not controlled by a user-customisable global variable.

That would be equivalent of wrapping some/many of the regexp searches
with with-case-table.  While I agree that it would be cleaner in the
code, the performance concerns Ihor expressed will still be there.

Btw, achieving what you want should be relatively easy: just make the
case-table the property of the regexp, and teach compile_pattern and
friends to look at that, in preference to the buffer's case-table,
when deciding which case-table to use.

> I'm just waiting for the horde of irate Turkish Pascal programmers to complain about broken font-lock.

That should have happened about 20 years ago.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> I'm just waiting for the horde of irate Turkish Pascal programmers to complain about broken font-lock.
>
> That should have happened about 20 years ago.

Or nobody bothers setting locale environment for Emacs.
As we saw in this report, the default locale captured by Emacs from
Windows only changed in Windows 11.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Mattias Engdegård <mattias.engdegard@gmail.com>,
>  senkijun@gmail.com,
>  ruijie@netyu.xyz, 64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 12:11:39 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I'm just waiting for the horde of irate Turkish Pascal programmers to complain about broken font-lock.
> >
> > That should have happened about 20 years ago.
> 
> Or nobody bothers setting locale environment for Emacs.

That's not possible: Emacs sets the default locale-environment (which
includes language-environment) during startup, using the system's
locale.  Emacs on Windows does the same, just using different system
APIs.

> As we saw in this report, the default locale captured by Emacs from
> Windows only changed in Windows 11.

I'm talking about users of Emacs on Posix machines.  I believe the
Turkish locale existed on GNU/Linux for a long time.

bug#64652: 28.2; Adding a property bug on Windows 11

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> As we saw in this report, the default locale captured by Emacs from
>> Windows only changed in Windows 11.
>
> I'm talking about users of Emacs on Posix machines.  I believe the
> Turkish locale existed on GNU/Linux for a long time.

Then, I am curious how Turkish locale did not cause a huge influx of bug
reports. Of course, this question is rather orthogonal to resolving this
bug report.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64652: 28.2; Adding a property bug on Windows 11

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: mattias.engdegard@gmail.com, senkijun@gmail.com, ruijie@netyu.xyz,
>  64652@debbugs.gnu.org
> Date: Sun, 16 Jul 2023 13:06:48 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> As we saw in this report, the default locale captured by Emacs from
> >> Windows only changed in Windows 11.
> >
> > I'm talking about users of Emacs on Posix machines.  I believe the
> > Turkish locale existed on GNU/Linux for a long time.
> 
> Then, I am curious how Turkish locale did not cause a huge influx of bug
> reports.

We did have bug reports about this, see bug#44604 as one example, and
https://lists.gnu.org/archive/html/emacs-devel/2007-03/msg01760.html
as another.  It isn't a flood, but the problem is not new, and people
did report it in the past.

bug#64652: 28.2; Adding a property bug on Windows 11

[Senki Jun]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Thank you so much for all your time and efforts improving emacs guys. You
deserve the best of all things.


On Sun, Jul 16, 2023 at 4:39 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Ihor Radchenko <yantar92@posteo.net>
> > Cc: mattias.engdegard@gmail.com, senkijun@gmail.com, ruijie@netyu.xyz,
> >  64652@debbugs.gnu.org
> > Date: Sun, 16 Jul 2023 13:06:48 +0000
> >
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> > >> As we saw in this report, the default locale captured by Emacs from
> > >> Windows only changed in Windows 11.
> > >
> > > I'm talking about users of Emacs on Posix machines.  I believe the
> > > Turkish locale existed on GNU/Linux for a long time.
> >
> > Then, I am curious how Turkish locale did not cause a huge influx of bug
> > reports.
>
> We did have bug reports about this, see bug#44604 as one example, and
> https://lists.gnu.org/archive/html/emacs-devel/2007-03/msg01760.html
> as another.  It isn't a flood, but the problem is not new, and people
> did report it in the past.
>

[-- Attachment #2: Type: text/html, Size: 1919 bytes --]