bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

[Mike Kupfer]

[-- Attachment #1: Type: text/plain, Size: 1379 bytes --]

From the top level directory of the git repository, running "emacs -Q":

1. C-c C-f etc/HELLO RET

2. M-x ediff-revision RET HELLO RET 0df73dabc16 RET 8721e87a6ec RET

Ediff says there are 6 diffs.  When I move through the diffs (e.g., by
typing SPC in the Ediff control buffer), the first 4 claim that a line
has been added, but that line already exists in the old revision (see
attached screenshot).  These are the lines for Batak, Efik, Hanifi
Rohingya, and Rejang.

Furthermore, ediff does not flag lines that actually are new: Adlam,
Coptic, Gothic, Mende Kikakui, and Wancho.  Nor does it catch the change
to Ukrainian.

Here's what I get after saving the 2 Ediff buffers to files and then
comparing with diff:

26a27
> Adlam (𞤀𞤣𞤤𞤢𞤥)	𞤅𞤢𞤤𞤢𞥄𞤥
42a44
> Coptic (ⲘⲉⲧⲢⲉⲙ̀ⲛⲭⲏⲙⲓ)	Ⲛⲟⲩϥⲣⲓ
58a61
> Gothic (𐌲𐌿𐍄𐌹𐍃𐌺𐌰)	𐌷𐌰𐌹𐌻𐍃 / 𐌷𐌰𐌹𐌻𐌰
86a90
> Mende Kikakui (𞠀𞠁𞠂)	𞠛𞠉
114c118
< Ukrainian (українська)	Вітаю
---
> Ukrainian (українська)	Вітаю / Добрий день! / Привіт
116c120
< 
---
> Wancho (𞋒𞋀𞋉𞋃𞋕)    	𞋂𞋈𞋛


The revision hash below includes some changes to MH-E and to the
function move-file-to-trash.  It is based on 8721e87a6ec.

regards,
mike


[-- Attachment #2: ediff buffers --]
[-- Type: image/png, Size: 115682 bytes --]

[-- Attachment #3: Type: text/plain, Size: 5086 bytes --]



In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw scroll bars) of 2022-11-06 built on alto
Repository revision: f749a1dd7875a83e6415d8c512c5659f0f23c834
Repository branch: master-mdk
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --prefix=/usr/new'

Configured features:
CAIRO FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX
LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LC_TIME: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  delete-selection-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  column-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(ediff ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
ediff-util thai-util thai-word lao-util enriched disp-table mh-search
shr-color color textsec uni-scripts idna-mapping ucs-normalize
uni-confusable textsec-check qp mm-archive mh-alias skeleton vc-filewise
vc-rcs log-view pcvs-util sgml-mode facemenu mail-extr info ispell
smerge-mode diff vc-git whitespace diff-mode easy-mmode pp shadow
mh-mime mh-identity mh-letter mh-show goto-addr gnus-cite emacsbug
tabify man display-line-numbers misearch multi-isearch shortdoc cl-print
help-fns radix-tree mule-util apropos image-file image-converter
image-dired image-dired-tags image-dired-external image-dired-util
dired-aux dired-x add-log org-element avl-tree generator cl-extra ol-eww
eww xdg url-queue thingatpt mm-url ol-rmail ol-mhe ol-irc ol-info
ol-gnus nnselect ol-docview doc-view filenotify jka-compr image-mode
exif ol-bibtex ol-bbdb ol-w3m ol-doi org-link-doi org ob ob-tangle
ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp
ob-core ob-eval org-table oc-basic bibtex ol rx org-keys oc org-compat
org-macs org-loaddefs format-spec cal-menu calendar cal-loaddefs
mh-thread mh-inc hl-line mh-tool-bar mh-acros mh-seq mh-xface mh-utils
mh-folder which-func imenu edebug debug backtrace help-mode find-func
files-x mdk-mail gnus-mh gnus-msg mh-comp mh-scan mh-gnus gnus-dup nnmh
gnus-score score-mode gnus-art mm-uu mml2015 mm-view mml-smime smime
gnutls dig gnus-sum shr pixel-fill kinsoku url-file svg dom browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map gv url-vars gnus-group gnus-undo
gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7
nnoo parse-time iso8601 gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range wid-edit mh-e mh-buffers mh-loaddefs message sendmail
mailcap yank-media puny dired dired-loaddefs rfc822 mml mml-sec
password-cache epa derived epg rfc6068 epg-config gnus-util
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev
mail-utils gmm-utils mailheader server noutline outline icons cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs byte-opt bytecomp byte-compile shell subr-x pcomplete comint
ansi-osc ansi-color ring xcscope advice delsel vc vc-dispatcher
timeclock cl-loaddefs cl-lib mdk-hacks rmc iso-transl tooltip cconv
eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type
elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads inotify dynamic-setting
system-font-setting font-render-setting cairo x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 605040 20381)
 (symbols 48 29639 15)
 (strings 32 132848 7954)
 (string-bytes 1 3807070)
 (vectors 16 82943)
 (vector-slots 8 1126283 27241)
 (floats 8 484 302)
 (intervals 56 19275 840)
 (buffers 984 48))

bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

[Eli Zaretskii]

> From: Mike Kupfer <mkupfer@alum.berkeley.edu>
> Date: Sun, 06 Nov 2022 21:29:06 -0800
> 
> From the top level directory of the git repository, running "emacs -Q":
> 
> 1. C-c C-f etc/HELLO RET
> 
> 2. M-x ediff-revision RET HELLO RET 0df73dabc16 RET 8721e87a6ec RET
> 
> Ediff says there are 6 diffs.  When I move through the diffs (e.g., by
> typing SPC in the Ediff control buffer), the first 4 claim that a line
> has been added, but that line already exists in the old revision (see
> attached screenshot).  These are the lines for Batak, Efik, Hanifi
> Rohingya, and Rejang.
> 
> Furthermore, ediff does not flag lines that actually are new: Adlam,
> Coptic, Gothic, Mende Kikakui, and Wancho.  Nor does it catch the change
> to Ukrainian.

I don't think Ediff can cope with portions of the file that disappear
from buffer text.  All the annotations that enriched.el eats up when
decoding the file are visible to Diff, and they get in the way of
counting characters.

Note that the real diffs also have 6 changes, just not where Ediff
shows them.

bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

[Mike Kupfer]

Eli Zaretskii wrote:

> I don't think Ediff can cope with portions of the file that disappear
> from buffer text.  All the annotations that enriched.el eats up when
> decoding the file are visible to Diff, and they get in the way of
> counting characters.

Ah, thanks, I hadn't realized that there were hidden annotations.  I
thought all the magic incantations were in the local variable settings
at the end of the buffer.

Are the charset tags (annotations) really necessary, given the setting
of 'coding' in the local variables?  (If the answer is "yes" and there
is documentation available that would explain why, feel free to simply
point me at the documentation.  If the answer is "yes" and there's no
documentation, feel free to just say "yes".  I understand that this area
of Emacs can be complicated, and I don't really need to know the
details.)  

The reason I ask is that the files that I saved last night from the old
and new Ediff buffers can be reloaded, and Ediff is able to correctly
show the changes.  Yet I don't see any visual distinctions between the
new file and etc/HELLO, even though the new file's mode is just
Fundamental, not Fundamental Enriched.  (Even the orange text on lines 1
and 34 is duplicated in the saved file, which I confess puzzles me.)

mike

bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

[Eli Zaretskii]

> From: Mike Kupfer <mkupfer@alum.berkeley.edu>
> cc: 59094@debbugs.gnu.org
> Date: Mon, 07 Nov 2022 07:49:36 -0800
> 
> Are the charset tags (annotations) really necessary, given the setting
> of 'coding' in the local variables?

Yes, they help Emacs choose suitable fonts where the same characters
should use different fonts depending on the character set.

> The reason I ask is that the files that I saved last night from the old
> and new Ediff buffers can be reloaded, and Ediff is able to correctly
> show the changes.  Yet I don't see any visual distinctions between the
> new file and etc/HELLO, even though the new file's mode is just
> Fundamental, not Fundamental Enriched.  (Even the orange text on lines 1
> and 34 is duplicated in the saved file, which I confess puzzles me.)

The effect of the charsets is really relevant to CJK cultures, and
even that is nowadays less important than 20 years ago.

bug#59094: 29.0.50; ediff-revision flags wrong lines in etc/HELLO

[Mike Kupfer]

Eli Zaretskii wrote:

> > Are the charset tags (annotations) really necessary, given the setting
> > of 'coding' in the local variables?
> 
> Yes, they help Emacs choose suitable fonts where the same characters
> should use different fonts depending on the character set.
[...]
> The effect of the charsets is really relevant to CJK cultures, and
> even that is nowadays less important than 20 years ago.

Got it.  Thanks.

So it looks like there's nothing to do here.  Closing.

mike