bug#23574: 24.5; Overzealous underlining in emacs-nox

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Colin Woodbury]

[-- Attachment #1: Type: text/plain, Size: 6107 bytes --]

I use ensime for Scala editing, and I've come across a strange display bug
 which is only present in `emacs -nw`, `emacsclient -nw` or just `emacs`
from
 the `emacs-nox` package. It has to do with some very jarring underlining
 that occurs during ensime's "semantic highlighting". The issue does not
 occur in normal GUI emacs.

 I've contacted the ensime maintainers, and while they can reproduce the
 bug, they claim it isn't an ensime problem. Hence I'm here.

 The ensime bug report (with pictures and instructions to reproduce) can be
found here:
 https://github.com/ensime/ensime-emacs/issues/440

 Thanks for any help you can give.

 In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu)
  of 2016-05-01 on svetlemodry
 System Description:     Arch Linux

 Configured using:
  `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
  --localstatedir=/var --without-x --without-sound 'CFLAGS=-march=x86-64
  -mtune=generic -O2 -pipe -fstack-protector-strong'
  CPPFLAGS=-D_FORTIFY_SOURCE=2
  LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

 Important settings:
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Scala

 Minor modes in effect:
   yas-minor-mode: t
   company-mode: t
   diff-auto-refine-mode: t
   ensime-mode: t
   helm-mode: t
   shell-dirtrack-mode: t
   async-bytecomp-package-mode: t
   electric-pair-mode: t
   tooltip-mode: t
   electric-indent-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   column-number-mode: t
   line-number-mode: t
   transient-mark-mode: t

 Recent messages:
 Wrote /home/colin/*message*-20160518-095148
 C-c k is undefined
 C-c C-n is undefined
 Mark saved where search started
 Quit
 ENSIME server starting...
 Connecting to Swank on port 38005..
 Connected to ENSIME speaking protocol 0.8.20, please wait while the
project is loaded.
 Initializing Analyzer. Please wait...
 ENSIME ready. Colin, this could be the start of a beautiful program.

 Load-path shadows:
 /home/colin/.emacs.d/elpa/helm-20160517.202/helm-multi-match hides
/home/colin/.emacs.d/elpa/helm-core-2\
 0160516.2343/helm-multi-match

 Features:
 (network-stream starttls tls ido vc-git ensime-company yasnippet company
 pcase scala-mode scala-mode-prettify-symbols scala-mode-imenu
 scala-mode-map scala-mode-fontlock scala-mode-indent
 scala-mode-paragraph scala-mode-lib image-file misearch multi-isearch
 shadow sort mail-extr emacsbug message idna rfc822 mml mml-sec mm-decode
 mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
 sendmail rfc2047 rfc2045 ietf-drums mail-utils winner helm-command
 helm-elisp helm-eval edebug eldoc help-mode org-clock diary-lib
 diary-loaddefs cal-iso org-element org-rmail org-mhe org-irc org-info
 org-gnus org-docview doc-view jka-compr image-mode image org-bibtex
 bibtex org-bbdb org-w3m org-agenda org org-macro org-footnote
 org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp
 ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
 ob-core ob-eval org-compat org-macs org-loaddefs find-func cal-menu
 calendar cal-loaddefs server ensime ensime-sbt sbt-mode sbt-mode-rgrep
 grep sbt-mode-comint sbt-mode-buffer sbt-mode-project sbt-mode-vars
 ensime-http ensime-ui ensime-semantic-highlight ensime-doc ensime-search
 ensime-undo ensime-startup ensime-refactor diff-mode ensime-popup
 ensime-notes ensime-model ensime-mode ensime-inspector imenu
 ensime-goto-testfile ensime-editor popup ensime-debug gdb-mi bindat gud
 ensime-stacktrace ensime-inf ensime-completion-util scala-mode-syntax
 ensime-config ensime-util ensime-client ensime-vars s ucs-normalize
 hideshow arc-mode archive-mode dash url-gw ensime-macros cl
 haskell-interactive-mode haskell-presentation-mode haskell-collapse
 haskell-process haskell-session json haskell-navigate-imports
 haskell-compile haskell-mode haskell-cabal haskell-utils
 haskell-font-lock haskell-indentation haskell-string
 haskell-sort-imports haskell-lexeme haskell-align-imports haskell-compat
 haskell-complete-module noutline outline flymake etags dabbrev
 haskell-customize helm-mode helm-files rx image-dired tramp tramp-compat
 tramp-loaddefs trampver shell pcomplete format-spec dired-x dired-aux
 ffap thingatpt helm-buffers helm-elscreen helm-tags helm-bookmark
 helm-adaptive helm-info bookmark pp helm-locate helm-grep helm-regexp
 helm-plugin helm-external helm-net browse-url xml url url-proxy
 url-privacy url-expand url-methods url-history url-cookie url-domsuf
 url-util url-parse auth-source gnus-util time-date mm-util mail-prsvr
 password-cache url-vars mailcap helm-utils compile comint regexp-opt
 ansi-color ring helm-help helm-types helm easy-mmode cl-macs gv
 helm-source eieio byte-opt bytecomp byte-compile cl-extra cconv
 eieio-core helm-multi-match helm-lib dired helm-config helm-easymenu
 edmacro kmacro async-bytecomp advice help-fns async cl-loaddefs cl-lib
 elec-pair info tool-bar easymenu package epg-config tooltip electric
 uniquify ediff-hook vc-hooks lisp-float-type tabulated-list newcomment
 lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
 mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
 utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
 japanese hebrew greek romanian slovak czech european ethiopic indian
 cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
 minibuffer nadvice loaddefs button faces cus-face macroexp files
 text-properties overlay sha1 md5 base64 format env code-pages mule
 custom widget hashtable-print-readable backquote make-network-process
 dbusbind gfilenotify multi-tty emacs)

Memory information:
 ((conses 16 493230 27709)
  (symbols 48 45671 8)
  (miscs 40 936 1876)
  (strings 32 102177 15065)
  (string-bytes 1 3052221)
  (vectors 16 60425)
  (vector-slots 8 931107 9519)
  (floats 8 234 1600)
  (intervals 56 2997 0)
  (buffers 960 37)
  (heap 1024 48683 1909))

-- 
Colin Woodbury
Geotrellis Team @ Azavea

[-- Attachment #2: Type: text/html, Size: 6880 bytes --]

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Colin Woodbury]

[-- Attachment #1: Type: text/plain, Size: 6631 bytes --]

Confirmed that this is definitely strictly a problem with emacs in the
terminal. Has anyone had a chance to look at this?

On Wed, May 18, 2016 at 10:03 AM, Colin Woodbury <cwoodbury@azavea.com>
wrote:

> I use ensime for Scala editing, and I've come across a strange display bug
>  which is only present in `emacs -nw`, `emacsclient -nw` or just `emacs`
> from
>  the `emacs-nox` package. It has to do with some very jarring underlining
>  that occurs during ensime's "semantic highlighting". The issue does not
>  occur in normal GUI emacs.
>
>  I've contacted the ensime maintainers, and while they can reproduce the
>  bug, they claim it isn't an ensime problem. Hence I'm here.
>
>  The ensime bug report (with pictures and instructions to reproduce) can
> be found here:
>  https://github.com/ensime/ensime-emacs/issues/440
>
>  Thanks for any help you can give.
>
>  In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu)
>   of 2016-05-01 on svetlemodry
>  System Description:     Arch Linux
>
>  Configured using:
>   `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
>   --localstatedir=/var --without-x --without-sound 'CFLAGS=-march=x86-64
>   -mtune=generic -O2 -pipe -fstack-protector-strong'
>   CPPFLAGS=-D_FORTIFY_SOURCE=2
>   LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'
>
>  Important settings:
>    value of $LANG: en_US.UTF-8
>    locale-coding-system: utf-8-unix
>
> Major mode: Scala
>
>  Minor modes in effect:
>    yas-minor-mode: t
>    company-mode: t
>    diff-auto-refine-mode: t
>    ensime-mode: t
>    helm-mode: t
>    shell-dirtrack-mode: t
>    async-bytecomp-package-mode: t
>    electric-pair-mode: t
>    tooltip-mode: t
>    electric-indent-mode: t
>    menu-bar-mode: t
>    file-name-shadow-mode: t
>    global-font-lock-mode: t
>    font-lock-mode: t
>    auto-composition-mode: t
>    auto-encryption-mode: t
>    auto-compression-mode: t
>    column-number-mode: t
>    line-number-mode: t
>    transient-mark-mode: t
>
>  Recent messages:
>  Wrote /home/colin/*message*-20160518-095148
>  C-c k is undefined
>  C-c C-n is undefined
>  Mark saved where search started
>  Quit
>  ENSIME server starting...
>  Connecting to Swank on port 38005..
>  Connected to ENSIME speaking protocol 0.8.20, please wait while the
> project is loaded.
>  Initializing Analyzer. Please wait...
>  ENSIME ready. Colin, this could be the start of a beautiful program.
>
>  Load-path shadows:
>  /home/colin/.emacs.d/elpa/helm-20160517.202/helm-multi-match hides
> /home/colin/.emacs.d/elpa/helm-core-2\
>  0160516.2343/helm-multi-match
>
>  Features:
>  (network-stream starttls tls ido vc-git ensime-company yasnippet company
>  pcase scala-mode scala-mode-prettify-symbols scala-mode-imenu
>  scala-mode-map scala-mode-fontlock scala-mode-indent
>  scala-mode-paragraph scala-mode-lib image-file misearch multi-isearch
>  shadow sort mail-extr emacsbug message idna rfc822 mml mml-sec mm-decode
>  mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
>  sendmail rfc2047 rfc2045 ietf-drums mail-utils winner helm-command
>  helm-elisp helm-eval edebug eldoc help-mode org-clock diary-lib
>  diary-loaddefs cal-iso org-element org-rmail org-mhe org-irc org-info
>  org-gnus org-docview doc-view jka-compr image-mode image org-bibtex
>  bibtex org-bbdb org-w3m org-agenda org org-macro org-footnote
>  org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp
>  ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
>  ob-core ob-eval org-compat org-macs org-loaddefs find-func cal-menu
>  calendar cal-loaddefs server ensime ensime-sbt sbt-mode sbt-mode-rgrep
>  grep sbt-mode-comint sbt-mode-buffer sbt-mode-project sbt-mode-vars
>  ensime-http ensime-ui ensime-semantic-highlight ensime-doc ensime-search
>  ensime-undo ensime-startup ensime-refactor diff-mode ensime-popup
>  ensime-notes ensime-model ensime-mode ensime-inspector imenu
>  ensime-goto-testfile ensime-editor popup ensime-debug gdb-mi bindat gud
>  ensime-stacktrace ensime-inf ensime-completion-util scala-mode-syntax
>  ensime-config ensime-util ensime-client ensime-vars s ucs-normalize
>  hideshow arc-mode archive-mode dash url-gw ensime-macros cl
>  haskell-interactive-mode haskell-presentation-mode haskell-collapse
>  haskell-process haskell-session json haskell-navigate-imports
>  haskell-compile haskell-mode haskell-cabal haskell-utils
>  haskell-font-lock haskell-indentation haskell-string
>  haskell-sort-imports haskell-lexeme haskell-align-imports haskell-compat
>  haskell-complete-module noutline outline flymake etags dabbrev
>  haskell-customize helm-mode helm-files rx image-dired tramp tramp-compat
>  tramp-loaddefs trampver shell pcomplete format-spec dired-x dired-aux
>  ffap thingatpt helm-buffers helm-elscreen helm-tags helm-bookmark
>  helm-adaptive helm-info bookmark pp helm-locate helm-grep helm-regexp
>  helm-plugin helm-external helm-net browse-url xml url url-proxy
>  url-privacy url-expand url-methods url-history url-cookie url-domsuf
>  url-util url-parse auth-source gnus-util time-date mm-util mail-prsvr
>  password-cache url-vars mailcap helm-utils compile comint regexp-opt
>  ansi-color ring helm-help helm-types helm easy-mmode cl-macs gv
>  helm-source eieio byte-opt bytecomp byte-compile cl-extra cconv
>  eieio-core helm-multi-match helm-lib dired helm-config helm-easymenu
>  edmacro kmacro async-bytecomp advice help-fns async cl-loaddefs cl-lib
>  elec-pair info tool-bar easymenu package epg-config tooltip electric
>  uniquify ediff-hook vc-hooks lisp-float-type tabulated-list newcomment
>  lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
>  mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
>  utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
>  japanese hebrew greek romanian slovak czech european ethiopic indian
>  cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
>  minibuffer nadvice loaddefs button faces cus-face macroexp files
>  text-properties overlay sha1 md5 base64 format env code-pages mule
>  custom widget hashtable-print-readable backquote make-network-process
>  dbusbind gfilenotify multi-tty emacs)
>
> Memory information:
>  ((conses 16 493230 27709)
>   (symbols 48 45671 8)
>   (miscs 40 936 1876)
>   (strings 32 102177 15065)
>   (string-bytes 1 3052221)
>   (vectors 16 60425)
>   (vector-slots 8 931107 9519)
>   (floats 8 234 1600)
>   (intervals 56 2997 0)
>   (buffers 960 37)
>   (heap 1024 48683 1909))
>
> --
> Colin Woodbury
> Geotrellis Team @ Azavea
>



-- 
Colin Woodbury
Geotrellis Team @ Azavea

[-- Attachment #2: Type: text/html, Size: 7606 bytes --]

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Mon, 30 May 2016 08:04:16 -0700
> From: Colin Woodbury <cwoodbury@azavea.com>
> 
> Confirmed that this is definitely strictly a problem with emacs in the terminal. Has anyone had a chance to
> look at this?

Given what I read in the original bug report you pointed to, I'm not
convinced it's an Emacs bug (or even that there is a bug at all).
There's an opinion there at the end that this is "strictly an Emacs
issue", but the example shown is the intended behavior: Emacs always
gives the empty screen space after the line end the same face as the
last character on the line.  You can easily see that by
shift-selecting text that spans more than one line.

Thanks.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Colin Woodbury <cwoodbury@azavea.com>
> Date: Sat, 4 Jun 2016 08:54:27 -0700
> 
> Thanks for the response, Eli. If it's not a bug in terminal emacs, then why do GUI emacs and terminal emacs
> display different things? Note from the images that the empty space in GUI is _not_ given the same
> overlay/face as the text.
> 
> Thoughts?

If you show me some Lisp which produces these different effects in X
and non-X sessions, I could try looking for the reason.  All I've seen
is a screenshot, from which I deduced (perhaps incorrectly) what was
done to produce it.

Thanks.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[John Mastro]

Eli Zaretskii <eliz@gnu.org> wrote:
>> Thanks for the response, Eli. If it's not a bug in terminal emacs,
>> then why do GUI emacs and terminal emacs display different things?
>> Note from the images that the empty space in GUI is _not_ given the
>> same overlay/face as the text.
>
> If you show me some Lisp which produces these different effects in X
> and non-X sessions, I could try looking for the reason.  All I've seen
> is a screenshot, from which I deduced (perhaps incorrectly) what was
> done to produce it.

Here's an example shows the effect Colin is seeing:

    (let (beg end ov)
      (defface example-underline-face
        '((t :underline t))
        "Example face with underlining")
      (goto-char (point-max))
      (newline)
      (setq beg (point))
      (insert "    foo\n    bar\n")
      (setq end (point))
      (setq ov (make-overlay beg end))
      (overlay-put ov 'face 'example-underline-face))

The result is the same with text properties instead of an overlay:

    (progn
      (defface example-underline-face
        '((t :underline t))
        "Example face with underlining")
      (goto-char (point-max))
      (newline)
      (insert (propertize "    foo\n    bar\n"
                          'font-lock-face
                          'example-underline-face)))

In a graphical frame, the underline only extends one character past the
visible text (this one extra character presumably being the newline).
However, in a text frame the underline extends all the way to the end of
the window.

        John

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: John Mastro <john.b.mastro@gmail.com>
> Date: Sat, 4 Jun 2016 14:37:28 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, Colin Woodbury <cwoodbury@azavea.com>
> 
>     (progn
>       (defface example-underline-face
>         '((t :underline t))
>         "Example face with underlining")
>       (goto-char (point-max))
>       (newline)
>       (insert (propertize "    foo\n    bar\n"
>                           'font-lock-face
>                           'example-underline-face)))
> 
> In a graphical frame, the underline only extends one character past the
> visible text (this one extra character presumably being the newline).
> However, in a text frame the underline extends all the way to the end of
> the window.

OK, then my guess was correct after all, and what you see is how Emacs
behaved since v21 at least.  If there is a problem here, it's in GUI
frames, not in TTY frames.  We always try to make the empty space
after the end of a screen line have the same face as the last
character of that line.  With background color, this works in both TTY
and GUI frames, but we cannot do that with underlining without
actually drawing something in that empty space.  While it should be
possible to have GUI frames display underline all the way to window
edge, no one has ever requested that, so we didn't bother.

In sum, this is the intended behavior, and if the application doesn't
like it, it should refrain from underlining more than one line.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Noam Postavsky]

On Sun, Jun 5, 2016 at 11:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> We always try to make the empty space
> after the end of a screen line have the same face as the last
> character of that line.

Just to clarify, "last character of that line" refers to the newline
character or the one before it?

> While it should be
> possible to have GUI frames display underline all the way to window
> edge, no one has ever requested that, so we didn't bother.

I think this would have been useful for magit to simplify the use of
overlays to display the region with horizontal lines. In
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21468#43 you suggested a
way using :align-to which turned out to have a bunch of complications
and magit ended up not using it (see
https://github.com/magit/magit/pull/2293).

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Sun, 5 Jun 2016 13:05:53 -0400
> Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org, cwoodbury@azavea.com
> 
> On Sun, Jun 5, 2016 at 11:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > We always try to make the empty space
> > after the end of a screen line have the same face as the last
> > character of that line.
> 
> Just to clarify, "last character of that line" refers to the newline
> character or the one before it?

The last displayed character of the line.  The newline is not
displayed, in the sense that it has no glyph, so it can have no face.

> > While it should be
> > possible to have GUI frames display underline all the way to window
> > edge, no one has ever requested that, so we didn't bother.
> 
> I think this would have been useful for magit to simplify the use of
> overlays to display the region with horizontal lines.

This is doable (and in fact we already do that in R2L paragraphs,
which you can observe if you change the recipe's text to use R2L
characters).  But note that the OP in this bug report wants the exact
opposite: to NOT have the underlining extended on TTYs.  So clearly
there's no "one size fits all" solution here.

> In http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21468#43 you
> suggested a way using :align-to which turned out to have a bunch of
> complications and magit ended up not using it (see
> https://github.com/magit/magit/pull/2293).

Did you try using the box attribute of a face?

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Colin Woodbury]

[-- Attachment #1: Type: text/plain, Size: 2001 bytes --]

Interesting that you consider the the GUI _not_ displaying the face all the
way to end of screen to be a bug. I think only displaying to the end of the
characters (as shown in images in the Github issue) is the expected
behaviour. Otherwise, the screen (at least with underlining) gets quite
noisy.

On Sun, Jun 5, 2016 at 10:56 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Noam Postavsky <npostavs@users.sourceforge.net>
> > Date: Sun, 5 Jun 2016 13:05:53 -0400
> > Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org,
> cwoodbury@azavea.com
> >
> > On Sun, Jun 5, 2016 at 11:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > > We always try to make the empty space
> > > after the end of a screen line have the same face as the last
> > > character of that line.
> >
> > Just to clarify, "last character of that line" refers to the newline
> > character or the one before it?
>
> The last displayed character of the line.  The newline is not
> displayed, in the sense that it has no glyph, so it can have no face.
>
> > > While it should be
> > > possible to have GUI frames display underline all the way to window
> > > edge, no one has ever requested that, so we didn't bother.
> >
> > I think this would have been useful for magit to simplify the use of
> > overlays to display the region with horizontal lines.
>
> This is doable (and in fact we already do that in R2L paragraphs,
> which you can observe if you change the recipe's text to use R2L
> characters).  But note that the OP in this bug report wants the exact
> opposite: to NOT have the underlining extended on TTYs.  So clearly
> there's no "one size fits all" solution here.
>
> > In http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21468#43 you
> > suggested a way using :align-to which turned out to have a bunch of
> > complications and magit ended up not using it (see
> > https://github.com/magit/magit/pull/2293).
>
> Did you try using the box attribute of a face?
>



-- 
Colin Woodbury
Geotrellis Team @ Azavea

[-- Attachment #2: Type: text/html, Size: 3128 bytes --]

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Colin Woodbury <cwoodbury@azavea.com>
> Date: Sun, 5 Jun 2016 11:20:24 -0700
> Cc: Noam Postavsky <npostavs@users.sourceforge.net>, john.b.mastro@gmail.com, 
> 	23574@debbugs.gnu.org
> 
> Interesting that you consider the the GUI _not_ displaying the face all the way to end of screen to be a bug.

It's not a bug, it's the intended behavior.  On GUI frames, the
display engine only makes sure to extend the face if it has one or
more of the following attributes:

  . background color
  . stipple
  . box

> I think only displaying to the end of the characters (as shown in images in the Github issue) is the expected
> behaviour. Otherwise, the screen (at least with underlining) gets quite noisy.

Why would you have underlining span several lines if you don't want
that effect?  (I admit I didn't really understand the original use
case which prompted this report.)

Anyway, you've just heard of another application which would like the
GUI frames behave like TTYs, not the other way around.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Noam Postavsky]

On Sun, Jun 5, 2016 at 1:56 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Noam Postavsky <npostavs@users.sourceforge.net>
>> Date: Sun, 5 Jun 2016 13:05:53 -0400
>> Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org, cwoodbury@azavea.com
>>
>> On Sun, Jun 5, 2016 at 11:54 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> > We always try to make the empty space
>> > after the end of a screen line have the same face as the last
>> > character of that line.
>>
>> Just to clarify, "last character of that line" refers to the newline
>> character or the one before it?
>
> The last displayed character of the line.  The newline is not
> displayed, in the sense that it has no glyph, so it can have no face.

That doesn't seem to be the case, with the following modification to
the recipe so that the newline characters are not underlined, the
underlining does not continue to the edge of the screen:

(let (beg end ov)
  (defface example-underline-face
    '((t :underline t))
    "Example face with underlining")
  (goto-char (point-max))
  (newline)
  (setq beg (point))
  (insert "    foo")
  (setq end (point))
  (setq ov (make-overlay beg end))
  (overlay-put ov 'face 'example-underline-face)
  (insert "\n")
  (setq beg (point))
  (insert "    bar")
  (setq end (point))
  (setq ov (make-overlay beg end))
  (overlay-put ov 'face 'example-underline-face)
  (insert "\n"))


>
>> > While it should be
>> > possible to have GUI frames display underline all the way to window
>> > edge, no one has ever requested that, so we didn't bother.
>>
>> I think this would have been useful for magit to simplify the use of
>> overlays to display the region with horizontal lines.
>
> This is doable (and in fact we already do that in R2L paragraphs,
> which you can observe if you change the recipe's text to use R2L
> characters).

Yes (though it seems R2L is disabled in programming modes by default,
needed to use fundamental-mode buffer to see the effect).

>> In http://debbugs.gnu.org/cgi/bugreport.cgi?bug=21468#43 you
>> suggested a way using :align-to which turned out to have a bunch of
>> complications and magit ended up not using it (see
>> https://github.com/magit/magit/pull/2293).
>
> Did you try using the box attribute of a face?

Yes, but magit actually wants a multiline box (without internal
lines), so it didn't really work out (anyway, this is getting
off-topic for this bug).

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Sun, 5 Jun 2016 15:13:00 -0400
> Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org, cwoodbury@azavea.com
> 
> >> Just to clarify, "last character of that line" refers to the newline
> >> character or the one before it?
> >
> > The last displayed character of the line.  The newline is not
> > displayed, in the sense that it has no glyph, so it can have no face.
> 
> That doesn't seem to be the case, with the following modification to
> the recipe so that the newline characters are not underlined, the
> underlining does not continue to the edge of the screen:

When the newline does not have the underline attribute, the underline
is not contiguous, so you are radically changing the use case.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Noam Postavsky]

tag 23574 + notabug
quit

On Sun, Jun 5, 2016 at 10:27 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> That doesn't seem to be the case, with the following modification to
>> the recipe so that the newline characters are not underlined, the
>> underlining does not continue to the edge of the screen:
>
> When the newline does not have the underline attribute, the underline
> is not contiguous, so you are radically changing the use case.

Yes. However, I believe that this is what the original ensime code
intended to do; it only underlines the newlines themselves because
it's easier to make 1 overlay for all the lines at once and the
programmer didn't notice it was wrong because it happens to give the
desired effect in GUI mode.

Regardless, by experimentation I find that the space at the edge of
the screen takes the face from the final newline, not the last
displayed glyph character in the line. Is this documented anywhere?

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Mon, 6 Jun 2016 07:42:37 -0400
> Cc: John Mastro <john.b.mastro@gmail.com>, 23574@debbugs.gnu.org, cwoodbury@azavea.com
> 
> > When the newline does not have the underline attribute, the underline
> > is not contiguous, so you are radically changing the use case.
> 
> Yes. However, I believe that this is what the original ensime code
> intended to do; it only underlines the newlines themselves because
> it's easier to make 1 overlay for all the lines at once and the
> programmer didn't notice it was wrong because it happens to give the
> desired effect in GUI mode.

I don't see how the effect on GUI frames could be considered
"desired".  What about the fact that the underline extends one space
beyond the line's text?  So on GUI frames we see the same problem,
just of smaller dimensions.  Or am I missing something?

> Regardless, by experimentation I find that the space at the edge of
> the screen takes the face from the final newline, not the last
> displayed glyph character in the line. Is this documented anywhere?

That space at the edge of the line is not a space at all, although it
looks like one.  It is a special glyph produced by the display engine,
primarily so that we could display the cursor at the end of the line.
Its attributes are invented by the display engine out of thin air for
its own purposes; for example, the buffer position recorded in that
glyph is zero, not the position of the newline.

As for your conclusion, I believe there's a misunderstanding here.
You are talking about the face of a buffer position, while I was
talking about the glyphs on the screen.  Other that this minor
disconnect, I don't see any contradiction between what we say.

Also note that the display engine doesn't examine each character's
face when it produces glyphs for display, it only examines the faces
where they change.  Which in this case means that the face of the
newline is immaterial; what matters is whether it is identical to that
of preceding characters.  To be precise, the face used for extension
is the one loaded into the iterator object when it hits the end of the
line.  As there are too many ways to specify faces in Emacs, I won't
risk confirming that your conclusion is 100% correct ;-)  My
description is correct, but perhaps less useful to a Lisp programmer.

As for documentation: these all are fine details of the display engine
that are not documented anywhere except in the code comments.  Even
the face extension itself isn't mentioned anywhere, I believe simply
because the effect is natural and expected, whereas its accurate
documentation is not simple at all.  Does it really make sense to
document just this specific subtlety?

IOW, if you are interested in these details, you should be hacking the
display engine code long ago ;-)

Going back to the bug report, there's still one issue to consider:
should we add underline (and then also overline and strikethrough) to
the list of face attributes that cause face extension on GUI frames.
The logic behind the current code seems to be to extend attributes
that are related to background of the text.  The above 3 seem to be a
kind-of background, so maybe we should add them.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 > Going back to the bug report, there's still one issue to consider:
 > should we add underline (and then also overline and strikethrough) to
 > the list of face attributes that cause face extension on GUI frames.
 > The logic behind the current code seems to be to extend attributes
 > that are related to background of the text.  The above 3 seem to be a
 > kind-of background, so maybe we should add them.

It would make my life much easier if face extension were, in general,
customizable.  I would immediately turn it off everywhere.

My motivation is that I have font-lock distinguish things like comments
and strings mainly by their background face.  Since these look awful
when extending to the end of a window, I have to provide my own
‘font-lock-fontify-syntactically-region’ function which assures that
newline characters within strings and comments never get the
corresponding face.  This consumes resources and, for example, disallows
using text properties to skip the rest of a comment or a string.  It
goes without saying, that my version of this function is never in synch
with the one of the repository.

I also use separate background colors for editable fields, buttons,
links and the like which also look awful when spanning two or more
lines.  No font-locking can help me here.

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Colin Woodbury]

[-- Attachment #1: Type: text/plain, Size: 2577 bytes --]

> Yes. However, I believe that this is what the original ensime code
> intended to do; it only underlines the newlines themselves because
> it's easier to make 1 overlay for all the lines at once and the
> programmer didn't notice it was wrong because it happens to give the
> desired effect in GUI mode.

I think that's the case. The purpose of the underlining is to show where
"Scala implicits" are occuring. An implicit is some code brought into scope
through imports, often used for silent conversion to inject methods into
preexisting datatypes. Essentially that with "traits" are how Scala hacks
Haskell typeclasses.

If your statement that uses an implicit happens to span multiple lines,
ensime (I believe) just finds the starting and ending points of the
statement and applies the face to the entire area. In the GUI, this happens
to produce the desired effect of only underlining where characters are
(with the newline as well, as mentioned previously). In TTY this places the
face over everything, which we don't agree is a bug or not.

On Mon, Jun 6, 2016 at 9:54 AM, martin rudalics <rudalics@gmx.at> wrote:

> > Going back to the bug report, there's still one issue to consider:
> > should we add underline (and then also overline and strikethrough) to
> > the list of face attributes that cause face extension on GUI frames.
> > The logic behind the current code seems to be to extend attributes
> > that are related to background of the text.  The above 3 seem to be a
> > kind-of background, so maybe we should add them.
>
> It would make my life much easier if face extension were, in general,
> customizable.  I would immediately turn it off everywhere.
>
> My motivation is that I have font-lock distinguish things like comments
> and strings mainly by their background face.  Since these look awful
> when extending to the end of a window, I have to provide my own
> ‘font-lock-fontify-syntactically-region’ function which assures that
> newline characters within strings and comments never get the
> corresponding face.  This consumes resources and, for example, disallows
> using text properties to skip the rest of a comment or a string.  It
> goes without saying, that my version of this function is never in synch
> with the one of the repository.
>
> I also use separate background colors for editable fields, buttons,
> links and the like which also look awful when spanning two or more
> lines.  No font-locking can help me here.
>
> martin
>
>


-- 
Colin Woodbury
Geotrellis Team @ Azavea

[-- Attachment #2: Type: text/html, Size: 3241 bytes --]

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Mon, 06 Jun 2016 18:54:38 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: 23574@debbugs.gnu.org, john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
>  > Going back to the bug report, there's still one issue to consider:
>  > should we add underline (and then also overline and strikethrough) to
>  > the list of face attributes that cause face extension on GUI frames.
>  > The logic behind the current code seems to be to extend attributes
>  > that are related to background of the text.  The above 3 seem to be a
>  > kind-of background, so maybe we should add them.
> 
> It would make my life much easier if face extension were, in general,
> customizable.  I would immediately turn it off everywhere.

Can you suggest a defcustom for that (what type? which values mean
what? etc)?  Then we can talk how hard it is to implement that.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Colin Woodbury <cwoodbury@azavea.com>
> Date: Mon, 6 Jun 2016 11:25:34 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, Noam Postavsky <npostavs@users.sourceforge.net>, 23574@debbugs.gnu.org, 
> 	John Mastro <john.b.mastro@gmail.com>
> 
> If your statement that uses an implicit happens to span multiple lines, ensime (I believe) just finds the starting
> and ending points of the statement and applies the face to the entire area. In the GUI, this happens to produce
> the desired effect of only underlining where characters are (with the newline as well, as mentioned
> previously).

Once again, there _is_ no newline.  It is not displayed.  What you see
is an empty character cell produced for displaying the cursor.  It has
no direct relation to the newline.

And I don't think what you get is the desired effect, you just get a
side effect of a particular implementation detail.  E.g., what happens
if a line fits exactly on a line, i.e. the cursor at its end will be
displayed on the fringe?

> In TTY this places the face over everything, which we don't agree is a bug or not.

It isn't a bug, because that's how the display engine was coded to
work.  Of course, we can make it behave differently if we want.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Noam Postavsky]

On Mon, Jun 6, 2016 at 3:18 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> And I don't think what you get is the desired effect, you just get a
> side effect of a particular implementation detail.  E.g., what happens
> if a line fits exactly on a line, i.e. the cursor at its end will be
> displayed on the fringe?

The underlining does reach the end of the screen in this case, but it
still looks okay because the underlining doesn't extend too far away
from the text.

>
>> In TTY this places the face over everything, which we don't agree is a bug or not.
>
> It isn't a bug, because that's how the display engine was coded to
> work.  Of course, we can make it behave differently if we want.

I think it makes sense to not do the face extension by default. If I
take a sheet of paper and underline lines 3 to 10, I'm going to stop
drawing at the end of the text, not go all the way to end of the
paper.

For the case I mentioned earlier, magit isn't actually underlining
text, it just wants to make some horizontal lines. It would be nice to
have some way to ask the display engine to do this directly.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 > Can you suggest a defcustom for that (what type? which values mean
 > what? etc)?  Then we can talk how hard it is to implement that.

(defcustom face-extend-to-window-edge nil
   "Non-nil means extend face of last character on line to window edge.
This variable specifies whether the space following the last
character on a line is decorated using properties of that
character.  nil means do not extend any properties of that
character.  t means extend all properties of that character.

A property list means to determine the value of any such property
from that list. If a property is not on that list, that property
is not extended.  The special value 'inherit' means to inherit
the value for that property from the last character on the line.
A function specifies the function to return the value of that
property.  Any other value means to extend that as value for that
property.")

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Tue, 07 Jun 2016 11:10:30 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
> (defcustom face-extend-to-window-edge nil
>    "Non-nil means extend face of last character on line to window edge.
> This variable specifies whether the space following the last
> character on a line is decorated using properties of that
> character.  nil means do not extend any properties of that
> character.  t means extend all properties of that character.
> 
> A property list means to determine the value of any such property
> from that list. If a property is not on that list, that property
> is not extended.  The special value 'inherit' means to inherit
> the value for that property from the last character on the line.
> A function specifies the function to return the value of that
> property.  Any other value means to extend that as value for that
> property.")

Talk about over-specification ;-)

I hope you realize how some of that will make redisplay much more
expensive?

How about the following more modest alternative?

(defcustom face-extend-to-window-edge t
   "Non-nil means extend face of last character on line to window edge.

Certain face attributes, if present in the face of the last character
of a line and different from those of the default face, cause the
empty space following the end of text on the line to be drawn with
those attributes, to give the empty space appearance similar to that
of the preceding text.  These attributes are those which affect the
background of a face: `:background', `:stipple', `:box', `:underline',
`:overline', and `:strike-through'.  By default, if the face of a
line's last character has any of these attributes, and the value is
different from that of the default face, the empty space following the
line's text will be drawn in the face of the last character.

This variable allows fine-tuning which attributes trigger the face
extension.  The default value of t means any of the mentioned
attributes will cause face extension.  The value of nil means face
extension is turned off.  A value that is a list of attributes will
extend the face only if any of the attributes from the list are
present in the last character's face.  Note that only attributes from
the above list are meaningful in list values of this variable.")

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Mon, 6 Jun 2016 20:18:06 -0400
> Cc: Colin Woodbury <cwoodbury@azavea.com>, martin rudalics <rudalics@gmx.at>, 23574@debbugs.gnu.org, 
> 	John Mastro <john.b.mastro@gmail.com>
> 
> I think it makes sense to not do the face extension by default.

Not even for the background color?  Or are you talking only about the
underline?

> For the case I mentioned earlier, magit isn't actually underlining
> text, it just wants to make some horizontal lines. It would be nice to
> have some way to ask the display engine to do this directly.

There's a way of doing that which we use in the VC commit log buffer,
I believe it uses the line-height property.  (I thought magit was
using that as well, no?)

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Noam Postavsky]

On Tue, Jun 7, 2016 at 11:55 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Noam Postavsky <npostavs@users.sourceforge.net>
>> Date: Mon, 6 Jun 2016 20:18:06 -0400
>> Cc: Colin Woodbury <cwoodbury@azavea.com>, martin rudalics <rudalics@gmx.at>, 23574@debbugs.gnu.org,
>>       John Mastro <john.b.mastro@gmail.com>
>>
>> I think it makes sense to not do the face extension by default.
>
> Not even for the background color?  Or are you talking only about the
> underline?

Well, I was thinking mostly about underlining, but imagining a
scenario with paper again, if I'm highlighting text with a marker, I
wouldn't go to edge of the page there either.

For some applications, the intention is to colour a whole block, not
just the text. So I think it's better to let the code making the faces
have a way to indicate which scenario is intended rather than relying
on user customization.

>
>> For the case I mentioned earlier, magit isn't actually underlining
>> text, it just wants to make some horizontal lines. It would be nice to
>> have some way to ask the display engine to do this directly.
>
> There's a way of doing that which we use in the VC commit log buffer,
> I believe it uses the line-height property.  (I thought magit was
> using that as well, no?)

Yes, and it caused some problems with cursor movement that you weren't
so happy about fixing.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 > Talk about over-specification ;-)
 >
 > I hope you realize how some of that will make redisplay much more
 > expensive?

Yes.  But the only value I really care about is nil ;-)

 > How about the following more modest alternative?
 >
 > (defcustom face-extend-to-window-edge t
 >     "Non-nil means extend face of last character on line to window edge.
 >
 > Certain face attributes, if present in the face of the last character
 > of a line and different from those of the default face, cause the
 > empty space following the end of text on the line to be drawn with
 > those attributes, to give the empty space appearance similar to that
 > of the preceding text.  These attributes are those which affect the
 > background of a face: `:background', `:stipple', `:box', `:underline',
 > `:overline', and `:strike-through'.  By default, if the face of a
 > line's last character has any of these attributes, and the value is
 > different from that of the default face, the empty space following the
 > line's text will be drawn in the face of the last character.
 >
 > This variable allows fine-tuning which attributes trigger the face
 > extension.  The default value of t means any of the mentioned
 > attributes will cause face extension.  The value of nil means face
 > extension is turned off.  A value that is a list of attributes will
 > extend the face only if any of the attributes from the list are
 > present in the last character's face.  Note that only attributes from
 > the above list are meaningful in list values of this variable.")

This has the advantage of a much better doc-string.  The only details
missing are whether the last character of a line is the newline
character, whether a non-printable character's attributes count, how
invisible characters are treated and whether the :display attribute has
any impact.  Since I have no good idea about all of these I deliberately
did not try to cover them.

I'm not sure whether your proposal (it obviously was my first idea as
well) is less expensive, though.  If the value of our variable is a
list, the display engine has to go through the properties of the last
character and compare them against the members of this list.  Or go
through all members of the list and compare them against the character's
properties - neither of these approaches is cheap even if optimizations
are applied.

And then I thought about the - possibly silly idea - that a user might
want to put a property like :background on all lines displayed,
regardless of the last character's attributes.  Such a user would have
to, before displaying the relevant buffer parts, go through all these
lines and add that property to the newline (?) character of each line.
Now I bet that the greater part of such users would put the property on
all newline characters of the buffer instead of using something like
‘pre-redisplay-functions’ where even I couldn't tell whether it gets the
window's start and end positions always right.

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Wed, 08 Jun 2016 08:33:30 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
> This has the advantage of a much better doc-string.  The only details
> missing are whether the last character of a line is the newline
> character, whether a non-printable character's attributes count, how
> invisible characters are treated and whether the :display attribute has
> any impact.  Since I have no good idea about all of these I deliberately
> did not try to cover them.

I do know about these, but I don't thunk we should document all those
details, since there are too many possible variations, and it's too
easy to be inaccurate.  "the last face used on the line" is good
enough, I think.

> I'm not sure whether your proposal (it obviously was my first idea as
> well) is less expensive, though.  If the value of our variable is a
> list, the display engine has to go through the properties of the last
> character and compare them against the members of this list.  Or go
> through all members of the list and compare them against the character's
> properties - neither of these approaches is cheap even if optimizations
> are applied.

No, it need not do any of that.  The list is fixed for each redisplay
cycle of each buffer, so the list can be processed only once into a
bitmap of flags that tell which face attributes trigger face
extension.  Then all redisplay needs to do is compare the attributes
of the face loaded into the iterator object at end of each line with
these flags.

> And then I thought about the - possibly silly idea - that a user might
> want to put a property like :background on all lines displayed,
> regardless of the last character's attributes.  Such a user would have
> to, before displaying the relevant buffer parts, go through all these
> lines and add that property to the newline (?) character of each line.
> Now I bet that the greater part of such users would put the property on
> all newline characters of the buffer instead of using something like
> ‘pre-redisplay-functions’ where even I couldn't tell whether it gets the
> window's start and end positions always right.

Hmm.. not sure how this is related.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 > I do know about these, but I don't thunk we should document all those
 > details, since there are too many possible variations, and it's too
 > easy to be inaccurate.  "the last face used on the line" is good
 > enough, I think.

Including the quotation marks?

 > No, it need not do any of that.  The list is fixed for each redisplay
 > cycle of each buffer, so the list can be processed only once into a
 > bitmap of flags that tell which face attributes trigger face
 > extension.  Then all redisplay needs to do is compare the attributes
 > of the face loaded into the iterator object at end of each line with
 > these flags.

OK.  But with my property list approach a once calculated bitmap would
have simply overridden the face of the iterator object.  Yet cheaper but
less versatile.

 >> And then I thought about the - possibly silly idea - that a user might
 >> want to put a property like :background on all lines displayed,
 >> regardless of the last character's attributes.  Such a user would have
 >> to, before displaying the relevant buffer parts, go through all these
 >> lines and add that property to the newline (?) character of each line.
 >> Now I bet that the greater part of such users would put the property on
 >> all newline characters of the buffer instead of using something like
 >> ‘pre-redisplay-functions’ where even I couldn't tell whether it gets the
 >> window's start and end positions always right.
 >
 > Hmm.. not sure how this is related.

Suppose a user wants to use the same background for all spaces at the
ends of all lines of a buffer regardless of "the last face used on the
line".  How would she specify that?

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Thu, 09 Jun 2016 10:38:48 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
>  > No, it need not do any of that.  The list is fixed for each redisplay
>  > cycle of each buffer, so the list can be processed only once into a
>  > bitmap of flags that tell which face attributes trigger face
>  > extension.  Then all redisplay needs to do is compare the attributes
>  > of the face loaded into the iterator object at end of each line with
>  > these flags.
> 
> OK.  But with my property list approach a once calculated bitmap would
> have simply overridden the face of the iterator object.

Your suggestion included a function, which could well return different
values on each call.

And creating new faces from arbitrary sets of attributes is no fun,
either.

> Suppose a user wants to use the same background for all spaces at the
> ends of all lines of a buffer regardless of "the last face used on the
> line".  How would she specify that?

By putting the proper face property on the newline.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 >> OK.  But with my property list approach a once calculated bitmap would
 >> have simply overridden the face of the iterator object.
 >
 > Your suggestion included a function, which could well return different
 > values on each call.
 >
 > And creating new faces from arbitrary sets of attributes is no fun,
 > either.

OK.  I was not fond of my proposal anyway.

 >> Suppose a user wants to use the same background for all spaces at the
 >> ends of all lines of a buffer regardless of "the last face used on the
 >> line".  How would she specify that?
 >
 > By putting the proper face property on the newline.

Which gets me back to my initial concern: If our user does that eagerly
for the entire buffer, the overhead might be non-negligible.  A more
lazy solution would require to hook into ‘pre-redisplay-functions’ or
something the like in order to know which lines get displayed.  Would
‘pre-redisplay-functions’ be a suitable place for that?

martin


A loosely related question: Does for R2L text row->pixel_width for each
glyph row indicate the width of that row occupied by text as it does for
L2R text?  Suppose we have a L2R line with dots indicating the empty
space at the end of that line:

TTTTTTTTTTTTT....

R2L this line would appear as

....TTTTTTTTTTTTT

Would these two lines have the same row->pixel_width?  Or, would the
length of the stretch glyph added at the left of the R2L line be that of
‘window-text-width’ minus row->pixel_width?

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Fri, 10 Jun 2016 09:16:02 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
>  >> Suppose a user wants to use the same background for all spaces at the
>  >> ends of all lines of a buffer regardless of "the last face used on the
>  >> line".  How would she specify that?
>  >
>  > By putting the proper face property on the newline.
> 
> Which gets me back to my initial concern: If our user does that eagerly
> for the entire buffer, the overhead might be non-negligible.

I don't see why.  Redisplay only considers the visible portion of the
buffer.

> A loosely related question: Does for R2L text row->pixel_width for each
> glyph row indicate the width of that row occupied by text as it does for
> L2R text?

row->pixel_width doesn't count text glyphs, it counts all of the
glyphs in a glyph row, including the glyphs produced by the display
engine for its own purposes.  E.g., it always includes the space glyph
produced at the end of a line, which is needed for displaying the
cursor.  In the R2L case the width includes the stretch glyph
prepended on the left.

> Suppose we have a L2R line with dots indicating the empty space at
> the end of that line:
> 
> TTTTTTTTTTTTT....
> 
> R2L this line would appear as
> 
> ....TTTTTTTTTTTTT
> 
> Would these two lines have the same row->pixel_width?  Or, would the
> length of the stretch glyph added at the left of the R2L line be that of
> ‘window-text-width’ minus row->pixel_width?

The latter.  row->pixel_width is computed in compute_line_metrics,
after the stretch glyph (and any other glyphs needed for line display)
were already inserted.  compute_line_metrics doesn't care about what
glyphs are there, it counts them all.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 >> Which gets me back to my initial concern: If our user does that eagerly
 >> for the entire buffer, the overhead might be non-negligible.
 >
 > I don't see why.  Redisplay only considers the visible portion of the
 > buffer.

I meant the overhead for adding the text property to every newline
character in the buffer.

 > row->pixel_width doesn't count text glyphs, it counts all of the
 > glyphs in a glyph row, including the glyphs produced by the display
 > engine for its own purposes.  E.g., it always includes the space glyph
 > produced at the end of a line, which is needed for displaying the
 > cursor.

Are there any other significant objects but that space glyph?  Is there
any other way to get the size of the empty space after text on each row?

 > row->pixel_width is computed in compute_line_metrics,
 > after the stretch glyph (and any other glyphs needed for line display)
 > were already inserted.  compute_line_metrics doesn't care about what
 > glyphs are there, it counts them all.

Hmm... How would I get the width of that stretch glyph then?

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Fri, 10 Jun 2016 10:24:30 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
>  >> Which gets me back to my initial concern: If our user does that eagerly
>  >> for the entire buffer, the overhead might be non-negligible.
>  >
>  > I don't see why.  Redisplay only considers the visible portion of the
>  > buffer.
> 
> I meant the overhead for adding the text property to every newline
> character in the buffer.

You mean, memory overhead?  I don't think it's significant.

>  > row->pixel_width doesn't count text glyphs, it counts all of the
>  > glyphs in a glyph row, including the glyphs produced by the display
>  > engine for its own purposes.  E.g., it always includes the space glyph
>  > produced at the end of a line, which is needed for displaying the
>  > cursor.
> 
> Are there any other significant objects but that space glyph?

Yes, a few.  Look at the comments at the beginning of 'struct glyph'
definition in dispextern.h.

> Is there any other way to get the size of the empty space after text
> on each row?

"Other way"? other than what?

>  > row->pixel_width is computed in compute_line_metrics,
>  > after the stretch glyph (and any other glyphs needed for line display)
>  > were already inserted.  compute_line_metrics doesn't care about what
>  > glyphs are there, it counts them all.
> 
> Hmm... How would I get the width of that stretch glyph then?

It's recorded in the glyph's pixel_width.  Or maybe I don't understand
the problem you are trying to solve.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[martin rudalics]

 >> I meant the overhead for adding the text property to every newline
 >> character in the buffer.
 >
 > You mean, memory overhead?  I don't think it's significant.

I meant the time overhead to find each newline (or the character before
it) in the buffer and put the property on it.

 > Yes, a few.  Look at the comments at the beginning of 'struct glyph'
 > definition in dispextern.h.

These ones

       glyph standing for newline at end of line    0
       empty space after the end of the line       -1
       overlay arrow on a TTY                      -1
       ...

?

 >> Is there any other way to get the size of the empty space after text
 >> on each row?
 >
 > "Other way"? other than what?

Other than subtracting the pixel_width from the window text width.  I
obviously want to just retrieve a calculated value, not recalculate it.

 >>   > row->pixel_width is computed in compute_line_metrics,
 >>   > after the stretch glyph (and any other glyphs needed for line display)
 >>   > were already inserted.  compute_line_metrics doesn't care about what
 >>   > glyphs are there, it counts them all.
 >>
 >> Hmm... How would I get the width of that stretch glyph then?
 >
 > It's recorded in the glyph's pixel_width.

So for R2L text to get the width of the empty space on the left of a row
I have to calculate the pixel_width of the leftmost character on that
row.  Something like the following for a given row?

   struct glyph *glyph = row->glyphs[TEXT_AREA];
   width = glyph->pixel_width;

 > Or maybe I don't understand
 > the problem you are trying to solve.

I want to put a mini-frame in the empty space on the right side of a L2R
window and accordingly on the left side of a R2L window.  I more or less
know how to do the former but have not yet tried the latter.

martin

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Eli Zaretskii]

> Date: Fri, 10 Jun 2016 15:59:41 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: npostavs@users.sourceforge.net, 23574@debbugs.gnu.org, 
>  john.b.mastro@gmail.com, cwoodbury@azavea.com
> 
> I meant the time overhead to find each newline (or the character before
> it) in the buffer and put the property on it.

If they decide to do this, they get what they deserve, no?

>  > Yes, a few.  Look at the comments at the beginning of 'struct glyph'
>  > definition in dispextern.h.
> 
> These ones
> 
>        glyph standing for newline at end of line    0
>        empty space after the end of the line       -1
>        overlay arrow on a TTY                      -1
>        ...
> 
> ?

Yes.

>  >> Is there any other way to get the size of the empty space after text
>  >> on each row?
>  >
>  > "Other way"? other than what?
> 
> Other than subtracting the pixel_width from the window text width.  I
> obviously want to just retrieve a calculated value, not recalculate it.

I guess that's the natural way, yes.  Of course, you can only use it
when the display is up to date, otherwise the glyph matrix cannot be
trusted, and you need the move_it_* functions instead.

>  >>   > row->pixel_width is computed in compute_line_metrics,
>  >>   > after the stretch glyph (and any other glyphs needed for line display)
>  >>   > were already inserted.  compute_line_metrics doesn't care about what
>  >>   > glyphs are there, it counts them all.
>  >>
>  >> Hmm... How would I get the width of that stretch glyph then?
>  >
>  > It's recorded in the glyph's pixel_width.
> 
> So for R2L text to get the width of the empty space on the left of a row
> I have to calculate the pixel_width of the leftmost character on that
> row.  Something like the following for a given row?
> 
>    struct glyph *glyph = row->glyphs[TEXT_AREA];
>    width = glyph->pixel_width;

Yes, modulo the usual caveats of hscroll etc.

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Ergus via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi:

In the new master branch there is a new feature (:extend attribute for
faces) that seems to fix also this issue. Could you try it to confirm if
we can close this as fixed??

Best,
Ergus

bug#23574: 24.5; Overzealous underlining in emacs-nox

[Lars Ingebrigtsen]

Ergus <spacibba@aol.com> writes:

> In the new master branch there is a new feature (:extend attribute for
> faces) that seems to fix also this issue. Could you try it to confirm if
> we can close this as fixed??

This was two years ago, and there was no response.  From skimming this
thread, I think the :extend attribute should have fixed the issue.  So
I'm closing this bug report.  If there's more to be done, please respond
to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no