Any objection to adding a unicode footnote style?

Any objection to adding a unicode footnote style?

[Leo]

[-- Attachment #1: Type: text/plain, Size: 70 bytes --]

Any objection to adding a new style `unicode' to footnote-mode?

Leo


[-- Attachment #2: 0001-Add-unicode-footnote-style.patch --]
[-- Type: text/x-diff, Size: 2866 bytes --]

From ff273b1cd36f9357d531dbb490008d13a2043bd9 Mon Sep 17 00:00:00 2001
Date: Thu, 5 May 2011 17:10:02 +0800
Subject: [PATCH] Add unicode footnote style

---
 lisp/mail/footnote.el |   27 +++++++++++++++++++++++----
 1 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/lisp/mail/footnote.el b/lisp/mail/footnote.el
index 82928642..fdeeb483 100644
--- a/lisp/mail/footnote.el
+++ b/lisp/mail/footnote.el
@@ -1,4 +1,4 @@
-;;; footnote.el --- footnote support for message mode  -*- coding: iso-latin-1;-*-
+;;; footnote.el --- footnote support for message mode  -*- coding: utf-8;-*-
 
 ;; Copyright (C) 1997, 2000, 2001, 2002, 2003, 2004,
 ;;   2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software Foundation, Inc.
@@ -279,7 +279,7 @@ (defun Footnote-roman-common (n footnote-roman-list)
 
 ;; Latin-1
 
-(defconst footnote-latin-string "ｹｲｳｺｪｧｶ"
+(defconst footnote-latin-string "ﾂｹﾂｲﾂｳﾂｺﾂｪﾂｧﾂｶ"
   "String of Latin-1 footnoting characters.")
 
 ;; Note not [...]+, because this style cycles.
@@ -292,6 +292,23 @@ (defun Footnote-latin (n)
   (string (aref footnote-latin-string
 		(mod (1- n) (length footnote-latin-string)))))
 
+;; Unicode
+
+(defconst footnote-unicode-string "竅ｰﾂｹﾂｲﾂｳ竅ｴ竅ｵ竅ｶ竅ｷ竅ｸ竅ｹ"
+  "String of unicode footnoting characters.")
+
+(defconst footnote-unicode-regexp (concat "[" footnote-unicode-string "]+")
+  "Regexp for unicode footnoting characters.")
+
+(defun Footnote-unicode (n)
+  (let (modulus result done)
+    (while (not done)
+      (setq modulus (mod n 10)
+            n (truncate n 10))
+      (and (zerop n) (setq done t))
+      (push (aref footnote-unicode-string modulus) result))
+    (apply #'string result)))
+
 ;;; list of all footnote styles
 (defvar footnote-style-alist
   `((numeric Footnote-numeric ,footnote-numeric-regexp)
@@ -299,7 +316,8 @@ (defvar footnote-style-alist
     (english-upper Footnote-english-upper ,footnote-english-upper-regexp)
     (roman-lower Footnote-roman-lower ,footnote-roman-lower-regexp)
     (roman-upper Footnote-roman-upper ,footnote-roman-upper-regexp)
-    (latin Footnote-latin ,footnote-latin-regexp))
+    (latin Footnote-latin ,footnote-latin-regexp)
+    (unicode Footnote-unicode ,footnote-unicode-regexp))
   "Styles of footnote tags available.
 By default only boring Arabic numbers, English letters and Roman Numerals
 are available.
@@ -313,7 +331,8 @@ (defcustom footnote-style 'numeric
 english-upper == A, B, C, ...
 roman-lower == i, ii, iii, iv, v, ...
 roman-upper == I, II, III, IV, V, ...
-latin == ｹ ｲ ｳ ｺ ｪ ｧ ｶ
+latin == ﾂｹ ﾂｲ ﾂｳ ﾂｺ ﾂｪ ﾂｧ ﾂｶ
+unicode == ﾂｹ, ﾂｲ, ﾂｳ, ...
 See also variables `footnote-start-tag' and `footnote-end-tag'.
 
 Customizing this variable has no effect on buffers already
-- 
1.7.5-rc2

Re: Any objection to adding a unicode footnote style?

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Sun, 08 May 2011 18:11:47 +0800
> 
> Any objection to adding a new style `unicode' to footnote-mode?

Maybe that's a good idea, but your attachment is illegible, precisely
in those lines which matter: it shows funny Japanese characters
instead of what you probably meant, and makes it very hard to judge
the patch.  For example, this character:

	  character: ﾂ (65410, #o177602, #xff82)
  preferred charset: unicode (Unicode (ISO10646))
	 code point: 0xFF82
	     syntax: w 	which means: word
	   category: .:Base, j:Japanese, k:Katakana, |:line breakable
	buffer code: #xEF #xBE #x82
	  file code: not encodable by coding system undecided
	    display: by this font (glyph code)
      uniscribe:-outline-Arial Unicode MS-normal-normal-normal-sans-15-*-*-*-p-*-jisx0208*-* (#x1A5C)

  Character code properties: customize what to show
    name: HALFWIDTH KATAKANA LETTER TU
    general-category: Lo (Letter, Other)
    decomposition: (narrow 12484) (narrow 'ツ')

Re: Any objection to adding a unicode footnote style?

[Leo]

On 2011-05-08 22:30 +0800, Eli Zaretskii wrote:
> Maybe that's a good idea, but your attachment is illegible, precisely
> in those lines which matter: it shows funny Japanese characters
> instead of what you probably meant, and makes it very hard to judge
> the patch.  For example, this character:
>
> 	  character: ﾂ (65410, #o177602, #xff82)
>   preferred charset: unicode (Unicode (ISO10646))
> 	 code point: 0xFF82
> 	     syntax: w 	which means: word
> 	   category: .:Base, j:Japanese, k:Katakana, |:line breakable
> 	buffer code: #xEF #xBE #x82
> 	  file code: not encodable by coding system undecided
> 	    display: by this font (glyph code)
>       uniscribe:-outline-Arial Unicode MS-normal-normal-normal-sans-15-*-*-*-p-*-jisx0208*-* (#x1A5C)
>
>   Character code properties: customize what to show
>     name: HALFWIDTH KATAKANA LETTER TU
>     general-category: Lo (Letter, Other)
>     decomposition: (narrow 12484) (narrow 'ツ')

Could you try viewing the patch in utf-8?

Leo

Re: Any objection to adding a unicode footnote style?

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Mon, 09 May 2011 00:43:30 +0800
> 
> Could you try viewing the patch in utf-8?

That's what I did.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Leo]

[-- Attachment #1: Type: text/plain, Size: 244 bytes --]

On 2011-05-09 01:14 +0800, Eli Zaretskii wrote:
[elide 5 lines]
> That's what I did.

Sorry for sending a diff with mixed encodings. It is now split into
two parts:
           1. convert footnote.el to utf-8
           2. diff (attached)

Leo


[-- Attachment #2: 0001-Add-a-new-footnote-style-unicode.patch --]
[-- Type: text/x-diff, Size: 2164 bytes --]

From d889e335e5587e1162cd2b4240c8239fabac4ce5 Mon Sep 17 00:00:00 2001
Date: Mon, 9 May 2011 12:22:13 +0800
Subject: [PATCH] Add a new footnote style `unicode'

---
 lisp/mail/footnote.el |   23 ++++++++++++++++++++++-
 1 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/lisp/mail/footnote.el b/lisp/mail/footnote.el
index c9899ca7..dc967ec8 100644
--- a/lisp/mail/footnote.el
+++ b/lisp/mail/footnote.el
@@ -292,6 +292,25 @@ (defun Footnote-latin (n)
   (string (aref footnote-latin-string
 		(mod (1- n) (length footnote-latin-string)))))
 
+;; Unicode
+
+(defconst footnote-unicode-string "⁰¹²³⁴⁵⁶⁷⁸⁹"
+  "String of unicode footnoting characters.")
+
+(defconst footnote-unicode-regexp (concat "[" footnote-unicode-string "]+")
+  "Regexp for unicode footnoting characters.")
+
+(defun Footnote-unicode (n)
+  "Unicode footnote style.
+Use unicode characters for footnoting."
+  (let (modulus result done)
+    (while (not done)
+      (setq modulus (mod n 10)
+            n (truncate n 10))
+      (and (zerop n) (setq done t))
+      (push (aref footnote-unicode-string modulus) result))
+    (apply #'string result)))
+
 ;;; list of all footnote styles
 (defvar footnote-style-alist
   `((numeric Footnote-numeric ,footnote-numeric-regexp)
@@ -299,7 +318,8 @@ (defvar footnote-style-alist
     (english-upper Footnote-english-upper ,footnote-english-upper-regexp)
     (roman-lower Footnote-roman-lower ,footnote-roman-lower-regexp)
     (roman-upper Footnote-roman-upper ,footnote-roman-upper-regexp)
-    (latin Footnote-latin ,footnote-latin-regexp))
+    (latin Footnote-latin ,footnote-latin-regexp)
+    (unicode Footnote-unicode ,footnote-unicode-regexp))
   "Styles of footnote tags available.
 By default only boring Arabic numbers, English letters and Roman Numerals
 are available.
@@ -314,6 +334,7 @@ (defcustom footnote-style 'numeric
 roman-lower == i, ii, iii, iv, v, ...
 roman-upper == I, II, III, IV, V, ...
 latin == ¹ ² ³ º ª § ¶
+unicode == ¹, ², ³, ...
 See also variables `footnote-start-tag' and `footnote-end-tag'.
 
 Customizing this variable has no effect on buffers already
-- 
1.7.5-rc2

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Mon, 09 May 2011 12:26:55 +0800
> 
> Sorry for sending a diff with mixed encodings. It is now split into
> two parts:
>            1. convert footnote.el to utf-8
>            2. diff (attached)

Thanks, now I can read it.

I don't really mind (as I don't use footnote-mode), but this will look
quite ugly if the default font doesn't have the Unicode superscript
characters.  In that case, Emacs will use a different font to draw
just the Unicode characters, which will result in 1, 2, and 3 drawn a
bit lower than the other digits, or wider, or both.  Depending on the
default font, it could also be that 0, 4, 5, 6, 7, 8, and 9 will be
drawn taller than the others.

Are we willing to bet that the default font is a Unicode font for most
users?  If not, perhaps at least some caveat in the doc string is in
order.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Ted Zlatanov]

On Mon, 09 May 2011 10:06:06 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Leo <sdl.web@gmail.com>
>> Date: Mon, 09 May 2011 12:26:55 +0800
>> 
>> Sorry for sending a diff with mixed encodings. It is now split into
>> two parts:
>> 1. convert footnote.el to utf-8
>> 2. diff (attached)

EZ> Thanks, now I can read it.

EZ> I don't really mind (as I don't use footnote-mode), but this will look
EZ> quite ugly if the default font doesn't have the Unicode superscript
EZ> characters.  In that case, Emacs will use a different font to draw
EZ> just the Unicode characters, which will result in 1, 2, and 3 drawn a
EZ> bit lower than the other digits, or wider, or both.  Depending on the
EZ> default font, it could also be that 0, 4, 5, 6, 7, 8, and 9 will be
EZ> drawn taller than the others.

EZ> Are we willing to bet that the default font is a Unicode font for most
EZ> users?  If not, perhaps at least some caveat in the doc string is in
EZ> order.

Wouldn't it look nicer if the Emacs superscript text properties were
used when the display supports them, and fall back to pure Unicode
characters when they're not available?

Ted

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 09 May 2011 10:39:24 -0500
> 
> Wouldn't it look nicer if the Emacs superscript text properties

What "superscript text properties"?

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Ted Zlatanov]

On Mon, 09 May 2011 19:05:15 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 09 May 2011 10:39:24 -0500
>> 
>> Wouldn't it look nicer if the Emacs superscript text properties

EZ> What "superscript text properties"?

from shr.el, used for HTML superscripts:

  (put-text-property start (point) 'display '(raise 0.5))

Would that work?  It's not my code and I don't know display properties
well but it seems like a sensible way to distinguish footnote references.

Ted

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Alp Aker]

> (put-text-property start (point) 'display '(raise 0.5))
> 
> Would that work? 

You can't test from within Lisp for the availability of that
display property.  The best you can do is check the display type
and infer from that whether it's available.  (E.g.,
(display-graphic-p) => nil implies that it's not.)

But that method's not reliable, since the implementation of the
raise property is inconsistent across ports.  (On NextStep, raise
properties doesn't raise the text; they just increase the overall
line height.)

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 09 May 2011 11:22:47 -0500
> 
> EZ> What "superscript text properties"?
> 
> from shr.el, used for HTML superscripts:
> 
>   (put-text-property start (point) 'display '(raise 0.5))
> 
> Would that work?

Oh, you mean `raise'...  Yes, that would work, although this is even
better:

  (put-text-property start (point) 'display '((height 0.8) (raise 0.5)))

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> From: Alp Aker <aker@pitt.edu>
> Date: Mon, 9 May 2011 16:50:02 +0000 (UTC)
> 
> > (put-text-property start (point) 'display '(raise 0.5))
> > 
> > Would that work? 
> 
> You can't test from within Lisp for the availability of that
> display property.  The best you can do is check the display type
> and infer from that whether it's available.  (E.g.,
> (display-graphic-p) => nil implies that it's not.)

This is supported on any GUI display, do display-graphic-p is up to
the job.

> But that method's not reliable, since the implementation of the
> raise property is inconsistent across ports.  (On NextStep, raise
> properties doesn't raise the text; they just increase the overall
> line height.)

AFAIK, what you see on NextStep is how it works on _any_ GUI display.
The implementation is device-independent.  Making the digit smaller
(with `height', as I show in my other message) alleviates the effect
on the line height to some degree.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Alp Aker]

> AFAIK, what you see on NextStep is how it works on _any_ GUI display.
> The implementation is device-independent.  Making the digit smaller
> (with `height', as I show in my other message) alleviates the effect
> on the line height to some degree.

I don't think you're right here.  A (raise 0.5) display spec on NextStep 
increases the line height, and does nothing else.  It doesn't change the 
vertical position of the character relative to the baseline.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Ted Zlatanov]

On Mon, 09 May 2011 19:55:49 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 09 May 2011 11:22:47 -0500
>> 
EZ> What "superscript text properties"?
>> 
>> from shr.el, used for HTML superscripts:
>> 
>> (put-text-property start (point) 'display '(raise 0.5))
>> 
>> Would that work?

EZ> Oh, you mean `raise'...  Yes, that would work, although this is even
EZ> better:

EZ>   (put-text-property start (point) 'display '((height 0.8) (raise 0.5)))

OK.  Leo, do you want to make this improvement happen when
`display-graphic-p' is true?

Ted

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Alp Aker]

> AFAIK, what you see on NextStep is how it works on _any_ GUI display.
> The implementation is device-independent.  Making the digit smaller
> (with `height', as I show in my other message) alleviates the effect
> on the line height to some degree.

I don't think you're right here.  A (raise 0.5) display spec on NextStep
increases the line height, and does nothing else.  It doesn't change the
vertical position of the character relative to the baseline.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Leo]

On 2011-05-10 01:20 +0800, Ted Zlatanov wrote:
> EZ> Oh, you mean `raise'...  Yes, that would work, although this is even
> EZ> better:
>
> EZ>   (put-text-property start (point) 'display '((height 0.8) (raise 0.5)))
>
> OK.  Leo, do you want to make this improvement happen when
> `display-graphic-p' is true?

This is only for those characters inside the doc-string, right? Because
email composed in emacs with unicode footnotes might be viewed in other
MUAs.

Leo

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> Date: Mon, 09 May 2011 13:06:54 -0400 (EDT)
> From: Alp Aker <aker@pitt.edu>
> Cc: emacs-devel@gnu.org
> 
> > AFAIK, what you see on NextStep is how it works on _any_ GUI display.
> > The implementation is device-independent.  Making the digit smaller
> > (with `height', as I show in my other message) alleviates the effect
> > on the line height to some degree.
> 
> I don't think you're right here.  A (raise 0.5) display spec on NextStep 
> increases the line height, and does nothing else.  It doesn't change the 
> vertical position of the character relative to the baseline.

Maybe it's a bug in NextStep, then.  The implementation is in xdisp.c,
at least most of it.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Ted Zlatanov]

On Tue, 10 May 2011 01:27:54 +0800 Leo <sdl.web@gmail.com> wrote: 

L> On 2011-05-10 01:20 +0800, Ted Zlatanov wrote:
EZ> Oh, you mean `raise'...  Yes, that would work, although this is even
EZ> better:
>> 
EZ> (put-text-property start (point) 'display '((height 0.8) (raise 0.5)))
>> 
>> OK.  Leo, do you want to make this improvement happen when
>> `display-graphic-p' is true?

L> This is only for those characters inside the doc-string, right? Because
L> email composed in emacs with unicode footnotes might be viewed in other
L> MUAs.

Oh, I was only thinking of using this inside Emacs.  Sorry, Unicode
characters are indeed better for your purpose.  Forget my suggestion.

Ted

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Alp Aker]

> > I don't think you're right here.  A (raise 0.5) display spec on NextStep 
> > increases the line height, and does nothing else.  It doesn't change the 
> > vertical position of the character relative to the baseline.
> 
> Maybe it's a bug in NextStep, then.  The implementation is in xdisp.c,
> at least most of it.

On a quick glance, it looks like the problem is with the glyph rendering in 
nsterm.m.  ns_draw_glyphs_foreground doesn't check s->ybase when 
deciding where to position the glyph, so it's essentially just discarding 
some of the information from the `raise' display spec that's processed by 
fill_composite_glyph_string. 

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Alp Aker]

> so it's essentially just discarding some of the information from 
> the `raise' display spec that's processed by fill_composite_glyph_string. 

And mutatis mutandis for non-composite glyphs.

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Leo]

On 2011-05-09 15:06 +0800, Eli Zaretskii wrote:
> Thanks, now I can read it.
>
> I don't really mind (as I don't use footnote-mode), but this will look
> quite ugly if the default font doesn't have the Unicode superscript
> characters.  In that case, Emacs will use a different font to draw
> just the Unicode characters, which will result in 1, 2, and 3 drawn a
> bit lower than the other digits, or wider, or both.  Depending on the
> default font, it could also be that 0, 4, 5, 6, 7, 8, and 9 will be
> drawn taller than the others.
>
> Are we willing to bet that the default font is a Unicode font for most
> users?  If not, perhaps at least some caveat in the doc string is in
> order.

How about adding something like this to the doc-string:

diff --git a/lisp/mail/footnote.el b/lisp/mail/footnote.el
index dc967ec8..1a60a333 100644
--- a/lisp/mail/footnote.el
+++ b/lisp/mail/footnote.el
@@ -337,6 +337,10 @@ (defcustom footnote-style 'numeric
 unicode == ¹, ², ³, ...
 See also variables `footnote-start-tag' and `footnote-end-tag'.
 
+NB: some characters in the latin and unicode style may not be
+shown properly if the default font does not contain those
+characters.
+
 Customizing this variable has no effect on buffers already
 displaying footnotes.  To change the style of footnotes in such a
 buffer use the command `Footnote-set-style'."

Leo

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Eli Zaretskii]

> From: Leo <sdl.web@gmail.com>
> Date: Tue, 10 May 2011 13:09:59 +0800
> 
> > Are we willing to bet that the default font is a Unicode font for most
> > users?  If not, perhaps at least some caveat in the doc string is in
> > order.
> 
> How about adding something like this to the doc-string:
> 
> diff --git a/lisp/mail/footnote.el b/lisp/mail/footnote.el
> index dc967ec8..1a60a333 100644
> --- a/lisp/mail/footnote.el
> +++ b/lisp/mail/footnote.el
> @@ -337,6 +337,10 @@ (defcustom footnote-style 'numeric
>  unicode == ¹, ², ³, ...
>  See also variables `footnote-start-tag' and `footnote-end-tag'.
>  
> +NB: some characters in the latin and unicode style may not be
> +shown properly if the default font does not contain those
> +characters.

Something like that, although I would suggest to mention only Unicode,
as Latin characters can be assumed to be universally available.  Also,
I think we use "Note", not "NB".

Re: Any objection to adding a unicode footnote style? (encoding fixed)

[Leo]

On 2011-05-10 15:31 +0800, Eli Zaretskii wrote:
> Something like that, although I would suggest to mention only Unicode,
> as Latin characters can be assumed to be universally available.  Also,
> I think we use "Note", not "NB".

Thanks for this. I have made the change and committed the patch.

Leo