Pretest next week

Pretest next week

[Chong Yidong]

Hi all,

I'm planning to make the 23.0.94 pretest next Wednesday, the 20th.

The release seems to be coming along nicely; the font code seems to have
converged at last (with the exception of #2667), and most of the
documentation work is now done.  I'm thinking of cutting the branch
after the next pretest, so that some of the pent-up patches can start
being applied.  I have not, however, discussed this in detail with
Stefan yet.  If you like, please feel free to weigh in with opinions
and/or concerns about the release process.

Thanks.

How to show *Warnings* buffer at startup time? [Re: Pretest next week]

[Kenichi Handa]

In article <8763g39jwd.fsf@cyd.mit.edu>, Chong Yidong <cyd@stupidchicken.com> writes:

> Hi all,
> I'm planning to make the 23.0.94 pretest next Wednesday, the 20th.

> The release seems to be coming along nicely; the font code seems to have
> converged at last (with the exception of #2667), and most of the
> documentation work is now done.  I'm thinking of cutting the branch
> after the next pretest, so that some of the pent-up patches can start
> being applied.  I have not, however, discussed this in detail with
> Stefan yet.  If you like, please feel free to weigh in with opinions
> and/or concerns about the release process.

I've just noticed that the subject issure is not yet
settled.  I think this is not a serious one but the attached
patch to startup.el is simple and it helps people identify a
fontset problem earlier.

---
Kenichi Handa
handa@m17n.org


In article <E1Ljviv-0002mh-Jc@etlken>, Kenichi Handa <handa@m17n.org> writes:

> I'm going to change create-fontset-from-x-resource
> (fontset.el) as this:

> @@ -1049,8 +1058,11 @@
>      (while (setq fontset-spec (x-get-resource (format "fontset-%d" idx)
>  					      (format "Fontset-%d" idx)))
>        (condition-case nil
> -	  (create-fontset-from-fontset-spec fontset-spec t 'noerror)
> -	(error (message "Fontset-%d: invalid specification in X resource" idx)))
> +	  (create-fontset-from-fontset-spec fontset-spec t)
> +	(error (display-warning
> +		'initialization
> +		(format "Fontset-%d: invalid specification in X resource" idx)
> +		:warning)))
>        (setq idx (1+ idx)))))
 
>  ;;

> But the *Warnings* buffer doesn't show up at startup time
> even if create-fontset-from-fontset-spec causes an error.
> Could someone tell me what is wrong with the above change?

> If it is because the X frame is not yet created at that
> time, don't we need something like the attached patch?

> ---
> Kenichi Handa
> handa@m17n.org

> --- startup.el.~1.527.~	2009-02-23 09:51:02.000000000 +0900
> +++ startup.el	2009-03-18 22:16:35.000000000 +0900
> @@ -1235,7 +1235,13 @@
>    (if (and (boundp 'x-session-previous-id)
>             (stringp x-session-previous-id))
>        (with-no-warnings
> -	(emacs-session-restore x-session-previous-id))))
> +	(emacs-session-restore x-session-previous-id)))
> +
> +  (let ((warning-buf (get-buffer "*Warnings*")))
> +    (if (and warning-buf
> +	     (not (get-buffer-window warning-buf))
> +	     (> (buffer-size warning-buf) 0))
> +	(display-buffer warning-buf))))
 
>  (defcustom initial-scratch-message (purecopy "\
>  ;; This buffer is for notes you don't want to save, and for Lisp evaluation.

Pretest next week

[Chong Yidong]

Barring unforseen problems, I will make the 24.2.90 pretest next week,
Saturday the 24th.  Please plan your commits accordingly, thanks.

This is a good time to concentrate on bugs that ought to be fixed for
24.3 (mainly, regressions against 23.1+).  If you know of a bug that is
not receiving enough attention, please point it out.

Re: Pretest next week

[Chong Yidong]

> I'm planning to make the 23.0.94 pretest next Wednesday, the 20th.

I'm delaying the pretest for a few days, because of Bug#2667, the
enable-character-translation issue, and a couple of other things.  Sorry
for the inconvenience.

Re: Pretest next week

[Chong Yidong]

The 23.0.94 pretest will be available shortly.  There was a minor
snafu---my internet connection died while I was uploading the tarfile to
ftp-upload.gnu.org, and now I can't continue uploading for some reason.
I've emailed the GNU webmasters for help.

The version number in CVS has already been bumped.

Re: Pretest next week

[Lennart Borgman]

On Sat, May 23, 2009 at 6:45 AM, Chong Yidong <cyd@stupidchicken.com> wrote:
> The 23.0.94 pretest will be available shortly.  There was a minor
> snafu---my internet connection died while I was uploading the tarfile to
> ftp-upload.gnu.org, and now I can't continue uploading for some reason.
> I've emailed the GNU webmasters for help.
>
> The version number in CVS has already been bumped.

If anyone is interested I just happened to upload some unpatched
binaries for w32 to my site just after I saw this. (They are compiled
from the sources in CVS though, which was feteched after pretest
23.0.94 was announced.)

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

 > In the reported case,
 > read_socket_hook is called from the select (ns_select, actually) call
 > in wait_reading_process_output, and a menu bar activation there leads
 > to Lisp evaluation containing Faccept_process_output.

It appears that a call verify-visited-file-modtime happens for each  
active
menu item, which then triggers tramp to go and check on the server.   
This
causes a reentrant call to wait_reading_process_output, thence a  
reentrant
call to ns_select.  I've added a check in the latter to shortcircuit  
in the
reentrant case.  There is still a delay when requesting a menu, but  
this is
probably unavoidable if the visited modtime must be checked for each  
menu
item.

I've looked at your code in the Carbon AppKit port and it seems like  
your
approach to handling menu events there could be applied to the NS  
port.  The
only difference I see in your event handling is processing all event  
types in
one function instead of passing them through NSApp-sendEvent: to go be
distributed through ordinary Cocoa channels to widgets.  But since - 
sendEvent:
is an interception point, the menu events could be taken there.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 5 Mar 2009 19:15:17 +0200, Adrian Robert <adrian.b.robert@gmail.com> said:

>> In the reported case, read_socket_hook is called from the select
>> (ns_select, actually) call in wait_reading_process_output, and a
>> menu bar activation there leads to Lisp evaluation containing
>> Faccept_process_output.

> It appears that a call verify-visited-file-modtime happens for each
> active menu item, which then triggers tramp to go and check on the
> server.  This causes a reentrant call to
> wait_reading_process_output, thence a reentrant call to ns_select.
> I've added a check in the latter to shortcircuit in the reentrant
> case.

I'm anxious whether the former reentrance (i.e.,
wait_reading_process_output) is safe.

> I've looked at your code in the Carbon AppKit port and it seems like
> your approach to handling menu events there could be applied to the
> NS port.  The only difference I see in your event handling is
> processing all event types in one function instead of passing them
> through NSApp-sendEvent: to go be distributed through ordinary Cocoa
> channels to widgets.  But since - sendEvent: is an interception
> point, the menu events could be taken there.

I vaguely remember that didn't work in some cases, maybe when Emacs is
used via "Screen Sharing.app" in Leopard.  Anyway you can try.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Fri, 06 Mar 2009 10:01:38 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

>> This causes a reentrant call to wait_reading_process_output, thence
>> a reentrant call to ns_select.  I've added a check in the latter to
>> shortcircuit in the reentrant case.

> I'm anxious whether the former reentrance (i.e.,
> wait_reading_process_output) is safe.

Because wait_reading_process_output also calls timer_check, its
reentrance would probably be OK.

Still, I think Feval calls from read_socket_hook and (emulated) select
are really bad.  Most developers assume that these functions (and some
higher-level ones such as detect_input_pending) don't call Feval, and
they may add some code without noticing that NS breaks such
assumptions.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> Still, I think Feval calls from read_socket_hook and (emulated) select
> are really bad.  Most developers assume that these functions (and some
> higher-level ones such as detect_input_pending) don't call Feval, and
> they may add some code without noticing that NS breaks such
> assumptions.

Although I am still unsure how often in practice there is a problem
(the tramp menu crash was not reported until now, despite the
situation causing it on the NS end existing for years), I fully agree
that the menu event handling under NS should be fixed.  Given the menu
update model in emacs, the best solution is likely to defer the event
as other ports including Carbon+AppKit have done.  The only thing
stopping me is my lack of time right now, and a hesitation to
implement such a change during pretest.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 07 Mar 2009 15:28:21 +0200, Adrian Robert <adrian.b.robert@gmail.com> said:

>> Still, I think Feval calls from read_socket_hook and (emulated)
>> select are really bad.  Most developers assume that these functions
>> (and some higher-level ones such as detect_input_pending) don't
>> call Feval, and they may add some code without noticing that NS
>> breaks such assumptions.

> Although I am still unsure how often in practice there is a problem
> (the tramp menu crash was not reported until now, despite the
> situation causing it on the NS end existing for years), I fully
> agree that the menu event handling under NS should be fixed.  Given
> the menu update model in emacs, the best solution is likely to defer
> the event as other ports including Carbon+AppKit have done.  The
> only thing stopping me is my lack of time right now, and a
> hesitation to implement such a change during pretest.

That's why I suggested putting off the pretest or removing the
Cocoa/GNUstep port from the pretest.

This is not the only thing.  The recent "fail on osx between 2/4/2009
and 2/5/2009" thread reminded me of several "kludges" that should be
fixed before the release:

1. Different interface for existing functionality.

  a. ns-read-file-name vs. x-file-dialog
  b. ns-drag-{file,color,text} event + own handlers vs.
     drag-n-drop event + dnd.el
  c. ns-expand-space vs. line-spacing frame parameter
  d. nsfont_make_fontset_for_font vs.
     :lang/:script/:registry properties in font-spec

2. Interface is the same, but implementation is based on own interpretation.

  a. tooltip (not being an Emacs frame, it cannot make use of Emacs
     display features such as images.)
  b. selection concepts (local/foreign selection, ownership)
  c. rightmost scroll-bar placement (it doesn't consider the case
     where scroll bars in different width, e.g., C-x 2 M-:
     (set-window-scroll-bars nil 11 t) RET)
  d. internal-border-width
  e. fringe and cursor drawing

3. NS-only implementation for features that are not inherently specific to NS.

  a. preferences panel
  b. alpha-component in color specification
  c. color image for stipple (cf. tiling patch by Miles Bader)

4. Suspected fundamental design flaw.

  a. C-g handling
  b. menu bar activation timing

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Stefan Monnier]

> stopping me is my lack of time right now, and a hesitation to
> implement such a change during pretest.

Please go ahead with it.  This is fixing a bug, not implementing any
new feature, it's quite within the scope of the pretest, although we
usually prefer not having to do such things.


        Stefan

Re: Pretest next week

[Chetan Pandya]

From: Stefan Monnier <monnier@iro.umontreal.ca>


> Yes, adding

>     else if (pending_atimers)                \
>       do_pending_atimers();                \

> at the end of QUIT allows poll_timer() to fire under SYNC_INPUT and Ctrl-g
> to be detected, with no apparent other ill effects (in very  limited
> testing).

Good.

>> If so, we should probably create a new
>> var `pending_signals', which should always reflect
>> "pending_timers || interrupt_input_pending"

> I'm not sure if the extra 0-comparison would significantly add to overhead
> but I guess code size could take a hit.

It's easy for the CPU to predict those jumps, but I still think the
current QUIT is already pretty costly, so I'd rather not make it worse.

> Though maybe the  part of QUIT above (when there is a quit_flag) could
> also be reduced to a function call to slim things down?

Yes, that would be good as well.


        Stefan

----------------
I am not sure code size is really an issue. Emacs executable is already huge.
On the other hand, merging flags set by different signal handlers is a problem.
It is quite possible for a signal handler to fire when another signal handler is 
running, unless other signals have been blocked. This can interfere with 
setting of those flags.

It is perhaps safer to leave the flags separate. Other option is to block all 
signals that can interfere, possibly using sigaction() instead of signal(), 
but I am not sure of availability on all supported platforms.

Chetan

Re: Pretest next week

[Stefan Monnier]

> Yes, signal() IS called with SIGIO, input_available_signal(), and the latter
> is never called (according to an fprintf which works fine  running under X).
> I also tried calling signal() at various later  times, in case the handler
> gets replaced.

What happens if you do a "kill -IO <...>" from the shell?
Is your fprintf properly executed?  I.e. is it that the proicess doesn't
receive any IO signal or is it that those signals don't invoke your
signal handler?  Also, could it be that the signals get delivered to
another thread?


        Stefan

Re: Pretest next week

[Stefan Monnier]

> I tried using special-event-map, following the example of delete-
> frame, but I still get a "Wrong type argument: commandp, ns-echo-
> working-text"  message.

Adding (interactive) will fix that, of course.


        Stefan

Pretest next week

[Chong Yidong]

The legal stuff that we were waiting on has been sorted out, so, barring
unforseen circumstances, we will begin the pretest next week.

There is one more task that needs to be done---moving the pmail files to
replace rmail.  Could someone please take care of that?  The pmail files
are now in a correct state, so this just involves replacing `pmail' with
`rmail' everywhere in those files, and changing the file names.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 22 Jan 2009 00:03:53 -0500, Chong Yidong <cyd@stupidchicken.com> said:

> The legal stuff that we were waiting on has been sorted out, so,
> barring unforseen circumstances, we will begin the pretest next
> week.

Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
this issue should be addressed before the pretest because it is
expected to require nontrivial changes in its event handling part.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Chong Yidong]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
> this issue should be addressed before the pretest because it is
> expected to require nontrivial changes in its event handling part.

Adrian, could you comment?  Thanks.

Re: Pretest next week

[Adrian Robert]

On Jan 22, 2009, at 3:49 PM, Chong Yidong wrote:

> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>
>> Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
>> this issue should be addressed before the pretest because it is
>> expected to require nontrivial changes in its event handling part.
>
> Adrian, could you comment?  Thanks.

Sure.  I don't think "nontrivial changes" would be required.  As I  
outlined earlier  (http://thread.gmane.org/gmane.emacs.devel/105688/ 
focus=105694):


1) Complete removal of all Feval() calls in bad places as discussed  
earlier on this list.  A quick check reveals four places, relating  
to: emacs termination, deadkey input handling, toolbar toggle, and  
preferences-help.

- Mostly straightforward (but taking some time) by using custom  
'nonascii-keystroke' events, except for the case of input-method  
handling when in isearch mode, where passing such an event breaks out  
of the isearch entry; I could use some help on this.


2) Go through the code comparing nsterm and macterm input handling to  
make sure all ctrl-g related processing is similar

- I did this at one point, but it should be re-checked if problems  
remain after (1) and (3).


3) Make sure some kind of interrupt mechanism is in place to pick up  
ctrl-g events when emacs core is busy processing and does not itself  
make passes through the NS code event loop (colored spinning disk  
shown in gui).

- I did some experimentation for (3) using a signal handler a while  
ago but something didn't seem to work, and some investigation is  
needed.  I don't think major changes to the port event-handling would  
be needed  since the issue is interrupts during emacs core  
processing, not inside the Cocoa processing.  It should just come  
down to getting whichever interrupt mechanism worked in the Carbon  
port on OS X during emacs-core processing to be active in the Cocoa  
port.  I would welcome help on this.


Recently I've been spending the time I've had for the port on fixing  
the bugs reported by users (http://emacsbugs.donarmstrong.com/cgi-bin/ 
pkgreport.cgi?package=ns), so I haven't made any progress on these yet.

Re: Pretest next week

[Adrian Robert]

On Jan 22, 2009, at 4:23 PM, Adrian Robert wrote:

> Recently I've been spending the time I've had for the port on  
> fixing the bugs reported by users (http:// 
> emacsbugs.donarmstrong.com/cgi-bin/pkgreport.cgi?package=ns), so I  
> haven't made any progress on these yet.

(BTW, wanted to mention that many listed on that page SHOULD be  
closed but not all the "###-done" messages I'm sending seem to get  
picked up.)

Re: Pretest next week

[Juanma Barranquero]

On Thu, Jan 22, 2009 at 15:37, Adrian Robert <adrian.b.robert@gmail.com> wrote:

> (BTW, wanted to mention that many listed on that page SHOULD be closed but
> not all the "###-done" messages I'm sending seem to get picked up.)

There are several bugs that I was unable to close by writing to
NNN-done@, and not even by sending "close NNN" commands to the control
address. I've been able to close them by sending a control message
referring to other bugs, for example

  reassign MMM spam
  close NNN
  quit

    Juanma

Re: Pretest next week

[Stefan Monnier]

> 1) Complete removal of all Feval() calls in bad places as discussed earlier
> on this list.  A quick check reveals four places, relating  to: emacs
> termination, deadkey input handling, toolbar toggle, and  preferences-help.

> - Mostly straightforward (but taking some time) by using custom
> nonascii-keystroke' events,

Sound right.

> except for the case of input-method handling when in isearch mode,
> where passing such an event breaks out of the isearch entry; I could
> use some help on this.

Most likely those events should not be handled by the global-map, but
instead either directly by the C code (e.g. in kbd_buffer_get_event), or
via special-event-map, or via input-decode-map.


        Stefan

Re: Pretest next week

[Adrian Robert]

On Jan 22, 2009, at 9:33 PM, Stefan Monnier wrote:

>> 1) Complete removal of all Feval() calls in bad places as  
>> discussed earlier
>> on this list.  A quick check reveals four places, relating  to: emacs
>> termination, deadkey input handling, toolbar toggle, and   
>> preferences-help.
>
>> - Mostly straightforward (but taking some time) by using custom
>> nonascii-keystroke' events,
>
> Sound right.
>
>> except for the case of input-method handling when in isearch mode,
>> where passing such an event breaks out of the isearch entry; I could
>> use some help on this.
>
> Most likely those events should not be handled by the global-map, but
> instead either directly by the C code (e.g. in  
> kbd_buffer_get_event), or
> via special-event-map, or via input-decode-map.

I tried using special-event-map, following the example of delete- 
frame, but I still get a "Wrong type argument: commandp, ns-echo- 
working-text"  message.  I'm trying to get ns-echo-working-text  
called non-interactively so it can make some text changes in the echo  
area.  Is there any way to do this?

(See ns-win.el for defn of ns-echo-working-text.)


Index: keyboard.c
===================================================================
RCS file: /sources/emacs/emacs/src/keyboard.c,v
retrieving revision 1.990
diff -u -p -r1.990 keyboard.c
--- keyboard.c  12 Jan 2009 09:21:19 -0000      1.990
+++ keyboard.c  24 Jan 2009 08:36:25 -0000
@@ -477,6 +477,9 @@ Lisp_Object Qsave_session;
  #ifdef HAVE_DBUS
  Lisp_Object Qdbus_event;
  #endif
+#ifdef HAVE_NS
+Lisp_Object Qns_echo_working_text;
+#endif
  /* Lisp_Object Qmouse_movement; - also an event header */

  /* Properties of event headers.  */
@@ -4113,6 +4116,14 @@ kbd_buffer_get_event (kbp, used_mouse_me
  #endif
         }

+#if defined (HAVE_NS)
+      else if (event->kind == NS_TEXT_EVENT)
+        {
+         obj = Fcons (intern ("ns-echo-working-text"), Qnil);
+         kbd_fetch_ptr = event + 1;
+        }
+#endif
+
  #if defined (HAVE_X11) || defined (HAVE_NTGUI) \
      || defined (HAVE_NS)
        else if (event->kind == DELETE_WINDOW_EVENT)
@@ -11596,6 +11607,9 @@ struct event_head head_table[] = {
    {&Qdelete_frame,        "delete-frame",        &Qdelete_frame},
    {&Qiconify_frame,       "iconify-frame",       &Qiconify_frame},
    {&Qmake_frame_visible,  "make-frame-visible",   
&Qmake_frame_visible},
+#ifdef HAVE_NS
+  {&Qns_echo_working_text,"ns-echo-working- 
text",&Qns_echo_working_text},
+#endif
    /* `select-window' should be handled just like `switch-frame'
       in read_key_sequence.  */
    {&Qselect_window,       "select-window",       &Qswitch_frame}
@@ -11682,6 +11696,11 @@ syms_of_keyboard ()
    staticpro (&Qdbus_event);
  #endif

+#ifdef HAVE_NS
+  Qns_echo_working_text = intern("ns-echo-working-text");
+  staticpro (&Qns_echo_working_text);
+#endif
+
    Qmenu_enable = intern ("menu-enable");
    staticpro (&Qmenu_enable);
    Qmenu_alias = intern ("menu-alias");
@@ -12382,6 +12401,8 @@ keys_of_keyboard ()

    initial_define_lispy_key (Vspecial_event_map, "delete-frame",
                             "handle-delete-frame");
+  initial_define_lispy_key (Vspecial_event_map, "ns-echo-working-text",
+                           "ns-echo-working-text");
    /* Here we used to use `ignore-event' which would simple set  
prefix-arg to
       current-prefix-arg, as is done in `handle-switch-frame'.
       But `handle-switch-frame is not run from the special-map.

Re: Pretest next week

[Adrian Robert]

On Jan 24, 2009, at 10:43 AM, Adrian Robert wrote:

>
> On Jan 22, 2009, at 9:33 PM, Stefan Monnier wrote:
>
>>> 1) Complete removal of all Feval() calls in bad places as  
>>> discussed earlier
>>> on this list.  A quick check reveals four places, relating  to:  
>>> emacs
>>> termination, deadkey input handling, toolbar toggle, and   
>>> preferences-help.
>>
>>> - Mostly straightforward (but taking some time) by using custom
>>> nonascii-keystroke' events,
>>
>> Sound right.
>>
>>> except for the case of input-method handling when in isearch mode,
>>> where passing such an event breaks out of the isearch entry; I could
>>> use some help on this.
>>
>> Most likely those events should not be handled by the global-map, but
>> instead either directly by the C code (e.g. in  
>> kbd_buffer_get_event), or
>> via special-event-map, or via input-decode-map.
>
> I tried using special-event-map, following the example of delete- 
> frame, but I still get a "Wrong type argument: commandp, ns-echo- 
> working-text"  message.  I'm trying to get ns-echo-working-text  
> called non-interactively so it can make some text changes in the  
> echo area.  Is there any way to do this?

Never mind, after more experimentation it seems that having an  
interactive function is needed and OK now -- the problem w/putting  
text in the echo area must be a difference between going through the  
special-event-map and NONASCII_KEYSTROKE.  I will try to clean up the  
patch below and repost before submitting.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 22 Jan 2009 16:23:40 +0200, Adrian Robert <adrian.b.robert@gmail.com> said:

>>> Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
>>> this issue should be addressed before the pretest because it is
>>> expected to require nontrivial changes in its event handling part.
>> 
>> Adrian, could you comment?  Thanks.

> Sure.  I don't think "nontrivial changes" would be required.  As I
> outlined earlier (http://thread.gmane.org/gmane.emacs.devel/105688/
> focus=105694):

But no progress (even the first step) has been observed in the CVS
since then.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

On Jan 22, 2009, at 4:23 PM, Adrian Robert wrote:
>
> 1) Complete removal of all Feval() calls in bad places as discussed  
> earlier on this list.  A quick check reveals four places, relating  
> to: emacs termination, deadkey input handling, toolbar toggle, and  
> preferences-help.

OK, this is done.


> 2) Go through the code comparing nsterm and macterm input handling  
> to make sure all ctrl-g related processing is similar
>
> 3) Make sure some kind of interrupt mechanism is in place to pick  
> up ctrl-g events when emacs core is busy processing and does not  
> itself make passes through the NS code event loop (colored spinning  
> disk shown in gui).

Here, the issue is that the SIGIO handler is never called, despite  
being registered.  It might be overridden by Cocoa in some way,  
though I haven't been able to fully confirm this.

In any case, I tried to use input polling (Fset_input_interrupt_mode  
(Qnil)), but poll_for_input() does not get called during, for  
example, (while t t).  It seems that the QUIT macro does nothing to  
update timers so the polling timer never fires.  Making the QUIT  
macro call handle_async_input() every time slows down emacs (esp  
during startup).

Does anyone have any suggestions?


thanks,
Adrian

Re: Pretest next week

[Chong Yidong]

Adrian Robert <adrian.b.robert@gmail.com> writes:

>> 3) Make sure some kind of interrupt mechanism is in place to pick up
>> ctrl-g events when emacs core is busy processing and does not
>> itself make passes through the NS code event loop (colored spinning
>> disk shown in gui).
>
> Here, the issue is that the SIGIO handler is never called, despite
> being registered.  It might be overridden by Cocoa in some way,
> though I haven't been able to fully confirm this.

Have you checked whether the SIGIO handler is registered in the first
place?  Also, have you confirmed that the handling function is indeed
never called (as opposed to a bug that makes it do nothing)?

Re: Pretest next week

[Adrian Robert]

On Jan 27, 2009, at 12:07 AM, Chong Yidong wrote:

> Adrian Robert <adrian.b.robert@gmail.com> writes:
>
>>> 3) Make sure some kind of interrupt mechanism is in place to pick up
>>> ctrl-g events when emacs core is busy processing and does not
>>> itself make passes through the NS code event loop (colored spinning
>>> disk shown in gui).
>>
>> Here, the issue is that the SIGIO handler is never called, despite
>> being registered.  It might be overridden by Cocoa in some way,
>> though I haven't been able to fully confirm this.
>
> Have you checked whether the SIGIO handler is registered in the first
> place?  Also, have you confirmed that the handling function is indeed
> never called (as opposed to a bug that makes it do nothing)?

Yes, signal() IS called with SIGIO, input_available_signal(), and the  
latter is never called (according to an fprintf which works fine  
running under X).  I also tried calling signal() at various later  
times, in case the handler gets replaced.

Re: Pretest next week

[Jason Rumney]

Adrian Robert wrote:
>> 3) Make sure some kind of interrupt mechanism is in place to pick up 
>> ctrl-g events when emacs core is busy processing and does not itself 
>> make passes through the NS code event loop (colored spinning disk 
>> shown in gui).
>
> Here, the issue is that the SIGIO handler is never called, despite 
> being registered.  It might be overridden by Cocoa in some way, though 
> I haven't been able to fully confirm this.
>
> In any case, I tried to use input polling (Fset_input_interrupt_mode 
> (Qnil)), but poll_for_input() does not get called during, for example, 
> (while t t).  It seems that the QUIT macro does nothing to update 
> timers so the polling timer never fires.  Making the QUIT macro call 
> handle_async_input() every time slows down emacs (esp during startup).
>
> Does anyone have any suggestions?

On w32, we set Vquit_flag directly when Ctrl-G (quit_char) is pressed. 
This is done in the asyncronous key handling where the event gets put on 
the Lisp input queue, so it does not have to wait for the input queue to 
be polled.

Re: Pretest next week

[Adrian Robert]

On Jan 27, 2009, at 4:10 AM, Jason Rumney wrote:

> On w32, we set Vquit_flag directly when Ctrl-G (quit_char) is  
> pressed. This is done in the asyncronous key handling where the  
> event gets put on the Lisp input queue, so it does not have to wait  
> for the input queue to be polled.

What is the entry point for detecting the Ctrl-G (or any other user  
keyboard input), when a tight loop is running, such as (while t t)?   
Is the SIGIO signal handler used, or is W32 itself asynchronously  
calling something in w32fns.c or w32term.c on a second thread?

thanks,
Adrian

Re: Pretest next week

[Chong Yidong]

Adrian Robert <adrian.b.robert@gmail.com> writes:

> On Jan 27, 2009, at 4:10 AM, Jason Rumney wrote:
>
>> On w32, we set Vquit_flag directly when Ctrl-G (quit_char) is
>> pressed. This is done in the asyncronous key handling where the
>> event gets put on the Lisp input queue, so it does not have to wait
>> for the input queue to be polled.
>
> What is the entry point for detecting the Ctrl-G (or any other user
> keyboard input), when a tight loop is running, such as (while t t)?
> Is the SIGIO signal handler used, or is W32 itself asynchronously
> calling something in w32fns.c or w32term.c on a second thread?

I think this is done in the SIGIO signal handler (at least, according to
the comment for w32_read_socket in w32fns.c).  Jason, could you confirm
this?

Re: Pretest next week

[Jason Rumney]

Chong Yidong wrote:
> Adrian Robert <adrian.b.robert@gmail.com> writes:
>
>   
>> What is the entry point for detecting the Ctrl-G (or any other user
>> keyboard input), when a tight loop is running, such as (while t t)?
>> Is the SIGIO signal handler used, or is W32 itself asynchronously
>> calling something in w32fns.c or w32term.c on a second thread?
>>     
>
> I think this is done in the SIGIO signal handler (at least, according to
> the comment for w32_read_socket in w32fns.c).  Jason, could you confirm
> this?
>   

W32 does not have a SIGIO signal handler, so that comment is probably a 
copy and paste error. On Windows, a second thread is listening for 
window system messages. w32_read_socket is the Lisp thread's message 
handler, C-g detection is done in post_character_message (w32fns.c) 
which is the point where the input message is posted from the window 
system message handling thread to the lisp thread.

Re: Pretest next week

[Adrian Robert]

On Jan 28, 2009, at 11:34 AM, Jason Rumney wrote:

> Chong Yidong wrote:
>> Adrian Robert <adrian.b.robert@gmail.com> writes:
>>
>>
>>> What is the entry point for detecting the Ctrl-G (or any other user
>>> keyboard input), when a tight loop is running, such as (while t t)?
>>> Is the SIGIO signal handler used, or is W32 itself asynchronously
>>> calling something in w32fns.c or w32term.c on a second thread?
>>>
>>
>> I think this is done in the SIGIO signal handler (at least,  
>> according to
>> the comment for w32_read_socket in w32fns.c).  Jason, could you  
>> confirm
>> this?
>>
>
> W32 does not have a SIGIO signal handler, so that comment is  
> probably a copy and paste error. On Windows, a second thread is  
> listening for window system messages. w32_read_socket is the Lisp  
> thread's message handler, C-g detection is done in  
> post_character_message (w32fns.c) which is the point where the  
> input message is posted from the window system message handling  
> thread to the lisp thread.

This could probably be done under NS as well.  There might be issues  
with one or the other thread needing to be the "primary" one, and  
some degree of rearchitecting would be needed, using the W32 port as  
a guide.  So, it might be simpler if there's any way to get the  
conventional keyboard.c timer-driven input polling done through the  
QUIT macro under NS.

Re: Pretest next week

[Stefan Monnier]

> might be simpler if there's any way to get the  conventional keyboard.c
> timer-driven input polling done through the  QUIT macro under NS.

There is no such thing as a timer-driven polling in the QUIT macro.
The polling done via the QUIT macro is "polling for some C var to
change", not "polling for some external thingy".


        Stefan

Re: Pretest next week

[Adrian Robert]

On Jan 28, 2009, at 4:08 PM, Stefan Monnier wrote:

>> might be simpler if there's any way to get the  conventional  
>> keyboard.c
>> timer-driven input polling done through the  QUIT macro under NS.
>
> There is no such thing as a timer-driven polling in the QUIT macro.
> The polling done via the QUIT macro is "polling for some C var to
> change", not "polling for some external thingy".

Right.  The polling I meant was running poll_for_input() in  
keyboard.c in response to the atimer 'poll_timer' firing, if running  
in polling mode (Fset_input_interrupt_mode(Qnil)).  I wasn't sure if  
atimers just weren't supposed to be called while tight loops are  
running, or if some code just wasn't working as expected under NS.   
My initial thought was to add something like

   do_pending_atimers();

to QUIT under HAVE_NS.

As it turns out, this isn't needed.  SIGALRM is triggering  
alarm_signal_handler() in atimer.c correctly, but run_timers() never  
gets called under SYNC_INPUT.  I don't fully understand why or where  
the problem is here.  But undefining SYNC_INPUT gets poll_for_input()  
called (and hence Ctrl-g detected) even in tight loops, with no ill  
effects so far.  Patch below.

It would be of interest to find out why it fails under the SYNC_INPUT  
case...

I also investigated the SIGIO thing a bit more.  I can trigger the  
handler using kill -IO from the command line.  But no SIGIO seems to  
be being sent to the process when keyboard input is given.


-Adrian



*** configure.in.~1.583.~	Thu Jan 22 15:08:19 2009
--- configure.in	Wed Jan 28 17:47:55 2009
***************
*** 2096,2101 ****
--- 2096,2103 ----
     fi
     # We also have mouse menus.
     HAVE_MENUS=yes
+   # SYNC_INPUT prevents Ctrl-g detection in many cases.
+   AC_DEFINE(SYNC_INPUT, 0, [Process async input synchronously.])
   fi


*** nsterm.m.~1.53.~	Sun Jan 25 21:23:24 2009
--- nsterm.m	Wed Jan 28 17:19:50 2009
***************
*** 3746,3752 ****
       
------------------------------------------------------------------------ 
-- */
   {
     baud_rate = 38400;
!   Fset_input_interrupt_mode (Qt);
   }


--- 3763,3769 ----
       
------------------------------------------------------------------------ 
-- */
   {
     baud_rate = 38400;
!   Fset_input_interrupt_mode (Qnil);
   }


***************

Re: Pretest next week

[Stefan Monnier]

> As it turns out, this isn't needed.  SIGALRM is triggering
> alarm_signal_handler() in atimer.c correctly, but run_timers() never  gets
> called under SYNC_INPUT.  I don't fully understand why or where  the problem
> is here.  But undefining SYNC_INPUT gets poll_for_input()  called (and hence
> Ctrl-g detected) even in tight loops, with no ill  effects so far.

I wrote SYNC_INPUT specifically for X11 input handling.  From what you
say above, there is apparently a bug in the way it handles SIGALRM.

Apparently the QUIT macro should check pending_atimers somehow.
Can you try to tweak QUIT so it does

   else if (pending_atimers)
      run_timers;

to see if it fixes your problem?  If so, we should probably create a new
var `pending_signals', which should always reflect
"pending_timers || interrupt_input_pending", then QUIT can check this
var, and if set, it can call a new function `process_pending_signals'
which will then look at pending_timers and interrupt_input_pending to
figure out which function(s) to call.

pending_timers and process_pending_timers are desired/needed to reduce
the code-size (the QUIT macro is expanded at many places), as well as to
reduce the polling overhead.


        Stefan

Re: Pretest next week

[Adrian Robert]

On Jan 28, 2009, at 7:40 PM, Stefan Monnier wrote:

>> As it turns out, this isn't needed.  SIGALRM is triggering
>> alarm_signal_handler() in atimer.c correctly, but run_timers()  
>> never  gets
>> called under SYNC_INPUT.  I don't fully understand why or where   
>> the problem
>> is here.  But undefining SYNC_INPUT gets poll_for_input()  called  
>> (and hence
>> Ctrl-g detected) even in tight loops, with no ill  effects so far.
>
> I wrote SYNC_INPUT specifically for X11 input handling.  From what you
> say above, there is apparently a bug in the way it handles SIGALRM.
>
> Apparently the QUIT macro should check pending_atimers somehow.

Yes, adding

     else if (pending_atimers)				\
       do_pending_atimers();				\

at the end of QUIT allows poll_timer() to fire under SYNC_INPUT and  
Ctrl-g to be detected, with no apparent other ill effects (in very  
limited testing).


>   If so, we should probably create a new
> var `pending_signals', which should always reflect
> "pending_timers || interrupt_input_pending"

I'm not sure if the extra 0-comparison would significantly add to  
overhead but I guess code size could take a hit.  Though maybe the  
part of QUIT above (when there is a quit_flag) could also be reduced  
to a function call to slim things down?

Re: Pretest next week

[Stefan Monnier]

> Yes, adding

>     else if (pending_atimers)				\
>       do_pending_atimers();				\

> at the end of QUIT allows poll_timer() to fire under SYNC_INPUT and Ctrl-g
> to be detected, with no apparent other ill effects (in very  limited
> testing).

Good.

>> If so, we should probably create a new
>> var `pending_signals', which should always reflect
>> "pending_timers || interrupt_input_pending"

> I'm not sure if the extra 0-comparison would significantly add to overhead
> but I guess code size could take a hit.

It's easy for the CPU to predict those jumps, but I still think the
current QUIT is already pretty costly, so I'd rather not make it worse.

> Though maybe the  part of QUIT above (when there is a quit_flag) could
> also be reduced to a function call to slim things down?

Yes, that would be good as well.


        Stefan

Re: Pretest next week

[Chong Yidong]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> we should probably create a new var `pending_signals', which should
> always reflect "pending_timers || interrupt_input_pending", then QUIT
> can check this var, and if set, it can call a new function
> `process_pending_signals' which will then look at pending_timers and
> interrupt_input_pending to figure out which function(s) to call.
>
> pending_timers and process_pending_timers are desired/needed to reduce
> the code-size (the QUIT macro is expanded at many places), as well as to
> reduce the polling overhead.

So basically any place in the code that sets pending_atimers or
interrupt_input_pending would have to set pending_signals as
well---something like in the attached patch?

It's not clear to me whether we want to process pending atimers in the
QUIT macro outside of NS.  Do we?



*** trunk/src/lisp.h.~1.649.~	2009-01-28 13:34:25.000000000 -0500
--- trunk/src/lisp.h	2009-01-28 15:46:38.000000000 -0500
***************
*** 1843,1850 ****
     and (in particular) cannot call arbitrary Lisp code.  */
  
  #ifdef SYNC_INPUT
! extern void handle_async_input P_ ((void));
! extern int interrupt_input_pending;
  
  #define QUIT						\
    do {							\
--- 1843,1850 ----
     and (in particular) cannot call arbitrary Lisp code.  */
  
  #ifdef SYNC_INPUT
! extern void process_pending_signals P_ ((void));
! extern int pending_signals;
  
  #define QUIT						\
    do {							\
***************
*** 1856,1863 ****
  	  Fthrow (Vthrow_on_input, Qt);			\
  	Fsignal (Qquit, Qnil);				\
        }							\
!     else if (interrupt_input_pending)			\
!       handle_async_input ();				\
    } while (0)
  
  #else  /* not SYNC_INPUT */
--- 1856,1863 ----
  	  Fthrow (Vthrow_on_input, Qt);			\
  	Fsignal (Qquit, Qnil);				\
        }							\
!     else if (pending_signals)				\
!       process_pending_signals ();			\
    } while (0)
  
  #else  /* not SYNC_INPUT */
*** trunk/src/blockinput.h.~1.28.~	2009-01-28 10:14:34.000000000 -0500
--- trunk/src/blockinput.h	2009-01-28 15:31:45.000000000 -0500
***************
*** 59,64 ****
--- 59,69 ----
  
  extern int pending_atimers;
  
+ /* This is equal to (interrupt_input_pending || pending_atimers).  */
+ 
+ extern int pending_signals;
+ 
+ 
  #if defined (HAVE_NS) && !defined (COCOA_EXPERIMENTAL_CTRL_G)
  /* NS does not use interrupt-driven input processing (yet), so this is
     unneeded and moreover was causing problems. */
*** trunk/src/keyboard.c.~1.991.~	2009-01-26 10:47:17.000000000 -0500
--- trunk/src/keyboard.c	2009-01-28 15:41:13.000000000 -0500
***************
*** 91,96 ****
--- 91,98 ----
     during the current critical section.  */
  int interrupt_input_pending;
  
+ int pending_signals;
+ 
  #define KBD_BUFFER_SIZE 4096
  
  KBOARD *initial_kboard;
***************
*** 2193,2203 ****
       struct atimer *timer;
  {
    if (poll_suppress_count == 0)
  #ifdef SYNC_INPUT
!     interrupt_input_pending = 1;
  #else
!     poll_for_input_1 ();
  #endif
  }
  
  #endif /* POLL_FOR_INPUT */
--- 2195,2208 ----
       struct atimer *timer;
  {
    if (poll_suppress_count == 0)
+     {
  #ifdef SYNC_INPUT
!       interrupt_input_pending = 1;
!       pending_signals = 1;
  #else
!       poll_for_input_1 ();
  #endif
+     }
  }
  
  #endif /* POLL_FOR_INPUT */
***************
*** 7261,7266 ****
--- 7266,7272 ----
  handle_async_input ()
  {
    interrupt_input_pending = 0;
+   pending_signals = pending_atimers;
  
    while (1)
      {
***************
*** 7274,7279 ****
--- 7280,7293 ----
      }
  }
  
+ void
+ process_pending_signals ()
+ {
+   if (interrupt_input_pending)
+     handle_async_input ();
+   do_pending_atimers ();
+ }
+ 
  #ifdef SIGIO   /* for entire page */
  /* Note SIGIO has been undef'd if FIONREAD is missing.  */
  
***************
*** 7291,7296 ****
--- 7305,7311 ----
  
  #ifdef SYNC_INPUT
    interrupt_input_pending = 1;
+   pending_signals = 1;
  #else
    SIGNAL_THREAD_CHECK (signo);
  #endif
***************
*** 11536,11541 ****
--- 11551,11557 ----
    input_pending = 0;
    interrupt_input_blocked = 0;
    interrupt_input_pending = 0;
+   pending_signals = pending_atimers;
  
    /* This means that command_loop_1 won't try to select anything the first
       time through.  */
*** trunk/src/atimer.c.~1.29.~	2009-01-08 06:46:21.000000000 -0500
--- trunk/src/atimer.c	2009-01-28 15:20:04.000000000 -0500
***************
*** 384,391 ****
        EMACS_GET_TIME (now);
      }
  
!   if (! pending_atimers)
!     set_alarm ();
  }
  
  
--- 384,396 ----
        EMACS_GET_TIME (now);
      }
  
!   if (pending_atimers)
!     pending_signals = 1;
!   else
!     {
!       pending_signals = interrupt_input_pending;
!       set_alarm ();
!     }
  }
  
  
***************
*** 397,402 ****
--- 402,408 ----
       int signo;
  {
    pending_atimers = 1;
+   pending_signals = 1;
  #ifndef SYNC_INPUT
    run_timers ();
  #endif
***************
*** 439,444 ****
--- 445,451 ----
  {
    free_atimers = atimers = NULL;
    pending_atimers = 0;
+   pending_signals = interrupt_input_pending;
    signal (SIGALRM, alarm_signal_handler);
  }
  
*** trunk/src/xterm.c.~1.1020.~	2009-01-16 09:48:32.000000000 -0500
--- trunk/src/xterm.c	2009-01-28 15:23:48.000000000 -0500
***************
*** 7138,7147 ****
--- 7138,7149 ----
    if (interrupt_input_blocked)
      {
        interrupt_input_pending = 1;
+       pending_signals = 1;
        return -1;
      }
  
    interrupt_input_pending = 0;
+   pending_signals = pending_atimers;
    BLOCK_INPUT;
  
    /* So people can tell when we have read the available input.  */
*** trunk/src/w32inevt.c.~1.44.~	2009-01-08 06:46:26.000000000 -0500
--- trunk/src/w32inevt.c	2009-01-28 15:25:12.000000000 -0500
***************
*** 651,660 ****
--- 651,662 ----
    if (interrupt_input_blocked)
      {
        interrupt_input_pending = 1;
+       pending_signals = 1;
        return -1;
      }
  
    interrupt_input_pending = 0;
+   pending_signals = pending_atimers;
    BLOCK_INPUT;
  
    for (;;)
*** trunk/src/w32term.c.~1.317.~	2009-01-10 07:53:18.000000000 -0500
--- trunk/src/w32term.c	2009-01-28 15:24:50.000000000 -0500
***************
*** 4078,4087 ****
--- 4078,4089 ----
    if (interrupt_input_blocked)
      {
        interrupt_input_pending = 1;
+       pending_signals = 1;
        return -1;
      }
  
    interrupt_input_pending = 0;
+   pending_signals = pending_atimers;
    BLOCK_INPUT;
  
    /* So people can tell when we have read the available input.  */

Re: Pretest next week

[Stefan Monnier]

> It's not clear to me whether we want to process pending atimers in the
> QUIT macro outside of NS.  Do we?

I think we do: the non-SYNC_INPUT code processes them directly from the
signal handler, AFAICT.


        Stefan

Re: Pretest next week

[Stefan Monnier]

>> The legal stuff that we were waiting on has been sorted out, so,
>> barring unforseen circumstances, we will begin the pretest next
>> week.

> Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
> this issue should be addressed before the pretest because it is
> expected to require nontrivial changes in its event handling part.

That's OK, I think.  The code is definitely not frozen once the pretest
starts.  But in any case, I urge you (or whoever is able to work on
this) to do it ASAP.


        Stefan

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 22 Jan 2009 09:44:00 -0500, "Stefan Monnier" <monnier@iro.umontreal.ca> said:

>> Still the Cocoa/GNUstep port doesn't handle C-g properly.  I think
>> this issue should be addressed before the pretest because it is
>> expected to require nontrivial changes in its event handling part.

> That's OK, I think.  The code is definitely not frozen once the
> pretest starts.  But in any case, I urge you (or whoever is able to
> work on this) to do it ASAP.

My concern is that the lack of proper C-g handling for such a long
term may imply a problem in the fundamental design of the port.  Of
course, no one can be sure unless someone refutes it by actually
implementing it.  I think maintainers should set some time limit to
wait such an implementation, and IMO the beginning of the pretest is
the most appropriate timing.  Even if "nontrivial changes" is not
necessary as Adrian says, there should be enough time to test the new
code before the release.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> writes:

> 
> My concern is that the lack of proper C-g handling for such a long
> term may imply a problem in the fundamental design of the port.  Of
> course, no one can be sure unless someone refutes it by actually
> implementing it.

I won't dispute the second sentence, but the lack for a long term is simply
because I've never found it a problem in my own usage patterns, and users
haven't been particularly vocal in complaining about it either.  There have
been other priorities.  But it's top of my list now.

BTW, the one other major gap in the port is dumping under GNUstep.  I posted a
suggestion for this a while ago, which came down to adding some
zone-allocation functionality to unexelf.c/NS_IMPL_GNUSTEP paralleling
unexmacosx.c.  If anyone is running under GNUstep and would like to take a
crack I can provide further advice offline.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 24 Jan 2009 08:51:46 +0000 (UTC), Adrian Robert <Adrian.B.Robert@gmail.com> said:

>> My concern is that the lack of proper C-g handling for such a long
>> term may imply a problem in the fundamental design of the port.  Of
>> course, no one can be sure unless someone refutes it by actually
>> implementing it.

> I won't dispute the second sentence, but the lack for a long term is
> simply because I've never found it a problem in my own usage
> patterns, and users haven't been particularly vocal in complaining
> about it either.  There have been other priorities.  But it's top of
> my list now.

I'm really surprised to hear that it hasn't have the top priority.
I've been thinking that proper C-g handling is a minimum requirement
to be a part of the official Emacs rather than an unofficial
distribution.  Without it, the port wouldn't be a "real Emacs".
Anyway, good to hear that now it has the top priority.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Chong Yidong]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> I'm really surprised to hear that it hasn't have the top priority.
> I've been thinking that proper C-g handling is a minimum requirement
> to be a part of the official Emacs rather than an unofficial
> distribution.  Without it, the port wouldn't be a "real Emacs".
> Anyway, good to hear that now it has the top priority.

I think it's already been fixed.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 26 Jan 2009 15:07:16 -0500, Chong Yidong <cyd@stupidchicken.com> said:

> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>> I'm really surprised to hear that it hasn't have the top priority.
>> I've been thinking that proper C-g handling is a minimum
>> requirement to be a part of the official Emacs rather than an
>> unofficial distribution.  Without it, the port wouldn't be a "real
>> Emacs".  Anyway, good to hear that now it has the top priority.

> I think it's already been fixed.

Not at all, actually.  I found that even Feval calls from
read_socket_hook are not completely removed (at least, menu bar
activation and OK button click in the preference panel.)

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> writes:

> Not at all, actually.  I found that even Feval calls from
> read_socket_hook are not completely removed (at least, menu bar
> activation and OK button click in the preference panel.)

I just noticed the menu one, actually.  Do you have an automated
way of finding these places, or is it just occurring during usage?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 27 Jan 2009 13:04:43 +0000 (UTC), Adrian Robert <Adrian.B.Robert@gmail.com> said:

>> Not at all, actually.  I found that even Feval calls from
>> read_socket_hook are not completely removed (at least, menu bar
>> activation and OK button click in the preference panel.)

> I just noticed the menu one, actually.  Do you have an automated way
> of finding these places, or is it just occurring during usage?

Not an automated way.  I just did some random experiment (with
--enable-cocoa-experimental-ctrl-g).

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Eli Zaretskii]

> Date: Mon, 26 Jan 2009 13:46:29 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> 
> I'm really surprised to hear that it hasn't have the top priority.
> I've been thinking that proper C-g handling is a minimum requirement
> to be a part of the official Emacs rather than an unofficial
> distribution.

What is "improper" in how C-g is handled on Cocoa?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 27 Jan 2009 00:36:31 +0200, Eli Zaretskii <eliz@gnu.org> said:

>> Date: Mon, 26 Jan 2009 13:46:29 +0900 From: YAMAMOTO Mitsuharu
>> <mituharu@math.s.chiba-u.ac.jp>
>> 
>> I'm really surprised to hear that it hasn't have the top priority.
>> I've been thinking that proper C-g handling is a minimum
>> requirement to be a part of the official Emacs rather than an
>> unofficial distribution.

> What is "improper" in how C-g is handled on Cocoa?

It makes almost all uses of QUIT macro throughout the source code
meaningless.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Eli Zaretskii]

> Date: Tue, 27 Jan 2009 08:27:33 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> Cc: emacs-devel@gnu.org
> 
> > What is "improper" in how C-g is handled on Cocoa?
> 
> It makes almost all uses of QUIT macro throughout the source code
> meaningless.

Could you please supply a few more technical details, assuming that I
know something about how C-g is supposed to work in Emacs?  What
doesn't work on Cocoa that works on GNU/Linux and other platforms?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 27 Jan 2009 05:28:15 +0200, Eli Zaretskii <eliz@gnu.org> said:

>> > What is "improper" in how C-g is handled on Cocoa?
>> 
>> It makes almost all uses of QUIT macro throughout the source code
>> meaningless.

> Could you please supply a few more technical details, assuming that
> I know something about how C-g is supposed to work in Emacs?  What
> doesn't work on Cocoa that works on GNU/Linux and other platforms?

If a Lisp expression doesn't contain any direct or indirect call to
some read operation, its evaluation cannot be quit in the
Cocoa/GNUstep port.  Simple examples are (while t) and (shell-command
"sleep 100").

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> writes:

> > What is "improper" in how C-g is handled on Cocoa?
> 
> It makes almost all uses of QUIT macro throughout the source code
> meaningless.

Hi,

Do you have any technical suggestions as to how to handle it given
that SIGIO seems not to be available?  As I posted, polling (keyboard.c
start_polling(), stop_polling(), etc.) seems not to happen through
the QUIT macro (perhaps it did at one time but the functionality was
lost after all terms went to SIGIO-based?).

One thing that would work would be to update the polling timer inside
the QUIT macro (under HAVE_NS of course), so that poll_for_input()
would eventually get called.  But I don't know if there are reasons
this would be dangerous or something, or if there would be another way.

thanks,
Adrian

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 27 Jan 2009 12:57:00 +0000 (UTC), Adrian Robert <Adrian.B.Robert@gmail.com> said:

> Do you have any technical suggestions as to how to handle it given
> that SIGIO seems not to be available?  As I posted, polling
> (keyboard.c start_polling(), stop_polling(), etc.) seems not to
> happen through the QUIT macro (perhaps it did at one time but the
> functionality was lost after all terms went to SIGIO-based?).

> One thing that would work would be to update the polling timer
> inside the QUIT macro (under HAVE_NS of course), so that
> poll_for_input() would eventually get called.  But I don't know if
> there are reasons this would be dangerous or something, or if there
> would be another way.

Those seem to be problems that are common to the systems that use
polling instead of SIGIO.  I think you can rather concentrate on Feval
removal for now, leaving those problems to others.  In particular, if
you are planning to call read_socket_hook at the QUIT timings, you
should never start menu bar tracking there.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Alex Ott]

Hello all

I have one issue with current cocoa/gnustep port - in some situations it
improperly show symbols on the screen when i used letter from different
charsets - some letter rendered higher than other.  I tried to explicitly
specify concrete fonts, instead of font classes, but it has same effects.

You can see this in the second header on the screenshot at
http://xtalk.msk.su/~ott/files/Emacs23.png called "Настройка Semantic для
работы с проектами на C & C++".  This screenshot was made from GNU Emacs
updated & built today

But in current Carbon Emacs (GNU Emacs 22.3.2 (i386-apple-darwin8.11.1,
Carbon Version 1.6.0)) all works fine, as you can see on the
http://xtalk.msk.su/~ott/files/CarbonEmacs.png

-- 
With best wishes, Alex Ott, MBA
http://alexott.blogspot.com/        http://xtalk.msk.su/~ott/
http://alexott-ru.blogspot.com/

Re: Pretest next week

[Bastien]

Chong Yidong <cyd@stupidchicken.com> writes:

> There is one more task that needs to be done---moving the pmail files to
> replace rmail.  Could someone please take care of that?  The pmail files
> are now in a correct state, so this just involves replacing `pmail' with
> `rmail' everywhere in those files, and changing the file names.

I will do this in the next 10 hours.

-- 
 Bastien

Re: Pretest next week

[Bastien]

Done.  I removed old rmail* files, except rmail-spam-filter.el.  
I renamed (i.e. deleted+added) pmail* files to rmail*.

I tested quickly, it looks okay.  Please test further.

I didn't move lisp/mail/Changelog.pmail yet.  Shall I merge this
changelog with lisp/ChangeLog?

Bastien <bastienguerry@googlemail.com> writes:

> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> There is one more task that needs to be done---moving the pmail files to
>> replace rmail.  Could someone please take care of that?  The pmail files
>> are now in a correct state, so this just involves replacing `pmail' with
>> `rmail' everywhere in those files, and changing the file names.
>
> I will do this in the next 10 hours.

-- 
 Bastien

Re: Pretest next week

[Stefan Monnier]

> I didn't move lisp/mail/Changelog.pmail yet.  Shall I merge this
> changelog with lisp/ChangeLog?

Yes, please.


        Stefan

Re: Pretest next week

[Glenn Morris]

Bastien wrote:

> Done.  I removed old rmail* files, except rmail-spam-filter.el.  
> I renamed (i.e. deleted+added) pmail* files to rmail*.

What was the point of cvs removing the old rmail files first?
AFAICS, it did nothing but send out two sets of massive, meaningless
mails to the diffs list.

Re: Pretest next week

[Bastien]

Glenn Morris <rgm@gnu.org> writes:

> Bastien wrote:
>
>> Done.  I removed old rmail* files, except rmail-spam-filter.el.  
>> I renamed (i.e. deleted+added) pmail* files to rmail*.
>
> What was the point of cvs removing the old rmail files first?

To win a new email address from Chong domain name. 

> AFAICS, it did nothing but send out two sets of massive, meaningless
> mails to the diffs list.

C'est la vie!

-- 
 Bastien

Re: Pretest next week

[Glenn Morris]

Bastien wrote:

>> What was the point of cvs removing the old rmail files first?
>
> To win a new email address from Chong domain name. 

I don't understand; but it's history now anyway. I wouldn't do it that
way in future is all.

Re: Pretest next week

[Bastien]

Glenn Morris <rgm@gnu.org> writes:

> Bastien wrote:
>
>>> What was the point of cvs removing the old rmail files first?
>>
>> To win a new email address from Chong domain name. 
>
> I don't understand; but it's history now anyway. 

M-x activate-sense-of-humor RET

> I wouldn't do it that way in future is all.

Noted, thanks!

"All work and no play makes Jack a dull boy."

-- 
 Bastien

Re: Pretest next week

[David Kastrup]

Bastien <bastienguerry@googlemail.com> writes:

> Glenn Morris <rgm@gnu.org> writes:
>
>> Bastien wrote:
>>
>>>> What was the point of cvs removing the old rmail files first?
>>>
>>> To win a new email address from Chong domain name. 
>>
>> I don't understand; but it's history now anyway. 
>
> M-x activate-sense-of-humor RET

I get [No match]

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Pretest next week

[Lennart Borgman]

On Sun, Jan 25, 2009 at 9:01 PM, David Kastrup <dak@gnu.org> wrote:
> Bastien <bastienguerry@googlemail.com> writes:
>> M-x activate-sense-of-humor RET
>
> I get [No match]

Ah, too bad for you ... ;-)

Re: Pretest next week

[Frank Schmitt]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> On Sun, Jan 25, 2009 at 9:01 PM, David Kastrup <dak@gnu.org> wrote:
>> Bastien <bastienguerry@googlemail.com> writes:
>>> M-x activate-sense-of-humor RET
>>
>> I get [No match]
>
> Ah, too bad for you ... ;-)

This must be due to the German locale...

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: Pretest next week

[Stefan Monnier]

>> M-x activate-sense-of-humor RET
> I get [No match]

Probably a problem in my new completion code,


        Stefan

Re: Pretest next week

[Chong Yidong]

Barring unforseen circumstances, I will roll the 23.0.90 pretest on
Sunday, the 1st of February.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 29 Jan 2009 10:29:01 -0500, Chong Yidong <cyd@stupidchicken.com> said:

> Barring unforseen circumstances, I will roll the 23.0.90 pretest on
> Sunday, the 1st of February.

How do you recognize the current status and perspective of the C-g
issue in the Cocoa/GNUstep port?  The changes made so far weren't
"nontrivial", indeed.  But I meant by "nontrivial" the still remaining
"Feval call in menu bar activation".  Actually, menu bar tracking
itself is problematic inside read_socket_hook if the Cocoa/GNUstep
port is trying to follow what the Carbon port does with respect to
C-g.)

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Chong Yidong]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> How do you recognize the current status and perspective of the C-g
> issue in the Cocoa/GNUstep port?  The changes made so far weren't
> "nontrivial", indeed.  But I meant by "nontrivial" the still remaining
> "Feval call in menu bar activation".  Actually, menu bar tracking
> itself is problematic inside read_socket_hook if the Cocoa/GNUstep
> port is trying to follow what the Carbon port does with respect to
> C-g.)

Work on this may still proceed during the pretest; we are not yet in a
hard code freeze.  It is not serious enough to delay the start of the
pretest.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 29 Jan 2009 20:42:26 -0500, Chong Yidong <cyd@stupidchicken.com> said:

>> How do you recognize the current status and perspective of the C-g
>> issue in the Cocoa/GNUstep port?  The changes made so far weren't
>> "nontrivial", indeed.  But I meant by "nontrivial" the still
>> remaining "Feval call in menu bar activation".  Actually, menu bar
>> tracking itself is problematic inside read_socket_hook if the
>> Cocoa/GNUstep port is trying to follow what the Carbon port does
>> with respect to C-g.)

> Work on this may still proceed during the pretest; we are not yet in
> a hard code freeze.  It is not serious enough to delay the start of
> the pretest.

Of course, it's up to you maintainers whether to start pretest at this
stage.  But I dare to say it's a serious issue from my experience.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Fri, 30 Jan 2009 10:46:59 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

>>> How do you recognize the current status and perspective of the C-g
>>> issue in the Cocoa/GNUstep port?  The changes made so far weren't
>>> "nontrivial", indeed.  But I meant by "nontrivial" the still
>>> remaining "Feval call in menu bar activation".  Actually, menu bar
>>> tracking itself is problematic inside read_socket_hook if the
>>> Cocoa/GNUstep port is trying to follow what the Carbon port does
>>> with respect to C-g.)

>> Work on this may still proceed during the pretest; we are not yet
>> in a hard code freeze.  It is not serious enough to delay the start
>> of the pretest.

> Of course, it's up to you maintainers whether to start pretest at
> this stage.  But I dare to say it's a serious issue from my
> experience.

If you want to start the pretest soon, I suggest putting off the
inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).  In
particular, the GNUstep port doesn't have enough quality to start
pretest.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Chong Yidong]

YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> If you want to start the pretest soon, I suggest putting off the
> inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).  In
> particular, the GNUstep port doesn't have enough quality to start
> pretest.

That would be counter-productive.  Consider the worst-case scenario of
keeping the Cocoa port: suppose that, just before the 23.1 release, the
port remains as irretrievably broken as you make it out to be.  In that
case, we can disable it by changing the build system, and mark it as
experimental/hackers-only.  So nothing is lost by keeping the port (and
I am optimistic that it will not come to that).

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 01 Feb 2009 09:34:06 -0500, Chong Yidong <cyd@stupidchicken.com> said:

>> If you want to start the pretest soon, I suggest putting off the
>> inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).
>> In particular, the GNUstep port doesn't have enough quality to
>> start pretest.

> That would be counter-productive.  Consider the worst-case scenario
> of keeping the Cocoa port: suppose that, just before the 23.1
> release, the port remains as irretrievably broken as you make it out
> to be.  In that case, we can disable it by changing the build
> system, and mark it as experimental/hackers-only.  So nothing is
> lost by keeping the port (and I am optimistic that it will not come
> to that).

OK.  Agreed except for the last parenthesis in the sense that I have
no idea how to solve the remaining problem without changing the
current design drastically.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Richard M Stallman]

    OK.  Agreed except for the last parenthesis in the sense that I have
    no idea how to solve the remaining problem without changing the
    current design drastically.

How would you change it?
Are you prepared to write the changes now?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Mon, 02 Feb 2009 20:42:38 -0500, Richard M Stallman <rms@gnu.org> said:

>     OK.  Agreed except for the last parenthesis in the sense that I have
>     no idea how to solve the remaining problem without changing the
>     current design drastically.

> How would you change it?
> Are you prepared to write the changes now?

What I would do can be found in my Carbon+AppKit port (*) for Emacs
22.  But that's not an "NextStep" port in the sense that it heavily
relies on Carbon on which the Cocoa event system is implemented.

*: ftp://ftp.math.s.chiba-u.ac.jp/emacs/emacs-22.3-appkit-1.2.tar.gz

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Richard M Stallman]

    What I would do can be found in my Carbon+AppKit port (*) for Emacs
    22.  But that's not an "NextStep" port

Is there ANY way to make the Cocoa port handle C-g correctly on
NextStep?

					   in the sense that it heavily
    relies on Carbon on which the Cocoa event system is implemented.

ISTR that the Mac support we had in Emacs 22 was based on Carbon, and
that we removed it because the interfaces it used were no longer
supported.  But now you're saying that what you would do is use those
interfaces.  I don't know how to reconcile these too.

Did I misunderstand part of this?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 04 Feb 2009 02:04:46 -0500, Richard M Stallman <rms@gnu.org> said:

>     What I would do can be found in my Carbon+AppKit port (*) for Emacs
>     22.  But that's not an "NextStep" port

> Is there ANY way to make the Cocoa port handle C-g correctly on
> NextStep?

I tried that with my Carbon+AppKit port first, but I abandoned it in
the very early development stage.  Because I'm relatively new to
Cocoa, there might be some way I don't know yet.  Anyway, I'm not 100%
sure.  If Adrian or someone else can find the way, that'll be fine.

> 					   in the sense that it heavily
>     relies on Carbon on which the Cocoa event system is implemented.

> ISTR that the Mac support we had in Emacs 22 was based on Carbon, and
> that we removed it because the interfaces it used were no longer
> supported.  But now you're saying that what you would do is use those
> interfaces.  I don't know how to reconcile these too.

> Did I misunderstand part of this?

Although almost all the GUI portion of Carbon will be deprecated,
that's not extended to the whole Carbon.  That's why I created the
Carbon+AppKit port from the Carbon port by only replacing its GUI
implementation (src/mactoolbox.c) with some Cocoa counterpart
(src/macappkit.m).

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

One more technical point: IMHO switching the Cocoa port to use the same
event loop integration method as W32 (a separate thread to run the GUI
toolkit event loop) would work as well under Cocoa as it does under W32.
But I'd prefer to avoid this sort of change unless the flaws in the
current integration, which has served a wide user base for many years,
prove unworkable.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 01 Feb 2009 09:34:06 -0500, Chong Yidong <cyd@stupidchicken.com> said:

> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>> If you want to start the pretest soon, I suggest putting off the
>> inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).
>> In particular, the GNUstep port doesn't have enough quality to
>> start pretest.

> That would be counter-productive.  Consider the worst-case scenario
> of keeping the Cocoa port: suppose that, just before the 23.1
> release, the port remains as irretrievably broken as you make it out
> to be.  In that case, we can disable it by changing the build
> system, and mark it as experimental/hackers-only.  So nothing is
> lost by keeping the port (and I am optimistic that it will not come
> to that).

Now I suppose we are at "just before the 23.1 release".  Will the NS
port be marked as experimental/hackers-only, or given a title of
"first-class" port as a part of the official release?

Besides stability/performance issues found in the bugtracker, it still
has several compatibility issues as I mentioned earlier (I added new
comments in brackets in the following):

1. Different interface for existing functionality.

  a. ns-read-file-name vs. x-file-dialog
  b. ns-drag-{file,color,text} event + own handlers vs.
     drag-n-drop event + dnd.el
  c. ns-expand-space vs. line-spacing frame parameter
     [No longer an issue: the former was removed.]
  d. nsfont_make_fontset_for_font vs.
     :lang/:script/:registry properties in font-spec
     [The former is removed, and :lang/:script support was added, but
      :registry support is still missing.  The ns font-backend driver
      doesn't have the "shape" function for Complex Text Layout.]

2. Interface is the same, but implementation is based on own interpretation.

  a. tooltip (not being an Emacs frame, it cannot make use of Emacs
     display features such as images.)
  b. selection concepts (local/foreign selection, ownership)
  c. rightmost scroll-bar placement (it doesn't consider the case
     where scroll bars in different width, e.g., C-x 2 M-:
     (set-window-scroll-bars nil 11 t) RET)
  d. internal-border-width
  e. fringe and cursor drawing

3. NS-only implementation for features that are not inherently specific to NS.

  a. preferences panel
     [No longer an issue: removed.]
  b. alpha-component in color specification
     [Even without alpha, many formats in the NS port are incompatible
      with the other platforms.  The only compatible one is #rrggbb,
      which is not encouraged according to the X11 man page.  In this
      sense, it can also be classified as Group 1.]
  c. color image for stipple (cf. tiling patch by Miles Bader)
     [Even for monochrome images, "stippling" in the NS port is
      actually implemented as tiling rather than stippling.  In this
      sense, it can also be classified as Group 2.]

4. Suspected fundamental design flaw.

  a. C-g handling
  b. menu bar activation timing

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Stefan Monnier]

> Now I suppose we are at "just before the 23.1 release".  Will the NS
> port be marked as experimental/hackers-only, or given a title of
> "first-class" port as a part of the official release?

I believe it is considered "first class port".  At least I do not think
it's "experimental".  [ Of course, the situation is completely
different for the GNUstep support ]

> Besides stability/performance issues found in the bugtracker, it still
> has several compatibility issues as I mentioned earlier (I added new
> comments in brackets in the following):

Yes, it still has a lot of room for improvement.  Help is welcome.
The points you raise all seem very relevant.  Maybe you should add them
in a new "Cocoa support" header in etc/TODO?


        Stefan

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Tue, 14 Jul 2009 14:40:01 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> Now I suppose we are at "just before the 23.1 release".  Will the
>> NS port be marked as experimental/hackers-only, or given a title of
>> "first-class" port as a part of the official release?

> I believe it is considered "first class port".  At least I do not
> think it's "experimental".  [ Of course, the situation is completely
> different for the GNUstep support ]

>> Besides stability/performance issues found in the bugtracker, it
>> still has several compatibility issues as I mentioned earlier (I
>> added new comments in brackets in the following):

> Yes, it still has a lot of room for improvement.  Help is welcome.
> The points you raise all seem very relevant.  Maybe you should add
> them in a new "Cocoa support" header in etc/TODO?

It's bad for the release version of the "first-class" port to keep
incompatible features that are to be removed/replaced later.  Maybe I
can help remove some of the incompatible features.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[David Reitter]

On Jul 14, 2009, at 7:40 PM, Stefan Monnier wrote:

>> Now I suppose we are at "just before the 23.1 release".  Will the NS
>> port be marked as experimental/hackers-only, or given a title of
>> "first-class" port as a part of the official release?
>
> I believe it is considered "first class port".  At least I do not  
> think
> it's "experimental".  [ Of course, the situation is completely
> different for the GNUstep support ]

The long list of bugs (including crashes) suggests to mark it as  
experimental.  We currently don't have enough people who contribute  
fixes.  The port doesn't have release quality.
Note that the port doesn't build on the developer-releases of OS X  
10.6 (dumping fails), so it is likely that it won't build on 10.6 once  
it is out.

On the other hand, we don't want to discourage people to tinker with  
it since we want them to contribute code.  So it would be important to  
get it out there for people to use.

Re: Pretest next week

[Chong Yidong]

I've added notes to etc/NEWS, nextstep/README, and doc/emacs/macos.texi
on the branch, stating that the Nextstep port is not as stable as on
other platforms.

Re: Pretest next week

[Stefan Monnier]

> If you want to start the pretest soon, I suggest putting off the
> inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).
> In particular, the GNUstep port doesn't have enough quality to start
> pretest.

Since there's no alternative code for Mac nor for GNUstep, there's no
harm in providing it and nothing to gain from removing it.  If it's not
robust enough, we'll just label it "experimental" (which we'll probably
have to do for GNUstep, sadly).
Let's move forward.


        Stefan

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sun, 01 Feb 2009 17:17:10 -0500, Stefan Monnier <monnier@iro.umontreal.ca> said:

>> If you want to start the pretest soon, I suggest putting off the
>> inclusion of the Cocoa/GNUstep port until Emacs 23.2 (or later).
>> In particular, the GNUstep port doesn't have enough quality to
>> start pretest.

> Since there's no alternative code for Mac nor for GNUstep,

Not actually.  Both users can use some of X11 builds in the meanwhile
if they want to use the new version.

> there's no harm in providing it and nothing to gain from removing
> it.  If it's not robust enough, we'll just label it "experimental"
> (which we'll probably have to do for GNUstep, sadly).  Let's move
> forward.

I thought it's shameful for Emacs to include a port that doesn't
handle C-g properly as a part of its official release.  Also, I was
not suggesting the removal of the port from the whole CVS branches.

			     YAMAMOTO Mitsuharu
			mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

YAMAMOTO Mitsuharu <mituharu <at> math.s.chiba-u.ac.jp> writes:

> I thought it's shameful for Emacs to include a port that doesn't
> handle C-g properly as a part of its official release.  Also, I was
> not suggesting the removal of the port from the whole CVS branches.

I believe it is working 100% now.  Please report any specific failure cases
you find through report-emacs-bug

The cost of the implementation has been that are some cases where menus are not
updated when clicked on; specifically, when ns_read_socket is called
asynchronously through handle_async_input or poll_for_input_1 (depending if
SYNC_INPUT is enabled), the menu update must be deferred.

I believe the effect of this on user experience will be rare to almost
nonexistent in practice, but let's find out empirically.  If it is a problem
there are a couple of approaches that could be tried, but I'd rather prioritize
other issues for now unless it proves to be a problem in actual use.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Wed, 4 Feb 2009 12:08:55 +0000 (UTC), Adrian Robert <Adrian.B.Robert@gmail.com> said:

>> I thought it's shameful for Emacs to include a port that doesn't
>> handle C-g properly as a part of its official release.  Also, I was
>> not suggesting the removal of the port from the whole CVS branches.

> I believe it is working 100% now.  Please report any specific
> failure cases you find through report-emacs-bug

> The cost of the implementation has been that are some cases where
> menus are not updated when clicked on; specifically, when
> ns_read_socket is called asynchronously through handle_async_input
> or poll_for_input_1 (depending if SYNC_INPUT is enabled), the menu
> update must be deferred.

> I believe the effect of this on user experience will be rare to
> almost nonexistent in practice, but let's find out empirically.  If
> it is a problem there are a couple of approaches that could be
> tried, but I'd rather prioritize other issues for now unless it
> proves to be a problem in actual use.

What happens if read_socket_hook is called from the QUIT macro in the
context of process filters or idle timers?

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Richard M Stallman]

    > I thought it's shameful for Emacs to include a port that doesn't
    > handle C-g properly as a part of its official release.  Also, I was
    > not suggesting the removal of the port from the whole CVS branches.

    I believe it is working 100% now.  Please report any specific failure cases
    you find through report-emacs-bug

Thank you.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 05 Feb 2009 00:40:10 -0500, Richard M Stallman <rms@gnu.org> said:

>> I thought it's shameful for Emacs to include a port that doesn't
>> handle C-g properly as a part of its official release.  Also, I was
>> not suggesting the removal of the port from the whole CVS branches.

>     I believe it is working 100% now.  Please report any specific failure cases
>     you find through report-emacs-bug

> Thank you.

That's actually a way I didn't adopt because it confuses the user: it
shows empty menu items if the user clicks the menu bar at the timing
of a read_socket_hook call from a QUIT macro in the context of process
filters or idle timers.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Adrian Robert]

On Feb 5, 2009, at 1:43 PM, YAMAMOTO Mitsuharu wrote:

>>>>>> On Thu, 05 Feb 2009 00:40:10 -0500, Richard M Stallman  
>>>>>> <rms@gnu.org> said:
>
>>> I thought it's shameful for Emacs to include a port that doesn't
>>> handle C-g properly as a part of its official release.  Also, I was
>>> not suggesting the removal of the port from the whole CVS branches.
>
>>     I believe it is working 100% now.  Please report any specific  
>> failure cases
>>     you find through report-emacs-bug
>
>> Thank you.
>
> That's actually a way I didn't adopt because it confuses the user: it
> shows empty menu items if the user clicks the menu bar at the timing
> of a read_socket_hook call from a QUIT macro in the context of process
> filters or idle timers.

In the current implementation it does this only if menus have not  
been clicked on before (else the previous items, albeit out-of-date  
ones) are shown.  This is better than nothing, though it should be  
better.  However I'm not really sure how often this "clicks the menu  
bar at the timing of a read_socket_hook call from a QUIT macro"  
occurs in practice.  Anyway, what about the second-thread approach  
like W32, are there any gotchas you know about?

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 5 Feb 2009 19:39:42 +0200, Adrian Robert <adrian.b.robert@gmail.com> said:

>> That's actually a way I didn't adopt because it confuses the user:
>> it shows empty menu items if the user clicks the menu bar at the
>> timing of a read_socket_hook call from a QUIT macro in the context
>> of process filters or idle timers.

> In the current implementation it does this only if menus have not
> been clicked on before (else the previous items, albeit out-of-date
> ones) are shown.  This is better than nothing, though it should be
> better.

What happens if the user selects that "bogus" item?  Doesn't it
generate a bogus Emacs event that might not happen if the menu bar
activation is deferred?

> However I'm not really sure how often this "clicks the menu bar at
> the timing of a read_socket_hook call from a QUIT macro" occurs in
> practice.

The process filters and idle timers are a kind of "background tasks".
Users will expect that they can continue normal editing work (with
some delay, sometimes).  Also, some process filter takes time to
complete, and an idle timer can be designed so it can do a long task
if no input is pending.

> Anyway, what about the second-thread approach like W32, are there
> any gotchas you know about?

I guess menu bar item calculation can be deferred but not for starting
menu bar tracking in the case of NS.  For example, I'm not sure if the
following scenario can work properly:

  1. Evaluate `(while t)'.
  2. The user clicks the menu bar.  The GUI thread starts to wait for
     menu items to be calculated in the Lisp thread.  The Lisp thread
     is not ready to do that because of the tight loop.
  3. The user types C-g to quit the loop.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Richard M Stallman]

    Work on this may still proceed during the pretest; we are not yet in a
    hard code freeze.  It is not serious enough to delay the start of the
    pretest.

If fixing this will require major changes, that may be a reason to
delay the pretest -- if someone is willing to working on these major
changes right away.

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 31 Jan 2009 01:44:22 -0500, Richard M Stallman <rms@gnu.org> said:

> If fixing this will require major changes, that may be a reason to
> delay the pretest -- if someone is willing to working on these major
> changes right away.

In the platforms other than the Cocoa/GNUstep port, menu bar is
uniformly activated by the x_activate_menubar call in
kbd_buffer_get_event, which is called from read_char.  However, the
Cocoa/GNUstep port activates the menu bar and starts mouse tracking in
the context of read_socket_hook, which is supposed to be called from
fairly random states of the Lisp interpreter.  (That port currently
calls read_socket_hook only from limited locations, and that makes
this problem obscured and the C-g handling improper.)

I think this difference becomes a major obstacle if that port is
planning to solve the C-g issue during the pretest period by following
the strategies that are used in some existing platforms.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Sat, 31 Jan 2009 16:35:27 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

>>>>> On Sat, 31 Jan 2009 01:44:22 -0500, Richard M Stallman <rms@gnu.org> said:
>> If fixing this will require major changes, that may be a reason to
>> delay the pretest -- if someone is willing to working on these
>> major changes right away.

> In the platforms other than the Cocoa/GNUstep port, menu bar is
> uniformly activated by the x_activate_menubar call in
> kbd_buffer_get_event, which is called from read_char.  However, the
> Cocoa/GNUstep port activates the menu bar and starts mouse tracking
> in the context of read_socket_hook, which is supposed to be called
> from fairly random states of the Lisp interpreter.  (That port
> currently calls read_socket_hook only from limited locations, and
> that makes this problem obscured and the C-g handling improper.)

Correction: bug #2564 shows that even "read_socket_hook only from
limited locations" is problematic.  In the reported case,
read_socket_hook is called from the select (ns_select, actually) call
in wait_reading_process_output, and a menu bar activation there leads
to Lisp evaluation containing Faccept_process_output.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 05 Mar 2009 09:56:01 +0900, YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> said:

> Correction: bug #2564 shows that even "read_socket_hook only from
> limited locations" is problematic.  In the reported case,
> read_socket_hook is called from the select (ns_select, actually)
> call in wait_reading_process_output, and a menu bar activation there
> leads to Lisp evaluation containing Faccept_process_output.

I think the bug #2432 is also caused by the same reason.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: Pretest next week

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Thu, 29 Jan 2009 10:29:01 -0500
> 
> Barring unforseen circumstances, I will roll the 23.0.90 pretest on
> Sunday, the 1st of February.

Bug #970 is still not fixed, as of today's CVS.  Is someone working on
it?  I don't think we can release Emacs 23 with this problem.

Also, I think we need to start the docs reviewing process ASAP.  Emacs
23 changes a lot, so finding all the obsolete and/or incorrect places
in the docs will take some time.  For past major releases, we used to
have at least 2 different people proofread each portion of the 2 main
manuals, so I think we need to have in admin/ some record of who
proofreads what.

Re: Pretest next week

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Chong Yidong <cyd@stupidchicken.com>
>> Date: Thu, 29 Jan 2009 10:29:01 -0500
>> 
>> Barring unforseen circumstances, I will roll the 23.0.90 pretest on
>> Sunday, the 1st of February.
>
> Bug #970 is still not fixed, as of today's CVS.  Is someone working on
> it?  I don't think we can release Emacs 23 with this problem.

I wonder if I'm the only one who is not able to compile emacs anymore
(bug #2097) when configuring with --with-libotf.

Bye,
Tassilo

Re: Pretest next week

[Kenichi Handa]

In article <87zlh9unfo.fsf@thinkpad.tsdh.de>, Tassilo Horn <tassilo@member.fsf.org> writes:

> I wonder if I'm the only one who is not able to compile
> emacs anymore (bug #2097) when configuring with
> --with-libotf.

I can compile Emacs with libotf successfully.

What is shown by this command?

% pkg-config --modversion libotf

And, does ALL_CFLAGS in src/Makefile contain the output of this?

% pkg-config --cflags libotf

---
Kenichi Handa
handa@m17n.org

Re: Pretest next week

[Tassilo Horn]

Kenichi Handa <handa@m17n.org> writes:

Hi!

[sorry for the repost, but I've sent a private message, not a wide one.]

>> I wonder if I'm the only one who is not able to compile emacs anymore
>> (bug #2097) when configuring with --with-libotf.
>
> I can compile Emacs with libotf successfully.
>
> What is shown by this command?
>
> % pkg-config --modversion libotf

0.9.8

> And, does ALL_CFLAGS in src/Makefile contain the output of this?
>
> % pkg-config --cflags libotf

That returns "-I/usr/include/freetype2" which is included in ALL_CFLAGS.

Bye,
Tassilo

Re: Pretest next week

[Kenichi Handa]

In article <87skn1ufzl.fsf@thinkpad.tsdh.de>, Tassilo Horn <tassilo@member.fsf.org> writes:

> > % pkg-config --modversion libotf

> 0.9.8

> > And, does ALL_CFLAGS in src/Makefile contain the output
> of this?
> >
> > % pkg-config --cflags libotf

> That returns "-I/usr/include/freetype2" which is included
> in ALL_CFLAGS.

Strange.  Please send me your /usr/include/otf.h.

---
Kenichi Handa
handa@m17n.org

Re: Pretest next week

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Kenichi Handa <handa@m17n.org> writes:

>> That returns "-I/usr/include/freetype2" which is included
>> in ALL_CFLAGS.
>
> Strange.  Please send me your /usr/include/otf.h.

Without the comments it's only:

--8<---------------cut here---------------start------------->8---
#ifndef OTF_H
#define OTF_H

#include "OTF_Definitions.h"
#include "OTF_FileManager.h"
#include "OTF_Filenames.h"
#include "OTF_HandlerArray.h"
#include "OTF_MasterControl.h"
#include "OTF_RStream.h"
#include "OTF_Reader.h"
#include "OTF_WStream.h"
#include "OTF_Writer.h"

#endif /* OTF_H */
--8<---------------cut here---------------end--------------->8---

The complete file + the referenced headers are in this tarball.

[-- Attachment #2: otf-headers.tar.gz --]
[-- Type: application/octet-stream, Size: 32080 bytes --]

[-- Attachment #3: Type: text/plain, Size: 13 bytes --]

Bye,
Tassilo

Re: Pretest next week

[Kenichi Handa]

In article <87ocxovroy.fsf@thinkpad.tsdh.de>, Tassilo Horn <tassilo@member.fsf.org> writes:

> > Strange.  Please send me your /usr/include/otf.h.

> Without the comments it's only:

> --8<---------------cut here---------------start------------->8---
> #ifndef OTF_H
> #define OTF_H

> #include "OTF_Definitions.h"
> #include "OTF_FileManager.h"
[...]

???  How did you install it?  It's completely different from
the header file that libotf requires (attached at the tail).

---
Kenichi Handa
handa@m17n.org

/* otf.h -- Header file for libotf (OpenType font library).

Copyright (C) 2003, 2004, 2005, 2006, 2007
  National Institute of Advanced Industrial Science and Technology (AIST)
  Registration Number H15PRO167

This file is part of libotf.

Libotf is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation; either version 2.1 of the License, or
(at your option) any later version.

Libotf is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public
License for more details.

You should have received a copy of the GNU Lesser General Public
License along with this library, in a file named COPYING; if not,
write to the Free Software Foundation, Inc., 59 Temple Place, Suite
330, Boston, MA 02111-1307, USA.  */

#ifndef _OTF_H_
#define _OTF_H_

#ifdef __cplusplus
extern "C" {
#endif

/* Version name of this library.  */
#define LIBOTF_VERSION "0.9.6"

/* Major version number.  */
#define LIBOTF_MAJOR_VERSION 0
/* Minor version number.  */
#define LIBOTF_MINOR_VERSION 9
/* Release (i.e. patch level) number.  */
#define LIBOTF_RELEASE_NUMBER 6

/***
    Table of contents:

    (1) Structures for OTF Layout tables and OTF itself
    (1-1) Basic types
    (1-2) "head" table
    (1-3) "name" table
    (1-4) "cmap" table
    (1-5) Structures common to GDEF, GSUB, and GPOS
    (1-6) "GDEF" table
    (1-7) Structures for ScriptList, FeatureList, and LookupList
    (1-8) Structures common to GSUB and GPOS
    (1-9) "GSUB" table
    (1-10) "GPOS" table
    (1-11) Structure for OTF

    (2) API for reading OTF
    (2-1) OTF_open(), OTF_open_ft_face()
    (2-2) OTF_close()
    (2-3) OTF_get_table()
    (2-4) OTF_check_table()

    (3) API for driving OTF
    (3-1) Structure for glyph string
    (3-2) OTF_drive_cmap()
    (3-3) OTF_drive_gdef()
    (3-4) OTF_drive_gsub()
    (3-5) OTF_drive_gpos()
    (3-6) OTF_drive_tables()
    (3-7) OTF_get_unicode()
    (3-8) OTF_drive_gsub_alternate()

    (4) API for error handling
    (4-1) Error codes
    (4-2) OTF_perror()

    (5) API miscellaneous

*/

\f
/*** (1) Structures for OTF Layout tables and OTF itself */

/*** (1-1) Basic types */

typedef unsigned OTF_Tag;
typedef unsigned OTF_GlyphID;
typedef unsigned OTF_Offset;

typedef struct
{
  unsigned high;
  unsigned low;
} OTF_Fixed;


/*** (1-2) "head" table */

typedef struct
{
  OTF_Fixed TableVersionNumber;
  OTF_Fixed fontRevision;
  unsigned checkSumAdjustment;
  unsigned magicNumber;
  unsigned flags;
  int unitsPerEm;
} OTF_head;


/*** (1-3) "name" table */

typedef struct
{
  int platformID;
  int encodingID;
  int languageID;
  int nameID;
  int length;
  int offset;

  /* If nonzero, NAME is an ASCII string.  */
  int ascii;
  unsigned char *name;
} OTF_NameRecord;

#define OTF_max_nameID 23

typedef struct
{
  int format;
  int count;
  int stringOffset;
  OTF_NameRecord *nameRecord;
  char *name[OTF_max_nameID + 1];
} OTF_name;


/*** (1-4) "cmap" table */

typedef struct
{
  unsigned char glyphIdArray[256];
} OTF_EncodingSubtable0;

typedef struct
{
  unsigned firstCode;
  unsigned entryCount;
  int idDelta;
  unsigned idRangeOffset;
} OTF_cmapSubHeader;

typedef struct
{
  unsigned short subHeaderKeys[256];
  int subHeaderCount;
  OTF_cmapSubHeader *subHeaders;
  int glyphIndexCount;
  OTF_GlyphID *glyphIndexArray;
} OTF_EncodingSubtable2;

typedef struct
{
  unsigned startCount;
  unsigned endCount;
  int idDelta;
  unsigned idRangeOffset;
} OTF_cmapSegment;

typedef struct
{
  unsigned segCountX2;
  unsigned searchRange;
  unsigned entrySelector;
  unsigned rangeShift;
  OTF_cmapSegment *segments;
  int GlyphCount;
  unsigned *glyphIdArray;
} OTF_EncodingSubtable4;

typedef struct
{
  unsigned firstCode;
  unsigned entryCount;
  unsigned *glyphIdArray;
} OTF_EncodingSubtable6;

typedef struct
{
  unsigned startCharCode;
  unsigned endCharCode;
  unsigned startGlyphID;
} OTF_cmapGroup;

typedef struct
{
  unsigned char is32[8192];
  unsigned nGroups;
  OTF_cmapGroup *Groups;
} OTF_EncodingSubtable8;

typedef struct
{
  unsigned startCharCode;
  unsigned numChars;
  unsigned *glyphs;
} OTF_EncodingSubtable10;

typedef struct
{
  unsigned nGroups;
  OTF_cmapGroup *Groups;
} OTF_EncodingSubtable12;

typedef struct
{
  unsigned unicodeValue;
  unsigned short glyphID;
} OTF_UVSMapping;

typedef struct
{
  unsigned startUnicodeValue;
  unsigned short additionalCount;
} OTF_UnicodeValueRange;

typedef struct
{
  unsigned varSelector;
  unsigned defaultUVSOffset;
  unsigned nonDefaultUVSOffset;
  /* DefaultUVS */
  unsigned numUnicodeValueRanges;
  OTF_UnicodeValueRange *unicodeValueRanges;
  /* NonDefaultUVS */
  unsigned numUVSMappings;
  OTF_UVSMapping *uvsMappings;
} OTF_VariationSelectorRecord;

typedef struct
{
  unsigned nRecords;
  OTF_VariationSelectorRecord *Records;
} OTF_EncodingSubtable14;

typedef struct
{
  unsigned format;
  unsigned length;
  unsigned language;
  union {
    OTF_EncodingSubtable0 *f0;
    OTF_EncodingSubtable2 *f2;
    OTF_EncodingSubtable4 *f4;
    OTF_EncodingSubtable6 *f6;
    OTF_EncodingSubtable8 *f8;
    OTF_EncodingSubtable10 *f10;
    OTF_EncodingSubtable12 *f12;
    OTF_EncodingSubtable14 *f14;
  }f;
} OTF_EncodingSubtable;

typedef struct
{
  unsigned platformID;
  unsigned encodingID;
  unsigned offset;
  OTF_EncodingSubtable subtable;
} OTF_EncodingRecord;

typedef struct
{
  unsigned version;
  unsigned numTables;
  OTF_EncodingRecord *EncodingRecord;
  /* Mapping table: Unicode->GlyphID */
  unsigned short *unicode_table;
  int max_glyph_id;
  /* Mapping table: GlyphID->Unicode */
  unsigned short *decode_table;
} OTF_cmap;


/*** (1-5) Structures common to GDEF, GSUB, GPOS */

typedef struct
{
  OTF_GlyphID Start;
  OTF_GlyphID End;
  unsigned StartCoverageIndex;
} OTF_RangeRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned CoverageFormat;
  unsigned Count;
  union {
    OTF_GlyphID *GlyphArray;
    OTF_RangeRecord *RangeRecord;
  } table;
} OTF_Coverage;

typedef struct
{
  OTF_Offset offset;
  unsigned StartSize;
  unsigned EndSize;
  unsigned DeltaFormat;
  char *DeltaValue;
} OTF_DeviceTable;

typedef struct
{
  OTF_GlyphID Start;
  OTF_GlyphID End;
  unsigned Class;
} OTF_ClassRangeRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned ClassFormat;
  union {
    struct {
      OTF_GlyphID StartGlyph;
      unsigned GlyphCount;
      unsigned *ClassValueArray;
    } f1;
    struct {
      unsigned ClassRangeCount;
      OTF_ClassRangeRecord *ClassRangeRecord;
    } f2;
  } f;
} OTF_ClassDef;


/*** (1-6) "GDEF" table */

typedef struct
{
  OTF_Fixed Version;
  OTF_Offset GlyphClassDef;
  OTF_Offset AttachList;
  OTF_Offset LigCaretList;
  OTF_Offset MarkAttachClassDef;
} OTF_GDEFHeader;

enum OTF_GlyphClassDef
  {
    OTF_GlyphClass0 = 0,
    OTF_GlyphClassBase = 1,
    OTF_GlyphClassLigature = 2,
    OTF_GlyphClassMark = 3,
    OTF_GlyphClassComponent = 4
  };

typedef struct
{
  OTF_Offset offset;
  unsigned PointCount;
  unsigned *PointIndex;
} OTF_AttachPoint;

typedef struct
{
  OTF_Coverage Coverage;
  unsigned GlyphCount;
  OTF_AttachPoint *AttachPoint;
} OTF_AttachList;

typedef struct
{
  OTF_Offset offset;
  unsigned CaretValueFormat;	/* 1, 2, or 3 */
  union {
    union {
      int Coordinate;
    } f1;
    union {
      unsigned CaretValuePoint;
    } f2;
    union {
      int Coordinate;
      OTF_DeviceTable DeviceTable;
    } f3;
  } f;
} OTF_CaretValue;

typedef struct
{
  OTF_Offset offset;
  unsigned CaretCount;
  OTF_CaretValue *CaretValue;
} OTF_LigGlyph;

typedef struct
{
  OTF_Coverage Coverage;
  unsigned LigGlyphCount;
  OTF_LigGlyph *LigGlyph;
} OTF_LigCaretList;

typedef struct
{
  OTF_GDEFHeader header;
  OTF_ClassDef glyph_class_def;
  OTF_AttachList attach_list;
  OTF_LigCaretList lig_caret_list;
  OTF_ClassDef mark_attach_class_def;
} OTF_GDEF;


/*** (1-7) Structures for ScriptList, FeatureList, and LookupList  */

/*** The structure hierarchy

   ScriptList
     ScriptRecord[]
       ScriptTag
     Script[]
       DefaultLangSys
       LangSysRecord[]
         LangSysTag
       LangSys[]
         LookupOrder
	 ReqFeatureIndex
	 FeatureIndex[]

  FeatureList
    FeatureRecored[]
      FeatureTag
    Feature[]
      FeatureParams
      LookupListIndex[]

  LookupList
    LookupOffset[]
    Lookup[]
      LookupType
      LookupFlag
      SubTableOffset[]
      SubTable.gsub[] or SubTable.gpos[]
*/


typedef struct
{
  OTF_Offset LookupOrder;
  unsigned ReqFeatureIndex;
  unsigned FeatureCount;
  unsigned *FeatureIndex;
} OTF_LangSys;

typedef struct
{
  OTF_Tag LangSysTag;
  OTF_Offset LangSys;
} OTF_LangSysRecord;

typedef struct
{
  OTF_Tag ScriptTag;
  OTF_Offset offset;
  OTF_Offset DefaultLangSysOffset;
  OTF_LangSys DefaultLangSys;
  unsigned LangSysCount;
  OTF_LangSysRecord *LangSysRecord;
  OTF_LangSys *LangSys;
} OTF_Script;

typedef struct
{
  OTF_Offset offset;
  unsigned ScriptCount;
  OTF_Script *Script;
} OTF_ScriptList;

typedef struct
{
  OTF_Tag FeatureTag;
  OTF_Offset offset;
  OTF_Offset FeatureParams;
  unsigned LookupCount;
  unsigned *LookupListIndex;
} OTF_Feature;

typedef struct
{
  OTF_Offset offset;
  unsigned FeatureCount;
  OTF_Feature *Feature;
} OTF_FeatureList;

typedef struct OTF_LookupSubTableGSUB OTF_LookupSubTableGSUB;
typedef struct OTF_LookupSubTableGPOS OTF_LookupSubTableGPOS;

enum OTF_LookupFlagBit
  {
    OTF_RightToLeft = 0x0001,
    OTF_IgnoreBaseGlyphs = 0x0002,
    OTF_IgnoreLigatures = 0x0004,
    OTF_IgnoreMarks = 0x0008,
    OTF_Reserved = 0x00F0,
    OTF_MarkAttachmentType = 0xFF00
  };

#define OTF_LookupFlagIgnoreMask \
  (OTF_IgnoreBaseGlyphs | OTF_IgnoreLigatures | OTF_IgnoreMarks)

typedef struct
{
  OTF_Offset offset;
  unsigned LookupType;
  unsigned LookupFlag;
  unsigned SubTableCount;
  OTF_Offset *SubTableOffset;
  union {
    OTF_LookupSubTableGSUB *gsub;
    OTF_LookupSubTableGPOS *gpos;
  } SubTable;
} OTF_Lookup;

typedef struct
{
  OTF_Offset offset;
  unsigned LookupCount;
  OTF_Lookup *Lookup;
} OTF_LookupList;


/*** (1-8) Structures common to GSUB and GPOS */

/* For SubstLookupRecord (GSUB) and PosLookupRecord (GPOS).  */

typedef struct
{
  unsigned SequenceIndex;
  unsigned LookupListIndex;
} OTF_LookupRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned GlyphCount;
  unsigned LookupCount;
  OTF_GlyphID *Input;		/* [<GlyphCount> - 1] */
  OTF_LookupRecord *LookupRecord; /* [<LookupCount>] */
} OTF_Rule;

typedef struct
{
  OTF_Offset offset;
  unsigned RuleCount;
  OTF_Rule *Rule;		/* [<RuleCount>] */
} OTF_RuleSet;

typedef struct
{
  OTF_Offset offset;
  unsigned GlyphCount;
  unsigned LookupCount;
  unsigned *Class;		/* [<GlyphCount> - 1] */
  OTF_LookupRecord *LookupRecord; /* [<LookupCount>] */
} OTF_ClassRule;

typedef struct
{
  OTF_Offset offset;
  unsigned ClassRuleCnt;
  OTF_ClassRule *ClassRule;	/* [<ClassRuleCnt>] */
} OTF_ClassSet;

typedef struct
{
  OTF_Offset offset;
  unsigned BacktrackGlyphCount;
  OTF_GlyphID *Backtrack;
  unsigned InputGlyphCount;
  OTF_GlyphID *Input;
  unsigned LookaheadGlyphCount;
  OTF_GlyphID *LookAhead;
  unsigned LookupCount;
  OTF_LookupRecord *LookupRecord;
} OTF_ChainRule;

typedef struct
{
  OTF_Offset offset;
  unsigned ChainRuleCount;
  OTF_ChainRule *ChainRule;
} OTF_ChainRuleSet;

typedef struct
{
  OTF_Offset offset;
  unsigned BacktrackGlyphCount;
  unsigned *Backtrack;
  unsigned InputGlyphCount;
  unsigned *Input;
  unsigned LookaheadGlyphCount;
  unsigned *LookAhead;
  unsigned LookupCount;
  OTF_LookupRecord *LookupRecord;
} OTF_ChainClassRule;

typedef struct
{
  OTF_Offset offset;
  unsigned ChainClassRuleCnt;
  OTF_ChainClassRule *ChainClassRule;
} OTF_ChainClassSet;


/* Common to OTF_GSUB/GPOS_Context1/2/3.  */

typedef struct
{
  unsigned RuleSetCount;
  OTF_RuleSet *RuleSet;		/* [<RuleSetCount>] */
} OTF_Context1;

typedef struct
{
  OTF_ClassDef ClassDef;
  unsigned ClassSetCnt;
  OTF_ClassSet *ClassSet;	/* [<ClassSetCnt>] */
} OTF_Context2;

typedef struct
{
  unsigned GlyphCount;
  unsigned LookupCount;
  OTF_Coverage *Coverage;	/* [<GlyphCount>] */
  OTF_LookupRecord *LookupRecord; /* [<LookupCount>] */
} OTF_Context3;


/* Common to OTF_GSUB/GPOS_ChainContext1/2/3.  */

typedef struct
{
  unsigned ChainRuleSetCount;
  OTF_ChainRuleSet *ChainRuleSet;
} OTF_ChainContext1;

typedef struct
{
  OTF_ClassDef BacktrackClassDef;
  OTF_ClassDef InputClassDef;
  OTF_ClassDef LookaheadClassDef;
  unsigned ChainClassSetCnt;
  OTF_ChainClassSet *ChainClassSet;
} OTF_ChainContext2;

typedef struct
{
  unsigned BacktrackGlyphCount;
  OTF_Coverage *Backtrack;
  unsigned InputGlyphCount;
  OTF_Coverage *Input;
  unsigned LookaheadGlyphCount;
  OTF_Coverage *LookAhead;
  unsigned LookupCount;
  OTF_LookupRecord *LookupRecord;
} OTF_ChainContext3;

/* Common to OTF_GSUB/GPOS.  */

typedef struct
{
  OTF_Fixed Version;
  OTF_ScriptList ScriptList;
  OTF_FeatureList FeatureList;
  OTF_LookupList LookupList;
} OTF_GSUB_GPOS;

/*** (1-9) "GSUB" table */

typedef struct
{
  int DeltaGlyphID;
} OTF_GSUB_Single1;

typedef struct
{
  unsigned GlyphCount;
  OTF_GlyphID *Substitute;
} OTF_GSUB_Single2;

typedef struct OTF_Sequence OTF_Sequence;

typedef struct
{
  unsigned SequenceCount;
  OTF_Sequence *Sequence;
} OTF_GSUB_Multiple1;

struct OTF_Sequence
{
  OTF_Offset offset;
  unsigned GlyphCount;
  OTF_GlyphID *Substitute;
};

typedef struct OTF_AlternateSet OTF_AlternateSet;

typedef struct
{
  unsigned AlternateSetCount;
  OTF_AlternateSet *AlternateSet;
} OTF_GSUB_Alternate1;

struct OTF_AlternateSet
{
  OTF_Offset offset;
  unsigned GlyphCount;
  OTF_GlyphID *Alternate;
};

typedef struct OTF_LigatureSet OTF_LigatureSet;
typedef struct OTF_Ligature OTF_Ligature;

typedef struct
{
  unsigned LigSetCount;
  OTF_LigatureSet *LigatureSet;
} OTF_GSUB_Ligature1;

struct OTF_LigatureSet
{
  OTF_Offset offset;
  unsigned LigatureCount;
  OTF_Ligature *Ligature;
};

struct OTF_Ligature
{
  OTF_Offset offset;
  OTF_GlyphID LigGlyph;
  unsigned CompCount;
  OTF_GlyphID *Component;
};

typedef OTF_Context1 OTF_GSUB_Context1;

typedef OTF_Context2 OTF_GSUB_Context2;

typedef OTF_Context3 OTF_GSUB_Context3;

typedef OTF_ChainContext1 OTF_GSUB_ChainContext1;

typedef OTF_ChainContext2 OTF_GSUB_ChainContext2;

typedef OTF_ChainContext3 OTF_GSUB_ChainContext3;

typedef struct
{
  unsigned ExtensionLookupType;
  unsigned ExtensionOffset;
  OTF_LookupSubTableGSUB *ExtensionSubtable;
} OTF_GSUB_Extension1;

typedef struct
{
  unsigned BacktrackGlyphCount;
  OTF_Coverage *Backtrack;
  unsigned LookaheadGlyphCount;
  OTF_Coverage *LookAhead;
  unsigned GlyphCount;
  OTF_GlyphID *Substitute;
} OTF_GSUB_ReverseChain1;

struct OTF_LookupSubTableGSUB
{
  unsigned Format;
  OTF_Coverage Coverage;
  union {
    /* LookupType 1 */
    OTF_GSUB_Single1 single1;
    OTF_GSUB_Single2 single2;
    /* LookupType 2 */
    OTF_GSUB_Multiple1 multiple1;
    /* LookupType 3 */
    OTF_GSUB_Alternate1 alternate1;
    /* LookupType 4 */
    OTF_GSUB_Ligature1 ligature1;
    /* LookupType 5 */
    OTF_GSUB_Context1 context1;
    OTF_GSUB_Context2 context2;
    OTF_GSUB_Context3 context3;
    /* LookupType 6 */
    OTF_GSUB_ChainContext1 chain_context1;
    OTF_GSUB_ChainContext2 chain_context2;
    OTF_GSUB_ChainContext3 chain_context3;
    /* LookupType 7 */
    OTF_GSUB_Extension1 extension1;
    /* LookupType 8 */
    OTF_GSUB_ReverseChain1 reverse_chain1;
  } u;
};

typedef OTF_GSUB_GPOS OTF_GSUB;

/*** (1-10) "GPOS" table */

enum OTF_ValueFormat
  {
    OTF_XPlacement = 0x0001,
    OTF_YPlacement = 0x0002,
    OTF_XAdvance = 0x0004,
    OTF_YAdvance = 0x0008,
    OTF_XPlaDevice = 0x0010,
    OTF_YPlaDevice = 0x0020,
    OTF_XAdvDevice = 0x0040,
    OTF_YAdvDevice = 0x0080
  };

typedef struct
{
  int XPlacement;
  int YPlacement;
  int XAdvance;
  int YAdvance;
  OTF_DeviceTable XPlaDevice;
  OTF_DeviceTable YPlaDevice;
  OTF_DeviceTable XAdvDevice;
  OTF_DeviceTable YAdvDevice;
} OTF_ValueRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned AnchorFormat;
  int XCoordinate;
  int YCoordinate;
  union {
    struct {
      unsigned AnchorPoint;
    } f1;
    struct {
      OTF_DeviceTable XDeviceTable;
      OTF_DeviceTable YDeviceTable;
    } f2;
  } f;
} OTF_Anchor;

typedef struct
{
  unsigned Class;
  OTF_Anchor MarkAnchor;
} OTF_MarkRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned MarkCount;
  OTF_MarkRecord *MarkRecord;
} OTF_MarkArray;

typedef struct
{
  unsigned ValueFormat;
  OTF_ValueRecord Value;
} OTF_GPOS_Single1;

typedef struct
{
  unsigned ValueFormat;
  unsigned ValueCount;
  OTF_ValueRecord *Value;  	/* [<ValueCount>] */
} OTF_GPOS_Single2;

typedef struct
{
  OTF_GlyphID SecondGlyph;
  OTF_ValueRecord Value1;
  OTF_ValueRecord Value2;
} OTF_PairValueRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned PairValueCount;
  OTF_PairValueRecord *PairValueRecord;
} OTF_PairSet;

typedef struct
{
  unsigned ValueFormat1;
  unsigned ValueFormat2;
  unsigned PairSetCount;
  OTF_PairSet *PairSet;
} OTF_GPOS_Pair1;

typedef struct
{
  OTF_ValueRecord Value1;
  OTF_ValueRecord Value2;
} OTF_Class2Record;

typedef struct
{
  OTF_Class2Record *Class2Record;
} OTF_Class1Record;

typedef struct
{
  unsigned ValueFormat1;
  unsigned ValueFormat2;
  OTF_ClassDef ClassDef1;
  OTF_ClassDef ClassDef2;
  unsigned Class1Count;
  unsigned Class2Count;
  OTF_Class1Record *Class1Record; /* size: <Class1Count> */
} OTF_GPOS_Pair2;

typedef struct
{
  OTF_Anchor EntryAnchor;
  OTF_Anchor ExitAnchor;
} OTF_EntryExitRecord;

typedef struct
{
  unsigned EntryExitCount;
  OTF_EntryExitRecord *EntryExitRecord;
} OTF_GPOS_Cursive1;

typedef struct
{
  OTF_Anchor *Anchor;
} OTF_AnchorRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned Count;
  OTF_AnchorRecord *AnchorRecord;
} OTF_AnchorArray;

typedef struct
{
  OTF_Coverage BaseCoverage;
  unsigned ClassCount;
  OTF_MarkArray MarkArray;
  OTF_AnchorArray BaseArray;
} OTF_GPOS_MarkBase1;

typedef struct
{
  OTF_Anchor *LigatureAnchor; /* [<ClassCount>] */
} OTF_ComponentRecord;

typedef struct
{
  OTF_Offset offset;
  unsigned ComponentCount;
  OTF_ComponentRecord *ComponentRecord; /* [<ComponentCount>] */
} OTF_LigatureAttach;

typedef struct
{
  OTF_Offset offset;
  unsigned LigatureCount;
  OTF_LigatureAttach *LigatureAttach; /* [<LiagureCount>] */
} OTF_LigatureArray;

typedef struct
{
  OTF_Coverage LigatureCoverage;
  unsigned ClassCount;
  OTF_MarkArray MarkArray;
  OTF_LigatureArray LigatureArray;
} OTF_GPOS_MarkLig1;

typedef struct
{
  OTF_Coverage Mark2Coverage;
  unsigned ClassCount;
  OTF_MarkArray Mark1Array;
  OTF_AnchorArray Mark2Array;
} OTF_GPOS_MarkMark1;

typedef OTF_Context1 OTF_GPOS_Context1;

typedef OTF_Context2 OTF_GPOS_Context2;

typedef OTF_Context3 OTF_GPOS_Context3;

typedef OTF_ChainContext1 OTF_GPOS_ChainContext1;

typedef OTF_ChainContext2 OTF_GPOS_ChainContext2;

typedef OTF_ChainContext3 OTF_GPOS_ChainContext3;

typedef struct
{
  unsigned ExtensionLookupType;
  unsigned ExtensionOffset;
  OTF_LookupSubTableGPOS *ExtensionSubtable;
} OTF_GPOS_Extension1;


struct OTF_LookupSubTableGPOS
{
  unsigned Format;
  OTF_Coverage Coverage;
  union {
    /* LookupType 1 */
    OTF_GPOS_Single1 single1;
    OTF_GPOS_Single2 single2;
    /* LookupType 2 */
    OTF_GPOS_Pair1 pair1;
    OTF_GPOS_Pair2 pair2;
    /* LookupType 3 */
    OTF_GPOS_Cursive1 cursive1;
    /* LookupType 4 */
    OTF_GPOS_MarkBase1 mark_base1;
    /* LookupType 5 */
    OTF_GPOS_MarkLig1 mark_lig1;
    /* LookupType 6 */
    OTF_GPOS_MarkMark1 mark_mark1;
    /* LookupType 7 */
    OTF_GPOS_Context1 context1;
    OTF_GPOS_Context2 context2;
    OTF_GPOS_Context3 context3;
    /* LookupType 8 */
    OTF_GPOS_ChainContext1 chain_context1;
    OTF_GPOS_ChainContext2 chain_context2;
    OTF_GPOS_ChainContext3 chain_context3;
    /* LookupType 9 */
    OTF_GPOS_Extension1 extension1;
  } u;
};

typedef OTF_GSUB_GPOS OTF_GPOS;

/*** (1-11) Structure for OTF */

typedef struct
{
  OTF_Fixed sfnt_version;
  unsigned numTables;
  unsigned searchRange;
  unsigned enterSelector;
  unsigned rangeShift;
} OTF_OffsetTable;

typedef struct
{
  OTF_Tag tag;
  char name[5];
  unsigned checkSum;
  unsigned offset;
  unsigned length;
} OTF_TableDirectory;

typedef struct OTF_InternalData  OTF_InternalData;

typedef struct
{
  char *filename;
  OTF_OffsetTable offset_table;
  OTF_TableDirectory *table_dirs;
  OTF_head *head;
  OTF_name *name;
  OTF_cmap *cmap;
  OTF_GDEF *gdef;
  OTF_GSUB *gsub;
  OTF_GPOS *gpos;
  /* The following tables are not yet supported.  */
  /* OTF_BASE *base; */
  /* OTF_JSTF *jstf; */
  OTF_InternalData *internal_data;
} OTF;

\f
/*** (2) API for reading OTF */

/*** (2-1) otf_open () */

/***
    Open OpenType font

    The OTF_open() function reads the OpenType font file whose name is
    $NAME, and return a pointer to the structure of type OTF.

    It setups these member of the structure OTF:
	filename, offset_table, table_dirs

    If the file can't be read or the file contains invalid data, NULL
    is returned, and the variable OTF_error is set to one of the
    following values.

	OTF_ERROR_MEMORY
	OTF_ERROR_FILE
	OTF_ERROR_TABLE

    See also OTF_get_table() and OTF_close().  */

extern OTF *OTF_open (const char *name);

#include <ft2build.h>
#include FT_FREETYPE_H

extern OTF *OTF_open_ft_face (FT_Face face);


/*** (2-2) OTF_close () */

/***
    Close OpenType font

    The OTF_close() function closes the OpenType font $OTF which must
    be what the OTF_open() function returned.

    See also OTF_open().  */

extern void OTF_close (OTF *otf);


/*** (2-3) OTF_get_table () */

/***
    Get OpenType font table

    The OTF_get_table() function setups one of the OTF table specified
    by $NAME in the OpenType font $OTF.

    $NAME must be one of "head", "name", "cmap", "GDEF", "GSUB", and
    "GPOS", and a member of the same name is setup.

    If the table is successfully setup, return 0.  Otherwise, return
    -1, and set the variable OTF_error to OTF_ERROR_TABLE.

    See also OTF_open().  */

extern int OTF_get_table (OTF *otf, const char *name);

/*** (2-4) OTF_check_table () */

/***
    Check the existence of OpenType font table

    The OTF_check_table() function checks if the the OTF table
    specified by $NAME exists in OpenType font $OTF.

    If the table exists, return 0, else return -1.

    See also OTF_open().  */

extern int OTF_check_table (OTF *otf, const char *name);

/*** (2-5) OTF_get_scripts () */

/***
    Get supported scripts.

    The OTF_get_scripts() function setups OTF_ScriptList of GSUB (if
    $GSUBP is nonzero) or GPOS (if $GSUBP is zero) table of the
    OpenType font $OTF.

    If the table is successfully setup, return 0.  Otherwise, retrun
    -1, and set the variable OTF_error to OTF_ERROR_TABLE.  */

extern int OTF_get_scripts (OTF *otf, int gsubp);

/*** (2-6) OTF_get_features () */

/***
    Get supported features.

    The OTF_get_features() function setups OTF_FeatureList of GSUB (if
    $GSUBP is nonzero) or GPOS (if $GSUBP is zero) table of the
    OpenType font $OTF.

    If the table is successfully setup, return 0.  Otherwise, retrun
    -1, and set the variable OTF_error to OTF_ERROR_TABLE.  */

extern int OTF_get_features (OTF *otf, int gsubp);

/*** (2-7) OTF_check_features  */

/***
    Check supported features.

    The OTF_check_features() function checks if or not the OpenType
    font $OTF has, for $SCRIPT and $LANGUAGE, all features in the
    array $FEATURES.  The array size is $N_FEATURES.  If $LANGUAGE is
    zero or $OTF doesn't have LangSys for $SCRIPT, the default LangSys
    is checked.

    If $OTF has all the features, return 1.  Otherwise, return 0.  If
    an error occurs, return -1, and set the variable OTF_error to
    OTF_ERROR_TABLE.   */

extern int OTF_check_features (OTF *otf, int gsubp,
			       OTF_Tag script, OTF_Tag language,
			       const OTF_Tag *features, int n_features);

/*** (3) API for driving OTF */

/*** (3-1) Structure for glyph string */

/***
    The structure OTF_Glyph contains information about each glyph in
    the structure OTF_GlyphString.  */

typedef struct
{
  /** The first two members must be set by a clinet before calling the
      function OTF_drive_XXX().  **/

  /* Character code of the glyph.  The value less than 32 is treated
     as a place-holder in a glyph string, and OTF_drive_XXX ()
     function just ignore the glyph as if it doesn't exist.  */
  int c;

  /* Glyph ID of the glyph.  If the value is 0, the library gets a
     correct value from the above character code via cmap if such a
     glyph is available in the font.

     The function OTF_drive_gpos2 may insert a glyph whose glyph_id is
     0 but positioning_type is positive.  It is not an actual glyph
     but just contains positioning information that should be
     accumulated to the positioning information of the previous
     glyphs.  */
  OTF_GlyphID glyph_id;

  /* GlyphClass of the glyph.  The value is extracted from the GDEF
     table.  */
  enum OTF_GlyphClassDef GlyphClass;

  /* MarkAttachClassDef of the glyph.  The value is extracted from the
     GDEF table.  */
  unsigned MarkAttachClass;

  /* Positioning format type of the glyph.  The value specifies how
     the glyph positioning information is encoded in the member <f>.
     If the value is N, the union member fN, is used.  If the value is
     zero, the glyph has no positioning information, i.e. it should be
     drawn at the normal position.  */
  int positioning_type;
  union {
    struct {
      int from, to;
    } index;
    struct {
      enum OTF_ValueFormat format;
      OTF_ValueRecord *value;
    } f1;
    struct {
      enum OTF_ValueFormat format;
      OTF_ValueRecord *value;
    } f2;
    struct {
      OTF_Anchor *entry_anchor;
      OTF_Anchor *exit_anchor;
    } f3;
    struct {
      OTF_Anchor *mark_anchor;
      OTF_Anchor *base_anchor;
    } f4;
    struct {
      OTF_Anchor *mark_anchor;
      OTF_Anchor *ligature_anchor;
    } f5;
    struct {
      OTF_Anchor *mark1_anchor;
      OTF_Anchor *mark2_anchor;
    } f6;
  } f;
} OTF_Glyph;

/***
    The structure OTF_GlyphString contains an array of glyphs (type
    OTF_Glyph).  It is used as arguments of otf_drive_XXX().  */

typedef struct
{
  /* How many glyphs are allocated at the memory pointed by the member
      <glyphs>.  */
  int size;
  /* How many glyphs contain valid information.  */
  int used;
  /* Array of glyphs.  It must be allocated by malloc().  The
     functions otf_drive_XXX() may reallocate it and increase the
     members <size> and <used>.  */
  OTF_Glyph *glyphs;
} OTF_GlyphString;


/*** (3-2) OTF_drive_cmap() */

/***
    Process glyph string by Unicode-based cmap table.

    The OTF_drive_cmap() function looks up a Unicode-based cmap table
    of OpenType font $OTF, and setup the member <glyph_id> of all
    glhphs in the glyph string $GSTRING if the value of the member is
    not zero.  */

extern int OTF_drive_cmap (OTF *otf, OTF_GlyphString *gstring);

/***
    Process glyph string by a specific cmap table.

    The OTF_drive_cmap2() function looks up a cmap table (whose
    Platform-ID is $PLATFORM_ID an Encoding-ID is $ENCODING_ID) of
    OpenType font $OTF, and setup the member <glyph_id> of all glhphs
    in the glyph string $GSTRING if the value of the member is not
    zero.  */

extern int OTF_drive_cmap2 (OTF *otf, OTF_GlyphString *gstring,
			    int platform_id, int encoding_id);


/***
    Store variable glyphs of character C in the array CODE.  The array
    size must be 256.  The Nth element of CODE is the glyph corresponding
    to the variation selector (N + 1).  The return value is the number
    of variation glyphs.  */

extern int OTF_get_variation_glyphs (OTF *otf, int c, OTF_GlyphID code[256]);


/*** (3-3) OTF_drive_gdef() */

/***
    Process glyph string by GDEF table.

    The OTF_drive_gdef() function looks up the GDEF table of OpenType
    font $OTF, and setup members <GlyphClass> and <MarkAttachClass> of
    all glhphs in the glyph string $GSTRING.  */

extern int OTF_drive_gdef (OTF *otf, OTF_GlyphString *gstring);


/*** (3-4) OTF_drive_gsub() */

/***
    Process glyph string by GSUB table.

    The OTF_drive_gsub() function looks up the GSUB table of OpenType
    font $OTF, and by using features the font has for script $SCRIPT
    and language system $LANGSYS, update member <glyphs> of the glyph
    string $GSTRING.  It may substitute, delete, insert glyphs in that
    array.  $FEATURES is a list of features to apply.  This doesn't
    perform a lookup of type 3 (Alternate Substitution).  For that,
    use OTF_drive_gsub_alternate().  */

extern int OTF_drive_gsub (OTF *otf, OTF_GlyphString *gstring,
			   const char *script, const char *language,
			   const char *features);

/*** (3-5) OTF_drive_gpos() */

/***
    Process glyph string by GPOS table.

    The OTF_drive_gpos() function is deprecated.  Use
    OTF_drive_gpos2() instread.  */

extern int OTF_drive_gpos (OTF *otf, OTF_GlyphString *gstring,
			   const char *script, const char *language,
			   const char *features);

/***
    Process glyph string by GPOS table.

    The OTF_drive_gpos2() function looks up the GPOS table of $OTF of
    OpenType font $OTF, and by using features the font has for script
    $SCRIPT and language system $LANGSYS, setup members
    <positioning_type> and <f> of all glhphs in the glyph string
    $GSTRING.  $FEATURES is a list of features to apply.  */

extern int OTF_drive_gpos2 (OTF *otf, OTF_GlyphString *gstring,
			    const char *script, const char *language,
			    const char *features);

/*** (3-6) OTF_drive_tables() */

/***
    Process glyph string by cmap, GDEF, GSUB, and GPOS tables.

    The OTF_drive_tables() function calls OTF_drive_cmap(),
    OTF_drive_gdef(), OTF_drive_gsub(), and OTF_drive_gpos() in this
    order, and update the glyphs string GSTRING.  */

extern int OTF_drive_tables (OTF *otf, OTF_GlyphString *gstring,
			     const char *script, const char *language,
			     const char *gsub_features,
			     const char *gpos_features);


/*** (3-7) OTF_get_unicode()  */

/***
    Return Unicode code point corresponding to the glyph-id CODE.
  */

extern int OTF_get_unicode (OTF *otf, OTF_GlyphID code);

/*** (3-8) OTF_drive_gsub_alternate() */

/***
    Find alternate glyphs.

    This is like OTF_drive_gsub(), but perform only a lookup of type 3
    (Alternate Substituion).  */

extern int OTF_drive_gsub_alternate (OTF *otf, OTF_GlyphString *gstring,
				     const char *script, const char *language,
				     const char *features);

/*** (4) API for error handling ***/

/*** (4-1) Error codes ***/

/***
    Global variable holding an error code.

    The variable OTF_error is set to one of OTF_ERROR_XXX macros when
    an error is detected in the OTF library.  */
extern int OTF_error;

/***
    Memory allocation error

    This error indicates that the library couldn't allocate
    memory.  */
#define OTF_ERROR_MEMORY	1

/***
    File error

    This error indicates that the library fails in opening, reading,
    or seeking an OTF file.  */
#define OTF_ERROR_FILE		2

/***
    Invalid table contents

    This error indicates that an OTF file contains invalid data.  */
#define OTF_ERROR_TABLE		3

/***
    CMAP driving error

    See the function otf_drive_cmap() for more detail.  */
#define OTF_ERROR_CMAP_DRIVE	4

/***
    GDEF driving error

    See the function OTF_drive_gdef() for more detail.  */
#define OTF_ERROR_GDEF_DRIVE	5

/***
    GSUB driving error

    See the function OTF_drive_gsub() for more detail.  */
#define OTF_ERROR_GSUB_DRIVE	6

/***
    GPOS driving error

    See the function OTF_drive_gpos() for more detail.  */
#define OTF_ERROR_GPOS_DRIVE	7

/***
    FT_Face access error.

    This error indicates that the library fails in accessing Sfnt
    tables via FT_Face.  */
#define OTF_ERROR_FT_FACE	8


/*** (4-2) OTF_perror() */

/***
    Print an OTF error message

    The OTF_perror() function produces a message on the standard error
    output, describing the last error encountered during a call to the
    OTF library function.  If $PREFIX is not NULL, it is printed
    first, followed by a colon and a blank.  Then the message and a
    newline.  */

extern void OTF_perror (const char *prefix);


/*** (5) API miscellaneous ***/

/***
    Return OTF tag of a specified name string.

    The OTF_tag() function returns OTF tag of name $NAME.  If $NAME is
    NULL, return 0.  Otherwise, $NAME must be at least 4-byte length.
    Only the first 4 characters are took into an account.  */

extern OTF_Tag OTF_tag (const char *name);

/***
    Convert OTF tag to name string.

    The OTF_tag_name() function converts OTF tag $TAG to a 5-byte name
    string (including the terminating NULL), and store it in $NAME.
    At least 5-byte space must be at $NAME.  */

extern void OTF_tag_name (OTF_Tag tag, char *name);

#ifdef __cplusplus
}
#endif

#endif /* not _OTF_H_ */

Re: Pretest next week

[Tassilo Horn]

Kenichi Handa <handa@m17n.org> writes:

> In article <87ocxovroy.fsf@thinkpad.tsdh.de>, Tassilo Horn
> <tassilo@member.fsf.org> writes:
>
>> > Strange.  Please send me your /usr/include/otf.h.
>
>> Without the comments it's only:
>
>> --8<---------------cut here---------------start------------->8---
>> #ifndef OTF_H
>> #define OTF_H
>
>> #include "OTF_Definitions.h"
>> #include "OTF_FileManager.h"
> [...]
>
> ???  How did you install it?  It's completely different from the
> header file that libotf requires (attached at the tail).

I installed it via paludis (Gentoo GNU/Linux).  I recompiled it (no
version change, still 0.9.8) and now /usr/include/otf.h is the same as
yours.

But looking at the old headers they state they're from some "Open Trace
Format" library.  I don't know how they got there.  Seems some broken
app I have to use at work had overwritten the original open type
header...

Thanks a lot for your assistance, emacs compiles again.

Bye,
Tassilo

Re: Pretest next week

[Kenichi Handa]

In article <ueiyl163e.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> > From: Chong Yidong <cyd@stupidchicken.com>
> > Date: Thu, 29 Jan 2009 10:29:01 -0500
> > 
> > Barring unforseen circumstances, I will roll the 23.0.90 pretest on
> > Sunday, the 1st of February.

> Bug #970 is still not fixed, as of today's CVS.  Is someone working on
> it?  I don't think we can release Emacs 23 with this problem.

I've just started to work on Bug #970.

---
Kenichi Handa
handa@m17n.org

Re: Pretest next week

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> CC: cyd@stupidchicken.com, emacs-devel@gnu.org
> Date: Fri, 30 Jan 2009 20:14:55 +0900
> 
> I've just started to work on Bug #970.

Thank you.

Re: Pretest next week

[Kenichi Handa]

In article <E1LSrKp-00005G-Jg@etlken>, Kenichi Handa <handa@m17n.org> writes:

> In article <ueiyl163e.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:
> > > From: Chong Yidong <cyd@stupidchicken.com>
> > > Date: Thu, 29 Jan 2009 10:29:01 -0500
> > > 
> > > Barring unforseen circumstances, I will roll the 23.0.90 pretest on
> > > Sunday, the 1st of February.

> > Bug #970 is still not fixed, as of today's CVS.  Is someone working on
> > it?  I don't think we can release Emacs 23 with this problem.

> I've just started to work on Bug #970.

I've just installed fixes.

---
Kenichi Handa
handa@m17n.org

Re: Pretest next week

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> CC: eliz@gnu.org, cyd@stupidchicken.com, emacs-devel@gnu.org
> Date: Wed, 04 Feb 2009 11:49:19 +0900
> 
> > > Bug #970 is still not fixed, as of today's CVS.  Is someone working on
> > > it?  I don't think we can release Emacs 23 with this problem.
> 
> > I've just started to work on Bug #970.
> 
> I've just installed fixes.

Thank you, I confirm that most of the problems with compositions seem
to be solved, at least in the HELLO file display.

There are still a few strange phenomena with terminal display,
although they seem unrelated to compositions.

For example, after typing "C-h H", go to the line that begins with
"CJK variety", and type "C-f": you will see that the cursor jumps past
some of the characters inside parentheses.  Is this a bug?

Re: Pretest next week

[Kenichi Handa]

In article <uljsjtvjh.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> There are still a few strange phenomena with terminal display,
> although they seem unrelated to compositions.

> For example, after typing "C-h H", go to the line that begins with
> "CJK variety", and type "C-f": you will see that the cursor jumps past
> some of the characters inside parentheses.  Is this a bug?

No.  Those CJK characters have width 2, and if they are not
supported by the terminal coding system,
encode_terminal_code produces two '?'s.

---
Kenichi Handa
handa@m17n.org

Re: Pretest next week

[Glenn Morris]

Eli Zaretskii wrote:

> Also, I think we need to start the docs reviewing process ASAP.  Emacs
> 23 changes a lot, so finding all the obsolete and/or incorrect places
> in the docs will take some time.  For past major releases, we used to
> have at least 2 different people proofread each portion of the 2 main
> manuals, so I think we need to have in admin/ some record of who
> proofreads what.

It's in admin/FOR-RELEASE (as always?).