bug#7030: 24.0.50; ns menus are all blank

bug#7030: 24.0.50; ns menus are all blank

[Derrell Piper]

[10.6.4, Xcode 3.2.4, MacOSX10.6.sdk]

Somewhere along the line, the version of 23.1 I had built for Snow
Leopard started showing me blank menus for everything.  The menu items
are present in the menu bar, but all blank.  If you click enough on a
menu item, you can usually get it to appear.  Once it does appear, it
stays for a while, but can go away later.

So I upgraded to the current trunk from bazaar and the problem's still
happening there too.  Imperically, the following patch to nsmenu.m seems
to fix this problem:

=== modified file 'src/nsmenu.m'
--- src/nsmenu.m	2010-08-11 12:34:46 +0000
+++ src/nsmenu.m	2010-09-13 19:04:37 +0000
@@ -568,18 +568,14 @@
   NSEvent *event;
   if (!FRAME_LIVE_P (frame))
     return;
+  /* Don't try this if from an event picked up asynchronously,
+     as lots of lisp evaluation happens in ns_update_menubar. */
+  if (handling_signal != 0)
+    return;
   event = [[FRAME_NS_VIEW (frame) window] currentEvent];
-  /* HACK: Cocoa/Carbon will request update on every keystroke
-     via IsMenuKeyEvent -> CheckMenusForKeyEvent.  These are not needed
-     since key equivalents are handled through emacs.
-     On Leopard, even keystroke events generate SystemDefined events, but
-     their subtype is 8. */
-  if ([event type] != NSSystemDefined || [event subtype] == 8
-      /* Also, don't try this if from an event picked up asynchronously,
-         as lots of lisp evaluation happens in ns_update_menubar. */
-      || handling_signal != 0)
-    return;
/*fprintf (stderr, "Updating menu '%s'\n", [[self title] UTF8String]); NSLog (@"%@\n", event); */
+  if ([event type] != NSApplicationDefined)
+    return;
   ns_update_menubar (frame, 1, self);
}

...however, I'm not familiar enough with this code (or with NSEvent) to
be sure that this fix is the correct one.  With this patch, the menus
all function and nothing untoward is getting logged to /var/log/system.

I have not tested this on prior versions of OS X, nor on GNUstep.


In GNU Emacs 24.0.50.14 (x86_64-apple-darwin10.4.0, NS apple-appkit-1038.32)
of 2010-09-13 on fluffy.local
Windowing system distributor `Apple', version 10.3.1038
configured using `configure  '--with-ns''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: ObjC/l

Minor modes in effect:
  tooltip-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  abbrev-mode: t

Recent input:
<backspace> <backspace> i s SPC a SPC v e r s i o n 
SPC b u i l t SPC w i t h SPC <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> C-p 
C-a C-o s-v <backspace> <backspace> C-a C-d [ C-e [ 
<backspace> ] <return> C-n C-n C-p C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-j C-j C-j C-j ESC q C-n C-e SPC SPC I t ' 
s SPC p r e s e n t SPC i n SPC t h e SPC a p p p <backspace> 
l i c a t i o n SPC m e n u S-SPC ( " E m a <backspace> 
<backspace> <backspace> <backspace> <backspace> b u 
t SPC <backspace> <backspace> <backspace> <backspace> 
a s SPC a SPC h e i r a r <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> b u t SPC <backspace> 
<backspace> <backspace> <backspace> <backspace> , SPC 
b u t SPC b l a n k . C-p C-n C-c C-c y C-p C-p C-p 
C-n C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-a C-SPC C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n <down-mouse-1> <mouse-1> 
M-x r e p r o t <backspace> <backspace> <backspace> 
o r t <tab> <return>

Recent messages:
Auto-saving...done
Mark set
No closing parenthesis found
Auto-saving...done
Send this bug report to the Emacs maintainers?  y
Sending...
Sending via mail...
Sending...done
Unable to load color "dark cyan"
Mark set

Load-path shadows:
~/src/el/custom hides /Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/custom
~/src/el/xscheme hides /Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/xscheme
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-vars hides /Users/ddp/src/el/cc-mode-5.31.3/cc-vars
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-styles hides /Users/ddp/src/el/cc-mode-5.31.3/cc-styles
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-mode hides /Users/ddp/src/el/cc-mode-5.31.3/cc-mode
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-menus hides /Users/ddp/src/el/cc-mode-5.31.3/cc-menus
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-langs hides /Users/ddp/src/el/cc-mode-5.31.3/cc-langs
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-fonts hides /Users/ddp/src/el/cc-mode-5.31.3/cc-fonts
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-engine hides /Users/ddp/src/el/cc-mode-5.31.3/cc-engine
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-defs hides /Users/ddp/src/el/cc-mode-5.31.3/cc-defs
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-compat hides /Users/ddp/src/el/cc-mode-5.31.3/cc-compat
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-cmds hides /Users/ddp/src/el/cc-mode-5.31.3/cc-cmds
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-bytecomp hides /Users/ddp/src/el/cc-mode-5.31.3/cc-bytecomp
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-awk hides /Users/ddp/src/el/cc-mode-5.31.3/cc-awk
/Users/ddp/src/trunk/nextstep/Emacs.app/Contents/Resources/lisp/progmodes/cc-align hides /Users/ddp/src/el/cc-mode-5.31.3/cc-align

Features:
(browse-url mailalias mailclient sendmail newcomment help-mode view
shadow sort gnus-util mail-extr message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader emacsbug
vc-git ebuff-menu electric multi-isearch vc-bzr sha1 hex-util cc-mode
cc-fonts cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
regexp-opt finder-inf package simple-wiki derived scheme-complete
git-blame git log-edit easy-mmode ring pcvs-util ewoc add-log avoid
server paren mic-paren cl cl-19 tooltip ediff-hook vc-hooks
lisp-float-type mwheel ns-win easymenu tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mldrag mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev loaddefs button minibuffer faces cus-face files text-properties
overlay md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process ns multi-tty
emacs)

bug#7030: update

[Derrell Piper]

I take it back, I seem to be crashing randomly with that fix in place (though the menus are now working reliably).  Investigation continues.

bug#7030:

[Derrell Piper]

More information...  I ultimately gave up trying to debug this because I've been living overseas and don't have sufficient bandwidth to download XCode.

This first came to light concurrent with upgrading to Snow Leopard (10.6), which happened when I upgraded my MacBook Air hardware.  This didn't seem to happen prior to 10.6 and FWIW, I was a beta tester of Adrian's nextstep branch prior to it getting checked into the GNU Emacs trunk.

I've lived with this problem for the last year and a half and finally have a little while to debug it some more.  First, I'm now on 10.7.2 on a MacBook Air (late 2010) w/ 4G and 256G SSD.  The problem is not related to my .emacs initialization file or any per-user or per-system customization, as far as I can determine from using DTrace 'opensnoop' and nulling everything out.  It does, however, seem to be related to my personal OS X environment, somehow.

I run a number of items at Login:

	Speach Events
	SpeachSynthesisServer
	Livescribe AutoLaunch
	Livescribe Connect AutoLaunch
	CoverSutra (2.2.2, pre-AppStore)			http://sophiestication.com/coversutra/
	iTunes
	FFHelperApp (2.2)						http://kevingessner.com/software/functionflip/
	Radium (2.8.3, pre-AppStore)				http://www.catpigstudios.com/

Observations:

1) if I create a new unprivileged test account and run Emacs out of there, it works
2) if I remove FFHelperApp (FunctionFlip.prefPane) *and* Radium from my Login items, it works
3) if I add Radium to the test account Login items, it works there
4) if I also add FunctionFlip.prefPane to the test account Login Items, it works there
5) if I put either Radium or FunctionFlip.prefPane back on my account, it fails when the Login Items fire; until then, it works
6) when it does fail, the Application and Help menus are always valid (possibly because they're baked into main application NIB?)
7) this happens on 23.n as well as the latest 24 nightly -- 24.0.92.1 (x86_64-apple-darwin, NS apple-appkit-1038.36) of 2011-12-02 on bob.porkrind.org
8) it works under Aquamacs (which is based on the same nextstep code), even with Radium and FunctionFlip present

I have been running without Radium and FunctionFlip for the last 24 hours or so and it has not failed since.

Thoughts:

I believe there was some controversy about inserting items into the OS X menu bar, circa Leopard or so, but it's a hard to Google this because of the noise using "crack" as a search term.  I could believe that FunctionFlip and Radium possibly share the same inherently buggy menu cracker, which is perhaps triggering a bug in how the dynamic menu code is functioning.  It's almost like it's a caching problem when it's happening because once you get the menu to drop, it's there for "a while."  In fact, if you keep flailing on a menu or two, even when Radium comes up, the menus you're flailing on often stay valid, while the others that you're ignoring go blank.  Or perhaps, the nextstep code simply has always had a day one bug that's just happening more often since 10.6.  A Google search for "emacs blank menus os x" shows that I'm not alone in seeing this problem, see also 8249 & 9206.

Wish I could be more help.  I might try doing some forensic analysis on Radium and FunctionFlip and see if I can find anything in common in their binaries.

bug#7030: more info

[Jan Djärv]

Hello.

11 dec 2011 kl. 20:18 skrev Derrell Piper:

> 8) it works under Aquamacs (which is based on the same nextstep code), even with Radium and FunctionFlip present
> 

I've included the relevant code from Aquamacs, please try it.

	Jan D.

bug#7030: more info

[Glenn Morris]

Jan Djärv wrote:

> I've included the relevant code from Aquamacs, please try it.

Do we know who the author of that code is?

bug#7030: more info

[Jan Djärv]

19 dec 2011 kl. 01:39 skrev Glenn Morris:

> Jan Djärv wrote:
> 
>> I've included the relevant code from Aquamacs, please try it.
> 
> Do we know who the author of that code is?

No, but the files I took it from says Copyright Free Software Foundation so I assumed it was OK, maybe that was wrong.  The snippets are so short they should be considered tiny changes.

	Jan D.

bug#7030: more info

[Stefan Monnier]

>>> I've included the relevant code from Aquamacs, please try it.
>> Do we know who the author of that code is?
> No, but the files I took it from says Copyright Free Software Foundation so
> I assumed it was OK, maybe that was wrong.  The snippets are so short they
> should be considered tiny changes.

Could you please try and figure out who they come from?  If they're
"tiny changes" it's important to know who they come from, because tiny
changes from the same author can accumulate to something non-tiny.


        Stefan

bug#7030: more info

[Jan Djärv]

19 dec 2011 kl. 12:30 skrev Stefan Monnier:

>>>> I've included the relevant code from Aquamacs, please try it.
>>> Do we know who the author of that code is?
>> No, but the files I took it from says Copyright Free Software Foundation so
>> I assumed it was OK, maybe that was wrong.  The snippets are so short they
>> should be considered tiny changes.
> 
> Could you please try and figure out who they come from?  If they're
> "tiny changes" it's important to know who they come from, because tiny
> changes from the same author can accumulate to something non-tiny.
> 

The git commit is done by David Reitter.  He is in copyright.list for Emacs.

	Jan D.

bug#7030: more info

[Glenn Morris]

Jan Djärv wrote:

> 19 dec 2011 kl. 12:30 skrev Stefan Monnier:
>
>>>>> I've included the relevant code from Aquamacs, please try it.
>>>> Do we know who the author of that code is?
>>> No, but the files I took it from says Copyright Free Software Foundation so
>>> I assumed it was OK, maybe that was wrong.  The snippets are so short they
>>> should be considered tiny changes.
>> 
>> Could you please try and figure out who they come from?  If they're
>> "tiny changes" it's important to know who they come from, because tiny
>> changes from the same author can accumulate to something non-tiny.
>
> The git commit is done by David Reitter.  He is in copyright.list for Emacs.

People who do the commits aren't always the author of the code.
I'm cc'ing David so he can confirm.
Could you adjust the ChangeLog entry so that the relevant pieces are in
his (or whoever's) name?
Thanks, and sorry for the extra bother.

bug#7030: more info

[David Reitter]

On Dec 19, 2011, at 8:38 PM, Glenn Morris wrote:
>> 
>> The git commit is done by David Reitter.  He is in copyright.list for Emacs.
> 
> People who do the commits aren't always the author of the code.
> I'm cc'ing David so he can confirm.
> Could you adjust the ChangeLog entry so that the relevant pieces are in
> his (or whoever's) name?
> Thanks, and sorry for the extra bother.

Yep - though generally, just because someone commits to Emacs doesn't mean they sign over all Emacs code they write (elsewhere) to the FSF, as far as I remember.  In this case however this is perfectly fine.

By the way, it's "Aquamacs", not "AquaEmacs".

- David

bug#7030: more info

[Jan D.]

Hello.

David Reitter skrev 2011-12-20 00:50:
> On Dec 19, 2011, at 8:38 PM, Glenn Morris wrote:
>>>
>>> The git commit is done by David Reitter.  He is in copyright.list
>>> for Emacs.
>>
>> People who do the commits aren't always the author of the code. I'm
>> cc'ing David so he can confirm. Could you adjust the ChangeLog
>> entry so that the relevant pieces are in his (or whoever's) name?
>> Thanks, and sorry for the extra bother.
>
> Yep - though generally, just because someone commits to Emacs doesn't
> mean they sign over all Emacs code they write (elsewhere) to the FSF,
> as far as I remember.  In this case however this is perfectly fine.

I'm not a copyrights expert, but if some parts aren't assigned to FSF, 
should not the files (nsmenu.m, nsterm.m in this case) have another 
Copyright statement besides the one for FSF?

>
> By the way, it's "Aquamacs", not "AquaEmacs".
>

I fixed it in the next commit, sorry for that.

	Jan D.

bug#7030: more info

[David Reitter]

On Dec 20, 2011, at 7:49 AM, Jan D. wrote:
> 
> I'm not a copyrights expert, but if some parts aren't assigned to FSF, should not the files (nsmenu.m, nsterm.m in this case) have another Copyright statement besides the one for FSF?

It may make sense, though I think that one can hold copyright without asserting it explicitly.  I think that may depend on jurisdiction. 

bug#7030: seems fixed in trunk

[Derrell Piper]

I pulled the trunk and rebuilt the world from scratch and with the latest changes, the problem no longer occurs.  It's fixed as far as I can tell.

Derrell

bug#7030: seems fixed in trunk

[Jan Djärv]

Hello.

21 dec 2011 kl. 00:39 skrev Derrell Piper:

> I pulled the trunk and rebuilt the world from scratch and with the latest changes, the problem no longer occurs.  It's fixed as far as I can tell.

Ok, closing this report.

Thanks for testing.

	Jan D.