bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Artemio González López]

[-- Attachment #1: Type: text/plain, Size: 4276 bytes --]


Emacs 26.1 menus don’t work correctly in macOS 10.14 Mojave (just released this week). More, precisely, to make a menu drop down you have to click twice on the corresponding menu title (except for the Emacs menu!). If you just click once nothing happens, but if you click a second time on a different menu that menu drops down.


In GNU Emacs 26.1 (build 1, x86_64-apple-darwin14.5.0, NS appkit-1348.17 Version 10.10.5 (Build 14F2511))
of 2018-05-31 built on builder10-10.porkrind.org
Windowing system distributor 'Apple', version 10.3.1671
Recent messages:
Ispell process killed
Starting new Ispell process aspell with castellano dictionary...
Applying style hooks...
Loading /Users/artemio/Documents/Coursework/Mecanica Clasica/MC 18-19/Apuntes/auto/chap2-1.el (source)...done
Sorting amsthm-newtheorem...done
Removing duplicates...done
Applying style hooks...done
Compiling label environment definitions...done
Sorting environment...done
Removing duplicates...done

Configured using:
'configure --with-ns '--enable-locallisppath=/Library/Application
Support/Emacs/${version}/site-lisp:/Library/Application
Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS

Important settings:
  value of $LANG: en_US@currency=EUR.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  shell-dirtrack-mode: t
  show-paren-mode: t
  delete-selection-mode: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec epa derived epg gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils reftex-parse texmathp preview prv-emacs
reftex-dcr reftex-auc reftex reftex-loaddefs reftex-vars bib-cite
flyspell ispell tex-bar toolbar-x noutline outline tex-buf font-latex
latex latex-flymake flymake-proc flymake warnings thingatpt tex-ispell
tex-style tex crm advice tex-mode compile shell pcomplete comint
ansi-color ring latexenc elec-pair paren delsel cus-start cus-load
edmacro kmacro tabbar easy-mmode session cl exec-path-from-shell
finder-inf info tex-site package easymenu epg-config url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache url-vars seq byte-opt gv bytecomp byte-compile cconv
cl-loaddefs cl-lib server time-date tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 353007 11527)
(symbols 48 30848 1)
(miscs 40 124 405)
(strings 32 64041 1865)
(string-bytes 1 1753106)
(vectors 16 45257)
(vector-slots 8 850451 24934)
(floats 8 199 309)
(intervals 56 1486 189)
(buffers 992 15))


Artemio Gonzalez Lopez
artemiog@mac.com


[-- Attachment #2: Type: text/html, Size: 7463 bytes --]

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Fri, Sep 28, 2018 at 07:40:31PM +0200, Artemio González López wrote:
> 
> Emacs 26.1 menus don’t work correctly in macOS 10.14 Mojave (just
> released this week). More, precisely, to make a menu drop down you
> have to click twice on the corresponding menu title (except for the
> Emacs menu!). If you just click once nothing happens, but if you
> click a second time on a different menu that menu drops down.
> 
> 
> In GNU Emacs 26.1 (build 1, x86_64-apple-darwin14.5.0, NS
> appkit-1348.17 Version 10.10.5 (Build 14F2511)) of 2018-05-31 built
> on builder10-10.porkrind.org

Thanks for the report. I wonder if this is specific to the
emacsformacosx.com builds or if a native 10.14 build would do the same
thing...?

Is there any chance you could build emacs 26 yourself to check? Or if
anyone else with 10.14 can confirm, that would be great.

The NS menus seem to be a bit kludgy, so they’re probably due for a
bit of refactoring anyway.
-- 
Alan Third

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Fri, Sep 28, 2018 at 09:43:55PM +0200, Artemio González López wrote:
> 
> > On Sep 28, 2018, at 9:40 PM, Alan Third <alan@idiocy.org> wrote:
> > 
> > On Fri, Sep 28, 2018 at 07:40:31PM +0200, Artemio González López wrote:
> >> 
> >> Emacs 26.1 menus don’t work correctly in macOS 10.14 Mojave (just
> >> released this week). More, precisely, to make a menu drop down you
> >> have to click twice on the corresponding menu title (except for the
> >> Emacs menu!). If you just click once nothing happens, but if you
> >> click a second time on a different menu that menu drops down.
> >> 
> >> 
> >> In GNU Emacs 26.1 (build 1, x86_64-apple-darwin14.5.0, NS
> >> appkit-1348.17 Version 10.10.5 (Build 14F2511)) of 2018-05-31 built
> >> on builder10-10.porkrind.org
> > 
> > Thanks for the report. I wonder if this is specific to the
> > emacsformacosx.com builds or if a native 10.14 build would do the same
> > thing...?
> > 
> > Is there any chance you could build emacs 26 yourself to check? Or if
> > anyone else with 10.14 can confirm, that would be great.
> > 
> > The NS menus seem to be a bit kludgy, so they’re probably due for a
> > bit of refactoring anyway.
> 
> Hi, Alan,
> 
> I’ll try to build Emacs.app myself. In the meantime, I can confirm
> that 1) I’ve had the problem with at least two builds of Emacs
> (Macport’s and Emacs for Mac OS X), and 2) a colleague of mine at
> work who just installed Mojave has had the same problem with the
> same builds of Emacs.

Thanks. Let know how it goes.

BTW, please leave the bug tracker email address in so your email
doesn’t get lost.
-- 
Alan Third

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Artemio González López]

[-- Attachment #1: Type: text/plain, Size: 2367 bytes --]



> On Sep 28, 2018, at 9:49 PM, Alan Third <alan@idiocy.org> wrote:
> 
> On Fri, Sep 28, 2018 at 09:43:55PM +0200, Artemio González López wrote:
>> 
>>> On Sep 28, 2018, at 9:40 PM, Alan Third <alan@idiocy.org> wrote:
>>> 
>>> On Fri, Sep 28, 2018 at 07:40:31PM +0200, Artemio González López wrote:
>>>> 
>>>> Emacs 26.1 menus don’t work correctly in macOS 10.14 Mojave (just
>>>> released this week). More, precisely, to make a menu drop down you
>>>> have to click twice on the corresponding menu title (except for the
>>>> Emacs menu!). If you just click once nothing happens, but if you
>>>> click a second time on a different menu that menu drops down.
>>>> 
>>>> 
>>>> In GNU Emacs 26.1 (build 1, x86_64-apple-darwin14.5.0, NS
>>>> appkit-1348.17 Version 10.10.5 (Build 14F2511)) of 2018-05-31 built
>>>> on builder10-10.porkrind.org
>>> 
>>> Thanks for the report. I wonder if this is specific to the
>>> emacsformacosx.com builds or if a native 10.14 build would do the same
>>> thing...?
>>> 
>>> Is there any chance you could build emacs 26 yourself to check? Or if
>>> anyone else with 10.14 can confirm, that would be great.
>>> 
>>> The NS menus seem to be a bit kludgy, so they’re probably due for a
>>> bit of refactoring anyway.
>> 
>> Hi, Alan,
>> 
>> I’ll try to build Emacs.app myself. In the meantime, I can confirm
>> that 1) I’ve had the problem with at least two builds of Emacs
>> (Macport’s and Emacs for Mac OS X), and 2) a colleague of mine at
>> work who just installed Mojave has had the same problem with the
>> same builds of Emacs.
> 
> Thanks. Let know how it goes.
> 
> BTW, please leave the bug tracker email address in so your email
> doesn’t get lost.
> -- 
> Alan Third

Hi, Alan,

I just compiled Emacs.app 26.1 on my own, and it has exactly the same problem. To be more precise, what seems to happen is that the first click on a menu title does nothing, and the second one drops the menu down. For instance, if you click on the File menu nothing happens, but if you then click on the Buffer menu it drops down normally. Thus, clicking twice on a menu drops it down. Strangely enough, the Emacs menu is an exception, since it works correctly (drops down after one click).

Thanks a lot for your help,

Artemio


Artemio Gonzalez Lopez
artemiog@mac.com


[-- Attachment #2: Type: text/html, Size: 8978 bytes --]

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Mon, Oct 01, 2018 at 03:12:59PM +0200, Artemio González López wrote:
> 
> I just compiled Emacs.app 26.1 on my own, and it has exactly the
> same problem. To be more precise, what seems to happen is that the
> first click on a menu title does nothing, and the second one drops
> the menu down. For instance, if you click on the File menu nothing
> happens, but if you then click on the Buffer menu it drops down
> normally. Thus, clicking twice on a menu drops it down. Strangely
> enough, the Emacs menu is an exception, since it works correctly
> (drops down after one click).

Hmm, that doesn’t surprise me a whole lot. IIRC the Emacs menu is
different from the others as it’s not built from elisp, it’s
hard‐coded. I suspect what’s happening is that when you click a menu
the first time it is ‘rebuilt’, and in old versions of macOS it then
opened, however for whatever reason it’s just rebuilding and not
opening in Mojave. The second click doesn’t need to rebuild it because
it’s not ‘changed’, so it just opens.

I’ve no idea why the menus are handled this way. Perhaps it’s normal,
but it seems odd to me. I’d think you’d build the whole menu when it
changed rather than when you try to open them. Perhaps it’s a
performance enhancement.
-- 
Alan Third

bug#32864: Run from Terminal

[Omari Norman]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

I had this same problem using either Emacs 26 from emacsforosx.com or Emacs
27 compiled using MacPorts.

By happenstance I discovered this problem does not exist if I run Emacs
from the Terminal (by running the executable).  The problem does exist if I
use "open" from the command line, if I run using Spotlight, or if I run it
from the Finder.

[-- Attachment #2: Type: text/html, Size: 439 bytes --]

bug#32864: Run from Terminal

[Omari Norman]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

To clarify, when I run from the Terminal, it still launches GUI Emacs (not
terminal Emacs.)

On Sat, Nov 17, 2018 at 12:15 PM Omari Norman <omari@smileystation.com>
wrote:

> I had this same problem using either Emacs 26 from emacsforosx.com or
> Emacs 27 compiled using MacPorts.
>
> By happenstance I discovered this problem does not exist if I run Emacs
> from the Terminal (by running the executable).  The problem does exist if I
> use "open" from the command line, if I run using Spotlight, or if I run it
> from the Finder.
>

[-- Attachment #2: Type: text/html, Size: 873 bytes --]

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[simonjgeorge]

Interestingly this bug disappears when you enable Emacs in Accessibility
under Security and Privacy in System Preferences.



--
Sent from: http://emacs.1067599.n8.nabble.com/Emacs-Bugs-f3.html

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Fri, Feb 08, 2019 at 01:15:35AM -0700, simonjgeorge wrote:
> Interestingly this bug disappears when you enable Emacs in Accessibility
> under Security and Privacy in System Preferences.

That’s interesting. I wonder if Emacs is using some accessibility API
to do its menu opening deferral.
-- 
Alan Third

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Robert Pluim]

Alan Third <alan@idiocy.org> writes:

> On Fri, Feb 08, 2019 at 01:15:35AM -0700, simonjgeorge wrote:
>> Interestingly this bug disappears when you enable Emacs in Accessibility
>> under Security and Privacy in System Preferences.
>
> That’s interesting. I wonder if Emacs is using some accessibility API
> to do its menu opening deferral.

This is emacs-26.1 built on 10.10, but running on 10.14, right? I
donʼt see such problems with my built on 10.14 26.1 (and I donʼt have
Emacs in the Accessibility settings).

Robert

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Mattias Engdegård]

Confirmed in 27.0 (master), built and run on Mojave (10.14.5).

The erroneous behaviour is there even if Emacs is started from a shell. Very annoying.

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Mattias Engdegård]

Bug explained:

When the user clicks on the menu bar, Emacs receives a menuWillOpen: message, and immediately cancels the menu by sending cancelTracking. It then returns from the event loop to rebuild the menu, but first synthesises a mouse click on the menu bar in the hope that this will make the menu actually open.

In MacOS Mojave, synthetic mouse events are blocked for security reasons, so this no longer works; the synthetic click is discarded and the menu doesn't open. When the user clicks on the menu bar a second time, Emacs believes it's the synthetic click that was acted upon and happily allows the menu to open.

So why does Emacs do this to begin with? Because the menu contents are generated dynamically from elisp code each time. The standard way to do this in Cocoa is to implement menuNeedsUpdate:, but this requires running arbitrary elisp code inside the event loop, which is undesirable for some reason.

The workarounds mentioned involved adding Emacs to some sort of whitelist for legacy applications, but this cannot be a solution. The synthetic mouse click hack must go away.

Could someone explain why, exactly, elisp code cannot be run inside the event loop? An alternative would be to run elisp code in a different thread, and let menuNeedsUpdate: block until the menu has been updated. I'm not sure what the difference would be.

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Eli Zaretskii]

> From: Mattias Engdegård <mattiase@acm.org>
> Date: Mon, 3 Jun 2019 19:25:27 +0200
> Cc: alan@idiocy.org, omari@smileystation.com, artemiog@mac.com,
> 	Robert Pluim <rpluim@gmail.com>, simon@simonscientific.com
> 
> Could someone explain why, exactly, elisp code cannot be run inside
> the event loop?

Because it's a different thread from the main one, where we run Lisp?
(I know nothing about macOS, so apologies if this makes no sense.)

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Mattias Engdegård]

3 juni 2019 kl. 20.20 skrev Eli Zaretskii <eliz@gnu.org>:
>> 
>> Could someone explain why, exactly, elisp code cannot be run inside
>> the event loop?
> 
> Because it's a different thread from the main one, where we run Lisp?
> (I know nothing about macOS, so apologies if this makes no sense.)

I thought so at first, but some printf debugging indicated that the main thread runs both lisp and the event loop. Perhaps there are circumstances where this isn't true.

Most likely the fake-menu-click system was inherited from the X11 back-end.
However, the win32 back-end seems to use distinct threads for lisp and event handling, perhaps out of necessity.

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Mon, Jun 03, 2019 at 07:25:27PM +0200, Mattias Engdegård wrote:
> 
> So why does Emacs do this to begin with? Because the menu contents
> are generated dynamically from elisp code each time. The standard
> way to do this in Cocoa is to implement menuNeedsUpdate:, but this
> requires running arbitrary elisp code inside the event loop, which
> is undesirable for some reason.
> 
> The workarounds mentioned involved adding Emacs to some sort of
> whitelist for legacy applications, but this cannot be a solution.
> The synthetic mouse click hack must go away.
> 
> Could someone explain why, exactly, elisp code cannot be run inside
> the event loop? An alternative would be to run elisp code in a
> different thread, and let menuNeedsUpdate: block until the menu has
> been updated. I'm not sure what the difference would be.

Elisp code doesn’t guarantee it will return, it can longjmp when you
hit C-g, for example. This means you can end up with the application
attempting to run the event loop while it is still INSIDE the previous
event loop, and the toolkit really doesn’t like that. It will, in
fact, kill Emacs on the spot.

It may also be the case that Emacs can try to run the event loop from
within elisp code as a matter of course.

Quite simply we’re, as you said, not able to handle running elisp from
within the NS event loop.

I’m not sure why it was written this way originally, I believe the NS
port is some twenty five years old now and I’ve only been working on
Emacs for three.

The best solution is, as you said, to separate the lisp and toolkit
calls into separate threads, but unfortunately it’s not a straight
forward job. I want to do it anyway as there are other benefits, but
it won’t be happening soon unless someone else wants to pick it up.

The other solution I found is to rebuild the menu completely whenever
lisp updates it. This is simple enough to do but rebuilding the menus
takes something like 40‐70ms every time, as opposed to 1‐2ms to just
rebuild the top level, and it can do it up to three times per
keypress. I think it may also do it sometimes while scrolling. It
didn’t seem like a good idea to me. On the other hand I don’t remember
actually having much trouble with it.

If anyone has any other ideas I’d be happy to hear them.
-- 
Alan Third

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[npostavs]

Alan Third <alan@idiocy.org> writes:

> Elisp code doesn’t guarantee it will return, it can longjmp when you
> hit C-g, for example. This means you can end up with the application
> attempting to run the event loop while it is still INSIDE the previous
> event loop, and the toolkit really doesn’t like that. It will, in
> fact, kill Emacs on the spot.

> If anyone has any other ideas I’d be happy to hear them.

Would using safe_call be enough to allow calling Elisp in this
situation?

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Mattias Engdegård]

4 juni 2019 kl. 18.44 skrev Alan Third <alan@idiocy.org>:
> 
> It may also be the case that Emacs can try to run the event loop from
> within elisp code as a matter of course.

I'm no Cocoa expert, but the docs explicitly state that recursively entering an event loop from an event handler is allowed.

> The other solution I found is to rebuild the menu completely whenever
> lisp updates it. This is simple enough to do but rebuilding the menus
> takes something like 40‐70ms every time, as opposed to 1‐2ms to just
> rebuild the top level, and it can do it up to three times per
> keypress. I think it may also do it sometimes while scrolling. It
> didn’t seem like a good idea to me. On the other hand I don’t remember
> actually having much trouble with it.

Is the entire menu rebuilt every time some part of it changes, or are the changes segregated by drop-down menu?
The Buffers menu probably sees a lot of traffic; it seems to be updated from menu-bar-update-hook, which fires a lot, even though most of the time the buffer list doesn't actually change.

It's probably a bad idea to add a latency of 40-70 ms several times per key press in any case.

> If anyone has any other ideas I’d be happy to hear them.

The emacs-mac port, which seems to be an AppKit/Carbon hybrid (?), does not exhibit this menu glitch. I'm not sure exactly how it does this, but the general approach looks roughly similar. Maybe we could ask its author for advice.

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Alan Third]

On Tue, Jun 04, 2019 at 11:08:10PM +0200, Mattias Engdegård wrote:
> 4 juni 2019 kl. 18.44 skrev Alan Third <alan@idiocy.org>:
> > 
> > It may also be the case that Emacs can try to run the event loop from
> > within elisp code as a matter of course.
> 
> I'm no Cocoa expert, but the docs explicitly state that recursively entering an event loop from an event handler is allowed.

Unfortunately I too am no expert. Perhaps I’ve misunderstood what was
going on.

> > The other solution I found is to rebuild the menu completely whenever
> > lisp updates it. This is simple enough to do but rebuilding the menus
> > takes something like 40‐70ms every time, as opposed to 1‐2ms to just
> > rebuild the top level, and it can do it up to three times per
> > keypress. I think it may also do it sometimes while scrolling. It
> > didn’t seem like a good idea to me. On the other hand I don’t remember
> > actually having much trouble with it.
> 
> Is the entire menu rebuilt every time some part of it changes, or
> are the changes segregated by drop-down menu? The Buffers menu
> probably sees a lot of traffic; it seems to be updated from
> menu-bar-update-hook, which fires a lot, even though most of the
> time the buffer list doesn't actually change.

IIRC set_frame_menubar is called with deep_p set to false and this
calls ns_update_menubar just recreating the top level.

When you click on a menu it does all the stuff you described
previously which ends up running ns_update_menubar with deep_p set to
true and this rebuilds the menu that was clicked on. I think. I don’t
think it rebuilds all menus.

> > If anyone has any other ideas I’d be happy to hear them.
> 
> The emacs-mac port, which seems to be an AppKit/Carbon hybrid (?),
> does not exhibit this menu glitch. I'm not sure exactly how it does
> this, but the general approach looks roughly similar. Maybe we could
> ask its author for advice.

Seems like a good idea.

Yamamoto‐san, I hope it’s OK to pull you into this discussion. Do you
have any thoughts on the issue described in:

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=32864#38

-- 
Alan Third

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[Tak Kunihiro]

I think I found workaround to have menu by single click.
Please ignore this message if this does not make sense.

> https://qiita.com/takaxp/items/2a0abaa6e5f1a7a9c440

1. Launch Keychain.app and create a certificate "StrayBuild"
2. sudo codesign --force --deep --sign "StrayBuild" Emacs.app

bug#32864: 26.1; menus don't work correctly in Mac OS Mojave

[pierrea]

I have just build GNU Emacs 28.0.50 on Mac OS X 10.11.6 El Capitan and the
Emacs Menu drop down menus don't work (even after clicking many times etc.).

Yet I have GNU Emacs 26.3 from emacsformacosx.com working perfectly.

I used this 

mac2011% ./configure --with-mailutils --with-gnutls=ifavailable --with-ns

I guess there is something with NSWindows.






--
Sent from: http://emacs.1067599.n8.nabble.com/Emacs-Bugs-f3.html

bug#32864: menus don't work in Mac OS: see similar issue 24472 and workaround there

[Marc Herbert]

> So why does Emacs do this to begin with? Because the menu contents are generated dynamically from elisp code each time.
> ...
> IIRC the Emacs menu is different from the others as it’s not built from elisp, it’s hard‐coded.

See similar issue https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24472
and successful, one-line .emacs workaround there.