bug#39890: 26.3; do-applescript undeterministic with long running scripts

bug#39890: 26.3; do-applescript undeterministic with long running scripts

[Alexander Adolf]

Dear Emacs Maintainers,

I use this function to query the Contacts app (which is part of macOS)
for names and email addresses via AppleScript:

----------------------------------------------------------------------
(defun eudc-macos-contacts-search-helper (str)
  "Helper function to query the Contacts app via AppleScript.
Searches for all persons with a case-insensitive substring match
of STR in any of their name fileds (first, middle, or last)."
  (insert (do-applescript (format "
set results to {}
tell application \"Contacts\"
	set pList to every person whose (name contains \"%s\")
	repeat with pers in pList
		repeat with emailAddr in emails of pers
			set results to results & {name of pers & \":\" & value of emailAddr & \"\n\"}
		end repeat
	end repeat
	get results as text
end tell" str))))
----------------------------------------------------------------------

This is part of a completion back-end.

When I do a query which has several hundred (or more) results, the
behaviour of (do-applescript) becomes unpredictable. Sometimes I get the
expected results, and sometimes I get an "AppleScript error
1". Unfortunately I have not been able to discern a usage pattern which
would reliably reproduce the error message. It never occurs for quick
queries (a couple dozen results). The longer the query runs, the more
likely the error message.

Such a big query (hundreds to thousands of results) will take 3 to 5
minutes to complete (AppleScript is not among the fastest script
languages...). After launching the query, Emacs becomes unresponsive,
but which is to be expected. After a few seconds, the mouse cursor
changes to the "spinning beach-ball" for some 20-ish seconds, and then
goes back to the arrow shape. During the query running, the cursor
changes shape forth and back in this way one or two more times. While
the query is running, I can bring Emacs frames to the front, move them
around, etc., but the buttons on the title bar for minimising, closing,
etc. are not active.

As a work-around, I have altered my function to use (call-process
"osascript") instead:

----------------------------------------------------------------------
(defun eudc-macos-contacts-search-helper (str)
  "Helper function to query the Contacts app via AppleScript.
Searches for all persons with a case-insensitive substring match
of STR in any of their name fileds (first, middle, or last)."
  (insert (do-applescript (format "
  (if (executable-find "osascript")
      (call-process "osascript" nil t nil
		    "-e"
		    (format "
set results to {}
tell application \"Contacts\"
	set pList to every person whose (name contains \"%s\")
	repeat with pers in pList
		repeat with emailAddr in emails of pers
			set results to results & {name of pers & \":\" & value of emailAddr & \"\n\"}
		end repeat
	end repeat
	get results as text
end tell" str))
    (message "[eudc] Error in macOS Contacts: `osascript' executable not found")))
----------------------------------------------------------------------

This turns out to be super reliable, and always delivers the results as
expected, even for thousands of them. As a positive (IMO) side-effect,
there is never any "spinning beach-ball" mouse pointer (which
implies that Emacs never gets flagged as "unresponsive" by macOS).


Many thanks and looking forward to your thoughts,

  --alexander


In GNU Emacs 26.3 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109))
 of 2019-09-02 built on builder10-14.porkrind.org
Windowing system distributor 'Apple', version 10.3.1894
Recent messages:
Mark set [4 times]
next-line: End of buffer [4 times]
Mark set
Auto-saving...done
Buffer eudc.el.gz modified; kill anyway? (y or n) y
mwheel-scroll: Beginning of buffer [5 times]
Quit [2 times]
Mark set
Quit [2 times]
mwheel-scroll: Beginning of buffer [26 times]

Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS

Important settings:
  value of $LC_ALL: C
  value of $LANG: en_DE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Message[Notmuch]

Minor modes in effect:
  flyspell-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-magit-file-mode: t
  diff-auto-refine-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  mml-mode: t
  async-bytecomp-package-mode: t
  global-company-mode: t
  company-mode: t
  modern-c++-font-lock-global-mode: t
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  shell-dirtrack-mode: t
  ivy-rich-mode: t
  ivy-mode: t
  recentf-mode: t
  persistent-scratch-autosave-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  show-paren-mode: t
  server-mode: t
  global-hl-line-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  auto-fill-function: yas--auto-fill
  visual-line-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/usr/local/share/emacs/site-lisp/cmake/cmake-mode hides /Users/alexanderadolf/.emacs.d/elpa/cmake-mode-20190710.1319/cmake-mode
/usr/local/share/emacs/site-lisp/notmuch/notmuch-mua hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-mua
/usr/local/share/emacs/site-lisp/notmuch/notmuch-tree hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-tree
/usr/local/share/emacs/site-lisp/notmuch/notmuch-show hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-show
/usr/local/share/emacs/site-lisp/notmuch/notmuch-tag hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-tag
/usr/local/share/emacs/site-lisp/notmuch/coolj hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/coolj
/usr/local/share/emacs/site-lisp/notmuch/notmuch-wash hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-wash
/usr/local/share/emacs/site-lisp/notmuch/notmuch-maildir-fcc hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-maildir-fcc
/usr/local/share/emacs/site-lisp/notmuch/notmuch-compat hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-compat
/usr/local/share/emacs/site-lisp/notmuch/notmuch hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch
/usr/local/share/emacs/site-lisp/notmuch/notmuch-address hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-address
/usr/local/share/emacs/site-lisp/notmuch/notmuch-lib hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-lib
/usr/local/share/emacs/site-lisp/notmuch/notmuch-print hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-print
/usr/local/share/emacs/site-lisp/notmuch/notmuch-draft hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-draft
/usr/local/share/emacs/site-lisp/notmuch/notmuch-jump hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-jump
/usr/local/share/emacs/site-lisp/notmuch/notmuch-parser hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-parser
/usr/local/share/emacs/site-lisp/notmuch/notmuch-query hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-query
/usr/local/share/emacs/site-lisp/notmuch/notmuch-message hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-message
/usr/local/share/emacs/site-lisp/notmuch/notmuch-hello hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-hello
/usr/local/share/emacs/site-lisp/notmuch/notmuch-crypto hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-crypto
/usr/local/share/emacs/site-lisp/notmuch/notmuch-company hides /Users/alexanderadolf/.emacs.d/elpa/notmuch-20200109.114/notmuch-company

Features:
(shadow emacsbug sendmail rect cl-print ffap tramp tramp-compat
tramp-loaddefs trampver sort mail-extr vc-mtn vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs vc vc-dispatcher magit-bookmark bookmark pp
eieio-opt speedbar sb-image ezimage dframe help-fns radix-tree cal-iso
flyspell ispell org-indent image-file org-rmail org-mhe org-irc org-info
org-gnus nnir gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range
gnus-win gnus nnheader org-docview doc-view jka-compr image-mode
org-bibtex bibtex org-bbdb org-w3m org-agenda company-oddmuse
company-keywords company-etags etags company-gtags company-dabbrev-code
company-dabbrev company-files company-capf company-cmake company-xcode
company-clang company-bbdb setup-server setup-patches patch-powerline
setup-hp48 rpl-tools sasm-mode sysrpl-mode rpl-edb rpl-base setup-backup
setup-calc setup-restclient restclient setup-nxml setup-completion
company-yasnippet company-auctex yasnippet-snippets yasnippet elec-pair
setup-notmuch company-eudc vc-git eudcb-macos-contacts eudc
eudc-options-file cus-edit eudc-vars counsel-notmuch s notmuch
notmuch-hello notmuch-tree notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc
notmuch-address notmuch-company notmuch-parser notmuch-wash coolj
notmuch-query goto-addr icalendar diary-lib diary-loaddefs notmuch-tag
notmuch-lib notmuch-version notmuch-compat cl mm-view mml-smime smime
dig setup-cedet forge-list forge-commands forge-semi forge-bitbucket
buck forge-gogs gogs forge-gitea gtea forge-gitlab glab forge-github
ghub-graphql treepy gsexp ghub let-alist json map forge-notify
forge-revnote forge-pullreq forge-issue forge-topic parse-time
bug-reference forge-post markdown-mode forge-repo forge forge-core
forge-db closql emacsql-sqlite emacsql emacsql-compiler url-http tls
gnutls url-auth url-gw nsm url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff smerge-mode diff-mode magit-core magit-autorevert autorevert
filenotify magit-margin magit-transient magit-process magit-mode
git-commit transient magit-git magit-section magit-utils log-edit
message rmc puny rfc822 mml mml-sec epa epg gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils
mailheader pcvs-util add-log with-editor async-bytecomp async cmake-mode
derived rx company-irony-c-headers company-irony company-template
company pcase irony-eldoc irony-completion irony-snippet irony
irony-iotask modern-cpp-font-lock setup-ibuffer ibuf-macs setup-auctex
auctex-latexmk tex-buf latex-mode-expansions latex latex-flymake
flymake-proc flymake warnings tex-ispell tex-style tex crm setup-org
org-mac-mail org-inlinetask org-element avl-tree generator
the-org-mode-expansions org org-macro org-footnote org-pcomplete
org-list org-faces org-entities noutline outline org-version ob-shell
shell pcomplete ob-perl ob-C cc-mode-expansions cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp
ob-comint ob-core ob-eval org-compat org-macs org-loaddefs format-spec
find-func cal-menu calendar cal-loaddefs setup-ivy counsel xdg xref
project dired dired-loaddefs compile comint ansi-color swiper cl-extra
help-mode ivy-rich subr-x ivy ring colir ivy-overlay setup-spell
setup-appearance recentf tree-widget wid-edit powerline
powerline-separators color powerline-themes setup-persistent-scratch
persistent-scratch setup-key-bindings two-column edmacro kmacro
setup-basic-editing expand-region text-mode-expansions
er-basic-expansions thingatpt expand-region-core advice
expand-region-custom ws-butler cua-base setup-file-encoding setup-env
exec-path-from-shell setup-packages auto-package-update easy-mmode dash
paren server hl-line delsel cus-start cus-load finder-inf tex-site info
package easymenu epg-config url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib time-date
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 1055266 90038)
 (symbols 48 57401 2)
 (miscs 40 3523 7094)
 (strings 32 217784 25504)
 (string-bytes 1 5825692)
 (vectors 16 88599)
 (vector-slots 8 1525104 40106)
 (floats 8 522 923)
 (intervals 56 66705 3323)
 (buffers 992 69))

bug#39890: 26.3; do-applescript undeterministic with long running scripts

[Lars Ingebrigtsen]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> As a work-around, I have altered my function to use (call-process
> "osascript") instead:
>
> ----------------------------------------------------------------------
> (defun eudc-macos-contacts-search-helper (str)
>   "Helper function to query the Contacts app via AppleScript.
> Searches for all persons with a case-insensitive substring match
> of STR in any of their name fileds (first, middle, or last)."
>   (insert (do-applescript (format "
>   (if (executable-find "osascript")
>       (call-process "osascript" nil t nil
> 		    "-e"

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I see that in the time since this was reported,
eudc-macos-contacts-search-helper has been changed to essentially be the
code suggested by Alexander here, so I assume that this fixed the
problem, and I'm closing this bug report.  If that's mistaken, please
respond to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#39890: 26.3; do-applescript undeterministic with long running scripts

[Alexander Adolf]

Hello Lars,

Lars Ingebrigtsen <larsi@gnus.org> writes:

> [...]
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> I see that in the time since this was reported,
> eudc-macos-contacts-search-helper has been changed to essentially be the
> code suggested by Alexander here, so I assume that this fixed the
> problem, and I'm closing this bug report.  If that's mistaken, please
> respond to the debbugs address and we'll reopen.
> [...]

The bug is about the behaviour of the elisp function's implementation
vs. the command line tool shipped with the OS. Thus, the code I
suggested is a workaround. I'm also the contributor of
eudc-macos-contacts-search-helper, btw.

That said, and as it doesn't seem to have bitten anyone else, it's
probably best to close the issue.


Many thanks and cheers,

  --alexander