bug#30364: 26.0.91; thread crash on macos

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

I'm trying out porting company-dabbrev to use a thread and things aren't
going well. The biggest problem so far is that Emacs occasionally
crashes while using threads.

The mac crash dump and super naive async company-dabbrev are here:

https://gist.github.com/aaronjensen/7b1685a7d275c20d8f0eb7170addf1d7

It seems to crash after a while of using it, though I'm also able to
repro the crash by evaling:

(company-dabbrev-thread (lambda (words) (message "%S" words)))

several times in a row

Also, if you uncomment out the (thread-yield) things get really
crazy--the point starts moving around in the current buffer when it runs
and it takes a very long time to finish.



In GNU Emacs 26.0.91 (build 1, x86_64-apple-darwin17.3.0, NS
appkit-1561.20 Version 10.13.2 (Build 17C205))
 of 2018-01-13 built on aaron-mbt.local
Repository revision: 5dd0e5c54d29e81c07798a124295c8c3f016d621
Windowing system distributor 'Apple', version 10.3.1561
Recent messages:
Spacemacs is ready.
Warning: desktop file appears to be in use by PID 84591.
Using it may cause conflicts.  Use it anyway? (y or n) y
Wrote /Users/aaronjensen/.emacs.d/.cache/.emacs.desktop.lock
Desktop: 1 frame, 0 buffers restored.
Loading /Users/aaronjensen/.emacs-private.el (source)...done
Starting new Ispell process aspell with default dictionary...
Loading /Users/aaronjensen/.emacs.d/.cache/recentf...done
Open the quickhelp.
Skipping check for new version (reason: dotfile)

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/info/emacs
 --prefix=/usr/local/Cellar/emacs-plus/HEAD-5dd0e5c --with-xml2
 --without-dbus --with-gnutls --with-imagemagick --with-modules
 --with-rsvg --with-ns --disable-ns-self-contained'

Configured features:
JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS MODULES LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Messages

Minor modes in effect:
  recentf-mode: t
  global-git-gutter+-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  desktop-save-mode: t
  auto-dim-other-buffers-mode: t
  global-wakatime-mode: t
  wakatime-mode: t
  global-spacemacs-whitespace-cleanup-mode: t
  spacemacs-whitespace-cleanup-mode: t
  ws-butler-global-mode: t
  ws-butler-mode: t
  winum-mode: t
  winner-mode: t
  pupo-mode: t
  purpose-mode: t
  volatile-highlights-mode: t
  global-vi-tilde-fringe-mode: t
  vi-tilde-fringe-mode: t
  spaceline-info-mode: t
  spaceline-helm-mode: t
  save-place-mode: t
  savehist-mode: t
  projectile-rails-global-mode: t
  projectile-mode: t
  persp-mode: t
  global-origami-mode: t
  origami-mode: t
  Info-breadcrumbs-in-mode-line-mode: t
  flycheck-pos-tip-mode: t
  global-flycheck-mode: t
  flx-ido-mode: t
  eyebrowse-mode: t
  global-evil-surround-mode: t
  evil-surround-mode: t
  global-evil-search-highlight-persist: t
  evil-search-highlight-persist: t
  show-smartparens-global-mode: t
  evil-lion-mode: t
  evil-escape-mode: t
  global-anzu-mode: t
  anzu-mode: t
  eval-sexp-fu-flash-mode: t
  editorconfig-mode: t
  diff-auto-refine-mode: t
  counsel-mode: t
  ivy-mode: t
  delete-selection-mode: t
  clean-aindent-mode: t
  hybrid-mode: t
  which-key-mode: t
  override-global-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  evil-mode: t
  evil-local-mode: t
  spacemacs-leader-override-mode: t
  global-spacemacs-leader-override-mode: t
  global-hl-line-mode: t
  xterm-mouse-mode: t
  global-auto-revert-mode: t
  shell-dirtrack-mode: t
  ido-vertical-mode: t
  global-page-break-lines-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/ht-20180129.1434/ht
hides /Users/aaronjensen/.emacs.d/core/libs/ht
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/inf-ruby-20180121.2300/inf-ruby
hides /usr/local/share/emacs/site-lisp/ruby/inf-ruby
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-stan
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-stan
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-exp
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-exp
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-J
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-J
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-eshell
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-eshell
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-emacs-lisp
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-emacs-lisp
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-gnus
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-gnus
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-css
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-css
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-lob
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-lob
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-forth
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-forth
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-macs
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-macs
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-version
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-version
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-scheme
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-scheme
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-abc
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-abc
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-C
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-C
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-capture
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-capture
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ref
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ref
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-clojure
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-clojure
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-mouse
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-mouse
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ledger
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ledger
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-ctags
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-ctags
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-entities
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-entities
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-archive
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-archive
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-screen
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-screen
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-haskell
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-haskell
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-asymptote
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-asymptote
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-mhe
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-mhe
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-table
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-table
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-keys
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-keys
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-org
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-org
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-plot
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-plot
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-awk
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-awk
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-groovy
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-groovy
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-octave
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-octave
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-faces
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-faces
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-colview
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-colview
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-R
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-R
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-timer
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-timer
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ebnf
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ebnf
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-mobile
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-mobile
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-fortran
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-fortran
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-shell
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-shell
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-perl
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-perl
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-sqlite
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-sqlite
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-sed
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-sed
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-list
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-list
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ruby
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ruby
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-eval
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-eval
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-habit
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-habit
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-clock
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-clock
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-html
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-html
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-src
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-src
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-lisp
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-lisp
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ditaa
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ditaa
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-pcomplete
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-pcomplete
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-lint
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-lint
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-rmail
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-rmail
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-latex
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-latex
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-sass
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-sass
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-io
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-io
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-tangle
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-tangle
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-calc
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-calc
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-java
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-java
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-icalendar
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-icalendar
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-eww
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-eww
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-md
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-md
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-beamer
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-beamer
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-element
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-element
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-protocol
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-protocol
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-mscgen
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-mscgen
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-gnuplot
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-gnuplot
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-latex
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-latex
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-id
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-id
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-vala
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-vala
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-man
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-man
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-feed
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-feed
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-lua
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-lua
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-table
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-table
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-ocaml
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-ocaml
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-coq
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-coq
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-picolisp
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-picolisp
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-indent
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-indent
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-lilypond
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-lilypond
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-matlab
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-matlab
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-datetree
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-datetree
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-python
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-python
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-bbdb
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-bbdb
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-makefile
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-makefile
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-duration
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-duration
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-agenda
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-agenda
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-dot
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-dot
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-js
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-js
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-publish
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-publish
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-inlinetask
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-inlinetask
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-org
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-org
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-core
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-core
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-compat
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-compat
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-docview
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-docview
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-odt
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-odt
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-plantuml
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-plantuml
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-ascii
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-ascii
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-loaddefs
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-loaddefs
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-w3m
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-w3m
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-bibtex
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-bibtex
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-info
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-info
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-hledger
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-hledger
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-maxima
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-maxima
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-macro
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-macro
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-sql
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-sql
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-attach
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-attach
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-processing
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-processing
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ox-texinfo
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ox-texinfo
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-irc
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-irc
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-crypt
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-crypt
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-footnote
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-footnote
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/org-install
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/org-install
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-comint
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-comint
/Users/aaronjensen/.emacs.d/elpa/26.0/develop/org-plus-contrib-20180129/ob-shen
hides /usr/local/Cellar/emacs-plus/HEAD-5dd0e5c/share/emacs/26.0.91/lisp/org/ob-shen

Features:
(shadow sort editorconfig-core editorconfig-core-handle
editorconfig-fnmatch mail-extr emacsbug sendmail colir smex recentf
tree-widget git-gutter-fringe+ fringe-helper git-gutter+ git-commit
with-editor magit-git magit-section magit-utils crm magit-popup
async-bytecomp async log-edit message rmc puny rfc822 mml mml-sec epa
gnus-util rmail rmail-loaddefs mailabbrev mail-utils gmm-utils
mailheader pcvs-util add-log desktop frameset face-remap
auto-dim-other-buffers wakatime-mode contextual-menubar quiet-emacs
fill-or-unfill init-macos-terminal-copy-paste init-flyspell
init-terminal-cursor evil-terminal-cursor-changer init-org init-magit
evil-mc evil-mc-command-execute evil-mc-command-record
evil-mc-cursor-make evil-mc-region evil-mc-cursor-state evil-mc-undo
evil-mc-vars evil-mc-known-commands evil-mc-common hl-todo
persistent-soft list-utils pcache eieio-base font-utils server zone
spacemacs-whitespace-cleanup ws-butler winum winner
spacemacs-purpose-popwin diminish window-purpose-x imenu-list imenu
window-purpose window-purpose-fixes window-purpose-prefix-overload
window-purpose-switch window-purpose-layout window-purpose-core
window-purpose-configuration window-purpose-utils volatile-highlights
vi-tilde-fringe unicode-fonts tmux string-inflection let-alist
spaceline-all-the-icons spaceline-all-the-icons-separators
spaceline-all-the-icons-segments all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons memoize spaceline-config
spaceline-segments spaceline powerline powerline-separators color
powerline-themes smartparens-config smartparens-text smartparens-ruby
saveplace savehist ruby-test-mode pcre2el rxt re-builder
projectile-rails rake inflections inf-ruby ruby-mode smie projectile
grep ibuf-ext ibuffer ibuffer-loaddefs popwin persp-mode osx-trash
origami origami-parsers linum ivy-hydra info+ image-mode
flycheck-pos-tip pos-tip flycheck-flow flycheck find-func flx-ido
eyebrowse evil-surround evil-search-highlight-persist evil-numbers
evil-lisp-state smartparens evil-lion evil-indent-plus evil-exchange
evil-escape evil-args evil-anzu anzu eval-sexp-fu highlight font-lock+
frame-fns avoid eterm-256color f term ehelp xterm-color editorconfig
noutline outline dtrt-indent diff-hl vc-dir ewoc vc vc-dispatcher
diff-mode counsel dired dired-loaddefs compile esh-util etags xref
project swiper ivy flx delsel ivy-overlay ffap clean-aindent-mode
adaptive-wrap gh-common gh-profile s marshal dash rx docker-tramp
tramp-cache hybrid-mode exec-path-from-shell evil-evilified-state
which-key use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key use-package-core
hydra lv cus-edit cus-start cus-load evil evil-integration undo-tree
diff evil-maps evil-commands reveal flyspell ispell evil-jumps
evil-command-window evil-types evil-search evil-ex evil-macros
evil-repeat evil-states evil-core evil-common windmove thingatpt rect
evil-digraphs evil-vars bind-map quelpa help-fns radix-tree
package-build mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 mm-util ietf-drums mail-prsvr json map lisp-mnt hl-line xt-mouse
autorevert filenotify cl-extra disp-table wid-edit monokai-theme info
finder-inf patch-server init-sass init-php init-html init-evil tramp
tramp-compat tramp-loaddefs trampver shell pcomplete comint ansi-color
ring parse-time format-spec ido-vertical-mode ido core-spacemacs
core-use-package-ext core-transient-state core-micro-state core-toggle
core-keybindings core-fonts-support core-themes-support
core-display-init core-jump core-release-management core-custom-settings
core-configuration-layer eieio-compat core-spacemacs-buffer core-funcs
core-dotspacemacs ht cl help-mode warnings package url-handlers
url-parse auth-source cl-seq password-cache url-vars seq eieio byte-opt
bytecomp byte-compile cconv eieio-core eieio-loaddefs epg epg-config
core-command-line pcase core-debug edmacro kmacro derived cl-macs gv
advice profiler easymenu cl-loaddefs cl-lib page-break-lines easy-mmode
core-emacs-backports subr-x time-date tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote kqueue cocoa ns lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 945418 925644)
 (symbols 48 62296 69)
 (miscs 40 332 623)
 (strings 32 181748 122734)
 (string-bytes 1 5947351)
 (vectors 16 88608)
 (vector-slots 8 1560060 539792)
 (floats 8 1090 1011)
 (intervals 56 3381 383)
 (buffers 992 12))

bug#30364: 26.0.91; thread crash on macos

[Noam Postavsky]

Aaron Jensen <aaronjensen@gmail.com> writes:

> I'm trying out porting company-dabbrev to use a thread and things aren't
> going well. The biggest problem so far is that Emacs occasionally
> crashes while using threads.
>
> The mac crash dump and super naive async company-dabbrev are here:
>
> https://gist.github.com/aaronjensen/7b1685a7d275c20d8f0eb7170addf1d7
>
> It seems to crash after a while of using it, though I'm also able to
> repro the crash by evaling:
>
> (company-dabbrev-thread (lambda (words) (message "%S" words)))
>
> several times in a row

I was going to check if this happens on systems other than macOS, but as
far as I can tell, the code you posted doesn't define the function
"company-dabbrev-thread".  So I guess what you posted and what are
running aren't quite the same...

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

From: Noam Postavsky (mailto:npostavs@users.sourceforge.net)
> I was going to check if this happens on systems other than macOS, but as
> far as I can tell, the code you posted doesn't define the function
> "company-dabbrev-thread". So I guess what you posted and what are
> running aren't quite the same…

Sorry, I think it was something like this:

(defun company-dabbrev-thread (arg callback)
  (make-thread
              (lambda ()
                (funcall callback
                         (let* ((case-fold-search company-dabbrev-ignore-case)
                                (words (company-dabbrev--search
(company-dabbrev--make-regexp)
                                                  company-dabbrev-time-limit
                                                  (pcase
company-dabbrev-other-buffers
                                                    (`t (list major-mode))
                                                    (`all `all))))
                                (downcase-p (if (eq
company-dabbrev-downcase 'case-replace)
                                                case-replace
                                              company-dabbrev-downcase)))
                           (setq words (company-dabbrev--filter arg words))
                           (if downcase-p
                               (mapcar 'downcase words)
                             words))))))

(company-dabbrev-thread "e" (lambda (words) (message "%S" words)))

I was going through multiple revisions and didn’t gist the right code.

bug#30364: 26.0.91; thread crash on macos

[Noam Postavsky]

Aaron Jensen <aaronjensen@gmail.com> writes:

> From: Noam Postavsky (mailto:npostavs@users.sourceforge.net)
>> I was going to check if this happens on systems other than macOS, but as
>> far as I can tell, the code you posted doesn't define the function
>> "company-dabbrev-thread". So I guess what you posted and what are
>> running aren't quite the same…
>
> Sorry, I think it was something like this:
>
> (defun company-dabbrev-thread (arg callback)
>   (make-thread
>               (lambda ()
>                 (funcall callback
>                          (let* ((case-fold-search company-dabbrev-ignore-case)
>                                 (words (company-dabbrev--search
> (company-dabbrev--make-regexp)
>                                                   company-dabbrev-time-limit
>                                                   (pcase
> company-dabbrev-other-buffers
>                                                     (`t (list major-mode))
>                                                     (`all `all))))
>                                 (downcase-p (if (eq
> company-dabbrev-downcase 'case-replace)
>                                                 case-replace
>                                               company-dabbrev-downcase)))
>                            (setq words (company-dabbrev--filter arg words))
>                            (if downcase-p
>                                (mapcar 'downcase words)
>                              words))))))
>
> (company-dabbrev-thread "e" (lambda (words) (message "%S" words)))

Okay, that doesn't crash for me on GNU/Linux (though it doesn't print
any message either).

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sat, Feb 17, 2018 at 6:44 AM, Noam Postavsky <npostavs@gmail.com> wrote:
> Okay, that doesn't crash for me on GNU/Linux (though it doesn't print
> any message either).

Ok, here's a more consistent repro for me:
https://gist.github.com/aaronjensen/aa4d7425f2ebef9ae05947529b42c404

Open init.el in a buffer so that all of those words are there for
dabbrev to search.

Then eval crash.el

It doesn't crash *every* time, but most of the times it does. It also
should actually print the messages.

Aaron

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 17 Feb 2018 09:58:42 -0800
> Cc: 30364@debbugs.gnu.org, Noam Postavsky <npostavs@users.sourceforge.net>
> 
> https://gist.github.com/aaronjensen/aa4d7425f2ebef9ae05947529b42c404
> 
> Open init.el in a buffer so that all of those words are there for
> dabbrev to search.
> 
> Then eval crash.el
> 
> It doesn't crash *every* time, but most of the times it does. It also
> should actually print the messages.

When this crashes, is one of the threads you launched always doing GC,
like in the first backtrace you've shown?

Btw, I'm not really sure what you are doing and why: what is the
purpose of launching 100 threads all doing the same job with the same
data?

bug#30364: 26.0.91; thread crash on macos

[Alan Third]

On Sat, Feb 17, 2018 at 09:58:42AM -0800, Aaron Jensen wrote:
> On Sat, Feb 17, 2018 at 6:44 AM, Noam Postavsky <npostavs@gmail.com> wrote:
> > Okay, that doesn't crash for me on GNU/Linux (though it doesn't print
> > any message either).
> 
> Ok, here's a more consistent repro for me:
> https://gist.github.com/aaronjensen/aa4d7425f2ebef9ae05947529b42c404
> 
> Open init.el in a buffer so that all of those words are there for
> dabbrev to search.
> 
> Then eval crash.el
> 
> It doesn't crash *every* time, but most of the times it does. It also
> should actually print the messages.

Can you try the Main Thread Checker to see if it’s doing some UI stuff
outside of the main thread?

I can’t find the instructions for doing it on the command line right
now, though...

-- 
Alan Third

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sat, Feb 17, 2018 at 10:17 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> When this crashes, is one of the threads you launched always doing GC,
> like in the first backtrace you've shown?

I can tell this by the mark_object/mark_vectorlike calls in the stack,
right? If so, yes, it appears so.

> Btw, I'm not really sure what you are doing and why: what is the
> purpose of launching 100 threads all doing the same job with the same
> data?

Ultimately, I was trying to port company-dabbrev to use threads. I
noticed the crash while working on it, so the 100 threads at the same
time was just a way of more consistently reproducing it. I could
probably have run the 100 threads serially to reproduce it, I don't
know.

On Sat, Feb 17, 2018 at 10:23 AM, Alan Third <alan@idiocy.org> wrote:
> Can you try the Main Thread Checker to see if it’s doing some UI stuff
> outside of the main thread?

I don't know what this is, but I do know that there are
(input-pending-p)'s in the thread from the current implementation.
They don't need to be there (i guess they'd be replaced by
thread-yields, but when I did that it got very slow). I don't have
time to test if removing those fixes the crashes just yet, but could
that be causing the crash? The thread also calls `message`, I don't
know if that's considered "UI stuff"

Aaron

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 17 Feb 2018 10:40:10 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>
> 
> On Sat, Feb 17, 2018 at 10:17 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > When this crashes, is one of the threads you launched always doing GC,
> > like in the first backtrace you've shown?
> 
> I can tell this by the mark_object/mark_vectorlike calls in the stack,
> right?

Yes.

> Ultimately, I was trying to port company-dabbrev to use threads. I
> noticed the crash while working on it, so the 100 threads at the same
> time was just a way of more consistently reproducing it. I could
> probably have run the 100 threads serially to reproduce it, I don't
> know.

They do run serially, the way you wrote the code.  The call to
make-thread creates the thread, but the thread doesn't run, it waits
for the currently-running thread to release the global lock.
Normally, I'd expect the first of the 100 threads to start running
almost immediately, because you don't do anything in the main thread.
But the other 99 will wait.  Whether some of them will start running
before the first one exits, depends on the code of the thread
function, and you didn't show all of it: I understand that the bulk is
in company-mode.

> I don't know what this is, but I do know that there are
> (input-pending-p)'s in the thread from the current implementation.

Where are they?  I don't see those input-pending-p calls in the code
on the page to which you referred. 

> They don't need to be there (i guess they'd be replaced by
> thread-yields, but when I did that it got very slow). I don't have
> time to test if removing those fixes the crashes just yet, but could
> that be causing the crash? The thread also calls `message`, I don't
> know if that's considered "UI stuff"

If we are to believe to the backtrace, the thread that crashed was the
main thread, and it crashed inside pthread_mutex_lock.  Which probably
means some snafu with pthreads synchronization functions.  But the NS
port has some complicated architecture wrt threads, so maybe that's
why it crashes.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sat, Feb 17, 2018 at 10:55 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> They do run serially, the way you wrote the code.  The call to
> make-thread creates the thread, but the thread doesn't run, it waits
> for the currently-running thread to release the global lock.
> Normally, I'd expect the first of the 100 threads to start running
> almost immediately, because you don't do anything in the main thread.
> But the other 99 will wait.  Whether some of them will start running
> before the first one exits, depends on the code of the thread
> function, and you didn't show all of it: I understand that the bulk is
> in company-mode.

Right, it is, in company-dabbrev.el

>> I don't know what this is, but I do know that there are
>> (input-pending-p)'s in the thread from the current implementation.
>
> Where are they?  I don't see those input-pending-p calls in the code
> on the page to which you referred.

They are here: https://github.com/company-mode/company-mode/blob/master/company-dabbrev.el#L121

My goal was to remove them and add thread-yields to the loops.

That said, I just tested removing them and it still crashed.

> If we are to believe to the backtrace, the thread that crashed was the
> main thread, and it crashed inside pthread_mutex_lock.  Which probably
> means some snafu with pthreads synchronization functions.  But the NS
> port has some complicated architecture wrt threads, so maybe that's
> why it crashes.

Is there anything else I can do to provide more information?

Alan, are you able to reproduce it?

Thanks,

Aaron

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 17 Feb 2018 17:58:28 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> > If we are to believe to the backtrace, the thread that crashed was the
> > main thread, and it crashed inside pthread_mutex_lock.  Which probably
> > means some snafu with pthreads synchronization functions.  But the NS
> > port has some complicated architecture wrt threads, so maybe that's
> > why it crashes.
> 
> Is there anything else I can do to provide more information?

A Lisp backtrace from the crashing thread would be nice.  Also, the
C-level backtrace from that thread seems truncated, so it would be
good to see all of it.

Thanks.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sat, Feb 17, 2018 at 8:51 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> A Lisp backtrace from the crashing thread would be nice.  Also, the
> C-level backtrace from that thread seems truncated, so it would be
> good to see all of it.

Pardon my inexperience, but are there instructions on how to get these
somewhere?

Thanks,

Aaron

bug#30364: 26.0.91; thread crash on macos

[Alan Third]

On Sat, Feb 17, 2018 at 05:58:28PM -0800, Aaron Jensen wrote:
> Alan, are you able to reproduce it?

sorry, no. I tried both Emacs 26 and master.

FWIW, the ‘lexical-let’ didn’t work for me at all and I had to change
it to ‘let’ (the buffer’s got lexical binding enabled anyway, so that
doesn’t make any difference?)
-- 
Alan Third

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sat, 17 Feb 2018 20:54:54 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> On Sat, Feb 17, 2018 at 8:51 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > A Lisp backtrace from the crashing thread would be nice.  Also, the
> > C-level backtrace from that thread seems truncated, so it would be
> > good to see all of it.
> 
> Pardon my inexperience, but are there instructions on how to get these
> somewhere?

I don't use LLDB, so I can be of limited help here.

For the second issue, I'm guessing that there's some limitation on
the number of call-stack frames LLDB shows by default, so a way to
show more would be to increase that limit.

For the first issue, you will have to evaluate expressions manually,
unfortunately, as LLDB doesn't support canned command sequences that
we define in src/.gdbinit for GDB.  You will find some starting point
here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#44
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#47
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#50

Thanks.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 09:15:11 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> Thanks, I believe this is the full trace ("bt all" is all of the
> threads and "crashing thread" is just the crashing thread):
> 
> https://gist.github.com/aaronjensen/2c765cfd08556f570a5b78b4c5f8e866

Ah, okay.  I see I've misinterpreted the reason for the crash: the
thread that crashes is the one that does GC.  And since GC is deeply
recursive, and you have 120 - 26 + 1 = 95 other threads all waiting
for the global lock, the first hypothesis I have is that the thread
which GCs hits stack overflow, because the other 95 threads use up (or
reserve) too much stack space.

Can you or someone else who knows about Darwin figure out what are the
limitations on stack space in multithreaded programs on macOS?

> > For the first issue, you will have to evaluate expressions manually,
> > unfortunately, as LLDB doesn't support canned command sequences that
> > we define in src/.gdbinit for GDB.  You will find some starting point
> > here:
> >
> >   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#44
> >   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#47
> >   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30320#50
> 
> I couldn't figure out how to do this. Nothing I do puts "args" in
> scope, so I'm not sure what I'm doing wrong.

You need to make the frame which calls Ffuncall current first.  Here's
one such frame:

  frame #2434: 0x000000010029a29e emacs`Ffuncall + 222

You will see in the sources that Ffuncall's first argument is the
args[] array.

However, since the crash is in GC, it's not important to see the Lisp
backtrace, as GC can be triggered by almost any Lisp code.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sun, Feb 18, 2018 at 10:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> Ah, okay.  I see I've misinterpreted the reason for the crash: the
> thread that crashes is the one that does GC.  And since GC is deeply
> recursive, and you have 120 - 26 + 1 = 95 other threads all waiting
> for the global lock, the first hypothesis I have is that the thread
> which GCs hits stack overflow, because the other 95 threads use up (or
> reserve) too much stack space.

FWIW, this happens even if I only have 1 thread active. The benchmark
100 is just for me to more easily repro it.

Here's a trace from doing one thread at a time:

https://gist.github.com/aaronjensen/cb91f0c7f63a28335a9cea3c88037d59

> Can you or someone else who knows about Darwin figure out what are the
> limitations on stack space in multithreaded programs on macOS?

I can probably look into it at some point if you think it is still
relevant given above.

> You need to make the frame which calls Ffuncall current first.  Here's
> one such frame:
>
>   frame #2434: 0x000000010029a29e emacs`Ffuncall + 222
>
> You will see in the sources that Ffuncall's first argument is the
> args[] array.

Perhaps I'm still doing something wrong:

(lldb) frame select 2449
frame #2449: 0x000000010029a4b7 emacs`Ffuncall + 759
emacs`Ffuncall:
    0x10029a4b7 <+759>: movq   %rax, -0x38(%rbp)
    0x10029a4bb <+763>: jmp    0x10029a528               ; <+872>
    0x10029a4c0 <+768>: movl   $0xc1, %edi
    0x10029a4c5 <+773>: movq   -0x28(%rbp), %rax
(lldb) p args
error: use of undeclared identifier 'args'

I'm not seeing sources, so I probably need to figure out how to load symbols.

> However, since the crash is in GC, it's not important to see the Lisp
> backtrace, as GC can be triggered by almost any Lisp code.

Got it.

Thanks,
Aaron

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sun, Feb 18, 2018 at 10:38 AM, Aaron Jensen <aaronjensen@gmail.com> wrote:
>> Can you or someone else who knows about Darwin figure out what are the
>> limitations on stack space in multithreaded programs on macOS?
>
> I can probably look into it at some point if you think it is still
> relevant given above.

This seems relevant:
https://developer.apple.com/library/content/qa/qa1419/_index.html

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sun, Feb 18, 2018 at 10:39 AM, Aaron Jensen <aaronjensen@gmail.com> wrote:
> On Sun, Feb 18, 2018 at 10:38 AM, Aaron Jensen <aaronjensen@gmail.com> wrote:
>>> Can you or someone else who knows about Darwin figure out what are the
>>> limitations on stack space in multithreaded programs on macOS?
>>
>> I can probably look into it at some point if you think it is still
>> relevant given above.
>
> This seems relevant:
> https://developer.apple.com/library/content/qa/qa1419/_index.html

I'm going to try out increasing the thread stack size and see if that helps.

Out of curiosity, would it be possible to limit GCs to only happen on
the main thread? They're stop the world any way, right, so does it
matter which thread they happen on?

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 10:38:46 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> On Sun, Feb 18, 2018 at 10:25 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > Ah, okay.  I see I've misinterpreted the reason for the crash: the
> > thread that crashes is the one that does GC.  And since GC is deeply
> > recursive, and you have 120 - 26 + 1 = 95 other threads all waiting
> > for the global lock, the first hypothesis I have is that the thread
> > which GCs hits stack overflow, because the other 95 threads use up (or
> > reserve) too much stack space.
> 
> FWIW, this happens even if I only have 1 thread active. The benchmark
> 100 is just for me to more easily repro it.
> 
> Here's a trace from doing one thread at a time:
> 
> https://gist.github.com/aaronjensen/cb91f0c7f63a28335a9cea3c88037d59

What do you mean by "one thread at a time"?  How was your program
changed for this run?

> > Can you or someone else who knows about Darwin figure out what are the
> > limitations on stack space in multithreaded programs on macOS?
> 
> I can probably look into it at some point if you think it is still
> relevant given above.

It crashes in a deeply recursive GC, so it might be relevant.

Can you figure out which object is being marked and causes the
exception?

> (lldb) frame select 2449
> frame #2449: 0x000000010029a4b7 emacs`Ffuncall + 759
> emacs`Ffuncall:
>     0x10029a4b7 <+759>: movq   %rax, -0x38(%rbp)
>     0x10029a4bb <+763>: jmp    0x10029a528               ; <+872>
>     0x10029a4c0 <+768>: movl   $0xc1, %edi
>     0x10029a4c5 <+773>: movq   -0x28(%rbp), %rax
> (lldb) p args
> error: use of undeclared identifier 'args'
> 
> I'm not seeing sources, so I probably need to figure out how to load symbols.

Yes, it looks like you don't have enough of the debug info in the
executable.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 10:39:58 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> This seems relevant:
> https://developer.apple.com/library/content/qa/qa1419/_index.html

If threads get only 512KB of stack, it's a small wonder that they
crash in GC.  IME, you need at least 3MB to endure a full GC in Emacs.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 10:59:02 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> Out of curiosity, would it be possible to limit GCs to only happen on
> the main thread? They're stop the world any way, right, so does it
> matter which thread they happen on?

A thread that runs Lisp can continue running Lisp for a long time, and
will generate a lot of garbage.  If GC is prevented, we will risk
running out of memory.

IOW, leaving GC for the main thread only makes sense if we can be sure
the main thread will run shortly.  But that cannot be guaranteed with
the current thread model, where a thread must exit or yield before
another thread can run.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

On Sun, Feb 18, 2018 at 11:32 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> A thread that runs Lisp can continue running Lisp for a long time, and
> will generate a lot of garbage.  If GC is prevented, we will risk
> running out of memory.
>
> IOW, leaving GC for the main thread only makes sense if we can be sure
> the main thread will run shortly.  But that cannot be guaranteed with
> the current thread model, where a thread must exit or yield before
> another thread can run.

I was actually imagining that when a thread needed a GC it would
request that the GC be done on the main thread (blocking while it
waits). I have no idea how difficult that would be to implement.

> If threads get only 512KB of stack, it's a small wonder that they
> crash in GC.  IME, you need at least 3MB to endure a full GC in Emacs.

I upped the required stack size to 5MB and it fixed the crash for me.
I don't know what the right size should bed, I could go w/ the 3MB you
call out here, but I went a little higher to be safe. See attached
patch. I'm assuming that pthread_attr_setstacksize is available
everywhere pthread is. I don't know if that's an ok assumption or not.

> What do you mean by "one thread at a time"?  How was your program
> changed for this run?

I just meant that I C-x C-e on (company-dabbrev-thread "e" (lambda
(words) (message "%S" words))), waited, did it again, etc, until
crash.

Aaron

[-- Attachment #2: 0001-Require-at-least-5MB-stack-size-for-a-thread-bug-303.patch --]
[-- Type: application/octet-stream, Size: 1231 bytes --]

From 6c56078197e97e3e33b0264c97c9807c7a9289e5 Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Sun, 18 Feb 2018 11:43:47 -0800
Subject: [PATCH] Require at least 5MB stack size for a thread (bug#30364)

* src/systhread.c (sys_thread_create): Require at least 5MB stack size
---
 src/systhread.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/src/systhread.c b/src/systhread.c
index 4ffb7db143..70feff29c1 100644
--- a/src/systhread.c
+++ b/src/systhread.c
@@ -94,6 +94,8 @@ sys_thread_yield (void)
 #include <sys/prctl.h>
 #endif
 
+#define REQUIRED_STACK_SIZE 5*1024*1024
+
 void
 sys_mutex_init (sys_mutex_t *mutex)
 {
@@ -161,10 +163,19 @@ sys_thread_create (sys_thread_t *thread_ptr, const char *name,
 {
   pthread_attr_t attr;
   int result = 0;
+  size_t stackSize = 0;
 
   if (pthread_attr_init (&attr))
     return 0;
 
+  if (pthread_attr_getstacksize (&attr, &stackSize))
+    return 0;
+
+  if (stackSize < REQUIRED_STACK_SIZE) {
+    if (pthread_attr_setstacksize (&attr, REQUIRED_STACK_SIZE))
+      return 0;
+  }
+
   if (!pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED))
     {
       result = pthread_create (thread_ptr, &attr, func, arg) == 0;
-- 
2.15.1

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 11:49:45 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> I was actually imagining that when a thread needed a GC it would
> request that the GC be done on the main thread (blocking while it
> waits). I have no idea how difficult that would be to implement.

Neither do I.

> > If threads get only 512KB of stack, it's a small wonder that they
> > crash in GC.  IME, you need at least 3MB to endure a full GC in Emacs.
> 
> I upped the required stack size to 5MB and it fixed the crash for me.
> I don't know what the right size should bed, I could go w/ the 3MB you
> call out here, but I went a little higher to be safe. See attached
> patch. I'm assuming that pthread_attr_setstacksize is available
> everywhere pthread is. I don't know if that's an ok assumption or not.

I think for 32-bit Emacs 3MB will do, but 64-bit build might need
more, like 4 or 5.  The MS-Windows build gives each thread the same
amount of stack as for the main program, which means 8MB.

I hope someone who is familiar with pthreads will chime in regarding
the portability issues.

Thanks.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Sun, Feb 18, 2018 at 12:15 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> I think for 32-bit Emacs 3MB will do, but 64-bit build might need
> more, like 4 or 5.  The MS-Windows build gives each thread the same
> amount of stack as for the main program, which means 8MB.

Would you like me to vary it based on 32-bit vs 64-bit?

> I hope someone who is familiar with pthreads will chime in regarding
> the portability issues.

Looks like it's part of POSIX.1-2001, so I'm guessing that means we
can assume it's there, yeah?

https://www.systutorials.com/docs/linux/man/3-pthread_attr_setstacksize/

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Sun, 18 Feb 2018 13:12:24 -0800
> Cc: Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> On Sun, Feb 18, 2018 at 12:15 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > I think for 32-bit Emacs 3MB will do, but 64-bit build might need
> > more, like 4 or 5.  The MS-Windows build gives each thread the same
> > amount of stack as for the main program, which means 8MB.
> 
> Would you like me to vary it based on 32-bit vs 64-bit?

I think so, but I'd like to hear from others about this.  Paul,
Andreas, could you please comment?

One thing that bothers me is whether giving threads more stack space
could adversely affect the memory available to the main thread.

> > I hope someone who is familiar with pthreads will chime in regarding
> > the portability issues.
> 
> Looks like it's part of POSIX.1-2001, so I'm guessing that means we
> can assume it's there, yeah?
> 
> https://www.systutorials.com/docs/linux/man/3-pthread_attr_setstacksize/

I think so, yes.  Still, I'd like to hear more opinions.

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

Eli Zaretskii wrote:
>> Would you like me to vary it based on 32-bit vs 64-bit?
> I think so, but I'd like to hear from others about this.  Paul,
> Andreas, could you please comment?

A 64-bit Emacs should need more stack space than a 32-bit Emacs, yes. It 
wouldn't be double the space, since not everything on the stack is a pointer or 
an EMACS_INT. If you really want to fine-tune it you can also depend on the 
width of EMACS_INT. Getting it "just right" would be a bit tricky.

8 MiB/thread for 64-bit Emacs sounds OK to me. I wouldn't cheap out on 64-bit 
platforms.

bug#30364: 26.0.91; thread crash on macos

[Daniel Colascione]

On 02/19/2018 09:24 AM, Paul Eggert wrote:
> Eli Zaretskii wrote:
>>> Would you like me to vary it based on 32-bit vs 64-bit?
>> I think so, but I'd like to hear from others about this.  Paul,
>> Andreas, could you please comment?
> 
> A 64-bit Emacs should need more stack space than a 32-bit Emacs, yes. It 
> wouldn't be double the space, since not everything on the stack is a 
> pointer or an EMACS_INT. If you really want to fine-tune it you can also 
> depend on the width of EMACS_INT. Getting it "just right" would be a bit 
> tricky.
> 
> 8 MiB/thread for 64-bit Emacs sounds OK to me. I wouldn't cheap out on 
> 64-bit platforms.
What about just dedicating a thread to GC? We'd create it and have it 
wait on a condition variable. Any thread could signal the CV, block, and 
wait for GC to complete. It'd be pretty simple, I think, and unlike the 
"always GC on main thread" proposal, it wouldn't force everything on the 
main thread to be interruptible.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> Cc: npostavs@gmail.com, 30364@debbugs.gnu.org,
>  npostavs@users.sourceforge.net, alan@idiocy.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 19 Feb 2018 09:24:32 -0800
> 
> A 64-bit Emacs should need more stack space than a 32-bit Emacs, yes. It 
> wouldn't be double the space, since not everything on the stack is a pointer or 
> an EMACS_INT. If you really want to fine-tune it you can also depend on the 
> width of EMACS_INT. Getting it "just right" would be a bit tricky.
> 
> 8 MiB/thread for 64-bit Emacs sounds OK to me. I wouldn't cheap out on 64-bit 
> platforms.

Thanks.  I guess we should go with 4MB for 32-bit builds and 8MB for
64-bit.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> Cc: 30364@debbugs.gnu.org, alan@idiocy.org, npostavs@gmail.com,
>  npostavs@users.sourceforge.net
> From: Daniel Colascione <dancol@dancol.org>
> Date: Mon, 19 Feb 2018 09:29:49 -0800
> 
> What about just dedicating a thread to GC?

If having several threads with 8MB of stack is asking for trouble,
then doing GC in a dedicated thread could be a better solution.

But I personally am undecided whether we should go that way as long as
threads are used as they are today (read: not used at all, AFAIK).
I'd suggest to wait until we see serious packages appear that are
based on Lisp threads, because if they never appear, we could declare
this experiment as failed, and then a separate GC thread might be just
a liability.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

On Mon, Feb 19, 2018 at 10:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> Thanks.  I guess we should go with 4MB for 32-bit builds and 8MB for
> 64-bit.

Updated patch attached, hopefully that's the right way of detecting x64/x86.

Aaron

[-- Attachment #2: 0001-Require-a-larger-stack-size-for-threads-bug-30364.patch --]
[-- Type: application/octet-stream, Size: 1333 bytes --]

From f1f091e3a9d364bdd0b9d935dbeede8f5708b744 Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Sun, 18 Feb 2018 11:43:47 -0800
Subject: [PATCH] Require a larger stack size for threads (bug#30364)

* src/systhread.c (sys_thread_create): Require at least 8MB stack size
  for x64 and 4MB for x86.
---
 src/systhread.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/src/systhread.c b/src/systhread.c
index 4ffb7db143..b2fa45fae9 100644
--- a/src/systhread.c
+++ b/src/systhread.c
@@ -94,6 +94,12 @@ sys_thread_yield (void)
 #include <sys/prctl.h>
 #endif
 
+#ifdef __x86_64__
+#define REQUIRED_STACK_SIZE 8*1024*1024
+#else
+#define REQUIRED_STACK_SIZE 4*1024*1024
+#endif
+
 void
 sys_mutex_init (sys_mutex_t *mutex)
 {
@@ -161,10 +167,19 @@ sys_thread_create (sys_thread_t *thread_ptr, const char *name,
 {
   pthread_attr_t attr;
   int result = 0;
+  size_t stackSize = 0;
 
   if (pthread_attr_init (&attr))
     return 0;
 
+  if (pthread_attr_getstacksize (&attr, &stackSize))
+    return 0;
+
+  if (stackSize < REQUIRED_STACK_SIZE) {
+    if (pthread_attr_setstacksize (&attr, REQUIRED_STACK_SIZE))
+      return 0;
+  }
+
   if (!pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED))
     {
       result = pthread_create (thread_ptr, &attr, func, arg) == 0;
-- 
2.15.1

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

Aaron Jensen wrote:
> hopefully that's the right way of detecting x64/x86

x86-64 is not the only 64-bit platform; I suggest making REQUIRED_STACK_SIZE 
equal to sizeof (void *) * 1024 * 1024. Also, no camelcase please. And why fail 
merely because pthread_attr_setstacksize fails?

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Mon, Feb 19, 2018 at 12:08 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> x86-64 is not the only 64-bit platform; I suggest making REQUIRED_STACK_SIZE
> equal to sizeof (void *) * 1024 * 1024. Also, no camelcase please. And why
> fail merely because pthread_attr_setstacksize fails?

Updated, thank you for the feedback. I'm mixed on failing when
setstacksize fails, but it seems like the worst case is it'd create a
thread w/ too small a stack size and it may crash on GC. Maybe that's
better than failing outright on some platforms.

Thanks,

Aaron

[-- Attachment #2: 0001-Require-a-larger-stack-size-for-threads-bug-30364.patch --]
[-- Type: application/octet-stream, Size: 1226 bytes --]

From 714beee58e1860a14c879915c57fc81d86bd0767 Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Sun, 18 Feb 2018 11:43:47 -0800
Subject: [PATCH] Require a larger stack size for threads (bug#30364)

* src/systhread.c (sys_thread_create): Require at least 8MB stack size
  for x64 and 4MB for x86.
---
 src/systhread.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/systhread.c b/src/systhread.c
index 4ffb7db143..c292343392 100644
--- a/src/systhread.c
+++ b/src/systhread.c
@@ -94,6 +94,8 @@ sys_thread_yield (void)
 #include <sys/prctl.h>
 #endif
 
+#define REQUIRED_STACK_SIZE sizeof (void *) * 1024 * 1024
+
 void
 sys_mutex_init (sys_mutex_t *mutex)
 {
@@ -161,10 +163,15 @@ sys_thread_create (sys_thread_t *thread_ptr, const char *name,
 {
   pthread_attr_t attr;
   int result = 0;
+  size_t stack_size = 0;
 
   if (pthread_attr_init (&attr))
     return 0;
 
+  if (!pthread_attr_getstacksize (&attr, &stack_size) &&
+      stack_size < REQUIRED_STACK_SIZE)
+    pthread_attr_setstacksize (&attr, REQUIRED_STACK_SIZE);
+
   if (!pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED))
     {
       result = pthread_create (thread_ptr, &attr, func, arg) == 0;
-- 
2.15.1

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

Hey all,

How is this looking? Anything else I can do on it, or are we waiting
on someone else's feedback?

Thanks,

Aaron

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

On 02/25/2018 12:52 PM, Aaron Jensen wrote:
> How is this looking? Anything else I can do on it, or are we waiting
> on someone else's feedback?

Nobody else commented, so I reformatted the patch a bit to be more in 
the Emacs style and installed the attached into master. (Also, I dropped 
the unnecessary initialization of 'stack_size'.) Thanks.


[-- Attachment #2: 0001-Require-a-larger-stack-size-for-threads-bug-30364.patch --]
[-- Type: text/x-patch, Size: 1155 bytes --]

From 09d71c66512934c5f7a4dd49779fd9cfabbcfe05 Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Tue, 27 Feb 2018 09:15:40 -0800
Subject: [PATCH] Require a larger stack size for threads (bug#30364)

* src/systhread.c (sys_thread_create) [THREADS_ENABLED && HAVE_PTHREAD]:
Require at least 8MB stack size for x64 and 4MB for x86.
---
 src/systhread.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/systhread.c b/src/systhread.c
index 3f162a2bcb..e972ed398a 100644
--- a/src/systhread.c
+++ b/src/systhread.c
@@ -177,6 +177,13 @@ sys_thread_create (sys_thread_t *thread_ptr, const char *name,
   if (pthread_attr_init (&attr))
     return 0;
 
+  /* Avoid crash on macOS with deeply nested GC (Bug#30364).  */
+  size_t stack_size;
+  size_t required_stack_size = sizeof (void *) * 1024 * 1024;
+  if (pthread_attr_getstacksize (&attr, &stack_size) == 0
+      && stack_size < required_stack_size)
+    pthread_attr_setstacksize (&attr, required_stack_size);
+
   if (!pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED))
     {
       result = pthread_create (thread_ptr, &attr, func, arg) == 0;
-- 
2.14.3

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Tue, Feb 27, 2018 at 9:30 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> Nobody else commented, so I reformatted the patch a bit to be more in the
> Emacs style and installed the attached into master. (Also, I dropped the
> unnecessary initialization of 'stack_size'.) Thanks.

That's great, thank you. Is this the best place for coding style
documentation? https://www.emacswiki.org/emacs/CodingStyle

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

On 02/27/2018 09:38 AM, Aaron Jensen wrote:
> Is this the best place for coding style
> documentation?https://www.emacswiki.org/emacs/CodingStyle

I don't recall reading that until now, and it doesn't seem to be all 
that helpful for the style used in C code in Emacs. We generally follow 
the GNU coding standards, which talks about some of the issues involved.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Tue, Feb 27, 2018 at 9:30 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> Nobody else commented, so I reformatted the patch a bit to be more in the
> Emacs style and installed the attached into master.

I forgot, but given that this is a show-stopper bug for a new
headline/experimental feature for Emacs 26, would it be worth applying
this to the emacs-26 branch as well?

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Tue, 27 Feb 2018 21:33:01 -0800
> Cc: Eli Zaretskii <eliz@gnu.org>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> On Tue, Feb 27, 2018 at 9:30 AM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> > Nobody else commented, so I reformatted the patch a bit to be more in the
> > Emacs style and installed the attached into master.
> 
> I forgot, but given that this is a show-stopper bug for a new
> headline/experimental feature for Emacs 26, would it be worth applying
> this to the emacs-26 branch as well?

What feature is that, and why is this a showstopper?

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Wed, Feb 28, 2018 at 7:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> What feature is that, and why is this a showstopper?

Threads, because the first time I tried to use it in earnest Emacs
crashed often.

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

Aaron Jensen wrote:
> would it be worth applying
> this to the emacs-26 branch as well?

I suppose so, but it's Eli's call. (Disclaimer: I am not a macOS user.)

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Wed, 28 Feb 2018 07:35:26 -0800
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> On Wed, Feb 28, 2018 at 7:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > What feature is that, and why is this a showstopper?
> 
> Threads, because the first time I tried to use it in earnest Emacs
> crashed often.

You mean, when you tried to start 100 threads at the same time?

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Wed, Feb 28, 2018 at 8:20 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> You mean, when you tried to start 100 threads at the same time?

No. I'm sorry I've apparently done such a terrible job of explaining
this. I ran into this when using a single thread. You only need one
thread that triggers a GC to cause this, it just needs to happen to
happen during that thread.

The 100 threads thing was just a repro that set it up in such a way
that I could repro it fairly reliably.

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Aaron Jensen <aaronjensen@gmail.com>
> Date: Wed, 28 Feb 2018 09:32:08 -0800
> Cc: Paul Eggert <eggert@cs.ucla.edu>, Andreas Schwab <schwab@linux-m68k.org>, 
> 	Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org, 
> 	Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> 
> No. I'm sorry I've apparently done such a terrible job of explaining
> this. I ran into this when using a single thread. You only need one
> thread that triggers a GC to cause this, it just needs to happen to
> happen during that thread.

But that is a macOS only problem, because the default stack size is
ridiculously small.  The fix that got installed, OTOH, is much more
general.

If you want to make a macOS-only change that enlarges its stack we
give to threads, I can live with that on emacs-26, provided that the
fix only affects threads other than the main one.

Thanks.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Wed, Feb 28, 2018 at 9:38 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> But that is a macOS only problem, because the default stack size is
> ridiculously small.  The fix that got installed, OTOH, is much more
> general.
>
> If you want to make a macOS-only change that enlarges its stack we
> give to threads, I can live with that on emacs-26, provided that the
> fix only affects threads other than the main one.

So wrap the new code in an #if DARWIN_OS?

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

[-- Attachment #1: Type: text/plain, Size: 337 bytes --]

On Wed, Feb 28, 2018 at 9:59 AM, Aaron Jensen <aaronjensen@gmail.com> wrote:
>> If you want to make a macOS-only change that enlarges its stack we
>> give to threads, I can live with that on emacs-26, provided that the
>> fix only affects threads other than the main one.

Attached. AFAICT sys_create_thread is only used by make-thread.

[-- Attachment #2: 0001-Require-a-larger-stack-size-for-threads-on-macOS-bug.patch --]
[-- Type: application/octet-stream, Size: 1208 bytes --]

From 56d9daa184f9e7fab34524863fdeaba52a742a8e Mon Sep 17 00:00:00 2001
From: Aaron Jensen <aaronjensen@gmail.com>
Date: Wed, 28 Feb 2018 10:05:30 -0800
Subject: [PATCH] Require a larger stack size for threads on macOS (bug#30364)

* src/systhread.c (sys_thread_create) [THREADS_ENABLED && HAVE_PTHREAD]:
Require at least 8MB stack size for x64 and 4MB for x86 on macOS.
---
 src/systhread.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/systhread.c b/src/systhread.c
index 4ffb7db143..550318d90f 100644
--- a/src/systhread.c
+++ b/src/systhread.c
@@ -165,6 +165,15 @@ sys_thread_create (sys_thread_t *thread_ptr, const char *name,
   if (pthread_attr_init (&attr))
     return 0;
 
+#if defined (DARWIN_OS)
+  /* Avoid crash on macOS with deeply nested GC (Bug#30364).  */
+  size_t stack_size;
+  size_t required_stack_size = sizeof (void *) * 1024 * 1024;
+  if (pthread_attr_getstacksize (&attr, &stack_size) == 0
+      && stack_size < required_stack_size)
+    pthread_attr_setstacksize (&attr, required_stack_size);
+#endif
+
   if (!pthread_attr_setdetachstate (&attr, PTHREAD_CREATE_DETACHED))
     {
       result = pthread_create (thread_ptr, &attr, func, arg) == 0;
-- 
2.15.1

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

On 02/28/2018 10:07 AM, Aaron Jensen wrote:
> Attached. AFAICT sys_create_thread is only used by make-thread.

This patch looks OK to me, to be applied to the emacs-26 branch (not 
master).

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> Cc: Andreas Schwab <schwab@linux-m68k.org>,
>  Noam Postavsky <npostavs@gmail.com>, 30364@debbugs.gnu.org,
>  Noam Postavsky <npostavs@users.sourceforge.net>, Alan Third <alan@idiocy.org>
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 28 Feb 2018 11:15:12 -0800
> 
> On 02/28/2018 10:07 AM, Aaron Jensen wrote:
> > Attached. AFAICT sys_create_thread is only used by make-thread.
> 
> This patch looks OK to me, to be applied to the emacs-26 branch (not 
> master).

I agree.  (The master branch already has a better fix.)

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

On 02/28/2018 12:36 PM, Eli Zaretskii wrote:
> I agree.  (The master branch already has a better fix.)

OK, I installed it into the emacs-26 branch.

bug#30364: 26.0.91; thread crash on macos

[Aaron Jensen]

On Wed, Feb 28, 2018 at 4:31 PM, Paul Eggert <eggert@cs.ucla.edu> wrote:
> On 02/28/2018 12:36 PM, Eli Zaretskii wrote:
>> I agree.  (The master branch already has a better fix.)
> OK, I installed it into the emacs-26 branch.

Thank you, Paul and Eli.

bug#30364: 26.0.91; thread crash on macos

[Noam Postavsky]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/28/2018 12:36 PM, Eli Zaretskii wrote:
>> I agree.  (The master branch already has a better fix.)
>
> OK, I installed it into the emacs-26 branch.

Should this bug be closed then?

bug#30364: 26.0.91; thread crash on macos

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  30364@debbugs.gnu.org,  alan@idiocy.org,  aaronjensen@gmail.com,  schwab@linux-m68k.org
> Date: Sun, 13 May 2018 11:13:36 -0400
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> 
> > On 02/28/2018 12:36 PM, Eli Zaretskii wrote:
> >> I agree.  (The master branch already has a better fix.)
> >
> > OK, I installed it into the emacs-26 branch.
> 
> Should this bug be closed then?

Yes, I think so.

bug#30364: 26.0.91; thread crash on macos

[Paul Eggert]

>> Should this bug be closed then? 
> Yes, I think so.

OK, closing.