bug#15695: 24.3; multiple async shell commands at once run randomly

[Noam Postavsky <npostavs@users.sourceforge.net>]

retitle 15695 Incomplete data from subprocesses with pty connections
quit

Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:

>   (defvar statistics nil "Hash table for keeping statistics")
>   (defun yf/keep-stats (process change) "Used as sentinel"
>     (with-current-buffer (process-buffer process)
>       (if (equal change "finished\n")
>           (puthash (buffer-string) (1+ (or (gethash (buffer-string) statistics) 0)) statistics)
>         (error "Unknown change for process %s: %s" process change))))
>   (defun yf/pp-hash-table (hashtable) "Pretty print hash table"
>     (let ((res))
>       (maphash (lambda (k v) (setq res (concat res (yf/pp-hash--line k v)))) hashtable)
>       (or res "")))
>   (defun yf/pp-hash--line (key value) "Pretty print one hash line"
>     (format "%s: %s\n" (pp key) (pp value)))
>   (progn 
>     (let ((n 820))
>       (setq statistics (make-hash-table :test 'equal))
>       (dotimes (i n)
>         (let* ((buffername (concat "Shell-test" (number-to-string i)))
>                (buffer (get-buffer-create buffername)))
>           (with-current-buffer buffer
>             (erase-buffer))
>           (set-process-sentinel
>            (start-process buffername
>                           buffer
>                           shell-file-name
>                           shell-command-switch "echo foo&echo bar&echo baz")
>            #'yf/keep-stats)))
>       t)
>     (sit-for 2)
>     (insert (yf/pp-hash-table statistics)))

If I set process-connection-type to nil, then I get expected results
(though I had to reduce n in order to avoid bumping into the file
descriptor limit):

"baz\nfoo\nbar\n": 58
"baz\nbar\nfoo\n": 41
"foo\nbaz\nbar\n": 1

Without setting process-connection-type (all tests with 26.0.91):

"baz\n": 86
"baz\nbar\n": 5
"baz\nfoo\n": 6
"foo\nbaz\n": 1
"baz\nfoo\nbar\n": 2

This problem is also talked about in [1] and [2].

[1]: http://nullprogram.com/blog/2018/01/17/
[2]: http://nullprogram.com/blog/2014/02/06/#pseudo-terminals