From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Helmut Eller Newsgroups: gmane.emacs.bugs Subject: bug#65211: 30.0.50; threads and accept-process-output Date: Thu, 10 Aug 2023 17:04:48 +0200 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10578"; mail-complaints-to="usenet@ciao.gmane.io" To: 65211@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Aug 10 17:06:25 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qU7F3-0002VU-It for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 10 Aug 2023 17:06:25 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qU7Ei-00060A-TG; Thu, 10 Aug 2023 11:06:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qU7Eg-000600-VV for bug-gnu-emacs@gnu.org; Thu, 10 Aug 2023 11:06:03 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qU7Eg-0005Vn-Md for bug-gnu-emacs@gnu.org; Thu, 10 Aug 2023 11:06:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qU7Eg-0007Si-Ak for bug-gnu-emacs@gnu.org; Thu, 10 Aug 2023 11:06:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Helmut Eller Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 10 Aug 2023 15:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 65211 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.169167991928627 (code B ref -1); Thu, 10 Aug 2023 15:06:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 10 Aug 2023 15:05:19 +0000 Original-Received: from localhost ([127.0.0.1]:43827 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qU7Dy-0007Rc-7n for submit@debbugs.gnu.org; Thu, 10 Aug 2023 11:05:18 -0400 Original-Received: from lists.gnu.org ([2001:470:142::17]:40436) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qU7Dt-0007RH-0B for submit@debbugs.gnu.org; Thu, 10 Aug 2023 11:05:16 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qU7Dn-0005K5-1z for bug-gnu-emacs@gnu.org; Thu, 10 Aug 2023 11:05:07 -0400 Original-Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qU7Dj-000568-0n for bug-gnu-emacs@gnu.org; Thu, 10 Aug 2023 11:05:06 -0400 Original-Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-3fe1e1142caso9245735e9.0 for ; Thu, 10 Aug 2023 08:05:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691679899; x=1692284699; h=mime-version:message-id:date:subject:to:from:from:to:cc:subject :date:message-id:reply-to; bh=IZK2EG0B2I8A4f5F0huuFX5YXUnCxk/18h+ZVbeYTME=; b=iyi7dh2tSrjTicxCPzeQmEYd5IMkF/ULo7uKwV/ddiIj963Sb9xYzoy8SOgeyNRgfh ug8X3sFH+NQMDNZJ2NUdhrxb26EYPyQNxYrDz6n8xywO0oCuwK6bkI0Ztrjzd4KF67Z5 JYSyWpGT9TbezmSc8JoISC+LgnGG0tR9znEi5qWo0Z9p68d14+zsfpdwjvMRJgj1wsNJ aToLCNSA3y05P2f4yis82SIzkZy9WRNzzVlRadsCbG36nc/2PFuT67LfrcDnBWEfdt6I poEEUyGvQXFjabVNVovm7NEovSQuRYMJpizG//KoZ6YSaWJCLJZ33i3kVINx+WJ2t6wV N+3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691679899; x=1692284699; h=mime-version:message-id:date:subject:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=IZK2EG0B2I8A4f5F0huuFX5YXUnCxk/18h+ZVbeYTME=; b=gpuLs1jyqik1JVfrVmto3Gi93OIKf8zRYMxmPmhRsHtQm+pgfzRX8r8koGSpXy7nwt Ya5HH5G0MEU4WDeDvP3T/RDM+c4zo7pNqHEYh8aQXAL0zk9zYmiVEEWPYU28j6joqpvn 8G7ZdAWvGWDb3ZA7OQ03rHDpb2WJt5YgzK9ELhmYkFTs1BHDeRZqH/DhUxvLy8Vkl+Wy g6uLYV7eMLYBJrzC73Znc1K+i8Fih2hHgRXK6FkBgb8LiOOpc6RqcpcGSXV/mQJoG30y BFzu3F1gXVRkK9jBGuX9eYQOJHAOfvJyRkVVQ01QtCK0TJKMrpB0fCkUylMVN+Ku07Rm b8sA== X-Gm-Message-State: AOJu0YxouJd7KtoGSVWSUPyxD9gUsOQmjKDuUOfb6zGdP6+ukh+9GL0j nxaE/pjzpf4J55jkoCGQzCA0HU57uCo= X-Google-Smtp-Source: AGHT+IEFQrGrYAlxiIQ5aB3drnI2D2R56X4zgWEhK1hB8fpH6COe4nfKaqCLU1dtLoJiJm2552IzxA== X-Received: by 2002:adf:fdc6:0:b0:317:731c:4d80 with SMTP id i6-20020adffdc6000000b00317731c4d80mr2099441wrs.24.1691679898899; Thu, 10 Aug 2023 08:04:58 -0700 (PDT) Original-Received: from caladan (dialin-228086.xdsl.raiffeisen.net. [195.254.228.86]) by smtp.gmail.com with ESMTPSA id b13-20020adfe30d000000b003113ed02080sm2420217wrj.95.2023.08.10.08.04.58 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Aug 2023 08:04:58 -0700 (PDT) Received-SPF: pass client-ip=2a00:1450:4864:20::332; envelope-from=eller.helmut@gmail.com; helo=mail-wm1-x332.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:267142 Archived-At: --=-=-= Content-Type: text/plain The test case from below hangs when executed with emacs -Q --batch -l apo-tests.el -f ert-run-tests-batch-and-exit It's not 100% reproducible, but it seems to hang 80% of the time. The test is fairly complicated and involves a TLS connection, a sub-process and a background thread. The background thread starts a sub-process and reads all its output until the process terminates. Then the main thread opens a TLS connection, sends a HTTP request and tries to read the response. At this point, accept-process-output hangs even though there is output available. I think the reason for this problem is that the sub-process uses a file descriptor (usually number 6) and sets fd_callback_info[6].waiting_thread to the background thread (in compute_non_keyboard_wait_mask) . Later, the TLS connection receives a file descriptor with the same number (6) because the sub-process closed it already. That would be fine, however fd_callback_info[6].waiting_thread is still set to the background thread. So this time compute_non_keyboard_wait_mask doesn't include 6 in the wait mask. Because 6 is not in the wait mask, emacs_gnutls_record_check_pending will not be called and Emacs doesn't notice the buffered output. The intention in the code seems to be that clear_waiting_thread_info resets fd_callback_info[6].waiting_thread to NULL, however by the time that clear_waiting_thread_info is called, max_desc was reduced to 4, because somebody closed file descriptors 5 and 6. So clear_waiting_thread_info doesn't touch fd_callback_info[6]. A possible solution would be to reset the .waiting_thread field in delete_read_fd, like so: diff --git a/src/process.c b/src/process.c index 08cb810ec13..74d0bf252ab 100644 --- a/src/process.c +++ b/src/process.c @@ -513,6 +513,9 @@ delete_read_fd (int fd) { fd_callback_info[fd].func = 0; fd_callback_info[fd].data = 0; + + if (fd_callback_info[fd].waiting_thread == current_thread) + fd_callback_info[fd].waiting_thread = NULL; } } With this change, the test doesn't hang. Helmut --=-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: inline; filename=apo-tests.el Content-Transfer-Encoding: quoted-printable ;; -*- lexical-binding: t -*- (require 'ert) (require 'gnutls) (defun t--read (proc nchars) "Read at least NCHARS characters from process PROC. Return 'ok on success; otherwise return 'not-live." (let* ((buf (process-buffer proc)) (size (+ (buffer-size buf) nchars)) (result nil)) (t--accept-process-output proc 0) (while (eq result nil) (cond ((>=3D (buffer-size buf) size) (setq result 'ok)) ((not (process-live-p proc)) (setq result 'not-live)) (t (t--accept-process-output proc nil)))) result)) ;; a wrapper around accept-process-output for debugging (defun t--accept-process-output (proc timeout) (let ((buf (process-buffer proc))) (message "%s accept-process-output %S %s %s =E2=80=A6" (current-thread) proc (process-status proc) timeout) (let ((val (accept-process-output proc timeout))) (message "%s accept-process-output %S %s %s =3D> %S %d" (current-thread) proc (process-status proc) timeout val (buffer-size buf)) val))) (defun t--read-all (proc) (while (pcase-exhaustive (t--read proc 1) ('ok t) ('not-live nil))) (with-current-buffer (process-buffer proc) (buffer-string))) (defun t--make-buffer (name) (with-current-buffer (generate-new-buffer name t) (buffer-disable-undo) (set-buffer-multibyte nil) (current-buffer))) (ert-deftest t-tls () (thread-join (make-thread (lambda () (let* ((proc (make-process :name "cat" :command (list "cat") :sentinel (lambda (_ _)) :buffer (t--make-buffer "cat")))) (process-send-eof proc) (should (equal (t--read-all proc) "")))))) (let* ((host "www.example.net") (proc (make-network-process :name "tls" :host host :service 443 :tls-parameters (cons 'gnutls-x509pki (gnutls-boot-parameters :hostname host)) :buffer (t--make-buffer "tls") :sentinel (lambda (_ _)))) (request (format "GET / HTTP/1.1\r\nHost: %s\r\n\r\n" host))) (process-send-string proc request) (process-send-eof proc) (let* ((response (t--read-all proc))) (should (string-match "^HTTP/1.1 200 OK" response))))) ;; Local Variables: ;; read-symbol-shorthands: (("t-" . "apo-tests-")) ;; End: --=-=-= Content-Type: text/plain In GNU Emacs 30.0.50 (build 71, x86_64-pc-linux-gnu, GTK+ Version 3.24.37, cairo version 1.16.0) of 2023-08-10 built on caladan Repository revision: 164588b174774eba0c3bd6999633a39bed748195 Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12101007 System Description: Debian GNU/Linux 12 (bookworm) Configured using: 'configure --enable-checking=yes --with-xpm=ifavailable --with-gif=ifavailable 'CFLAGS=-g -O1'' Configured features: CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM XINPUT2 GTK3 ZLIB Important settings: value of $LANG: C.UTF-8 locale-coding-system: utf-8-unix --=-=-=--