bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[dick.r.chiang]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: 0001-TLS-bureaucracy-flusters-test.patch --]
[-- Type: text/x-diff, Size: 2833 bytes --]

From f84829428d2e912f925ceccd95c1d81aebe64108 Mon Sep 17 00:00:00 2001
From: dickmao <none>
Date: Wed, 13 Oct 2021 16:39:05 -0400
Subject: [PATCH] TLS bureaucracy flusters test

It's never ideal when a test pings the internet (messes up statistics,
pointlessly adds to WAN congestion), but I get it's a pain to stand up
a toy https server.  See also

https://www.reddit.com/r/emacs/comments/pyevj8/what_should_i_do_cant_use_the_package_manager_it/?utm_source=share&utm_medium=web2x&context=3

* test/src/process-tests.el (process-async-https-with-delay):
Dial down the security.
---
 test/src/process-tests.el | 38 +++++++++++++++++---------------------
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/test/src/process-tests.el b/test/src/process-tests.el
index 44f3ea2fbb..ee64a145b7 100644
--- a/test/src/process-tests.el
+++ b/test/src/process-tests.el
@@ -920,31 +920,27 @@ process-sentinel-interrupt-event
 (ert-deftest process-async-https-with-delay ()
   "Bug#49449: asynchronous TLS connection with delayed completion."
   (skip-unless (and internet-is-working (gnutls-available-p)))
-  (let* ((status nil)
+  (let* (status
+         (network-security-level 'low)
          (buf (url-http
-                 #s(url "https" nil nil "elpa.gnu.org" nil
-                        "/packages/archive-contents" nil nil t silent t t)
-                 (lambda (s) (setq status s))
-                 '(nil) nil 'tls)))
+               #s(url "https" nil nil "elpa.gnu.org" nil
+                      "/packages/archive-contents" nil nil t silent t t)
+               (lambda (s) (setq status s))
+               '(nil) nil 'tls)))
     (unwind-protect
         (progn
-          ;; Busy-wait for 1 s to allow for the TCP connection to complete.
-          (let ((delay 1.0)
-                (t0 (float-time)))
-            (while (< (float-time) (+ t0 delay))))
-          ;; Wait for the entire operation to finish.
-          (let ((limit 4.0)
-                (t0 (float-time)))
-            (while (and (null status)
-                        (< (float-time) (+ t0 limit)))
-              (sit-for 0.1)))
+          (catch 'done
+            (dotimes (_i 40)
+              (when status
+                (throw 'done status))
+              (accept-process-output nil 0.1)))
           (should status)
-          (should-not (assq :error status))
-          (should buf)
-          (should (> (buffer-size buf) 0))
-          )
-      (when buf
-        (kill-buffer buf)))))
+          (should-not (plist-get status :error))
+          (should (buffer-live-p buf))
+          (should (> (buffer-size buf) 0)))
+      (when (buffer-live-p buf)
+        (let (kill-buffer-query-functions)
+          (kill-buffer buf))))))
 
 (ert-deftest process-num-processors ()
   "Sanity checks for num-processors."
-- 
2.26.2


[-- Attachment #2: Type: text/plain, Size: 7117 bytes --]




In Commercial Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.10)
 of 2021-10-13 built on dick
Repository revision: cfc2ffe8668b87f4951e73230db3703b2c251d24
Repository branch: process
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.4 LTS

Configured using:
 'configure --prefix=/home/dick/.local --enable-dumping-overwrite CC=gcc-10
 PKG_CONFIG_PATH=/home/dick/.local/lib/pkgconfig'
Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON LCMS2
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB
Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  gnus-undo-mode: t
  shell-dirtrack-mode: t
  projectile-mode: t
  flx-ido-mode: t
  override-global-mode: t
  global-hl-line-mode: t
  winner-mode: t
  tooltip-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/dick/gomacro-mode/gomacro-mode hides /home/dick/.emacs.d/elpa/gomacro-mode-20200326.1103/gomacro-mode
/home/dick/.emacs.d/elpa/hydra-20170924.2259/lv hides /home/dick/.emacs.d/elpa/lv-20191106.1238/lv
/home/dick/.emacs.d/elpa/magit-3.3.0/magit-section-pkg hides /home/dick/.emacs.d/elpa/magit-section-3.3.0/magit-section-pkg
/home/dick/org-gcal.el/org-gcal hides /home/dick/.emacs.d/elpa/org-gcal-0.3/org-gcal
/home/dick/.emacs.d/lisp/json hides /home/dick/.local/share/emacs/28.0.50/lisp/json
/home/dick/.emacs.d/elpa/transient-0.3.6/transient hides /home/dick/.local/share/emacs/28.0.50/lisp/transient
/home/dick/.emacs.d/elpa/hierarchy-20171221.1151/hierarchy hides /home/dick/.local/share/emacs/28.0.50/lisp/emacs-lisp/hierarchy

Features:
(shadow sort bbdb-message footnote mail-extr emacsbug sendmail
gnus-notifications gnus-fun notifications gnus-kill gnus-dup disp-table utf-7
mm-archive url-cache nnrss nnfolder nndiscourse benchmark rbenv nnhackernews
nntwitter nntwitter-api bbdb-gnus gnus-demon nntp nnmairix nnml nnreddit
gnus-topic url-http url-auth url-gw network-stream gnutls nsm
virtualenvwrapper gud s json-rpc python tramp-sh gnus-score score-mode
gnus-bcklg gnus-srvr gnus-cite bbdb-mua bbdb-com crm bbdb bbdb-site timezone
gnus-delay gnus-draft gnus-cache gnus-agent gnus-msg nndraft nnmh use-package
use-package-delight use-package-diminish vc-git diff-mode vc-dispatcher
bug-reference org-element avl-tree ol-eww eww xdg url-queue ol-rmail ol-mhe
ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar ezimage dframe
gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum gnus-group mm-url
gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo gnus-spec gnus-int gnus-range message rmc rfc822 mml mml-sec epa epg
rfc6068 epg-config mm-decode mm-bodies mm-encode mailabbrev gmm-utils gnus-win
ol-docview doc-view jka-compr image-mode exif dired-x dired dired-loaddefs
ol-bibtex bibtex ol-bbdb ol-w3m ol-doi org-link-doi org-tempo tempo org
org-macro org-footnote org-pcomplete org-list org-faces org-entities
org-version ob-R ob-emacs-lisp ob-ein ein-cell ein-output-area shr kinsoku
puny svg dom ein-kernel ein-ipdb ein-query ein-events ein-websocket websocket
bindat ein-node ewoc ein-log ein-classes ein-core request mailheader
autorevert filenotify anaphora ein ein-utils deferred dash cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs ob
ob-tangle org-src ob-ref ob-lob ob-table ob-exp ob-comint ob-core ob-eval
org-table ol org-keys oc org-compat org-macs org-loaddefs find-func cal-menu
calendar cal-loaddefs tramp-archive tramp-gvfs tramp-cache zeroconf dbus xml
tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
pcomplete parse-time iso8601 ls-lisp format-spec paredit-ext paredit subed
subed-vtt subed-srt subed-common subed-mpv subed-debug subed-config inf-ruby
ruby-mode smie company pcase haskell-interactive-mode
haskell-presentation-mode haskell-process haskell-session haskell-compile
haskell-mode haskell-cabal haskell-utils haskell-font-lock haskell-indentation
haskell-string haskell-sort-imports haskell-lexeme rx haskell-align-imports
haskell-complete-module haskell-ghc-support noutline outline flymake-proc
flymake warnings etags fileloop generator xref project dabbrev
haskell-customize hydra lv use-package-ensure solarized-theme
solarized-definitions projectile lisp-mnt mail-parse rfc2231 ibuf-ext ibuffer
ibuffer-loaddefs thingatpt grep compile comint ansi-color gnus nnheader
gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils text-property-search time-date flx-ido flx
google-translate-default-ui google-translate-core-ui facemenu color ido
google-translate-core google-translate-tk google-translate-backend
use-package-bind-key bind-key auto-complete easy-mmode advice popup cus-edit
pp cus-load wid-edit emms-player-mplayer emms-player-simple emms emms-compat
cl-extra help-mode use-package-core derived hl-line winner ring edmacro kmacro
finder-inf json-reformat-autoloads json-snatcher-autoloads sml-mode-autoloads
tornado-template-mode-autoloads info package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode
register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj
charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev
obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind inotify
lcms2 dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 805970 97599)
 (symbols 48 43370 1)
 (strings 32 174394 19790)
 (string-bytes 1 6003490)
 (vectors 16 87423)
 (vector-slots 8 1641774 40738)
 (floats 8 1842 315)
 (intervals 56 684 0)
 (buffers 992 25))

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[Lars Ingebrigtsen]

dick.r.chiang@gmail.com writes:

> * test/src/process-tests.el (process-async-https-with-delay):
> Dial down the security.

[...]

> +         (network-security-level 'low)

Well, I see your point -- this test isn't about testing security stuff,
but only a specific TLS negotiating thing, so disabling the security
stuff makes some sense.  But on the other hand, we're then not testing
the complete normal code path, so...

So I'm not sure.  I'm leaning towards applying the patch, but does
anybody else have an opinion here first?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[dick]

> Well, I see your point -- this test isn't about testing security stuff

My larger point is `make check` fails as of October 13 because some
certificate in the ELPA tls chain has fallen out of favor with the notoriously
capricious security gods.

It's becoming increasingly obvious to me that I'm the only person in the
emacsverse who regularly runs `make check`, as no one else, including the
maintainers, seems to notice `make check` failures as quickly as I do.

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[Lars Ingebrigtsen]

dick <dick.r.chiang@gmail.com> writes:

>> Well, I see your point -- this test isn't about testing security stuff
>
> My larger point is `make check` fails as of October 13 because some
> certificate in the ELPA tls chain has fallen out of favor with the notoriously
> capricious security gods.

This means that your TLS toolchain is out of date and you should update
it.  (Google for "Let's Encrypt meltdown of September 30th 2021".)

> It's becoming increasingly obvious to me that I'm the only person in the
> emacsverse who regularly runs `make check`, as no one else, including the
> maintainers, seems to notice `make check` failures as quickly as I do.

I run "make check" every day.  The test works fine on Debian/bullseye.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[dick]

>>>>> Lars Ingebrigtsen <larsi@gnus.org> writes:

> (Google for "Let's Encrypt meltdown of September 30th 2021".)

My first Google hit was a Tolstoyesque treatise from "catchpoint.com".
Related to our earlier discussion about length versus comprehensibility, it
really is remarkable how deluded IT people are about the layperson who
perceives security as a tedious and probabalistically unnnecessary precaution.
Doctors and lawyers do not hold forth about the viscissitudes of their daily
work.  Perhaps that is why they are paid more.

I then googled "ubuntu ca-certificates update" and got what I needed.

I shall add crow to my grocery list for having mistaken this for a bug.  I'll
close this issue presently.

bug#51197: 28.0.50; [PATCH] get process-async-https-with-delay to pass

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Date: Thu, 14 Oct 2021 18:50:24 -0400
> Cc: 51197@debbugs.gnu.org
> 
> It's becoming increasingly obvious to me that I'm the only person in the
> emacsverse who regularly runs `make check`, as no one else, including the
> maintainers, seems to notice `make check` failures as quickly as I do.

No, the rest solved the problem with that certificate, so the failure
doesn't happen for them.  The Internet is full with recipes for how to
solve this.