* bug#72358: 29.4; oauth2.el improvements @ 2024-07-29 21:25 Xiyue Deng 2024-07-30 7:46 ` Robert Pluim ` (3 more replies) 0 siblings, 4 replies; 49+ messages in thread From: Xiyue Deng @ 2024-07-29 21:25 UTC (permalink / raw) To: 72358 [-- Attachment #1: Type: text/plain, Size: 72931 bytes --] Hi, I have been trying out using oauth2.el to enable OAuth2-based authentication for email service providers and had some success for Gmail. During this process, I have made a few changes to oauth2.el that enables it to use with Gmail OAuth2 as well as some usability and debugging improvements, which I'm sharing below. This is a series of five patches, which are attached. The first patch shows the authentication URL in the minibuffer window alongside the prompt accepting the authorization code. This helps when a user has multiple accounts from the same provider but is logged into a different account than the one that the user is trying to set up. If the user use the link (or through `browse-url') it will use the active account instead of the one intended. By showing the URL in the minibuffer, the user can choose other ways to get the authorization code (e.g. using another browser, using private/encognito mode, etc.) The second patch adds the parameters `access_type=offline' and `prompt=consent' to the authorization URL, which is required for Gmail OAuth2 to get the refresh token. Without these 2 parameters, Gmail response will only contain the access token which expires in one hour. They should also be compatible with other OAuth2 authentication process. (Though I am currently having trouble to get outlook.com to work regardless of these parameters, which I'll ask in a separate thread.) Note that the second patch depends on the first patch as they modify the same part of the code. The third patch encodes the parameters for requesting refreshing access token, which is recommended because the client secret and other parameters may contain characters that may break parameter parsing. The fourth patch may need a bit of background: oauth2.el (optionally) uses plstore to save authentication data for future reuse, and the plstore id for an account is computed using a combination of `auth-url', `token-url', and `scope'. However, this combination of data doesn't guarantee uniqueness for accounts for a same provider, e.g. for Gmail, the three parameters are the same for different accounts, and hence storing a second account information will override the first one. This fourth patch adds `client-id' to the calculation of plstore id to ensure its uniqueness. This may cause a few concerns: - This will invalidate all existing entries and a user will have to redo the authorization process again to get a new refresh token. However, I think it's more important to ensure that oauth2.el works correctly for multiple accounts of the same provider, or a user may suffer from confusion when adding a new account invalidates a previous account. - Adding `client-id' to the calculation of plstore id may provoke suspicion of leaking it as the hash calculation uses md5. In most cases, requesting a refresh token requires both `client-id' and `client-secret', so without including the latter it should be safe. There are cases when requesting only the access token may work with `client-id' along. Still, I think this should not be a big concern as the data is combined with `auth-url', `token-url', and `scope' which provides sufficient salt. Alternatively, we can also choose to use a more secure hash function, e.g. SHA2 or better, given that existing entries will be invalidated anyway. The fifth patch adds debug messages when doing a URL query which records the request URL, the request data, and the response data, and provide a custom variable to enable this. This provides a way to help debugging the requests, and I find it handy when testing oauth2 against different providers. Please review and comment. TIA! In GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.16.0) of 2024-07-02, modified by Debian built on debian-hx90 System Description: Debian GNU/Linux 12 (bookworm) Configured using: 'configure --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/libexec --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --with-libsystemd --with-pop=yes --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.4/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.4/site-lisp:/usr/share/emacs/site-lisp --with-sound=alsa --without-gconf --with-mailutils --with-native-compilation --build x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/libexec --localstatedir=/var/lib --infodir=/usr/share/info --mandir=/usr/share/man --with-libsystemd --with-pop=yes --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.4/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.4/site-lisp:/usr/share/emacs/site-lisp --with-sound=alsa --without-gconf --with-mailutils --with-native-compilation --with-cairo --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2 -ffile-prefix-map=/build/emacs-UNWIcy/emacs-29.4+1=. -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB Important settings: value of $LANG: en_US.UTF-8 locale-coding-system: utf-8-unix Major mode: VTerm Minor modes in effect: global-git-commit-mode: t magit-auto-revert-mode: t shell-dirtrack-mode: t mu4e-modeline-mode: t windmove-mode: t rcirc-track-minor-mode: t server-mode: t xclip-mode: t global-treesit-auto-mode: t treemacs-project-follow-mode: t treemacs-follow-mode: t treemacs-git-mode: t treemacs-fringe-indicator-mode: t corfu-terminal-mode: t corfu-popupinfo-mode: t corfu-echo-mode: t global-corfu-mode: t corfu-mode: t activities-tabs-mode: t activities-mode: t fido-vertical-mode: t icomplete-vertical-mode: t icomplete-mode: t fido-mode: t override-global-mode: t global-display-line-numbers-mode: t display-line-numbers-mode: t global-auto-revert-mode: t tooltip-mode: t global-eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t tab-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t buffer-read-only: t column-number-mode: t line-number-mode: t transient-mark-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t Load-path shadows: /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-autoloads hides /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/debian-autoloads /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-section hides /usr/share/emacs/site-lisp/elpa/magit-section-3.3.0/magit-section /usr/share/emacs/site-lisp/elpa/ace-window-0.10.0/ace-window-autoloads hides /usr/share/emacs/site-lisp/elpa-src/ace-window-0.10.0/ace-window-autoloads /usr/share/emacs/site-lisp/elpa/ace-window-0.10.0/ace-window-pkg hides /usr/share/emacs/site-lisp/elpa-src/ace-window-0.10.0/ace-window-pkg /usr/share/emacs/site-lisp/elpa/ace-window-0.10.0/ace-window hides /usr/share/emacs/site-lisp/elpa-src/ace-window-0.10.0/ace-window /usr/share/emacs/site-lisp/elpa/activities-0.7/activities-autoloads hides /usr/share/emacs/site-lisp/elpa-src/activities-0.7/activities-autoloads /usr/share/emacs/site-lisp/elpa/activities-0.7/activities-pkg hides /usr/share/emacs/site-lisp/elpa-src/activities-0.7/activities-pkg /usr/share/emacs/site-lisp/elpa/activities-0.7/activities-list hides /usr/share/emacs/site-lisp/elpa-src/activities-0.7/activities-list /usr/share/emacs/site-lisp/elpa/activities-0.7/activities hides /usr/share/emacs/site-lisp/elpa-src/activities-0.7/activities /usr/share/emacs/site-lisp/elpa/activities-0.7/activities-tabs hides /usr/share/emacs/site-lisp/elpa-src/activities-0.7/activities-tabs /usr/share/emacs/site-lisp/elpa/apache-mode-2.2.0/apache-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/apache-mode-2.2.0/apache-mode-autoloads /usr/share/emacs/site-lisp/elpa/apache-mode-2.2.0/apache-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/apache-mode-2.2.0/apache-mode-pkg /usr/share/emacs/site-lisp/elpa/apache-mode-2.2.0/apache-mode hides /usr/share/emacs/site-lisp/elpa-src/apache-mode-2.2.0/apache-mode /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-info hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-info /usr/share/emacs/site-lisp/elpa/auctex-13.3/latex-flymake hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/latex-flymake /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-site hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-site /usr/share/emacs/site-lisp/elpa/auctex-13.3/texmathp hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/texmathp /usr/share/emacs/site-lisp/elpa/auctex-13.3/toolbar-x hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/toolbar-x /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-style hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-style /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-font hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-font /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-jp hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-jp /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-mik hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-mik /usr/share/emacs/site-lisp/elpa/auctex-13.3/plain-tex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/plain-tex /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-fold hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-fold /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-ispell hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-ispell /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-bar hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-bar /usr/share/emacs/site-lisp/elpa/auctex-13.3/preview-latex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/preview-latex /usr/share/emacs/site-lisp/elpa/auctex-13.3/bib-cite hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/bib-cite /usr/share/emacs/site-lisp/elpa/auctex-13.3/preview hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/preview /usr/share/emacs/site-lisp/elpa/auctex-13.3/context-nl hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/context-nl /usr/share/emacs/site-lisp/elpa/auctex-13.3/auto-loads hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/auto-loads /usr/share/emacs/site-lisp/elpa/auctex-13.3/multi-prompt hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/multi-prompt /usr/share/emacs/site-lisp/elpa/auctex-13.3/context-en hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/context-en /usr/share/emacs/site-lisp/elpa/auctex-13.3/lpath hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/lpath /usr/share/emacs/site-lisp/elpa/auctex-13.3/auctex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/auctex /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex /usr/share/emacs/site-lisp/elpa/auctex-13.3/auctex-autoloads hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/auctex-autoloads /usr/share/emacs/site-lisp/elpa/auctex-13.3/font-latex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/font-latex /usr/share/emacs/site-lisp/elpa/auctex-13.3/auctex-pkg hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/auctex-pkg /usr/share/emacs/site-lisp/elpa/auctex-13.3/context hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/context /usr/share/emacs/site-lisp/elpa/auctex-13.3/latex hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/latex /usr/share/emacs/site-lisp/elpa/auctex-13.3/tex-wizard hides /usr/share/emacs/site-lisp/elpa-src/auctex-13.3/tex-wizard /usr/share/emacs/site-lisp/elpa/avy-0.5.0/avy hides /usr/share/emacs/site-lisp/elpa-src/avy-0.5.0/avy /usr/share/emacs/site-lisp/elpa/avy-0.5.0/avy-autoloads hides /usr/share/emacs/site-lisp/elpa-src/avy-0.5.0/avy-autoloads /usr/share/emacs/site-lisp/elpa/avy-0.5.0/avy-pkg hides /usr/share/emacs/site-lisp/elpa-src/avy-0.5.0/avy-pkg /usr/share/emacs/site-lisp/elpa/bazel-0/bazel-autoloads hides /usr/share/emacs/site-lisp/elpa-src/bazel-0/bazel-autoloads /usr/share/emacs/site-lisp/elpa/bazel-0/bazel hides /usr/share/emacs/site-lisp/elpa-src/bazel-0/bazel /usr/share/emacs/site-lisp/elpa/bazel-0/test hides /usr/share/emacs/site-lisp/elpa-src/bazel-0/test /usr/share/emacs/site-lisp/elpa/bazel-0/bazel-pkg hides /usr/share/emacs/site-lisp/elpa-src/bazel-0/bazel-pkg /usr/share/emacs/site-lisp/elpa/bison-mode-0.3/bison-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/bison-mode-0.3/bison-mode-pkg /usr/share/emacs/site-lisp/elpa/bison-mode-0.3/bison-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/bison-mode-0.3/bison-mode-autoloads /usr/share/emacs/site-lisp/elpa/bison-mode-0.3/bison-mode hides /usr/share/emacs/site-lisp/elpa-src/bison-mode-0.3/bison-mode /usr/share/emacs/site-lisp/elpa/boxquote-2.3/boxquote hides /usr/share/emacs/site-lisp/elpa-src/boxquote-2.3/boxquote /usr/share/emacs/site-lisp/elpa/boxquote-2.3/boxquote-autoloads hides /usr/share/emacs/site-lisp/elpa-src/boxquote-2.3/boxquote-autoloads /usr/share/emacs/site-lisp/elpa/boxquote-2.3/boxquote-pkg hides /usr/share/emacs/site-lisp/elpa-src/boxquote-2.3/boxquote-pkg /usr/share/emacs/site-lisp/elpa/buttercup-1.26/buttercup-pkg hides /usr/share/emacs/site-lisp/elpa-src/buttercup-1.26/buttercup-pkg /usr/share/emacs/site-lisp/elpa/buttercup-1.26/buttercup hides /usr/share/emacs/site-lisp/elpa-src/buttercup-1.26/buttercup /usr/share/emacs/site-lisp/elpa/buttercup-1.26/buttercup-autoloads hides /usr/share/emacs/site-lisp/elpa-src/buttercup-1.26/buttercup-autoloads /usr/share/emacs/site-lisp/elpa/buttercup-1.26/buttercup-compat hides /usr/share/emacs/site-lisp/elpa-src/buttercup-1.26/buttercup-compat /usr/share/emacs/site-lisp/elpa/cfrs-1.6.0/cfrs-pkg hides /usr/share/emacs/site-lisp/elpa-src/cfrs-1.6.0/cfrs-pkg /usr/share/emacs/site-lisp/elpa/cfrs-1.6.0/cfrs hides /usr/share/emacs/site-lisp/elpa-src/cfrs-1.6.0/cfrs /usr/share/emacs/site-lisp/elpa/cfrs-1.6.0/cfrs-autoloads hides /usr/share/emacs/site-lisp/elpa-src/cfrs-1.6.0/cfrs-autoloads /usr/share/emacs/site-lisp/elpa/clojure-mode-5.19.0/clojure-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-5.19.0/clojure-mode-pkg /usr/share/emacs/site-lisp/elpa/clojure-mode-5.19.0/clojure-mode hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-5.19.0/clojure-mode /usr/share/emacs/site-lisp/elpa/clojure-mode-5.19.0/clojure-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-5.19.0/clojure-mode-autoloads /usr/share/emacs/site-lisp/elpa/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking-pkg hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking-pkg /usr/share/emacs/site-lisp/elpa/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking-autoloads hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking-autoloads /usr/share/emacs/site-lisp/elpa/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking hides /usr/share/emacs/site-lisp/elpa-src/clojure-mode-extra-font-locking-3.0.0/clojure-mode-extra-font-locking /usr/share/emacs/site-lisp/elpa/cmake-mode-3.29.0/cmake-mode hides /usr/share/emacs/site-lisp/elpa-src/cmake-mode-3.29.0/cmake-mode /usr/share/emacs/site-lisp/elpa/cmake-mode-3.29.0/cmake-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/cmake-mode-3.29.0/cmake-mode-autoloads /usr/share/emacs/site-lisp/elpa/cmake-mode-3.29.0/cmake-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/cmake-mode-3.29.0/cmake-mode-pkg /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-dabbrev hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-dabbrev /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-capf hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-capf /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-yasnippet hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-yasnippet /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-ispell hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-ispell /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-etags hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-etags /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-template hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-template /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-abbrev hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-abbrev /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-files hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-files /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-css hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-css /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-tests /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-dabbrev-code hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-dabbrev-code /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-pkg hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-pkg /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-oddmuse hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-oddmuse /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-bbdb hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-bbdb /usr/share/emacs/site-lisp/elpa/company-0.10.2/company hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-clang hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-clang /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-nxml hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-nxml /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-gtags hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-gtags /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-tempo hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-tempo /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-autoloads hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-autoloads /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-cmake hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-cmake /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-tng hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-tng /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-elisp hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-elisp /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-semantic hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-semantic /usr/share/emacs/site-lisp/elpa/company-0.10.2/company-keywords hides /usr/share/emacs/site-lisp/elpa-src/company-0.10.2/company-keywords /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-26 hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-26 /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-28 hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-28 /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-macs hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-macs /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-autoloads hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-autoloads /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-pkg hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-pkg /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-25 hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-25 /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-29 hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-29 /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat-27 hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat-27 /usr/share/emacs/site-lisp/elpa/compat-29.1.4.5/compat hides /usr/share/emacs/site-lisp/elpa-src/compat-29.1.4.5/compat /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-quick hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-quick /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-info hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-info /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-history hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-history /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-popupinfo hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-popupinfo /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-indexed hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-indexed /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-pkg hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-pkg /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-echo hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-echo /usr/share/emacs/site-lisp/elpa/corfu-1.4/corfu-autoloads hides /usr/share/emacs/site-lisp/elpa-src/corfu-1.4/corfu-autoloads /usr/share/emacs/site-lisp/elpa/corfu-terminal-0.7/corfu-terminal-autoloads hides /usr/share/emacs/site-lisp/elpa-src/corfu-terminal-0.7/corfu-terminal-autoloads /usr/share/emacs/site-lisp/elpa/corfu-terminal-0.7/corfu-terminal-pkg hides /usr/share/emacs/site-lisp/elpa-src/corfu-terminal-0.7/corfu-terminal-pkg /usr/share/emacs/site-lisp/elpa/corfu-terminal-0.7/corfu-terminal hides /usr/share/emacs/site-lisp/elpa-src/corfu-terminal-0.7/corfu-terminal /usr/share/emacs/site-lisp/elpa/csv-mode-1.23/csv-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/csv-mode-1.23/csv-mode-autoloads /usr/share/emacs/site-lisp/elpa/csv-mode-1.23/csv-mode hides /usr/share/emacs/site-lisp/elpa-src/csv-mode-1.23/csv-mode /usr/share/emacs/site-lisp/elpa/csv-mode-1.23/csv-mode-tests hides /usr/share/emacs/site-lisp/elpa-src/csv-mode-1.23/csv-mode-tests /usr/share/emacs/site-lisp/elpa/csv-mode-1.23/csv-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/csv-mode-1.23/csv-mode-pkg /usr/share/emacs/site-lisp/elpa/dart-mode-1.0.7/dart-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/dart-mode-1.0.7/dart-mode-autoloads /usr/share/emacs/site-lisp/elpa/dart-mode-1.0.7/dart-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/dart-mode-1.0.7/dart-mode-pkg /usr/share/emacs/site-lisp/elpa/dart-mode-1.0.7/dart-mode hides /usr/share/emacs/site-lisp/elpa-src/dart-mode-1.0.7/dart-mode /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash-pkg hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash-pkg /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash-autoloads hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash-autoloads /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-el-autoloads hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/debian-el-autoloads /usr/share/emacs/site-lisp/elpa/debian-el-37.16/apt-sources hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/apt-sources /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-bug hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/debian-bug /usr/share/emacs/site-lisp/elpa/debian-el-37.16/apt-utils hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/apt-utils /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-el-pkg hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/debian-el-pkg /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-autoloads hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/debian-autoloads /usr/share/emacs/site-lisp/elpa/debian-el-37.16/gnus-BTS hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/gnus-BTS /usr/share/emacs/site-lisp/elpa/debian-el-37.16/deb-view hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/deb-view /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-el hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/debian-el /usr/share/emacs/site-lisp/elpa/debian-el-37.16/preseed hides /usr/share/emacs/site-lisp/elpa-src/debian-el-37.16/preseed /usr/share/emacs/site-lisp/elpa/debpaste-0.1.5/debpaste hides /usr/share/emacs/site-lisp/elpa-src/debpaste-0.1.5/debpaste /usr/share/emacs/site-lisp/elpa/debpaste-0.1.5/debpaste-pkg hides /usr/share/emacs/site-lisp/elpa-src/debpaste-0.1.5/debpaste-pkg /usr/share/emacs/site-lisp/elpa/debpaste-0.1.5/debpaste-autoloads hides /usr/share/emacs/site-lisp/elpa-src/debpaste-0.1.5/debpaste-autoloads /usr/share/emacs/site-lisp/elpa/devscripts-40/devscripts hides /usr/share/emacs/site-lisp/elpa-src/devscripts-40/devscripts /usr/share/emacs/site-lisp/elpa/devscripts-40/devscripts-autoloads hides /usr/share/emacs/site-lisp/elpa-src/devscripts-40/devscripts-autoloads /usr/share/emacs/site-lisp/elpa/devscripts-40/pbuilder-mode hides /usr/share/emacs/site-lisp/elpa-src/devscripts-40/pbuilder-mode /usr/share/emacs/site-lisp/elpa/devscripts-40/devscripts-pkg hides /usr/share/emacs/site-lisp/elpa-src/devscripts-40/devscripts-pkg /usr/share/emacs/site-lisp/elpa/devscripts-40/pbuilder-log-view-mode hides /usr/share/emacs/site-lisp/elpa-src/devscripts-40/pbuilder-log-view-mode /usr/share/emacs/site-lisp/elpa/dockerfile-mode-1.7/dockerfile-mode hides /usr/share/emacs/site-lisp/elpa-src/dockerfile-mode-1.7/dockerfile-mode /usr/share/emacs/site-lisp/elpa/dockerfile-mode-1.7/dockerfile-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/dockerfile-mode-1.7/dockerfile-mode-autoloads /usr/share/emacs/site-lisp/elpa/dockerfile-mode-1.7/dockerfile-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/dockerfile-mode-1.7/dockerfile-mode-pkg /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/debian-bts-control hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/debian-bts-control /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/debian-changelog-mode hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/debian-changelog-mode /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/debian-autopkgtest-control-mode hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/debian-autopkgtest-control-mode /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/dpkg-dev-el-autoloads hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/dpkg-dev-el-autoloads /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/dpkg-dev-el-pkg hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/dpkg-dev-el-pkg /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/dpkg-dev-el hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/dpkg-dev-el /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/debian-control-mode hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/debian-control-mode /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/debian-copyright hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/debian-copyright /usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.16/readme-debian hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.16/readme-debian /usr/share/emacs/site-lisp/elpa/exec-path-from-shell-2.1/exec-path-from-shell hides /usr/share/emacs/site-lisp/elpa-src/exec-path-from-shell-2.1/exec-path-from-shell /usr/share/emacs/site-lisp/elpa/exec-path-from-shell-2.1/exec-path-from-shell-autoloads hides /usr/share/emacs/site-lisp/elpa-src/exec-path-from-shell-2.1/exec-path-from-shell-autoloads /usr/share/emacs/site-lisp/elpa/exec-path-from-shell-2.1/exec-path-from-shell-pkg hides /usr/share/emacs/site-lisp/elpa-src/exec-path-from-shell-2.1/exec-path-from-shell-pkg /usr/share/emacs/site-lisp/elpa/format-all-0.6.0/format-all hides /usr/share/emacs/site-lisp/elpa-src/format-all-0.6.0/format-all /usr/share/emacs/site-lisp/elpa/format-all-0.6.0/format-all-pkg hides /usr/share/emacs/site-lisp/elpa-src/format-all-0.6.0/format-all-pkg /usr/share/emacs/site-lisp/elpa/format-all-0.6.0/format-all-autoloads hides /usr/share/emacs/site-lisp/elpa-src/format-all-0.6.0/format-all-autoloads /usr/share/emacs/site-lisp/elpa/git-commit-3.3.0/git-commit hides /usr/share/emacs/site-lisp/elpa-src/git-commit-3.3.0/git-commit /usr/share/emacs/site-lisp/elpa/git-commit-3.3.0/git-commit-autoloads hides /usr/share/emacs/site-lisp/elpa-src/git-commit-3.3.0/git-commit-autoloads /usr/share/emacs/site-lisp/elpa/git-commit-3.3.0/git-commit-pkg hides /usr/share/emacs/site-lisp/elpa-src/git-commit-3.3.0/git-commit-pkg /usr/share/emacs/site-lisp/elpa/git-modes-1.4.2/git-modes hides /usr/share/emacs/site-lisp/elpa-src/git-modes-1.4.2/git-modes /usr/share/emacs/site-lisp/elpa/git-modes-1.4.2/git-modes-pkg hides /usr/share/emacs/site-lisp/elpa-src/git-modes-1.4.2/git-modes-pkg /usr/share/emacs/site-lisp/elpa/git-modes-1.4.2/git-modes-autoloads hides /usr/share/emacs/site-lisp/elpa-src/git-modes-1.4.2/git-modes-autoloads /usr/share/emacs/site-lisp/elpa/gitattributes-mode-1.4.2/gitattributes-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/gitattributes-mode-1.4.2/gitattributes-mode-pkg /usr/share/emacs/site-lisp/elpa/gitattributes-mode-1.4.2/gitattributes-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/gitattributes-mode-1.4.2/gitattributes-mode-autoloads /usr/share/emacs/site-lisp/elpa/gitattributes-mode-1.4.2/gitattributes-mode hides /usr/share/emacs/site-lisp/elpa-src/gitattributes-mode-1.4.2/gitattributes-mode /usr/share/emacs/site-lisp/elpa/gitconfig-mode-1.4.2/gitconfig-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/gitconfig-mode-1.4.2/gitconfig-mode-autoloads /usr/share/emacs/site-lisp/elpa/gitconfig-mode-1.4.2/gitconfig-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/gitconfig-mode-1.4.2/gitconfig-mode-pkg /usr/share/emacs/site-lisp/elpa/gitconfig-mode-1.4.2/gitconfig-mode hides /usr/share/emacs/site-lisp/elpa-src/gitconfig-mode-1.4.2/gitconfig-mode /usr/share/emacs/site-lisp/elpa/gitignore-mode-1.4.2/gitignore-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/gitignore-mode-1.4.2/gitignore-mode-pkg /usr/share/emacs/site-lisp/elpa/gitignore-mode-1.4.2/gitignore-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/gitignore-mode-1.4.2/gitignore-mode-autoloads /usr/share/emacs/site-lisp/elpa/gitignore-mode-1.4.2/gitignore-mode hides /usr/share/emacs/site-lisp/elpa-src/gitignore-mode-1.4.2/gitignore-mode /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/gnuplot hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/gnuplot /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/gnuplot-autoloads hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/gnuplot-autoloads /usr/share/emacs/site-lisp/elpa/debian-el-37.16/debian-autoloads hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/debian-autoloads /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/gnuplot-pkg hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/gnuplot-pkg /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/gnuplot-context hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/gnuplot-context /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/gnuplot-gui hides /usr/share/emacs/site-lisp/elpa-src/gnuplot-0.8.1/gnuplot-gui /usr/share/emacs/site-lisp/elpa/go-mode-1.6.0/go-mode hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.6.0/go-mode /usr/share/emacs/site-lisp/elpa/go-mode-1.6.0/go-guru hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.6.0/go-guru /usr/share/emacs/site-lisp/elpa/go-mode-1.6.0/go-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.6.0/go-mode-pkg /usr/share/emacs/site-lisp/elpa/go-mode-1.6.0/go-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.6.0/go-mode-autoloads /usr/share/emacs/site-lisp/elpa/go-mode-1.6.0/go-rename hides /usr/share/emacs/site-lisp/elpa-src/go-mode-1.6.0/go-rename /usr/share/emacs/site-lisp/elpa/graphviz-dot-mode-0.4.2/graphviz-dot-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/graphviz-dot-mode-0.4.2/graphviz-dot-mode-autoloads /usr/share/emacs/site-lisp/elpa/graphviz-dot-mode-0.4.2/graphviz-dot-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/graphviz-dot-mode-0.4.2/graphviz-dot-mode-pkg /usr/share/emacs/site-lisp/elpa/graphviz-dot-mode-0.4.2/graphviz-dot-mode hides /usr/share/emacs/site-lisp/elpa-src/graphviz-dot-mode-0.4.2/graphviz-dot-mode /usr/share/emacs/site-lisp/elpa/ht-2.3/ht-pkg hides /usr/share/emacs/site-lisp/elpa-src/ht-2.3/ht-pkg /usr/share/emacs/site-lisp/elpa/ht-2.3/ht-autoloads hides /usr/share/emacs/site-lisp/elpa-src/ht-2.3/ht-autoloads /usr/share/emacs/site-lisp/elpa/ht-2.3/ht hides /usr/share/emacs/site-lisp/elpa-src/ht-2.3/ht /usr/share/emacs/site-lisp/elpa/hydra-0.15.0/hydra-ox hides /usr/share/emacs/site-lisp/elpa-src/hydra-0.15.0/hydra-ox /usr/share/emacs/site-lisp/elpa/hydra-0.15.0/hydra-autoloads hides /usr/share/emacs/site-lisp/elpa-src/hydra-0.15.0/hydra-autoloads /usr/share/emacs/site-lisp/elpa/hydra-0.15.0/hydra-pkg hides /usr/share/emacs/site-lisp/elpa-src/hydra-0.15.0/hydra-pkg /usr/share/emacs/site-lisp/elpa/hydra-0.15.0/hydra-examples hides /usr/share/emacs/site-lisp/elpa-src/hydra-0.15.0/hydra-examples /usr/share/emacs/site-lisp/elpa/hydra-0.15.0/hydra hides /usr/share/emacs/site-lisp/elpa-src/hydra-0.15.0/hydra /usr/share/emacs/site-lisp/elpa/inheritenv-0.2/inheritenv hides /usr/share/emacs/site-lisp/elpa-src/inheritenv-0.2/inheritenv /usr/share/emacs/site-lisp/elpa/inheritenv-0.2/inheritenv-autoloads hides /usr/share/emacs/site-lisp/elpa-src/inheritenv-0.2/inheritenv-autoloads /usr/share/emacs/site-lisp/elpa/inheritenv-0.2/inheritenv-pkg hides /usr/share/emacs/site-lisp/elpa-src/inheritenv-0.2/inheritenv-pkg /usr/share/emacs/site-lisp/elpa/inheritenv-0.2/inheritenv-tests hides /usr/share/emacs/site-lisp/elpa-src/inheritenv-0.2/inheritenv-tests /usr/share/emacs/site-lisp/elpa/language-id-0.20/language-id hides /usr/share/emacs/site-lisp/elpa-src/language-id-0.20/language-id /usr/share/emacs/site-lisp/elpa/language-id-0.20/language-id-pkg hides /usr/share/emacs/site-lisp/elpa-src/language-id-0.20/language-id-pkg /usr/share/emacs/site-lisp/elpa/language-id-0.20/language-id-autoloads hides /usr/share/emacs/site-lisp/elpa-src/language-id-0.20/language-id-autoloads /usr/share/emacs/site-lisp/elpa/lintian-0.1/lintian-pkg hides /usr/share/emacs/site-lisp/elpa-src/lintian-0.1/lintian-pkg /usr/share/emacs/site-lisp/elpa/lintian-0.1/lintian-autoloads hides /usr/share/emacs/site-lisp/elpa-src/lintian-0.1/lintian-autoloads /usr/share/emacs/site-lisp/elpa/lintian-0.1/lintian hides /usr/share/emacs/site-lisp/elpa-src/lintian-0.1/lintian /usr/share/emacs/site-lisp/elpa/lv-0.15.0/lv-autoloads hides /usr/share/emacs/site-lisp/elpa-src/lv-0.15.0/lv-autoloads /usr/share/emacs/site-lisp/elpa/lv-0.15.0/lv hides /usr/share/emacs/site-lisp/elpa-src/lv-0.15.0/lv /usr/share/emacs/site-lisp/elpa/lv-0.15.0/lv-pkg hides /usr/share/emacs/site-lisp/elpa-src/lv-0.15.0/lv-pkg /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-remote hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-remote /usr/share/emacs/site-lisp/elpa/magit-3.3.0/git-rebase hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/git-rebase /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-bisect hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-bisect /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-margin hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-margin /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-merge hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-merge /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-section hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-section /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-patch hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-patch /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-commit hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-commit /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-autoloads hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-autoloads /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-files hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-files /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-stash hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-stash /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-bookmark hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-bookmark /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-submodule hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-submodule /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-apply hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-apply /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-repos hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-repos /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-core hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-core /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-subtree hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-subtree /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-autorevert hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-autorevert /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-gitignore hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-gitignore /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-transient hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-transient /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-extras hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-extras /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-git hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-git /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-notes hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-notes /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-reflog hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-reflog /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-mode hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-mode /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-push hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-push /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-tag hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-tag /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-process hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-process /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-ediff hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-ediff /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-imenu hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-imenu /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-diff hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-diff /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-clone hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-clone /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-log hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-log /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-utils hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-utils /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-wip hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-wip /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-branch hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-branch /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-pull hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-pull /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-reset hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-reset /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-sequence hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-sequence /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-status hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-status /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-refs hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-refs /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-obsolete hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-obsolete /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-fetch hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-fetch /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-worktree hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-worktree /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-blame hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-blame /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-pkg hides /usr/share/emacs/site-lisp/elpa-src/magit-3.3.0/magit-pkg /usr/share/emacs/site-lisp/elpa/magit-section-3.3.0/magit-section-autoloads hides /usr/share/emacs/site-lisp/elpa-src/magit-section-3.3.0/magit-section-autoloads /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-section hides /usr/share/emacs/site-lisp/elpa-src/magit-section-3.3.0/magit-section /usr/share/emacs/site-lisp/elpa/magit-section-3.3.0/magit-section-pkg hides /usr/share/emacs/site-lisp/elpa-src/magit-section-3.3.0/magit-section-pkg /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-cgen hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-cgen /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-scan hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-scan /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-publish hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-publish /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-complete hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-complete /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-syntax hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-syntax /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/tlc hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/tlc /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-netshell hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-netshell /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/semantic-matlab hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/semantic-matlab /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/mlint hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/mlint /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/company-matlab-shell hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/company-matlab-shell /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-shell-gud hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-shell-gud /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/mlgud hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/mlgud /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/srecode-matlab hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/srecode-matlab /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-compat hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-compat /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-mode-pkg /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-maint hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-maint /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/cedet-matlab hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/cedet-matlab /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-topic hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-topic /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/linemark hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/linemark /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-shell hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-shell /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/semanticdb-matlab hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/semanticdb-matlab /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-load hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-load /usr/share/emacs/site-lisp/elpa/matlab-mode-4.0/matlab-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/matlab-mode-4.0/matlab-mode-autoloads /usr/share/emacs/site-lisp/elpa/meson-mode-0.2/meson-mode hides /usr/share/emacs/site-lisp/elpa-src/meson-mode-0.2/meson-mode /usr/share/emacs/site-lisp/elpa/meson-mode-0.2/utils hides /usr/share/emacs/site-lisp/elpa-src/meson-mode-0.2/utils /usr/share/emacs/site-lisp/elpa/meson-mode-0.2/meson-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/meson-mode-0.2/meson-mode-autoloads /usr/share/emacs/site-lisp/elpa/meson-mode-0.2/meson-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/meson-mode-0.2/meson-mode-pkg /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-draft hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-draft /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-modeline hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-modeline /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-view hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-view /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-message hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-message /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-helpers hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-helpers /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-pkg hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-pkg /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-bookmarks hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-bookmarks /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-thread hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-thread /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-mime-parts hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-mime-parts /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-server hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-server /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-query-items hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-query-items /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-contrib hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-contrib /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-window hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-window /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-config hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-config /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-autoloads hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-autoloads /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-icalendar hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-icalendar /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-mark hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-mark /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-headers hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-headers /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-org hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-org /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-contacts hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-contacts /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-speedbar hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-speedbar /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-obsolete hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-obsolete /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-vars hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-vars /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-actions hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-actions /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-main hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-main /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-search hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-search /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-notification hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-notification /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-context hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-context /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-compose hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-compose /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-lists hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-lists /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-folders hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-folders /usr/share/emacs/site-lisp/elpa/mu4e-1.12.5/mu4e-update hides /usr/share/emacs/site-lisp/elpa-src/mu4e-1.12.5/mu4e-update /usr/share/emacs/site-lisp/elpa/nginx-mode-1.1.9/nginx-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/nginx-mode-1.1.9/nginx-mode-autoloads /usr/share/emacs/site-lisp/elpa/nginx-mode-1.1.9/nginx-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/nginx-mode-1.1.9/nginx-mode-pkg /usr/share/emacs/site-lisp/elpa/nginx-mode-1.1.9/nginx-mode hides /usr/share/emacs/site-lisp/elpa-src/nginx-mode-1.1.9/nginx-mode /usr/share/emacs/site-lisp/elpa/paredit-26/paredit-autoloads hides /usr/share/emacs/site-lisp/elpa-src/paredit-26/paredit-autoloads /usr/share/emacs/site-lisp/elpa/paredit-26/paredit-pkg hides /usr/share/emacs/site-lisp/elpa-src/paredit-26/paredit-pkg /usr/share/emacs/site-lisp/elpa/paredit-26/paredit hides /usr/share/emacs/site-lisp/elpa-src/paredit-26/paredit /usr/share/emacs/site-lisp/elpa/persist-0.6.1/persist hides /usr/share/emacs/site-lisp/elpa-src/persist-0.6.1/persist /usr/share/emacs/site-lisp/elpa/persist-0.6.1/persist-pkg hides /usr/share/emacs/site-lisp/elpa-src/persist-0.6.1/persist-pkg /usr/share/emacs/site-lisp/elpa/persist-0.6.1/persist-autoloads hides /usr/share/emacs/site-lisp/elpa-src/persist-0.6.1/persist-autoloads /usr/share/emacs/site-lisp/elpa/pfuture-1.9/pfuture hides /usr/share/emacs/site-lisp/elpa-src/pfuture-1.9/pfuture /usr/share/emacs/site-lisp/elpa/pfuture-1.9/pfuture-autoloads hides /usr/share/emacs/site-lisp/elpa-src/pfuture-1.9/pfuture-autoloads /usr/share/emacs/site-lisp/elpa/pfuture-1.9/pfuture-pkg hides /usr/share/emacs/site-lisp/elpa-src/pfuture-1.9/pfuture-pkg /usr/share/emacs/site-lisp/elpa/po-mode-0.21/po-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/po-mode-0.21/po-mode-pkg /usr/share/emacs/site-lisp/elpa/po-mode-0.21/po-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/po-mode-0.21/po-mode-autoloads /usr/share/emacs/site-lisp/elpa/po-mode-0.21/po-mode hides /usr/share/emacs/site-lisp/elpa-src/po-mode-0.21/po-mode /usr/share/emacs/site-lisp/elpa/popon-0.13/popon hides /usr/share/emacs/site-lisp/elpa-src/popon-0.13/popon /usr/share/emacs/site-lisp/elpa/popon-0.13/popon-autoloads hides /usr/share/emacs/site-lisp/elpa-src/popon-0.13/popon-autoloads /usr/share/emacs/site-lisp/elpa/popon-0.13/popon-pkg hides /usr/share/emacs/site-lisp/elpa-src/popon-0.13/popon-pkg /usr/share/emacs/site-lisp/elpa/posframe-1.1.7/posframe-pkg hides /usr/share/emacs/site-lisp/elpa-src/posframe-1.1.7/posframe-pkg /usr/share/emacs/site-lisp/elpa/posframe-1.1.7/posframe-autoloads hides /usr/share/emacs/site-lisp/elpa-src/posframe-1.1.7/posframe-autoloads /usr/share/emacs/site-lisp/elpa/posframe-1.1.7/posframe hides /usr/share/emacs/site-lisp/elpa-src/posframe-1.1.7/posframe /usr/share/emacs/site-lisp/elpa/projectile-2.8.0/projectile-pkg hides /usr/share/emacs/site-lisp/elpa-src/projectile-2.8.0/projectile-pkg /usr/share/emacs/site-lisp/elpa/projectile-2.8.0/projectile-autoloads hides /usr/share/emacs/site-lisp/elpa-src/projectile-2.8.0/projectile-autoloads /usr/share/emacs/site-lisp/elpa/projectile-2.8.0/projectile hides /usr/share/emacs/site-lisp/elpa-src/projectile-2.8.0/projectile /usr/share/emacs/site-lisp/elpa/py-isort-2016.1/py-isort hides /usr/share/emacs/site-lisp/elpa-src/py-isort-2016.1/py-isort /usr/share/emacs/site-lisp/elpa/py-isort-2016.1/py-isort-autoloads hides /usr/share/emacs/site-lisp/elpa-src/py-isort-2016.1/py-isort-autoloads /usr/share/emacs/site-lisp/elpa/py-isort-2016.1/py-isort-pkg hides /usr/share/emacs/site-lisp/elpa-src/py-isort-2016.1/py-isort-pkg /usr/share/emacs/site-lisp/elpa/pyvenv-1.21/pyvenv hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.21/pyvenv /usr/share/emacs/site-lisp/elpa/pyvenv-1.21/pyvenv-pkg hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.21/pyvenv-pkg /usr/share/emacs/site-lisp/elpa/pyvenv-1.21/pyvenv-autoloads hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.21/pyvenv-autoloads /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-common hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-common /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-mode-tests hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-mode-tests /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-mode-treesitter hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-mode-treesitter /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-cargo hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-cargo /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-mode-autoloads /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-utils hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-utils /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-rustfmt hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-rustfmt /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-mode hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-mode /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-playpen hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-playpen /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-prog-mode hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-prog-mode /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-mode-pkg /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-cargo-tests hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-cargo-tests /usr/share/emacs/site-lisp/elpa/rust-mode-1.0.5/rust-compile hides /usr/share/emacs/site-lisp/elpa-src/rust-mode-1.0.5/rust-compile /usr/share/emacs/site-lisp/elpa/s-1.12.0/s-autoloads hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s-autoloads /usr/share/emacs/site-lisp/elpa/s-1.12.0/s-pkg hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s-pkg /usr/share/emacs/site-lisp/elpa/s-1.12.0/s hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-map hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-map /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-indent hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-indent /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-syntax hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-syntax /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-lib hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-lib /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-pkg /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-prettify-symbols hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-prettify-symbols /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-compile hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-compile /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-organise hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-organise /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-autoloads /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-paragraph hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-paragraph /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-imenu hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-imenu /usr/share/emacs/site-lisp/elpa/scala-mode-1.1.0/scala-mode-fontlock hides /usr/share/emacs/site-lisp/elpa-src/scala-mode-1.1.0/scala-mode-fontlock /usr/share/emacs/site-lisp/elpa/seq-2.24/seq-24 hides /usr/share/emacs/site-lisp/elpa-src/seq-2.24/seq-24 /usr/share/emacs/site-lisp/elpa/seq-2.24/seq-autoloads hides /usr/share/emacs/site-lisp/elpa-src/seq-2.24/seq-autoloads /usr/share/emacs/site-lisp/elpa/seq-2.24/seq hides /usr/share/emacs/site-lisp/elpa-src/seq-2.24/seq /usr/share/emacs/site-lisp/elpa/seq-2.24/seq-pkg hides /usr/share/emacs/site-lisp/elpa-src/seq-2.24/seq-pkg /usr/share/emacs/site-lisp/elpa/seq-2.24/seq-25 hides /usr/share/emacs/site-lisp/elpa-src/seq-2.24/seq-25 /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-project-follow-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-project-follow-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-dom hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-dom /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-follow-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-follow-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-logging hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-logging /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-autoloads hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-autoloads /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-compatibility hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-compatibility /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-async hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-async /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-themes hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-themes /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-mouse-interface hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-mouse-interface /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-tag-follow-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-tag-follow-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-bookmarks hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-bookmarks /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-fringe-indicator hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-fringe-indicator /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-filewatch-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-filewatch-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-interface hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-interface /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-tags hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-tags /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-visuals hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-visuals /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-core-utils hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-core-utils /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-peek-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-peek-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-annotations hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-annotations /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-icons hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-icons /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-persistence hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-persistence /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-diagnostics hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-diagnostics /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-scope hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-scope /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-extensions hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-extensions /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-rendering hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-rendering /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-customization hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-customization /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-file-management hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-file-management /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-faces hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-faces /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-macros hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-macros /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-git-commit-diff-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-git-commit-diff-mode /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-treelib hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-treelib /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-hydras hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-hydras /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-pkg hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-pkg /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-header-line hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-header-line /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-icons-dired hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-icons-dired /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-workspaces hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-workspaces /usr/share/emacs/site-lisp/elpa/treemacs-3.1/treemacs-mode hides /usr/share/emacs/site-lisp/elpa-src/treemacs-3.1/treemacs-mode /usr/share/emacs/site-lisp/elpa/treemacs-magit-3.1/treemacs-magit hides /usr/share/emacs/site-lisp/elpa-src/treemacs-magit-3.1/treemacs-magit /usr/share/emacs/site-lisp/elpa/treemacs-magit-3.1/treemacs-magit-pkg hides /usr/share/emacs/site-lisp/elpa-src/treemacs-magit-3.1/treemacs-magit-pkg /usr/share/emacs/site-lisp/elpa/treemacs-magit-3.1/treemacs-magit-autoloads hides /usr/share/emacs/site-lisp/elpa-src/treemacs-magit-3.1/treemacs-magit-autoloads /usr/share/emacs/site-lisp/elpa/treemacs-projectile-3.1/treemacs-projectile-pkg hides /usr/share/emacs/site-lisp/elpa-src/treemacs-projectile-3.1/treemacs-projectile-pkg /usr/share/emacs/site-lisp/elpa/treemacs-projectile-3.1/treemacs-projectile-autoloads hides /usr/share/emacs/site-lisp/elpa-src/treemacs-projectile-3.1/treemacs-projectile-autoloads /usr/share/emacs/site-lisp/elpa/treemacs-projectile-3.1/treemacs-projectile hides /usr/share/emacs/site-lisp/elpa-src/treemacs-projectile-3.1/treemacs-projectile /usr/share/emacs/site-lisp/elpa/vterm-0.0.2/vterm-load-path hides /usr/share/emacs/site-lisp/elpa-src/vterm-0.0.2/vterm-load-path /usr/share/emacs/site-lisp/elpa/vterm-0.0.2/vterm hides /usr/share/emacs/site-lisp/elpa-src/vterm-0.0.2/vterm /usr/share/emacs/site-lisp/elpa/vterm-0.0.2/vterm-pkg hides /usr/share/emacs/site-lisp/elpa-src/vterm-0.0.2/vterm-pkg /usr/share/emacs/site-lisp/elpa/vterm-0.0.2/vterm-autoloads hides /usr/share/emacs/site-lisp/elpa-src/vterm-0.0.2/vterm-autoloads /usr/share/emacs/site-lisp/elpa/web-mode-17.3.13/web-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/web-mode-17.3.13/web-mode-pkg /usr/share/emacs/site-lisp/elpa/web-mode-17.3.13/web-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/web-mode-17.3.13/web-mode-autoloads /usr/share/emacs/site-lisp/elpa/web-mode-17.3.13/web-mode hides /usr/share/emacs/site-lisp/elpa-src/web-mode-17.3.13/web-mode /usr/share/emacs/site-lisp/elpa/with-editor-3.3.2/with-editor-pkg hides /usr/share/emacs/site-lisp/elpa-src/with-editor-3.3.2/with-editor-pkg /usr/share/emacs/site-lisp/elpa/with-editor-3.3.2/with-editor-autoloads hides /usr/share/emacs/site-lisp/elpa-src/with-editor-3.3.2/with-editor-autoloads /usr/share/emacs/site-lisp/elpa/with-editor-3.3.2/with-editor hides /usr/share/emacs/site-lisp/elpa-src/with-editor-3.3.2/with-editor /usr/share/emacs/site-lisp/elpa/xml-rpc-1.6.17/xml-rpc hides /usr/share/emacs/site-lisp/elpa-src/xml-rpc-1.6.17/xml-rpc /usr/share/emacs/site-lisp/elpa/xml-rpc-1.6.17/xml-rpc-pkg hides /usr/share/emacs/site-lisp/elpa-src/xml-rpc-1.6.17/xml-rpc-pkg /usr/share/emacs/site-lisp/elpa/xml-rpc-1.6.17/xml-rpc-autoloads hides /usr/share/emacs/site-lisp/elpa-src/xml-rpc-1.6.17/xml-rpc-autoloads /usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.16/yaml-mode-pkg hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.16/yaml-mode-pkg /usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.16/yaml-mode hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.16/yaml-mode /usr/share/emacs/site-lisp/elpa/yaml-mode-0.0.16/yaml-mode-autoloads hides /usr/share/emacs/site-lisp/elpa-src/yaml-mode-0.0.16/yaml-mode-autoloads /usr/share/emacs/site-lisp/elpa/yasnippet-0.14.1/yasnippet-autoloads hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.14.1/yasnippet-autoloads /usr/share/emacs/site-lisp/elpa/yasnippet-0.14.1/yasnippet-pkg hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.14.1/yasnippet-pkg /usr/share/emacs/site-lisp/elpa/yasnippet-0.14.1/yasnippet hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.14.1/yasnippet /usr/share/emacs/site-lisp/elpa/yasnippet-snippets-20220713/yasnippet-snippets hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-snippets-20220713/yasnippet-snippets /usr/share/emacs/site-lisp/elpa/yasnippet-snippets-20220713/yasnippet-snippets-pkg hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-snippets-20220713/yasnippet-snippets-pkg /usr/share/emacs/site-lisp/elpa/yasnippet-snippets-20220713/yasnippet-snippets-autoloads hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-snippets-20220713/yasnippet-snippets-autoloads /usr/share/emacs/site-lisp/elpa/zenburn-theme-2.8.0/zenburn-theme hides /usr/share/emacs/site-lisp/elpa-src/zenburn-theme-2.8.0/zenburn-theme /usr/share/emacs/site-lisp/elpa/zenburn-theme-2.8.0/zenburn-theme-pkg hides /usr/share/emacs/site-lisp/elpa-src/zenburn-theme-2.8.0/zenburn-theme-pkg /usr/share/emacs/site-lisp/elpa/zenburn-theme-2.8.0/zenburn-theme-autoloads hides /usr/share/emacs/site-lisp/elpa-src/zenburn-theme-2.8.0/zenburn-theme-autoloads /usr/share/emacs/site-lisp/elpa/seq-2.24/seq hides /usr/share/emacs/29.4/lisp/emacs-lisp/seq Features: (shadow emacsbug novice debian-copyright shr-color json-ts-mode git-rebase reporter debian-bts-control vterm tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat term ehelp vterm-module debian-changelog-mode debian-bug conf-mode make-mode goto-addr misearch multi-isearch url-queue magit-extras magit-bookmark magit-submodule magit-obsolete magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit magit-sequence magit-notes magit-worktree magit-tag magit-merge magit-branch magit-reset magit-files magit-refs magit-status magit magit-repos magit-apply magit-wip magit-log which-func magit-diff git-commit log-edit add-log magit-core magit-autorevert magit-margin magit-transient magit-process with-editor shell magit-mode transient magit-git magit-section magit-utils crm smerge-mode diff mailalias face-remap dired-aux mm-archive qp sort gnus-cite mail-extr textsec uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check gnus-async gnus-bcklg gnus-ml gnus-topic cursor-sensor timezone utf-7 url-cache nnfolder gnus-demon nnml ezgnus gnus-delay gnus-draft gnus-agent gnus-srvr gnus-score score-mode nnvirtual nntp gnus-cache nndraft nnmh auth-source-xoauth2-plugin oauth2 url-http url-auth url-gw plstore eglot external-completion array jsonrpc ert ewoc debug backtrace xref pcase imenu matlab matlab-scan matlab-syntax matlab-compat jka-compr mu4e mu4e-org org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete pcomplete org-list org-footnote org-faces org-entities noutline outline ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys oc org-loaddefs find-func org-version org-compat org-macs format-spec mu4e-notification notifications mu4e-main smtpmail mu4e-view mu4e-mime-parts cal-menu calendar cal-loaddefs mu4e-headers mu4e-thread mu4e-actions mu4e-compose mu4e-draft gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader range mu4e-search mu4e-lists mu4e-bookmarks mu4e-mark mu4e-message shr pixel-fill kinsoku url-file svg xml dom flow-fill mule-util mu4e-contacts mu4e-update mu4e-folders mu4e-context mu4e-query-items mu4e-server mu4e-modeline mu4e-vars mu4e-helpers mu4e-config mu4e-window ido message sendmail yank-media rfc822 mml mml-sec gnus-util mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader mu4e-obsolete windmove flyspell ispell gnutls network-stream puny nsm epa-file epa derived epg rfc6068 epg-config rcirc parse-time iso8601 time-date term/xterm xterm comp comp-cstr server cap-words superword subword vc-hg vc-git diff-mode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view pcvs-util vc vc-dispatcher bug-reference disp-table whitespace yasnippet-snippets yasnippet cus-edit cus-start wid-edit init mu4e-debian-hx90 zenburn-theme xclip treesit-auto treesit treemacs-project-follow-mode treemacs-follow-mode treemacs-rendering treemacs-annotations treemacs-async treemacs-visuals treemacs-fringe-indicator pulse color treemacs-workspaces treemacs-dom treemacs-icons treemacs-themes treemacs-scope treemacs-core-utils treemacs-logging treemacs-customization pfuture inline ht s hl-line dash keychain-environment exec-path-from-shell corfu-terminal popon corfu-popupinfo corfu-echo corfu compat activities-tabs activities persist bookmark pp edmacro kmacro advice icomplete cus-load flymake-proc flymake project compile text-property-search comint ansi-osc ansi-color ring warnings icons thingatpt cl-extra help-mode use-package use-package-ensure use-package-delight use-package-diminish use-package-bind-key bind-key easy-mmode use-package-core display-line-numbers autorevert filenotify keychain-environment-autoloads treesit-auto-autoloads xclip-autoloads rx info debian-el dired dired-loaddefs finder-inf package browse-url url url-proxy url-privacy url-expand url-methods url-history url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic indonesian philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget keymap hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 1385704 157406) (symbols 48 42601 43) (strings 32 181071 16554) (string-bytes 1 5513742) (vectors 16 115417) (vector-slots 8 2872186 223315) (floats 8 1025 2197) (intervals 56 42184 12669) (buffers 984 116)) -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2055 bytes --] From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/5] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Add-parameters-required-by-Google-OAuth2-to-get-refr.patch --] [-- Type: text/x-diff, Size: 1264 bytes --] From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:52:02 -0700 Subject: [PATCH 2/5] Add parameters required by Google OAuth2 to get refresh_token * packages/oauth2/oauth2.el (oauth2-request-authorization): add `access_type=offline' and `prompt=consent' when requesting token to receive refresh_token. --- oauth2.el | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 3a3e50ad2b..9780ac3a1d 100644 --- a/oauth2.el +++ b/oauth2.el @@ -63,7 +63,9 @@ It returns the code provided by the service." "&response_type=code" "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) "")))) + (if state (concat "&state=" (url-hexify-string state)) "") + "&access_type=offline" + "&prompt=consent"))) (browse-url url) (read-string (concat "Follow the instruction on your default browser, or " "visit:\n" url -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Encode-parameters-when-requesting-access.patch --] [-- Type: text/x-diff, Size: 1194 bytes --] From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:54:08 -0700 Subject: [PATCH 3/5] Encode parameters when requesting access * packages/oauth2/oauth2.el (oauth2-request-access): encode all parameters which may contain characters that breaks URL. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index 9780ac3a1d..b035742fc1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." (oauth2-make-access-request token-url (concat - "client_id=" client-id + "client_id=" (url-hexify-string client-id) (when client-secret - (concat "&client_secret=" client-secret)) - "&code=" code + (concat "&client_secret=" (url-hexify-string client-secret))) + "&code=" (url-hexify-string code) "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) "&grant_type=authorization_code")))) (make-oauth2-token :client-id client-id -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Support-storing-data-for-multiple-accounts-of-the-sa.patch --] [-- Type: text/x-diff, Size: 1987 bytes --] From eb4afbc2f21c65417de5ca8e06309fd4c53cc1ab Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:00:04 -0700 Subject: [PATCH 4/5] Support storing data for multiple accounts of the same provider Currently the plstore id computed by `oauth2-compute-id' only takes `auth-url', `token-url', and `scope' into account, which could be the same for the same provider (e.g. Gmail). This prevents storing information for multiple accounts of the same service for some providers. This patch adds `client-id' to the calculation of plstore id to make sure that it is unique for different accounts of the same provider. * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a parameter of `oauth2-compute-id' to ensure unique id amount multiple accounts of the same provider. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index b035742fc1..ea012d4d38 100644 --- a/oauth2.el +++ b/oauth2.el @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." :group 'oauth2 :type 'file) -(defun oauth2-compute-id (auth-url token-url scope) +(defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. This allows to store the token in an unique way." - (secure-hash 'md5 (concat auth-url token-url scope))) + (secure-hash 'md5 (concat auth-url token-url scope client-id))) ;;;###autoload (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) "Request access to a resource and store it using `plstore'." ;; We store a MD5 sum of all URL (let* ((plstore (plstore-open oauth2-token-file)) - (id (oauth2-compute-id auth-url token-url scope)) + (id (oauth2-compute-id auth-url token-url scope client-id)) (plist (cdr (plstore-get plstore id)))) ;; Check if we found something matching this access (if plist -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-Add-debug-messages-and-provide-a-switch-for-enabling.patch --] [-- Type: text/x-diff, Size: 2174 bytes --] From 6cf0ab190d24b6fc2a7335d02a8aef29ab24c622 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/5] Add debug messages and provide a switch for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add debug message and option for enabling. --- oauth2.el | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/oauth2.el b/oauth2.el index ea012d4d38..06a880532e 100644 --- a/oauth2.el +++ b/oauth2.el @@ -40,6 +40,7 @@ (require 'plstore) (require 'json) (require 'url-http) +(require 'pp) (defvar url-http-data) (defvar url-http-method) @@ -53,6 +54,15 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defcustom oauth2-debug nil + "Enable debug messages." + :type 'boolean) + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (if oauth2-debug + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -79,6 +89,8 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) (let ((url-request-method "POST") (url-request-data data) (url-request-extra-headers @@ -86,6 +98,8 @@ It returns the code provided by the service." (with-current-buffer (url-retrieve-synchronously url) (let ((data (oauth2-request-access-parse))) (kill-buffer (current-buffer)) + (oauth2--do-debug "oauth2-make-access-request: response: %s" + (pp-to-string data)) data)))) (cl-defstruct oauth2-token -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-29 21:25 bug#72358: 29.4; oauth2.el improvements Xiyue Deng @ 2024-07-30 7:46 ` Robert Pluim 2024-07-30 14:05 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors ` (2 more replies) 2024-07-30 14:08 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors ` (2 subsequent siblings) 3 siblings, 3 replies; 49+ messages in thread From: Robert Pluim @ 2024-07-30 7:46 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 >>>>> On Mon, 29 Jul 2024 14:25:01 -0700, Xiyue Deng <manphiz@gmail.com> said: Xiyue> Hi, Xiyue> I have been trying out using oauth2.el to enable OAuth2-based Xiyue> authentication for email service providers and had some success for Xiyue> Gmail. During this process, I have made a few changes to oauth2.el that Xiyue> enables it to use with Gmail OAuth2 as well as some usability and Xiyue> debugging improvements, which I'm sharing below. Thank you for this. This support is becoming more necessary as time goes on. I even wonder if we should bring oauth2.el into emacs instead of it being a package. Xiyue> This is a series of five patches, which are attached. Xiyue> The first patch shows the authentication URL in the minibuffer window Xiyue> alongside the prompt accepting the authorization code. This helps when Xiyue> a user has multiple accounts from the same provider but is logged into a Xiyue> different account than the one that the user is trying to set up. If Xiyue> the user use the link (or through `browse-url') it will use the active Xiyue> account instead of the one intended. By showing the URL in the Xiyue> minibuffer, the user can choose other ways to get the authorization code Xiyue> (e.g. using another browser, using private/encognito mode, etc.) OK. This fixes one of my irritations with oauth2.el 🙂 Xiyue> The second patch adds the parameters `access_type=offline' and Xiyue> `prompt=consent' to the authorization URL, which is required for Gmail Xiyue> OAuth2 to get the refresh token. Without these 2 parameters, Gmail Xiyue> response will only contain the access token which expires in one hour. Xiyue> They should also be compatible with other OAuth2 authentication process. Xiyue> (Though I am currently having trouble to get outlook.com to work Xiyue> regardless of these parameters, which I'll ask in a separate thread.) Xiyue> Note that the second patch depends on the first patch as they modify the same Xiyue> part of the code. OK. Iʼm assuming oauth2.el can use the refresh token next time it needs to authorize? (Iʼve been avoiding actually using oauth2.el in anger, since app passwords still work) Xiyue> The third patch encodes the parameters for requesting refreshing access Xiyue> token, which is recommended because the client secret and other Xiyue> parameters may contain characters that may break parameter parsing. OK Xiyue> The fourth patch may need a bit of background: oauth2.el (optionally) Xiyue> uses plstore to save authentication data for future reuse, and the Xiyue> plstore id for an account is computed using a combination of `auth-url', Xiyue> `token-url', and `scope'. However, this combination of data doesn't Xiyue> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, Xiyue> the three parameters are the same for different accounts, and hence Xiyue> storing a second account information will override the first one. Xiyue> This fourth patch adds `client-id' to the calculation of plstore id to Xiyue> ensure its uniqueness. This may cause a few concerns: Xiyue> - This will invalidate all existing entries and a user will have to redo Xiyue> the authorization process again to get a new refresh token. However, Xiyue> I think it's more important to ensure that oauth2.el works correctly Xiyue> for multiple accounts of the same provider, or a user may suffer from Xiyue> confusion when adding a new account invalidates a previous account. I donʼt think thatʼs too big a concern. 'modern' authentication flows regularly re-prompt, so this will not be too surprising (although maybe call it out in the packageʼs NEWS or README). Xiyue> - Adding `client-id' to the calculation of plstore id may provoke Xiyue> suspicion of leaking it as the hash calculation uses md5. In most Xiyue> cases, requesting a refresh token requires both `client-id' and Xiyue> `client-secret', so without including the latter it should be safe. Xiyue> There are cases when requesting only the access token may work with Xiyue> `client-id' along. Still, I think this should not be a big concern as Xiyue> the data is combined with `auth-url', `token-url', and `scope' which Xiyue> provides sufficient salt. Alternatively, we can also choose to use a Xiyue> more secure hash function, e.g. SHA2 or better, given that existing Xiyue> entries will be invalidated anyway. If the existing entries are going to become invalid anyway, you might as well take the opportunity to move away from md5 at the same time. git picked SHA-256, but that was a while ago, so maybe SHA-512? Xiyue> The fifth patch adds debug messages when doing a URL query which records Xiyue> the request URL, the request data, and the response data, and provide a Xiyue> custom variable to enable this. This provides a way to help debugging Xiyue> the requests, and I find it handy when testing oauth2 against different Xiyue> providers. OK (although perhaps make it a defvar rather than a defcustom, to avoid people accidentally enabling it). Robert -- ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 7:46 ` Robert Pluim @ 2024-07-30 14:05 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-07-30 19:37 ` Xiyue Deng [not found] ` <66a8f323.170a0220.9172c.8e28SMTPIN_ADDED_BROKEN@mx.google.com> 2 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 14:05 UTC (permalink / raw) To: Robert Pluim; +Cc: 72358, Xiyue Deng Robert Pluim <rpluim@gmail.com> writes: > Xiyue> - This will invalidate all existing entries and a user will have to redo > Xiyue> the authorization process again to get a new refresh token. However, > Xiyue> I think it's more important to ensure that oauth2.el works correctly > Xiyue> for multiple accounts of the same provider, or a user may suffer from > Xiyue> confusion when adding a new account invalidates a previous account. > > I donʼt think thatʼs too big a concern. 'modern' authentication flows > regularly re-prompt, so this will not be too surprising (although > maybe call it out in the packageʼs NEWS or README). In many cases the refreshing of tokens is transparent to the user there doesn't have to be a re-prompt to refresh the token if the OAuth provider support it. Micrsofts OAuth workflow is quite good in this regard as there's a non-standard error to indicate when the user has to re-authorize the application. I assume all implementation of OAuth have their quirks. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 7:46 ` Robert Pluim 2024-07-30 14:05 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 19:37 ` Xiyue Deng 2024-07-31 8:54 ` Robert Pluim [not found] ` <66a8f323.170a0220.9172c.8e28SMTPIN_ADDED_BROKEN@mx.google.com> 2 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-07-30 19:37 UTC (permalink / raw) To: Robert Pluim; +Cc: 72358 [-- Attachment #1: Type: text/plain, Size: 6343 bytes --] Robert Pluim <rpluim@gmail.com> writes: >>>>>> On Mon, 29 Jul 2024 14:25:01 -0700, Xiyue Deng <manphiz@gmail.com> said: > > Xiyue> Hi, > Xiyue> I have been trying out using oauth2.el to enable OAuth2-based > Xiyue> authentication for email service providers and had some success for > Xiyue> Gmail. During this process, I have made a few changes to oauth2.el that > Xiyue> enables it to use with Gmail OAuth2 as well as some usability and > Xiyue> debugging improvements, which I'm sharing below. > > Thank you for this. This support is becoming more necessary as time > goes on. I even wonder if we should bring oauth2.el into emacs instead > of it being a package. > I would also love that to happen. In fact, I have another addon inspired by auth-source-xoauth2 and makes use of oauth2.el and auth-source backend, though it requires advising the auth-source-search-backends and kind of awkward. I'll post that through another bug report for comments after this one gets submitted. > Xiyue> This is a series of five patches, which are attached. > > Xiyue> The first patch shows the authentication URL in the minibuffer window > Xiyue> alongside the prompt accepting the authorization code. This helps when > Xiyue> a user has multiple accounts from the same provider but is logged into a > Xiyue> different account than the one that the user is trying to set up. If > Xiyue> the user use the link (or through `browse-url') it will use the active > Xiyue> account instead of the one intended. By showing the URL in the > Xiyue> minibuffer, the user can choose other ways to get the authorization code > Xiyue> (e.g. using another browser, using private/encognito mode, etc.) > > OK. This fixes one of my irritations with oauth2.el 🙂 > Glad to hear I'm not the only one. > Xiyue> The second patch adds the parameters `access_type=offline' and > Xiyue> `prompt=consent' to the authorization URL, which is required for Gmail > Xiyue> OAuth2 to get the refresh token. Without these 2 parameters, Gmail > Xiyue> response will only contain the access token which expires in one hour. > Xiyue> They should also be compatible with other OAuth2 authentication process. > Xiyue> (Though I am currently having trouble to get outlook.com to work > Xiyue> regardless of these parameters, which I'll ask in a separate thread.) > > Xiyue> Note that the second patch depends on the first patch as they modify the same > Xiyue> part of the code. > > OK. Iʼm assuming oauth2.el can use the refresh token next time it > needs to authorize? (Iʼve been avoiding actually using oauth2.el in > anger, since app passwords still work) > Yes, with a valid refresh token you can login without any manual steps. Unless the refresh_token itself is expired, in which case you'll have to re-authorize. > Xiyue> The third patch encodes the parameters for requesting refreshing access > Xiyue> token, which is recommended because the client secret and other > Xiyue> parameters may contain characters that may break parameter parsing. > > OK > > Xiyue> The fourth patch may need a bit of background: oauth2.el (optionally) > Xiyue> uses plstore to save authentication data for future reuse, and the > Xiyue> plstore id for an account is computed using a combination of `auth-url', > Xiyue> `token-url', and `scope'. However, this combination of data doesn't > Xiyue> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, > Xiyue> the three parameters are the same for different accounts, and hence > Xiyue> storing a second account information will override the first one. > > Xiyue> This fourth patch adds `client-id' to the calculation of plstore id to > Xiyue> ensure its uniqueness. This may cause a few concerns: > > Xiyue> - This will invalidate all existing entries and a user will have to redo > Xiyue> the authorization process again to get a new refresh token. However, > Xiyue> I think it's more important to ensure that oauth2.el works correctly > Xiyue> for multiple accounts of the same provider, or a user may suffer from > Xiyue> confusion when adding a new account invalidates a previous account. > > I donʼt think thatʼs too big a concern. 'modern' authentication flows > regularly re-prompt, so this will not be too surprising (although > maybe call it out in the packageʼs NEWS or README). > I have added a NEWS file in patch 6 documenting this for 0.17 (which should be the version of the next release.) > Xiyue> - Adding `client-id' to the calculation of plstore id may provoke > Xiyue> suspicion of leaking it as the hash calculation uses md5. In most > Xiyue> cases, requesting a refresh token requires both `client-id' and > Xiyue> `client-secret', so without including the latter it should be safe. > Xiyue> There are cases when requesting only the access token may work with > Xiyue> `client-id' along. Still, I think this should not be a big concern as > Xiyue> the data is combined with `auth-url', `token-url', and `scope' which > Xiyue> provides sufficient salt. Alternatively, we can also choose to use a > Xiyue> more secure hash function, e.g. SHA2 or better, given that existing > Xiyue> entries will be invalidated anyway. > > If the existing entries are going to become invalid anyway, you might > as well take the opportunity to move away from md5 at the same > time. git picked SHA-256, but that was a while ago, so maybe SHA-512? > Makes sense. I have updated the hash function to use SHA-512 in patch 5. > Xiyue> The fifth patch adds debug messages when doing a URL query which records > Xiyue> the request URL, the request data, and the response data, and provide a > Xiyue> custom variable to enable this. This provides a way to help debugging > Xiyue> the requests, and I find it handy when testing oauth2 against different > Xiyue> providers. > > OK (although perhaps make it a defvar rather than a defcustom, to > avoid people accidentally enabling it). > Done also in patch 5. > Robert -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2055 bytes --] From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/6] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Add-parameters-required-by-Google-OAuth2-to-get-refr.patch --] [-- Type: text/x-diff, Size: 1264 bytes --] From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:52:02 -0700 Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get refresh_token * packages/oauth2/oauth2.el (oauth2-request-authorization): add `access_type=offline' and `prompt=consent' when requesting token to receive refresh_token. --- oauth2.el | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 3a3e50ad2b..9780ac3a1d 100644 --- a/oauth2.el +++ b/oauth2.el @@ -63,7 +63,9 @@ It returns the code provided by the service." "&response_type=code" "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) "")))) + (if state (concat "&state=" (url-hexify-string state)) "") + "&access_type=offline" + "&prompt=consent"))) (browse-url url) (read-string (concat "Follow the instruction on your default browser, or " "visit:\n" url -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Encode-parameters-when-requesting-access.patch --] [-- Type: text/x-diff, Size: 1194 bytes --] From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:54:08 -0700 Subject: [PATCH 3/6] Encode parameters when requesting access * packages/oauth2/oauth2.el (oauth2-request-access): encode all parameters which may contain characters that breaks URL. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index 9780ac3a1d..b035742fc1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." (oauth2-make-access-request token-url (concat - "client_id=" client-id + "client_id=" (url-hexify-string client-id) (when client-secret - (concat "&client_secret=" client-secret)) - "&code=" code + (concat "&client_secret=" (url-hexify-string client-secret))) + "&code=" (url-hexify-string code) "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) "&grant_type=authorization_code")))) (make-oauth2-token :client-id client-id -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Support-storing-data-for-multiple-accounts-of-the-sa.patch --] [-- Type: text/x-diff, Size: 2090 bytes --] From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:00:04 -0700 Subject: [PATCH 4/6] Support storing data for multiple accounts of the same provider Currently the plstore id computed by `oauth2-compute-id' only takes `auth-url', `token-url', and `scope' into account, which could be the same for the same provider (e.g. Gmail). This prevents storing information for multiple accounts of the same service for some providers. This patch adds `client-id' to the calculation of plstore id to make sure that it is unique for different accounts of the same provider. It also changes the hash function to sha512 to be more secure. * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a parameter of `oauth2-compute-id' to ensure unique id amount multiple accounts of the same provider, and change hash function to sha512. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index b035742fc1..035971ac85 100644 --- a/oauth2.el +++ b/oauth2.el @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." :group 'oauth2 :type 'file) -(defun oauth2-compute-id (auth-url token-url scope) +(defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. This allows to store the token in an unique way." - (secure-hash 'md5 (concat auth-url token-url scope))) + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) ;;;###autoload (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) "Request access to a resource and store it using `plstore'." ;; We store a MD5 sum of all URL (let* ((plstore (plstore-open oauth2-token-file)) - (id (oauth2-compute-id auth-url token-url scope)) + (id (oauth2-compute-id auth-url token-url scope client-id)) (plist (cdr (plstore-get plstore id)))) ;; Check if we found something matching this access (if plist -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-Add-debug-messages-and-provide-a-switch-variable-for.patch --] [-- Type: text/x-diff, Size: 2751 bytes --] From 08b04a07aa399bc756f1a8a00ae17cf333be02d9 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/6] Add debug messages and provide a switch variable for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add support for debug messages and a switch variable for enabling. --- oauth2.el | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/oauth2.el b/oauth2.el index 035971ac85..aca48f9ce1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -40,6 +40,7 @@ (require 'plstore) (require 'json) (require 'url-http) +(require 'pp) (defvar url-http-data) (defvar url-http-method) @@ -53,6 +54,15 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defcustom oauth2-debug nil + "Enable debug messages." + :type 'boolean) + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (if oauth2-debug + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -79,6 +89,8 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) (let ((url-request-method "POST") (url-request-data data) (url-request-extra-headers @@ -86,6 +98,8 @@ It returns the code provided by the service." (with-current-buffer (url-retrieve-synchronously url) (let ((data (oauth2-request-access-parse))) (kill-buffer (current-buffer)) + (oauth2--do-debug "oauth2-make-access-request: response: %s" + (pp-to-string data)) data)))) (cl-defstruct oauth2-token @@ -158,10 +172,8 @@ TOKEN should be obtained with `oauth2-request-access'." auth-url client-id scope state redirect-uri) redirect-uri)) -(defcustom oauth2-token-file (concat user-emacs-directory "oauth2.plstore") - "File path where store OAuth tokens." - :group 'oauth2 - :type 'file) +(defvar oauth2-token-file (concat user-emacs-directory "oauth2.plstore") + "File path where store OAuth tokens.") (defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-Add-NEWS-file-to-document-the-changes-to-plstore-id-.patch --] [-- Type: text/x-diff, Size: 1350 bytes --] From 2fbc85ee128cfdf7a1521bbc9554424e9ba510da Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Tue, 30 Jul 2024 03:46:57 -0700 Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id generation --- NEWS | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 NEWS diff --git a/NEWS b/NEWS new file mode 100644 index 0000000000..6715a1914a --- /dev/null +++ b/NEWS @@ -0,0 +1,23 @@ +Summary of changes to oauth2.el +------------------------------- + +For changes of 0.16 and older or full changes please check the git +history of the repository of oauth2.el. + +* 0.17 + +** Changes to plstore id generation and needs to reacquire refresh_token + +The generation of plstore id used to include `auth-url', `token-url', +and `scope'. Now `client-id' is also included. This is required to +support multiple accounts of some providers which use the same +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the +generated plstore id is not unique amount accounts. Adding +`client-id' solves this problem. + +The hash function of calculating the plstore id has also changed from +MD5 to SHA512 to be more secure. + +As a result, users of oauth2.el will need to redo the authentication +process to get a new refresh_token when upgrading from older version +to 0.17. -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 19:37 ` Xiyue Deng @ 2024-07-31 8:54 ` Robert Pluim 2024-07-31 11:13 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Robert Pluim @ 2024-07-31 8:54 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 >>>>> On Tue, 30 Jul 2024 12:37:05 -0700, Xiyue Deng <manphiz@gmail.com> said: Xiyue> The fifth patch adds debug messages when doing a URL query which records Xiyue> the request URL, the request data, and the response data, and provide a Xiyue> custom variable to enable this. This provides a way to help debugging Xiyue> the requests, and I find it handy when testing oauth2 against different Xiyue> providers. >> >> OK (although perhaps make it a defvar rather than a defcustom, to >> avoid people accidentally enabling it). >> Xiyue> Done also in patch 5. I see you changed `oauth2-token-file' to a `defvar', which I donʼt think you should do. I was talking about making `oauth2-debug' a defvar. Robert -- ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-31 8:54 ` Robert Pluim @ 2024-07-31 11:13 ` Xiyue Deng 2024-08-02 8:15 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-07-31 11:13 UTC (permalink / raw) To: Robert Pluim; +Cc: 72358 [-- Attachment #1: Type: text/plain, Size: 957 bytes --] Robert Pluim <rpluim@gmail.com> writes: >>>>>> On Tue, 30 Jul 2024 12:37:05 -0700, Xiyue Deng <manphiz@gmail.com> said: > Xiyue> The fifth patch adds debug messages when doing a URL query which records > Xiyue> the request URL, the request data, and the response data, and provide a > Xiyue> custom variable to enable this. This provides a way to help debugging > Xiyue> the requests, and I find it handy when testing oauth2 against different > Xiyue> providers. > >> > >> OK (although perhaps make it a defvar rather than a defcustom, to > >> avoid people accidentally enabling it). > >> > > Xiyue> Done also in patch 5. > > I see you changed `oauth2-token-file' to a `defvar', which I donʼt > think you should do. I was talking about making `oauth2-debug' a > defvar. > > Robert Oops. Another reminder not to write code after 2am. The fixed patch 5 is attached. -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0005-Add-debug-messages-and-provide-a-switch-variable-for.patch --] [-- Type: text/x-diff, Size: 2186 bytes --] From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/6] Add debug messages and provide a switch variable for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add support for debug messages and a switch variable for enabling. --- oauth2.el | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/oauth2.el b/oauth2.el index 035971ac85..ce7a835100 100644 --- a/oauth2.el +++ b/oauth2.el @@ -40,6 +40,7 @@ (require 'plstore) (require 'json) (require 'url-http) +(require 'pp) (defvar url-http-data) (defvar url-http-method) @@ -53,6 +54,14 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defvar oauth2-debug nil + "Enable debug messages.") + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (if oauth2-debug + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -79,6 +88,8 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) (let ((url-request-method "POST") (url-request-data data) (url-request-extra-headers @@ -86,6 +97,8 @@ It returns the code provided by the service." (with-current-buffer (url-retrieve-synchronously url) (let ((data (oauth2-request-access-parse))) (kill-buffer (current-buffer)) + (oauth2--do-debug "oauth2-make-access-request: response: %s" + (pp-to-string data)) data)))) (cl-defstruct oauth2-token -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-31 11:13 ` Xiyue Deng @ 2024-08-02 8:15 ` Xiyue Deng 2024-08-02 8:38 ` Robert Pluim 2024-08-03 5:52 ` Eli Zaretskii 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-02 8:15 UTC (permalink / raw) To: Robert Pluim; +Cc: 72358 Xiyue Deng <manphiz@gmail.com> writes: > Robert Pluim <rpluim@gmail.com> writes: > >>>>>>> On Tue, 30 Jul 2024 12:37:05 -0700, Xiyue Deng <manphiz@gmail.com> said: >> Xiyue> The fifth patch adds debug messages when doing a URL query which records >> Xiyue> the request URL, the request data, and the response data, and provide a >> Xiyue> custom variable to enable this. This provides a way to help debugging >> Xiyue> the requests, and I find it handy when testing oauth2 against different >> Xiyue> providers. >> >> >> >> OK (although perhaps make it a defvar rather than a defcustom, to >> >> avoid people accidentally enabling it). >> >> >> >> Xiyue> Done also in patch 5. >> >> I see you changed `oauth2-token-file' to a `defvar', which I donʼt >> think you should do. I was talking about making `oauth2-debug' a >> defvar. >> >> Robert > > Oops. Another reminder not to write code after 2am. > > The fixed patch 5 is attached. More reviews welcome! BTW, is there any dev available to commit the changes once it's in a good shape? -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-02 8:15 ` Xiyue Deng @ 2024-08-02 8:38 ` Robert Pluim 2024-08-03 0:04 ` Xiyue Deng 2024-08-03 5:52 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Robert Pluim @ 2024-08-02 8:38 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 >>>>> On Fri, 02 Aug 2024 01:15:22 -0700, Xiyue Deng <manphiz@gmail.com> said: Xiyue> BTW, is there any dev available to commit the changes once it's in a Xiyue> good shape? You could request commit access. I see you have a number of contributions to Emacs already. Robert -- ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-02 8:38 ` Robert Pluim @ 2024-08-03 0:04 ` Xiyue Deng 0 siblings, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-03 0:04 UTC (permalink / raw) To: Robert Pluim; +Cc: 72358 Hi Robert, Robert Pluim <rpluim@gmail.com> writes: >>>>>> On Fri, 02 Aug 2024 01:15:22 -0700, Xiyue Deng <manphiz@gmail.com> said: > > Xiyue> BTW, is there any dev available to commit the changes once it's in a > Xiyue> good shape? > > You could request commit access. I see you have a number of > contributions to Emacs already. > > Robert That would be great to have. Do you know how I can apply for it (I have already done the paperwork)? -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-02 8:15 ` Xiyue Deng 2024-08-02 8:38 ` Robert Pluim @ 2024-08-03 5:52 ` Eli Zaretskii 2024-08-03 9:26 ` Xiyue Deng 2024-08-13 22:03 ` Xiyue Deng 1 sibling, 2 replies; 49+ messages in thread From: Eli Zaretskii @ 2024-08-03 5:52 UTC (permalink / raw) To: Xiyue Deng; +Cc: rpluim, 72358 > Cc: 72358@debbugs.gnu.org > From: Xiyue Deng <manphiz@gmail.com> > Date: Fri, 02 Aug 2024 01:15:22 -0700 > > BTW, is there any dev available to commit the changes once it's in a > good shape? All the 3 co-maintainers are tracking these discussions and install changes that are ready to be installed. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-03 5:52 ` Eli Zaretskii @ 2024-08-03 9:26 ` Xiyue Deng 2024-08-13 22:03 ` Xiyue Deng 1 sibling, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-03 9:26 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rpluim, 72358 Eli Zaretskii <eliz@gnu.org> writes: >> Cc: 72358@debbugs.gnu.org >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Fri, 02 Aug 2024 01:15:22 -0700 >> >> BTW, is there any dev available to commit the changes once it's in a >> good shape? > > All the 3 co-maintainers are tracking these discussions and install > changes that are ready to be installed. That's good to know! Thanks Eli! -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-03 5:52 ` Eli Zaretskii 2024-08-03 9:26 ` Xiyue Deng @ 2024-08-13 22:03 ` Xiyue Deng 2024-08-14 5:28 ` Eli Zaretskii 1 sibling, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-13 22:03 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rpluim, 72358 Hi Eli, Eli Zaretskii <eliz@gnu.org> writes: >> Cc: 72358@debbugs.gnu.org >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Fri, 02 Aug 2024 01:15:22 -0700 >> >> BTW, is there any dev available to commit the changes once it's in a >> good shape? > > All the 3 co-maintainers are tracking these discussions and install > changes that are ready to be installed. It's been a few days since the last time I received feedback for improvements regarding my patches. Is there any other feedbacks/reviews I am expecting from the co-maintainers? Please also let me know when it's time to ask for merging and requesting a new tagged release. Thanks in advance. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-13 22:03 ` Xiyue Deng @ 2024-08-14 5:28 ` Eli Zaretskii 2024-08-14 8:23 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2024-08-14 5:28 UTC (permalink / raw) To: Xiyue Deng; +Cc: Björn Bidar, rpluim, Thomas Fitzsimmons, 72358 > From: Xiyue Deng <manphiz@gmail.com> > Cc: rpluim@gmail.com, 72358@debbugs.gnu.org > Date: Tue, 13 Aug 2024 15:03:30 -0700 > > Hi Eli, > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Cc: 72358@debbugs.gnu.org > >> From: Xiyue Deng <manphiz@gmail.com> > >> Date: Fri, 02 Aug 2024 01:15:22 -0700 > >> > >> BTW, is there any dev available to commit the changes once it's in a > >> good shape? > > > > All the 3 co-maintainers are tracking these discussions and install > > changes that are ready to be installed. > > It's been a few days since the last time I received feedback for > improvements regarding my patches. Is there any other feedbacks/reviews > I am expecting from the co-maintainers? Please also let me know when > it's time to ask for merging and requesting a new tagged release. ?? The last message in this discussion was just yesterday evening, and my understanding is that you are still discussing the issues and did not reach the final conclusion. If I'm mistaken, my apologies; please describe your conclusion and post the patch that you-all agree would solve the issues, and let's take it from there. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-14 5:28 ` Eli Zaretskii @ 2024-08-14 8:23 ` Xiyue Deng 2024-08-14 8:40 ` Xiyue Deng 2024-08-14 9:13 ` Eli Zaretskii 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-14 8:23 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Björn Bidar, rpluim, Thomas Fitzsimmons, 72358 [-- Attachment #1: Type: text/plain, Size: 1717 bytes --] Eli Zaretskii <eliz@gnu.org> writes: >> From: Xiyue Deng <manphiz@gmail.com> >> Cc: rpluim@gmail.com, 72358@debbugs.gnu.org >> Date: Tue, 13 Aug 2024 15:03:30 -0700 >> >> Hi Eli, >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> Cc: 72358@debbugs.gnu.org >> >> From: Xiyue Deng <manphiz@gmail.com> >> >> Date: Fri, 02 Aug 2024 01:15:22 -0700 >> >> >> >> BTW, is there any dev available to commit the changes once it's in a >> >> good shape? >> > >> > All the 3 co-maintainers are tracking these discussions and install >> > changes that are ready to be installed. >> >> It's been a few days since the last time I received feedback for >> improvements regarding my patches. Is there any other feedbacks/reviews >> I am expecting from the co-maintainers? Please also let me know when >> it's time to ask for merging and requesting a new tagged release. > > ?? The last message in this discussion was just yesterday evening, and > my understanding is that you are still discussing the issues and did > not reach the final conclusion. If I'm mistaken, my apologies; The recent communication was not related to my patches but to check whether it is possible to support outlook.com OAuth2 login (and the conclusion was no because refreshing access token was disabled as confirmed by MS representative during an online chat.) > please describe your conclusion and post the patch that you-all agree > would solve the issues, and let's take it from there. I actually only received comments from Robert and I have updated my patches according in [1][2] (also attached in EOM). [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2055 bytes --] From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/6] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-14 8:23 ` Xiyue Deng @ 2024-08-14 8:40 ` Xiyue Deng 2024-08-14 9:13 ` Eli Zaretskii 1 sibling, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-14 8:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Björn Bidar, rpluim, Thomas Fitzsimmons, 72358 [-- Attachment #1: Type: text/plain, Size: 2595 bytes --] Xiyue Deng <manphiz@gmail.com> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >>> From: Xiyue Deng <manphiz@gmail.com> >>> Cc: rpluim@gmail.com, 72358@debbugs.gnu.org >>> Date: Tue, 13 Aug 2024 15:03:30 -0700 >>> >>> Hi Eli, >>> >>> Eli Zaretskii <eliz@gnu.org> writes: >>> >>> >> Cc: 72358@debbugs.gnu.org >>> >> From: Xiyue Deng <manphiz@gmail.com> >>> >> Date: Fri, 02 Aug 2024 01:15:22 -0700 >>> >> >>> >> BTW, is there any dev available to commit the changes once it's in a >>> >> good shape? >>> > >>> > All the 3 co-maintainers are tracking these discussions and install >>> > changes that are ready to be installed. >>> >>> It's been a few days since the last time I received feedback for >>> improvements regarding my patches. Is there any other feedbacks/reviews >>> I am expecting from the co-maintainers? Please also let me know when >>> it's time to ask for merging and requesting a new tagged release. >> >> ?? The last message in this discussion was just yesterday evening, and >> my understanding is that you are still discussing the issues and did >> not reach the final conclusion. If I'm mistaken, my apologies; > > The recent communication was not related to my patches but to check > whether it is possible to support outlook.com OAuth2 login (and the > conclusion was no because refreshing access token was disabled as > confirmed by MS representative during an online chat.) > >> please describe your conclusion and post the patch that you-all agree >> would solve the issues, and let's take it from there. > > I actually only received comments from Robert and I have updated my > patches according in [1][2] (also attached in EOM). > > [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 > [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 Sorry I accidentally hit sent for my previous mail (which actually ended on a full sentence, what are the chances?) To continue, the description of the patches is in the first message[3] and not changed much except the added 6th patch which added a NEWS file. TL;DR the patches include * adding authorization request parameters required for Gmail OAuth2 to work, * updating key calculation in oauth2.plstore to include `client-id' so that it can store multiple accounts for a given service, and * usability and debugging improvements. I haven't received more comments from Robert so I hope the patches now look good to him. Still, I would like to invite for more reviews (if any) and submit the patches when ready. Thanks! [3] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#5 -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2055 bytes --] From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/6] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Add-parameters-required-by-Google-OAuth2-to-get-refr.patch --] [-- Type: text/x-diff, Size: 1264 bytes --] From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:52:02 -0700 Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get refresh_token * packages/oauth2/oauth2.el (oauth2-request-authorization): add `access_type=offline' and `prompt=consent' when requesting token to receive refresh_token. --- oauth2.el | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 3a3e50ad2b..9780ac3a1d 100644 --- a/oauth2.el +++ b/oauth2.el @@ -63,7 +63,9 @@ It returns the code provided by the service." "&response_type=code" "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) "")))) + (if state (concat "&state=" (url-hexify-string state)) "") + "&access_type=offline" + "&prompt=consent"))) (browse-url url) (read-string (concat "Follow the instruction on your default browser, or " "visit:\n" url -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Encode-parameters-when-requesting-access.patch --] [-- Type: text/x-diff, Size: 1194 bytes --] From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:54:08 -0700 Subject: [PATCH 3/6] Encode parameters when requesting access * packages/oauth2/oauth2.el (oauth2-request-access): encode all parameters which may contain characters that breaks URL. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index 9780ac3a1d..b035742fc1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." (oauth2-make-access-request token-url (concat - "client_id=" client-id + "client_id=" (url-hexify-string client-id) (when client-secret - (concat "&client_secret=" client-secret)) - "&code=" code + (concat "&client_secret=" (url-hexify-string client-secret))) + "&code=" (url-hexify-string code) "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) "&grant_type=authorization_code")))) (make-oauth2-token :client-id client-id -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Support-storing-data-for-multiple-accounts-of-the-sa.patch --] [-- Type: text/x-diff, Size: 2090 bytes --] From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:00:04 -0700 Subject: [PATCH 4/6] Support storing data for multiple accounts of the same provider Currently the plstore id computed by `oauth2-compute-id' only takes `auth-url', `token-url', and `scope' into account, which could be the same for the same provider (e.g. Gmail). This prevents storing information for multiple accounts of the same service for some providers. This patch adds `client-id' to the calculation of plstore id to make sure that it is unique for different accounts of the same provider. It also changes the hash function to sha512 to be more secure. * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a parameter of `oauth2-compute-id' to ensure unique id amount multiple accounts of the same provider, and change hash function to sha512. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index b035742fc1..035971ac85 100644 --- a/oauth2.el +++ b/oauth2.el @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." :group 'oauth2 :type 'file) -(defun oauth2-compute-id (auth-url token-url scope) +(defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. This allows to store the token in an unique way." - (secure-hash 'md5 (concat auth-url token-url scope))) + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) ;;;###autoload (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) "Request access to a resource and store it using `plstore'." ;; We store a MD5 sum of all URL (let* ((plstore (plstore-open oauth2-token-file)) - (id (oauth2-compute-id auth-url token-url scope)) + (id (oauth2-compute-id auth-url token-url scope client-id)) (plist (cdr (plstore-get plstore id)))) ;; Check if we found something matching this access (if plist -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-Add-debug-messages-and-provide-a-switch-variable-for.patch --] [-- Type: text/x-diff, Size: 2186 bytes --] From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/6] Add debug messages and provide a switch variable for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add support for debug messages and a switch variable for enabling. --- oauth2.el | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/oauth2.el b/oauth2.el index 035971ac85..ce7a835100 100644 --- a/oauth2.el +++ b/oauth2.el @@ -40,6 +40,7 @@ (require 'plstore) (require 'json) (require 'url-http) +(require 'pp) (defvar url-http-data) (defvar url-http-method) @@ -53,6 +54,14 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defvar oauth2-debug nil + "Enable debug messages.") + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (if oauth2-debug + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -79,6 +88,8 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) (let ((url-request-method "POST") (url-request-data data) (url-request-extra-headers @@ -86,6 +97,8 @@ It returns the code provided by the service." (with-current-buffer (url-retrieve-synchronously url) (let ((data (oauth2-request-access-parse))) (kill-buffer (current-buffer)) + (oauth2--do-debug "oauth2-make-access-request: response: %s" + (pp-to-string data)) data)))) (cl-defstruct oauth2-token -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-Add-NEWS-file-to-document-the-changes-to-plstore-id-.patch --] [-- Type: text/x-diff, Size: 1350 bytes --] From e8735da21ac82b0698edad1796ddf4a1b8eb4bb2 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Tue, 30 Jul 2024 03:46:57 -0700 Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id generation --- NEWS | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 NEWS diff --git a/NEWS b/NEWS new file mode 100644 index 0000000000..6715a1914a --- /dev/null +++ b/NEWS @@ -0,0 +1,23 @@ +Summary of changes to oauth2.el +------------------------------- + +For changes of 0.16 and older or full changes please check the git +history of the repository of oauth2.el. + +* 0.17 + +** Changes to plstore id generation and needs to reacquire refresh_token + +The generation of plstore id used to include `auth-url', `token-url', +and `scope'. Now `client-id' is also included. This is required to +support multiple accounts of some providers which use the same +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the +generated plstore id is not unique amount accounts. Adding +`client-id' solves this problem. + +The hash function of calculating the plstore id has also changed from +MD5 to SHA512 to be more secure. + +As a result, users of oauth2.el will need to redo the authentication +process to get a new refresh_token when upgrading from older version +to 0.17. -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-14 8:23 ` Xiyue Deng 2024-08-14 8:40 ` Xiyue Deng @ 2024-08-14 9:13 ` Eli Zaretskii 2024-08-21 18:22 ` Xiyue Deng 1 sibling, 1 reply; 49+ messages in thread From: Eli Zaretskii @ 2024-08-14 9:13 UTC (permalink / raw) To: Xiyue Deng, Philip Kaludercic; +Cc: bjorn.bidar, rpluim, fitzsim, 72358 > From: Xiyue Deng <manphiz@gmail.com> > Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar > <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org > Date: Wed, 14 Aug 2024 01:23:19 -0700 > > >> It's been a few days since the last time I received feedback for > >> improvements regarding my patches. Is there any other feedbacks/reviews > >> I am expecting from the co-maintainers? Please also let me know when > >> it's time to ask for merging and requesting a new tagged release. > > > > ?? The last message in this discussion was just yesterday evening, and > > my understanding is that you are still discussing the issues and did > > not reach the final conclusion. If I'm mistaken, my apologies; > > The recent communication was not related to my patches but to check > whether it is possible to support outlook.com OAuth2 login (and the > conclusion was no because refreshing access token was disabled as > confirmed by MS representative during an online chat.) > > > please describe your conclusion and post the patch that you-all agree > > would solve the issues, and let's take it from there. > > I actually only received comments from Robert and I have updated my > patches according in [1][2] (also attached in EOM). > > [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 > [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 Thanks. Philip, could you please DTRT here? This seems to be an ELPA package. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-14 9:13 ` Eli Zaretskii @ 2024-08-21 18:22 ` Xiyue Deng 2024-08-21 19:42 ` Philip Kaludercic 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-21 18:22 UTC (permalink / raw) To: Eli Zaretskii; +Cc: bjorn.bidar, Philip Kaludercic, fitzsim, rpluim, 72358 Eli Zaretskii <eliz@gnu.org> writes: >> From: Xiyue Deng <manphiz@gmail.com> >> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >> Date: Wed, 14 Aug 2024 01:23:19 -0700 >> >> >> It's been a few days since the last time I received feedback for >> >> improvements regarding my patches. Is there any other feedbacks/reviews >> >> I am expecting from the co-maintainers? Please also let me know when >> >> it's time to ask for merging and requesting a new tagged release. >> > >> > ?? The last message in this discussion was just yesterday evening, and >> > my understanding is that you are still discussing the issues and did >> > not reach the final conclusion. If I'm mistaken, my apologies; >> >> The recent communication was not related to my patches but to check >> whether it is possible to support outlook.com OAuth2 login (and the >> conclusion was no because refreshing access token was disabled as >> confirmed by MS representative during an online chat.) >> >> > please describe your conclusion and post the patch that you-all agree >> > would solve the issues, and let's take it from there. >> >> I actually only received comments from Robert and I have updated my >> patches according in [1][2] (also attached in EOM). >> >> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 > > Thanks. > > Philip, could you please DTRT here? This seems to be an ELPA package. Friendly ping. Please also let me know if there are more review comments. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-21 18:22 ` Xiyue Deng @ 2024-08-21 19:42 ` Philip Kaludercic 2024-08-21 22:11 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Philip Kaludercic @ 2024-08-21 19:42 UTC (permalink / raw) To: Xiyue Deng; +Cc: bjorn.bidar, Eli Zaretskii, fitzsim, rpluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >>> From: Xiyue Deng <manphiz@gmail.com> >>> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >>> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >>> Date: Wed, 14 Aug 2024 01:23:19 -0700 >>> >>> >> It's been a few days since the last time I received feedback for >>> >> improvements regarding my patches. Is there any other feedbacks/reviews >>> >> I am expecting from the co-maintainers? Please also let me know when >>> >> it's time to ask for merging and requesting a new tagged release. >>> > >>> > ?? The last message in this discussion was just yesterday evening, and >>> > my understanding is that you are still discussing the issues and did >>> > not reach the final conclusion. If I'm mistaken, my apologies; >>> >>> The recent communication was not related to my patches but to check >>> whether it is possible to support outlook.com OAuth2 login (and the >>> conclusion was no because refreshing access token was disabled as >>> confirmed by MS representative during an online chat.) >>> >>> > please describe your conclusion and post the patch that you-all agree >>> > would solve the issues, and let's take it from there. >>> >>> I actually only received comments from Robert and I have updated my >>> patches according in [1][2] (also attached in EOM). >>> >>> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 >> >> Thanks. >> >> Philip, could you please DTRT here? This seems to be an ELPA package. > > Friendly ping. Please also let me know if there are more review > comments. I'm sorry, this was a rather long thread and I didn't have the time yet to follow up on it. Can you confirm that you want me to review the patches attached to this message: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 ? -- Philip Kaludercic on peregrine ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-21 19:42 ` Philip Kaludercic @ 2024-08-21 22:11 ` Xiyue Deng 2024-08-29 6:58 ` Xiyue Deng 2024-08-29 14:14 ` Philip Kaludercic 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-21 22:11 UTC (permalink / raw) To: Philip Kaludercic; +Cc: bjorn.bidar, Eli Zaretskii, fitzsim, rpluim, 72358 [-- Attachment #1: Type: text/plain, Size: 2230 bytes --] Hi Philip, Philip Kaludercic <philipk@posteo.net> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Eli Zaretskii <eliz@gnu.org> writes: >> >>>> From: Xiyue Deng <manphiz@gmail.com> >>>> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >>>> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >>>> Date: Wed, 14 Aug 2024 01:23:19 -0700 >>>> >>>> >> It's been a few days since the last time I received feedback for >>>> >> improvements regarding my patches. Is there any other feedbacks/reviews >>>> >> I am expecting from the co-maintainers? Please also let me know when >>>> >> it's time to ask for merging and requesting a new tagged release. >>>> > >>>> > ?? The last message in this discussion was just yesterday evening, and >>>> > my understanding is that you are still discussing the issues and did >>>> > not reach the final conclusion. If I'm mistaken, my apologies; >>>> >>>> The recent communication was not related to my patches but to check >>>> whether it is possible to support outlook.com OAuth2 login (and the >>>> conclusion was no because refreshing access token was disabled as >>>> confirmed by MS representative during an online chat.) >>>> >>>> > please describe your conclusion and post the patch that you-all agree >>>> > would solve the issues, and let's take it from there. >>>> >>>> I actually only received comments from Robert and I have updated my >>>> patches according in [1][2] (also attached in EOM). >>>> >>>> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>>> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 >>> >>> Thanks. >>> >>> Philip, could you please DTRT here? This seems to be an ELPA package. >> >> Friendly ping. Please also let me know if there are more review >> comments. > > I'm sorry, this was a rather long thread and I didn't have the time yet > to follow up on it. Can you confirm that you want me to review the > patches attached to this message: > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 > > ? Almost with a later update for patch 5. I am now attaching the latest patches here to avoid any confusions. Thanks! -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2055 bytes --] From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/6] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Add-parameters-required-by-Google-OAuth2-to-get-refr.patch --] [-- Type: text/x-diff, Size: 1264 bytes --] From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:52:02 -0700 Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get refresh_token * packages/oauth2/oauth2.el (oauth2-request-authorization): add `access_type=offline' and `prompt=consent' when requesting token to receive refresh_token. --- oauth2.el | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 3a3e50ad2b..9780ac3a1d 100644 --- a/oauth2.el +++ b/oauth2.el @@ -63,7 +63,9 @@ It returns the code provided by the service." "&response_type=code" "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) "")))) + (if state (concat "&state=" (url-hexify-string state)) "") + "&access_type=offline" + "&prompt=consent"))) (browse-url url) (read-string (concat "Follow the instruction on your default browser, or " "visit:\n" url -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Encode-parameters-when-requesting-access.patch --] [-- Type: text/x-diff, Size: 1194 bytes --] From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:54:08 -0700 Subject: [PATCH 3/6] Encode parameters when requesting access * packages/oauth2/oauth2.el (oauth2-request-access): encode all parameters which may contain characters that breaks URL. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index 9780ac3a1d..b035742fc1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." (oauth2-make-access-request token-url (concat - "client_id=" client-id + "client_id=" (url-hexify-string client-id) (when client-secret - (concat "&client_secret=" client-secret)) - "&code=" code + (concat "&client_secret=" (url-hexify-string client-secret))) + "&code=" (url-hexify-string code) "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) "&grant_type=authorization_code")))) (make-oauth2-token :client-id client-id -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Support-storing-data-for-multiple-accounts-of-the-sa.patch --] [-- Type: text/x-diff, Size: 2090 bytes --] From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:00:04 -0700 Subject: [PATCH 4/6] Support storing data for multiple accounts of the same provider Currently the plstore id computed by `oauth2-compute-id' only takes `auth-url', `token-url', and `scope' into account, which could be the same for the same provider (e.g. Gmail). This prevents storing information for multiple accounts of the same service for some providers. This patch adds `client-id' to the calculation of plstore id to make sure that it is unique for different accounts of the same provider. It also changes the hash function to sha512 to be more secure. * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a parameter of `oauth2-compute-id' to ensure unique id amount multiple accounts of the same provider, and change hash function to sha512. --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index b035742fc1..035971ac85 100644 --- a/oauth2.el +++ b/oauth2.el @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." :group 'oauth2 :type 'file) -(defun oauth2-compute-id (auth-url token-url scope) +(defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. This allows to store the token in an unique way." - (secure-hash 'md5 (concat auth-url token-url scope))) + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) ;;;###autoload (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) "Request access to a resource and store it using `plstore'." ;; We store a MD5 sum of all URL (let* ((plstore (plstore-open oauth2-token-file)) - (id (oauth2-compute-id auth-url token-url scope)) + (id (oauth2-compute-id auth-url token-url scope client-id)) (plist (cdr (plstore-get plstore id)))) ;; Check if we found something matching this access (if plist -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-Add-debug-messages-and-provide-a-switch-variable-for.patch --] [-- Type: text/x-diff, Size: 2186 bytes --] From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/6] Add debug messages and provide a switch variable for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add support for debug messages and a switch variable for enabling. --- oauth2.el | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/oauth2.el b/oauth2.el index 035971ac85..ce7a835100 100644 --- a/oauth2.el +++ b/oauth2.el @@ -40,6 +40,7 @@ (require 'plstore) (require 'json) (require 'url-http) +(require 'pp) (defvar url-http-data) (defvar url-http-method) @@ -53,6 +54,14 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defvar oauth2-debug nil + "Enable debug messages.") + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (if oauth2-debug + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -79,6 +88,8 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) (let ((url-request-method "POST") (url-request-data data) (url-request-extra-headers @@ -86,6 +97,8 @@ It returns the code provided by the service." (with-current-buffer (url-retrieve-synchronously url) (let ((data (oauth2-request-access-parse))) (kill-buffer (current-buffer)) + (oauth2--do-debug "oauth2-make-access-request: response: %s" + (pp-to-string data)) data)))) (cl-defstruct oauth2-token -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-Add-NEWS-file-to-document-the-changes-to-plstore-id-.patch --] [-- Type: text/x-diff, Size: 1350 bytes --] From e8735da21ac82b0698edad1796ddf4a1b8eb4bb2 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Tue, 30 Jul 2024 03:46:57 -0700 Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id generation --- NEWS | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 NEWS diff --git a/NEWS b/NEWS new file mode 100644 index 0000000000..6715a1914a --- /dev/null +++ b/NEWS @@ -0,0 +1,23 @@ +Summary of changes to oauth2.el +------------------------------- + +For changes of 0.16 and older or full changes please check the git +history of the repository of oauth2.el. + +* 0.17 + +** Changes to plstore id generation and needs to reacquire refresh_token + +The generation of plstore id used to include `auth-url', `token-url', +and `scope'. Now `client-id' is also included. This is required to +support multiple accounts of some providers which use the same +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the +generated plstore id is not unique amount accounts. Adding +`client-id' solves this problem. + +The hash function of calculating the plstore id has also changed from +MD5 to SHA512 to be more secure. + +As a result, users of oauth2.el will need to redo the authentication +process to get a new refresh_token when upgrading from older version +to 0.17. -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-21 22:11 ` Xiyue Deng @ 2024-08-29 6:58 ` Xiyue Deng 2024-08-29 14:14 ` Philip Kaludercic 1 sibling, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-29 6:58 UTC (permalink / raw) To: Philip Kaludercic; +Cc: bjorn.bidar, Eli Zaretskii, fitzsim, rpluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Hi Philip, > > Philip Kaludercic <philipk@posteo.net> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Eli Zaretskii <eliz@gnu.org> writes: >>> >>>>> From: Xiyue Deng <manphiz@gmail.com> >>>>> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >>>>> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >>>>> Date: Wed, 14 Aug 2024 01:23:19 -0700 >>>>> >>>>> >> It's been a few days since the last time I received feedback for >>>>> >> improvements regarding my patches. Is there any other feedbacks/reviews >>>>> >> I am expecting from the co-maintainers? Please also let me know when >>>>> >> it's time to ask for merging and requesting a new tagged release. >>>>> > >>>>> > ?? The last message in this discussion was just yesterday evening, and >>>>> > my understanding is that you are still discussing the issues and did >>>>> > not reach the final conclusion. If I'm mistaken, my apologies; >>>>> >>>>> The recent communication was not related to my patches but to check >>>>> whether it is possible to support outlook.com OAuth2 login (and the >>>>> conclusion was no because refreshing access token was disabled as >>>>> confirmed by MS representative during an online chat.) >>>>> >>>>> > please describe your conclusion and post the patch that you-all agree >>>>> > would solve the issues, and let's take it from there. >>>>> >>>>> I actually only received comments from Robert and I have updated my >>>>> patches according in [1][2] (also attached in EOM). >>>>> >>>>> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>>>> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 >>>> >>>> Thanks. >>>> >>>> Philip, could you please DTRT here? This seems to be an ELPA package. >>> >>> Friendly ping. Please also let me know if there are more review >>> comments. >> >> I'm sorry, this was a rather long thread and I didn't have the time yet >> to follow up on it. Can you confirm that you want me to review the >> patches attached to this message: >> >> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >> >> ? > > Almost with a later update for patch 5. I am now attaching the latest > patches here to avoid any confusions. Thanks! Friendly ping. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-21 22:11 ` Xiyue Deng 2024-08-29 6:58 ` Xiyue Deng @ 2024-08-29 14:14 ` Philip Kaludercic 2024-08-29 15:18 ` Robert Pluim 2024-08-29 23:54 ` Xiyue Deng 1 sibling, 2 replies; 49+ messages in thread From: Philip Kaludercic @ 2024-08-29 14:14 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 Xiyue Deng <manphiz@gmail.com> writes: > Hi Philip, > > Philip Kaludercic <philipk@posteo.net> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Eli Zaretskii <eliz@gnu.org> writes: >>> >>>>> From: Xiyue Deng <manphiz@gmail.com> >>>>> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >>>>> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >>>>> Date: Wed, 14 Aug 2024 01:23:19 -0700 >>>>> >>>>> >> It's been a few days since the last time I received feedback for >>>>> >> improvements regarding my patches. Is there any other >>>>> >> feedbacks/reviews >>>>> >> I am expecting from the co-maintainers? Please also let me know when >>>>> >> it's time to ask for merging and requesting a new tagged release. >>>>> > >>>>> > ?? The last message in this discussion was just yesterday evening, and >>>>> > my understanding is that you are still discussing the issues and did >>>>> > not reach the final conclusion. If I'm mistaken, my apologies; >>>>> >>>>> The recent communication was not related to my patches but to check >>>>> whether it is possible to support outlook.com OAuth2 login (and the >>>>> conclusion was no because refreshing access token was disabled as >>>>> confirmed by MS representative during an online chat.) >>>>> >>>>> > please describe your conclusion and post the patch that you-all agree >>>>> > would solve the issues, and let's take it from there. >>>>> >>>>> I actually only received comments from Robert and I have updated my >>>>> patches according in [1][2] (also attached in EOM). >>>>> >>>>> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>>>> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 >>>> >>>> Thanks. >>>> >>>> Philip, could you please DTRT here? This seems to be an ELPA package. >>> >>> Friendly ping. Please also let me know if there are more review >>> comments. >> >> I'm sorry, this was a rather long thread and I didn't have the time yet >> to follow up on it. Can you confirm that you want me to review the >> patches attached to this message: >> >> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >> >> ? > > Almost with a later update for patch 5. I am now attaching the latest > patches here to avoid any confusions. Thanks! > > -- > Xiyue Deng > > From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Sun, 21 Jul 2024 14:50:56 -0700 > Subject: [PATCH 1/6] Show full authentication URL to let user choose how to > visit it > > * packages/oauth2/oauth2.el (oauth2-request-authorization): show full > authentication URL in user prompt. Generally it would be nice if you could attach a "(Bug#72358)" to the end of each commit. Just add it like a new sentence. [...] authentication URL in user prompt. (Bug#72358) > --- > oauth2.el | 19 +++++++++++-------- > 1 file changed, 11 insertions(+), 8 deletions(-) > > diff --git a/oauth2.el b/oauth2.el > index 7da9702004..3a3e50ad2b 100644 > --- a/oauth2.el > +++ b/oauth2.el > @@ -57,14 +57,17 @@ > "Request OAuth authorization at AUTH-URL by launching `browse-url'. > CLIENT-ID is the client id provided by the provider. > It returns the code provided by the service." > - (browse-url (concat auth-url > - (if (string-match-p "\?" auth-url) "&" "?") > - "client_id=" (url-hexify-string client-id) > - "&response_type=code" > - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) > - (if scope (concat "&scope=" (url-hexify-string scope)) "") > - (if state (concat "&state=" (url-hexify-string state)) ""))) > - (read-string "Enter the code your browser displayed: ")) > + (let ((url (concat auth-url > + (if (string-match-p "\?" auth-url) "&" "?") > + "client_id=" (url-hexify-string client-id) > + "&response_type=code" > + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) > + (if scope (concat "&scope=" (url-hexify-string scope)) "") > + (if state (concat "&state=" (url-hexify-string state)) "")))) > + (browse-url url) > + (read-string (concat "Follow the instruction on your default browser, or " > + "visit:\n" url > + "\nEnter the code your browser displayed: ")))) LGTM. > > (defun oauth2-request-access-parse () > "Parse the result of an OAuth request." > -- > 2.39.2 > > > From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Sun, 21 Jul 2024 14:52:02 -0700 > Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get > refresh_token > > * packages/oauth2/oauth2.el (oauth2-request-authorization): add > `access_type=offline' and `prompt=consent' when requesting token to I wouldn't use `...' syntax in commit ChangeLog messages. > receive refresh_token. > --- > oauth2.el | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/oauth2.el b/oauth2.el > index 3a3e50ad2b..9780ac3a1d 100644 > --- a/oauth2.el > +++ b/oauth2.el > @@ -63,7 +63,9 @@ It returns the code provided by the service." > "&response_type=code" > "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) > (if scope (concat "&scope=" (url-hexify-string scope)) "") > - (if state (concat "&state=" (url-hexify-string state)) "")))) > + (if state (concat "&state=" (url-hexify-string state)) "") > + "&access_type=offline" > + "&prompt=consent"))) You do what you say, but perhaps a comment explaining why you have these queries might be useful? > (browse-url url) > (read-string (concat "Follow the instruction on your default browser, or " > "visit:\n" url > -- > 2.39.2 > > > From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Sun, 21 Jul 2024 14:54:08 -0700 > Subject: [PATCH 3/6] Encode parameters when requesting access > > * packages/oauth2/oauth2.el (oauth2-request-access): encode all > parameters which may contain characters that breaks URL. > --- > oauth2.el | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/oauth2.el b/oauth2.el > index 9780ac3a1d..b035742fc1 100644 > --- a/oauth2.el > +++ b/oauth2.el > @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." > (oauth2-make-access-request > token-url > (concat > - "client_id=" client-id > + "client_id=" (url-hexify-string client-id) > (when client-secret > - (concat "&client_secret=" client-secret)) > - "&code=" code > + (concat "&client_secret=" (url-hexify-string client-secret))) > + "&code=" (url-hexify-string code) > "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) > "&grant_type=authorization_code")))) > (make-oauth2-token :client-id client-id > -- > 2.39.2 1+ Though one might also consider using something like `url-encode-url' to generate the entire URL. > > > From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Sun, 28 Jul 2024 03:00:04 -0700 > Subject: [PATCH 4/6] Support storing data for multiple accounts of the same > provider > > Currently the plstore id computed by `oauth2-compute-id' only takes > `auth-url', `token-url', and `scope' into account, which could be the > same for the same provider (e.g. Gmail). This prevents storing > information for multiple accounts of the same service for some > providers. > > This patch adds `client-id' to the calculation of plstore id to make > sure that it is unique for different accounts of the same provider. > > It also changes the hash function to sha512 to be more secure. > > * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a > parameter of `oauth2-compute-id' to ensure unique id amount multiple > accounts of the same provider, and change hash function to sha512. > --- > oauth2.el | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/oauth2.el b/oauth2.el > index b035742fc1..035971ac85 100644 > --- a/oauth2.el > +++ b/oauth2.el > @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." > :group 'oauth2 > :type 'file) > > -(defun oauth2-compute-id (auth-url token-url scope) > +(defun oauth2-compute-id (auth-url token-url scope client-id) > "Compute an unique id based on URLs. > This allows to store the token in an unique way." > - (secure-hash 'md5 (concat auth-url token-url scope))) > + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) > > ;;;###autoload > (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) > "Request access to a resource and store it using `plstore'." > ;; We store a MD5 sum of all URL > (let* ((plstore (plstore-open oauth2-token-file)) > - (id (oauth2-compute-id auth-url token-url scope)) > + (id (oauth2-compute-id auth-url token-url scope client-id)) > (plist (cdr (plstore-get plstore id)))) > ;; Check if we found something matching this access > (if plist > -- > 2.39.2 LGTM > > From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Sun, 28 Jul 2024 03:41:20 -0700 > Subject: [PATCH 5/6] Add debug messages and provide a switch variable for > enabling > > This helps debugging whether the authorization and refresh requests > were successful and inspecting the responses. > > * packages/oauth2/oauth2.el: add support for debug messages and a > switch variable for enabling. > --- > oauth2.el | 13 +++++++++++++ > 1 file changed, 13 insertions(+) > > diff --git a/oauth2.el b/oauth2.el > index 035971ac85..ce7a835100 100644 > --- a/oauth2.el > +++ b/oauth2.el > @@ -40,6 +40,7 @@ > (require 'plstore) > (require 'json) > (require 'url-http) > +(require 'pp) > > (defvar url-http-data) > (defvar url-http-method) > @@ -53,6 +54,14 @@ > :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") > :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) > > +(defvar oauth2-debug nil > + "Enable debug messages.") I'd slightly expand this comment in case someone wants to use something like `apropos-documentation'. Also, why is this not a user option? > + > +(defun oauth2--do-debug (&rest msg) > + "Output debug messages when `oauth2-debug' is enabled." > + (if oauth2-debug I'd use `when' here. > + (apply #'message msg))) ... and perhaps prefix the message with a string like "[oauth2] ". I also notice that you manually mention the function issuing the debug message below. It is possible to infer this using `backtrace-frame' (but that is really something that should be added to the core, instead of packages having to re-implement it themselves.) > + > (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) > "Request OAuth authorization at AUTH-URL by launching `browse-url'. > CLIENT-ID is the client id provided by the provider. > @@ -79,6 +88,8 @@ It returns the code provided by the service." > > (defun oauth2-make-access-request (url data) > "Make an access request to URL using DATA in POST." > + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) > + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) > (let ((url-request-method "POST") > (url-request-data data) > (url-request-extra-headers > @@ -86,6 +97,8 @@ It returns the code provided by the service." > (with-current-buffer (url-retrieve-synchronously url) > (let ((data (oauth2-request-access-parse))) > (kill-buffer (current-buffer)) > + (oauth2--do-debug "oauth2-make-access-request: response: %s" > + (pp-to-string data)) Is pp-to-string here really much better than prin1-to-string? Note that 'oauth2--do-debug' is a function, so the arguments are always evaluated. I have experienced pp being significantly slower than the core print functions, so this is something that is worth thinking about IMO. > data)))) > > (cl-defstruct oauth2-token > -- > 2.39.2 > > > From e8735da21ac82b0698edad1796ddf4a1b8eb4bb2 Mon Sep 17 00:00:00 2001 > From: Xiyue Deng <manphiz@gmail.com> > Date: Tue, 30 Jul 2024 03:46:57 -0700 > Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id > generation > > --- > NEWS | 23 +++++++++++++++++++++++ > 1 file changed, 23 insertions(+) > create mode 100644 NEWS > > diff --git a/NEWS b/NEWS > new file mode 100644 > index 0000000000..6715a1914a > --- /dev/null > +++ b/NEWS > @@ -0,0 +1,23 @@ > +Summary of changes to oauth2.el > +------------------------------- > + > +For changes of 0.16 and older or full changes please check the git > +history of the repository of oauth2.el. > + > +* 0.17 > + > +** Changes to plstore id generation and needs to reacquire refresh_token > + > +The generation of plstore id used to include `auth-url', `token-url', > +and `scope'. Now `client-id' is also included. This is required to > +support multiple accounts of some providers which use the same > +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the > +generated plstore id is not unique amount accounts. Adding > +`client-id' solves this problem. > + > +The hash function of calculating the plstore id has also changed from > +MD5 to SHA512 to be more secure. > + > +As a result, users of oauth2.el will need to redo the authentication > +process to get a new refresh_token when upgrading from older version > +to 0.17. Perhaps you can add a file local variable to enable outline-mode? All in all the patches look more than fine though, nothing I mentioned is pressing. If you don't have the time and the changes are urgent, then I can also apply them as such. Xiyue Deng <manphiz@gmail.com> writes: [...] > Friendly ping. Friendly pong. -- Philip Kaludercic on peregrine ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-29 14:14 ` Philip Kaludercic @ 2024-08-29 15:18 ` Robert Pluim 2024-08-29 23:54 ` Xiyue Deng 1 sibling, 0 replies; 49+ messages in thread From: Robert Pluim @ 2024-08-29 15:18 UTC (permalink / raw) To: Philip Kaludercic; +Cc: 72358, Xiyue Deng >>>>> On Thu, 29 Aug 2024 14:14:17 +0000, Philip Kaludercic <philipk@posteo.net> said: >> +(defvar oauth2-debug nil >> + "Enable debug messages.") Philip> I'd slightly expand this comment in case someone wants to use something Philip> like `apropos-documentation'. Also, why is this not a user option? Because I asked him to make it a `defvar' :-) I donʼt think debug options should be customizable, theyʼre not user-level. Robert -- ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-29 14:14 ` Philip Kaludercic 2024-08-29 15:18 ` Robert Pluim @ 2024-08-29 23:54 ` Xiyue Deng 2024-08-30 7:09 ` Philip Kaludercic 1 sibling, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-29 23:54 UTC (permalink / raw) To: Philip Kaludercic; +Cc: 72358 [-- Attachment #1: Type: text/plain, Size: 16503 bytes --] Hi Philip, Thanks for the review! Please see my replies below inline. Philip Kaludercic <philipk@posteo.net> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Hi Philip, >> >> Philip Kaludercic <philipk@posteo.net> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>>> Eli Zaretskii <eliz@gnu.org> writes: >>>> >>>>>> From: Xiyue Deng <manphiz@gmail.com> >>>>>> Cc: Thomas Fitzsimmons <fitzsim@fitzsim.org>, Björn Bidar >>>>>> <bjorn.bidar@thaodan.de>, rpluim@gmail.com, 72358@debbugs.gnu.org >>>>>> Date: Wed, 14 Aug 2024 01:23:19 -0700 >>>>>> >>>>>> >> It's been a few days since the last time I received feedback for >>>>>> >> improvements regarding my patches. Is there any other >>>>>> >> feedbacks/reviews >>>>>> >> I am expecting from the co-maintainers? Please also let me know when >>>>>> >> it's time to ask for merging and requesting a new tagged release. >>>>>> > >>>>>> > ?? The last message in this discussion was just yesterday evening, and >>>>>> > my understanding is that you are still discussing the issues and did >>>>>> > not reach the final conclusion. If I'm mistaken, my apologies; >>>>>> >>>>>> The recent communication was not related to my patches but to check >>>>>> whether it is possible to support outlook.com OAuth2 login (and the >>>>>> conclusion was no because refreshing access token was disabled as >>>>>> confirmed by MS representative during an online chat.) >>>>>> >>>>>> > please describe your conclusion and post the patch that you-all agree >>>>>> > would solve the issues, and let's take it from there. >>>>>> >>>>>> I actually only received comments from Robert and I have updated my >>>>>> patches according in [1][2] (also attached in EOM). >>>>>> >>>>>> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>>>>> [2] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#44 >>>>> >>>>> Thanks. >>>>> >>>>> Philip, could you please DTRT here? This seems to be an ELPA package. >>>> >>>> Friendly ping. Please also let me know if there are more review >>>> comments. >>> >>> I'm sorry, this was a rather long thread and I didn't have the time yet >>> to follow up on it. Can you confirm that you want me to review the >>> patches attached to this message: >>> >>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=72358#20 >>> >>> ? >> >> Almost with a later update for patch 5. I am now attaching the latest >> patches here to avoid any confusions. Thanks! >> >> -- >> Xiyue Deng >> >> From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Sun, 21 Jul 2024 14:50:56 -0700 >> Subject: [PATCH 1/6] Show full authentication URL to let user choose how to >> visit it >> >> * packages/oauth2/oauth2.el (oauth2-request-authorization): show full >> authentication URL in user prompt. > > Generally it would be nice if you could attach a "(Bug#72358)" to the > end of each commit. Just add it like a new sentence. > > [...] > authentication URL in user prompt. (Bug#72358) > Done for all patches. >> --- >> oauth2.el | 19 +++++++++++-------- >> 1 file changed, 11 insertions(+), 8 deletions(-) >> >> diff --git a/oauth2.el b/oauth2.el >> index 7da9702004..3a3e50ad2b 100644 >> --- a/oauth2.el >> +++ b/oauth2.el >> @@ -57,14 +57,17 @@ >> "Request OAuth authorization at AUTH-URL by launching `browse-url'. >> CLIENT-ID is the client id provided by the provider. >> It returns the code provided by the service." >> - (browse-url (concat auth-url >> - (if (string-match-p "\?" auth-url) "&" "?") >> - "client_id=" (url-hexify-string client-id) >> - "&response_type=code" >> - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >> - (if scope (concat "&scope=" (url-hexify-string scope)) "") >> - (if state (concat "&state=" (url-hexify-string state)) ""))) >> - (read-string "Enter the code your browser displayed: ")) >> + (let ((url (concat auth-url >> + (if (string-match-p "\?" auth-url) "&" "?") >> + "client_id=" (url-hexify-string client-id) >> + "&response_type=code" >> + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >> + (if scope (concat "&scope=" (url-hexify-string scope)) "") >> + (if state (concat "&state=" (url-hexify-string state)) "")))) >> + (browse-url url) >> + (read-string (concat "Follow the instruction on your default browser, or " >> + "visit:\n" url >> + "\nEnter the code your browser displayed: ")))) > > LGTM. > >> >> (defun oauth2-request-access-parse () >> "Parse the result of an OAuth request." >> -- >> 2.39.2 >> >> >> From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Sun, 21 Jul 2024 14:52:02 -0700 >> Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get >> refresh_token >> >> * packages/oauth2/oauth2.el (oauth2-request-authorization): add >> `access_type=offline' and `prompt=consent' when requesting token to > > I wouldn't use `...' syntax in commit ChangeLog messages. > Ah I saw this used in the texinfo doc extensively so I thought it was preferred. I have changed all these to double quotes, though do let me know if there is a better preference. >> receive refresh_token. >> --- >> oauth2.el | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/oauth2.el b/oauth2.el >> index 3a3e50ad2b..9780ac3a1d 100644 >> --- a/oauth2.el >> +++ b/oauth2.el >> @@ -63,7 +63,9 @@ It returns the code provided by the service." >> "&response_type=code" >> "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >> (if scope (concat "&scope=" (url-hexify-string scope)) "") >> - (if state (concat "&state=" (url-hexify-string state)) "")))) >> + (if state (concat "&state=" (url-hexify-string state)) "") >> + "&access_type=offline" >> + "&prompt=consent"))) > > You do what you say, but perhaps a comment explaining why you have these > queries might be useful? > Added a comment above to explain the reasons. >> (browse-url url) >> (read-string (concat "Follow the instruction on your default browser, or " >> "visit:\n" url >> -- >> 2.39.2 >> >> >> From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Sun, 21 Jul 2024 14:54:08 -0700 >> Subject: [PATCH 3/6] Encode parameters when requesting access >> >> * packages/oauth2/oauth2.el (oauth2-request-access): encode all >> parameters which may contain characters that breaks URL. >> --- >> oauth2.el | 6 +++--- >> 1 file changed, 3 insertions(+), 3 deletions(-) >> >> diff --git a/oauth2.el b/oauth2.el >> index 9780ac3a1d..b035742fc1 100644 >> --- a/oauth2.el >> +++ b/oauth2.el >> @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." >> (oauth2-make-access-request >> token-url >> (concat >> - "client_id=" client-id >> + "client_id=" (url-hexify-string client-id) >> (when client-secret >> - (concat "&client_secret=" client-secret)) >> - "&code=" code >> + (concat "&client_secret=" (url-hexify-string client-secret))) >> + "&code=" (url-hexify-string code) >> "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >> "&grant_type=authorization_code")))) >> (make-oauth2-token :client-id client-id >> -- >> 2.39.2 > > 1+ Though one might also consider using something like `url-encode-url' > to generate the entire URL. > It is indeed much cleaner. Changed accordingly. >> >> >> From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Sun, 28 Jul 2024 03:00:04 -0700 >> Subject: [PATCH 4/6] Support storing data for multiple accounts of the same >> provider >> >> Currently the plstore id computed by `oauth2-compute-id' only takes >> `auth-url', `token-url', and `scope' into account, which could be the >> same for the same provider (e.g. Gmail). This prevents storing >> information for multiple accounts of the same service for some >> providers. >> >> This patch adds `client-id' to the calculation of plstore id to make >> sure that it is unique for different accounts of the same provider. >> >> It also changes the hash function to sha512 to be more secure. >> >> * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a >> parameter of `oauth2-compute-id' to ensure unique id amount multiple >> accounts of the same provider, and change hash function to sha512. >> --- >> oauth2.el | 6 +++--- >> 1 file changed, 3 insertions(+), 3 deletions(-) >> >> diff --git a/oauth2.el b/oauth2.el >> index b035742fc1..035971ac85 100644 >> --- a/oauth2.el >> +++ b/oauth2.el >> @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." >> :group 'oauth2 >> :type 'file) >> >> -(defun oauth2-compute-id (auth-url token-url scope) >> +(defun oauth2-compute-id (auth-url token-url scope client-id) >> "Compute an unique id based on URLs. >> This allows to store the token in an unique way." >> - (secure-hash 'md5 (concat auth-url token-url scope))) >> + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) >> >> ;;;###autoload >> (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) >> "Request access to a resource and store it using `plstore'." >> ;; We store a MD5 sum of all URL >> (let* ((plstore (plstore-open oauth2-token-file)) >> - (id (oauth2-compute-id auth-url token-url scope)) >> + (id (oauth2-compute-id auth-url token-url scope client-id)) >> (plist (cdr (plstore-get plstore id)))) >> ;; Check if we found something matching this access >> (if plist >> -- >> 2.39.2 > > LGTM > >> >> From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Sun, 28 Jul 2024 03:41:20 -0700 >> Subject: [PATCH 5/6] Add debug messages and provide a switch variable for >> enabling >> >> This helps debugging whether the authorization and refresh requests >> were successful and inspecting the responses. >> >> * packages/oauth2/oauth2.el: add support for debug messages and a >> switch variable for enabling. >> --- >> oauth2.el | 13 +++++++++++++ >> 1 file changed, 13 insertions(+) >> >> diff --git a/oauth2.el b/oauth2.el >> index 035971ac85..ce7a835100 100644 >> --- a/oauth2.el >> +++ b/oauth2.el >> @@ -40,6 +40,7 @@ >> (require 'plstore) >> (require 'json) >> (require 'url-http) >> +(require 'pp) >> >> (defvar url-http-data) >> (defvar url-http-method) >> @@ -53,6 +54,14 @@ >> :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") >> :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) >> >> +(defvar oauth2-debug nil >> + "Enable debug messages.") > > I'd slightly expand this comment in case someone wants to use something > like `apropos-documentation'. Expanded the doc string to explain the purpose. > Also, why is this not a user option? > As Robert mentioned in the follow-up mail it was his suggestion (thanks Robert!) Do let me know if `defcustom' is preferred. >> + >> +(defun oauth2--do-debug (&rest msg) >> + "Output debug messages when `oauth2-debug' is enabled." >> + (if oauth2-debug > > I'd use `when' here. > Done >> + (apply #'message msg))) > > ... and perhaps prefix the message with a string like "[oauth2] ". > Done. I used `setcar' but let me know if there is a more elegant way to do this. > I also notice that you manually mention the function issuing the debug > message below. It is possible to infer this using `backtrace-frame' > (but that is really something that should be added to the core, instead > of packages having to re-implement it themselves.) > Added a let-bounded `func-name' by getting function name from backtrace-frame. And it would be great if there is a built-in macro like `__PRETTY_FUNCTION__' and `__LINE__'. >> + >> (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) >> "Request OAuth authorization at AUTH-URL by launching `browse-url'. >> CLIENT-ID is the client id provided by the provider. >> @@ -79,6 +88,8 @@ It returns the code provided by the service." >> >> (defun oauth2-make-access-request (url data) >> "Make an access request to URL using DATA in POST." >> + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) >> + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) >> (let ((url-request-method "POST") >> (url-request-data data) >> (url-request-extra-headers >> @@ -86,6 +97,8 @@ It returns the code provided by the service." >> (with-current-buffer (url-retrieve-synchronously url) >> (let ((data (oauth2-request-access-parse))) >> (kill-buffer (current-buffer)) >> + (oauth2--do-debug "oauth2-make-access-request: response: %s" >> + (pp-to-string data)) > > Is pp-to-string here really much better than prin1-to-string? Note that > 'oauth2--do-debug' is a function, so the arguments are always evaluated. > I have experienced pp being significantly slower than the core print > functions, so this is something that is worth thinking about IMO. > It's just a personal preference. And it look like the values are mostly key-value pairs so they are not that hard to read through `prin1-to-string' so I changed to it. >> data)))) >> >> (cl-defstruct oauth2-token >> -- >> 2.39.2 >> >> >> From e8735da21ac82b0698edad1796ddf4a1b8eb4bb2 Mon Sep 17 00:00:00 2001 >> From: Xiyue Deng <manphiz@gmail.com> >> Date: Tue, 30 Jul 2024 03:46:57 -0700 >> Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id >> generation >> >> --- >> NEWS | 23 +++++++++++++++++++++++ >> 1 file changed, 23 insertions(+) >> create mode 100644 NEWS >> >> diff --git a/NEWS b/NEWS >> new file mode 100644 >> index 0000000000..6715a1914a >> --- /dev/null >> +++ b/NEWS >> @@ -0,0 +1,23 @@ >> +Summary of changes to oauth2.el >> +------------------------------- >> + >> +For changes of 0.16 and older or full changes please check the git >> +history of the repository of oauth2.el. >> + >> +* 0.17 >> + >> +** Changes to plstore id generation and needs to reacquire refresh_token >> + >> +The generation of plstore id used to include `auth-url', `token-url', >> +and `scope'. Now `client-id' is also included. This is required to >> +support multiple accounts of some providers which use the same >> +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the >> +generated plstore id is not unique amount accounts. Adding >> +`client-id' solves this problem. >> + >> +The hash function of calculating the plstore id has also changed from >> +MD5 to SHA512 to be more secure. >> + >> +As a result, users of oauth2.el will need to redo the authentication >> +process to get a new refresh_token when upgrading from older version >> +to 0.17. > > Perhaps you can add a file local variable to enable outline-mode? > Done. Also improved the wording. > All in all the patches look more than fine though, nothing I mentioned > is pressing. If you don't have the time and the changes are urgent, > then I can also apply them as such. > > Xiyue Deng <manphiz@gmail.com> writes: > > [...] > >> Friendly ping. > > Friendly pong. The updated patches are attached. Thanks again! -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Show-full-authentication-URL-to-let-user-choose-how-.patch --] [-- Type: text/x-diff, Size: 2068 bytes --] From dc1930f5b4828eb285cb28ad0138e67158003c22 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:50:56 -0700 Subject: [PATCH 1/6] Show full authentication URL to let user choose how to visit it * packages/oauth2/oauth2.el (oauth2-request-authorization): show full authentication URL in user prompt. (Bug#72358) --- oauth2.el | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index 7da9702004..3a3e50ad2b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -57,14 +57,17 @@ "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. It returns the code provided by the service." - (browse-url (concat auth-url - (if (string-match-p "\?" auth-url) "&" "?") - "client_id=" (url-hexify-string client-id) - "&response_type=code" - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) ""))) - (read-string "Enter the code your browser displayed: ")) + (let ((url (concat auth-url + (if (string-match-p "\?" auth-url) "&" "?") + "client_id=" (url-hexify-string client-id) + "&response_type=code" + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) + (if scope (concat "&scope=" (url-hexify-string scope)) "") + (if state (concat "&state=" (url-hexify-string state)) "")))) + (browse-url url) + (read-string (concat "Follow the instruction on your default browser, or " + "visit:\n" url + "\nEnter the code your browser displayed: ")))) (defun oauth2-request-access-parse () "Parse the result of an OAuth request." -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Add-parameters-required-by-Google-OAuth2-to-get-refr.patch --] [-- Type: text/x-diff, Size: 1419 bytes --] From 4f6628439bdee67025719314bab5b824c86d19e9 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:52:02 -0700 Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get refresh_token * packages/oauth2/oauth2.el (oauth2-request-authorization): add "access_type=offline" and "prompt=consent" when requesting token to receive refresh_token. (Bug#72358) --- oauth2.el | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 3a3e50ad2b..8371e73ffb 100644 --- a/oauth2.el +++ b/oauth2.el @@ -63,7 +63,11 @@ It returns the code provided by the service." "&response_type=code" "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) (if scope (concat "&scope=" (url-hexify-string scope)) "") - (if state (concat "&state=" (url-hexify-string state)) "")))) + (if state (concat "&state=" (url-hexify-string state)) "") + ;; The following two parameters are required for Gmail + ;; OAuth2 to generate the refresh token + "&access_type=offline" + "&prompt=consent"))) (browse-url url) (read-string (concat "Follow the instruction on your default browser, or " "visit:\n" url -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: 0003-Encode-URL-when-requesting-access.patch --] [-- Type: text/x-diff, Size: 1524 bytes --] From 5f2877930ac69bef64b2d53e7ded4913e13c50b6 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 21 Jul 2024 14:54:08 -0700 Subject: [PATCH 3/6] Encode URL when requesting access * packages/oauth2/oauth2.el (oauth2-request-access): encode the whole URL to avoid having characters that breaks URL parsing. (Bug#72358) --- oauth2.el | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/oauth2.el b/oauth2.el index 8371e73ffb..54ca61e1cf 100644 --- a/oauth2.el +++ b/oauth2.el @@ -108,13 +108,14 @@ Return an `oauth2-token' structure." (let ((result (oauth2-make-access-request token-url - (concat - "client_id=" client-id - (when client-secret - (concat "&client_secret=" client-secret)) - "&code=" code - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) - "&grant_type=authorization_code")))) + (url-encode-url + (concat + "client_id=" client-id + (when client-secret + (concat "&client_secret=" client-secret)) + "&code=" code + "&redirect_uri=" (or redirect-uri "urn:ietf:wg:oauth:2.0:oob") + "&grant_type=authorization_code"))))) (make-oauth2-token :client-id client-id :client-secret client-secret :access-token (cdr (assoc 'access_token result)) -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #5: 0004-Support-storing-data-for-multiple-accounts-of-the-sa.patch --] [-- Type: text/x-diff, Size: 2103 bytes --] From a5241f369f5ea255fcb6f36c0b52829745d71ca8 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:00:04 -0700 Subject: [PATCH 4/6] Support storing data for multiple accounts of the same provider Currently the plstore id computed by "oauth2-compute-id" only takes "auth-url", "token-url", and "scope" into account, which could be the same for the same provider (e.g. Gmail). This prevents storing information for multiple accounts of the same service for some providers. This patch adds "client-id" to the calculation of plstore id to make sure that it is unique for different accounts of the same provider. It also changes the hash function to sha512 to be more secure. * packages/oauth2/oauth2.el (oauth2-compute-id): add "client-id" as a parameter of "oauth2-compute-id" to ensure unique id amount multiple accounts of the same provider, and change hash function to sha512. (Bug#72358) --- oauth2.el | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oauth2.el b/oauth2.el index 54ca61e1cf..c8011ebf9b 100644 --- a/oauth2.el +++ b/oauth2.el @@ -166,17 +166,17 @@ TOKEN should be obtained with `oauth2-request-access'." :group 'oauth2 :type 'file) -(defun oauth2-compute-id (auth-url token-url scope) +(defun oauth2-compute-id (auth-url token-url scope client-id) "Compute an unique id based on URLs. This allows to store the token in an unique way." - (secure-hash 'md5 (concat auth-url token-url scope))) + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) ;;;###autoload (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) "Request access to a resource and store it using `plstore'." ;; We store a MD5 sum of all URL (let* ((plstore (plstore-open oauth2-token-file)) - (id (oauth2-compute-id auth-url token-url scope)) + (id (oauth2-compute-id auth-url token-url scope client-id)) (plist (cdr (plstore-get plstore id)))) ;; Check if we found something matching this access (if plist -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #6: 0005-Add-debug-messages-and-provide-an-option-variable-fo.patch --] [-- Type: text/x-diff, Size: 2540 bytes --] From e2ece9aee7f43ec35789823576dc5390d3384dc6 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Sun, 28 Jul 2024 03:41:20 -0700 Subject: [PATCH 5/6] Add debug messages and provide an option variable for enabling This helps debugging whether the authorization and refresh requests were successful and inspecting the responses. * packages/oauth2/oauth2.el: add support for debug messages and an option variable for enabling. (Bug#72358) --- oauth2.el | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/oauth2.el b/oauth2.el index c8011ebf9b..9c91bb08a1 100644 --- a/oauth2.el +++ b/oauth2.el @@ -53,6 +53,15 @@ :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) +(defvar oauth2-debug nil + "Enable verbose logging in oauth2 to help debugging.") + +(defun oauth2--do-debug (&rest msg) + "Output debug messages when `oauth2-debug' is enabled." + (when oauth2-debug + (setcar msg (concat "[oauth2] " (car msg))) + (apply #'message msg))) + (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) "Request OAuth authorization at AUTH-URL by launching `browse-url'. CLIENT-ID is the client id provided by the provider. @@ -81,14 +90,18 @@ It returns the code provided by the service." (defun oauth2-make-access-request (url data) "Make an access request to URL using DATA in POST." - (let ((url-request-method "POST") - (url-request-data data) - (url-request-extra-headers - '(("Content-Type" . "application/x-www-form-urlencoded")))) - (with-current-buffer (url-retrieve-synchronously url) - (let ((data (oauth2-request-access-parse))) - (kill-buffer (current-buffer)) - data)))) + (let ((func-name (nth 1 (backtrace-frame 3)))) + (oauth2--do-debug "%s: url: %s" func-name url) + (oauth2--do-debug "%s: data: %s" func-name data) + (let ((url-request-method "POST") + (url-request-data data) + (url-request-extra-headers + '(("Content-Type" . "application/x-www-form-urlencoded")))) + (with-current-buffer (url-retrieve-synchronously url) + (let ((data (oauth2-request-access-parse))) + (kill-buffer (current-buffer)) + (oauth2--do-debug "%s: response: %s" func-name (prin1-to-string data)) + data))))) (cl-defstruct oauth2-token plstore -- 2.39.2 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #7: 0006-Add-NEWS-file-to-document-the-changes-to-plstore-id-.patch --] [-- Type: text/x-diff, Size: 1444 bytes --] From e99ec9b3f209312bfa0048c5ae94876e03654c18 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Tue, 30 Jul 2024 03:46:57 -0700 Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id generation * Also add local variables to enable outline-mode. (Bug#72358) --- NEWS | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 NEWS diff --git a/NEWS b/NEWS new file mode 100644 index 0000000000..72b0606659 --- /dev/null +++ b/NEWS @@ -0,0 +1,27 @@ +Summary of changes to oauth2.el +------------------------------- + +(For changes of 0.16 and older please check the git history of the +repository of oauth2.el.) + +* 0.17 + +** Changes to plstore id generation and needs to reacquire refresh_token + +The generation of plstore id used to include `auth-url', `token-url', +and `scope'. Now `client-id' is also included. This is required to +support multiple accounts of some providers which use the same +`auth-url', `token-url', and `scope' (e.g. Gmail), or the generated +plstore id is not unique amount accounts. Adding `client-id' solves +this problem. + +The hash function of calculating the plstore id has also changed from +MD5 to SHA512 to be more secure. + +As a result, users of oauth2.el will need to redo the authentication +process to get a new refresh_token when upgrading from older version +to 0.17. + +# Local variables: +# mode: outline +# End: -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-29 23:54 ` Xiyue Deng @ 2024-08-30 7:09 ` Philip Kaludercic 2024-08-30 8:32 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Philip Kaludercic @ 2024-08-30 7:09 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358-done Xiyue Deng <manphiz@gmail.com> writes: > Hi Philip, > > Thanks for the review! Please see my replies below inline. > [...] >>> From 2b9e50cb0948e0b4f28883042109994ffa295d3d Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Sun, 21 Jul 2024 14:50:56 -0700 >>> Subject: [PATCH 1/6] Show full authentication URL to let user choose how to >>> visit it >>> >>> * packages/oauth2/oauth2.el (oauth2-request-authorization): show full >>> authentication URL in user prompt. >> >> Generally it would be nice if you could attach a "(Bug#72358)" to the >> end of each commit. Just add it like a new sentence. >> >> [...] >> authentication URL in user prompt. (Bug#72358) >> > > Done for all patches. 1+ >>> --- >>> oauth2.el | 19 +++++++++++-------- >>> 1 file changed, 11 insertions(+), 8 deletions(-) >>> >>> diff --git a/oauth2.el b/oauth2.el >>> index 7da9702004..3a3e50ad2b 100644 >>> --- a/oauth2.el >>> +++ b/oauth2.el >>> @@ -57,14 +57,17 @@ >>> "Request OAuth authorization at AUTH-URL by launching `browse-url'. >>> CLIENT-ID is the client id provided by the provider. >>> It returns the code provided by the service." >>> - (browse-url (concat auth-url >>> - (if (string-match-p "\?" auth-url) "&" "?") >>> - "client_id=" (url-hexify-string client-id) >>> - "&response_type=code" >>> - "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >>> - (if scope (concat "&scope=" (url-hexify-string scope)) "") >>> - (if state (concat "&state=" (url-hexify-string state)) ""))) >>> - (read-string "Enter the code your browser displayed: ")) >>> + (let ((url (concat auth-url >>> + (if (string-match-p "\?" auth-url) "&" "?") >>> + "client_id=" (url-hexify-string client-id) >>> + "&response_type=code" >>> + "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >>> + (if scope (concat "&scope=" (url-hexify-string scope)) "") >>> + (if state (concat "&state=" (url-hexify-string state)) "")))) >>> + (browse-url url) >>> + (read-string (concat "Follow the instruction on your default browser, or " >>> + "visit:\n" url >>> + "\nEnter the code your browser displayed: ")))) >> >> LGTM. >> >>> >>> (defun oauth2-request-access-parse () >>> "Parse the result of an OAuth request." >>> -- >>> 2.39.2 >>> >>> >>> From 26ed9886bd9d3970d55cf76e4269cef3998503a7 Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Sun, 21 Jul 2024 14:52:02 -0700 >>> Subject: [PATCH 2/6] Add parameters required by Google OAuth2 to get >>> refresh_token >>> >>> * packages/oauth2/oauth2.el (oauth2-request-authorization): add >>> `access_type=offline' and `prompt=consent' when requesting token to >> >> I wouldn't use `...' syntax in commit ChangeLog messages. >> > > Ah I saw this used in the texinfo doc extensively so I thought it was > preferred. I have changed all these to double quotes, though do let me > know if there is a better preference. I think that double or single quotes are both fine. >>> receive refresh_token. >>> --- >>> oauth2.el | 4 +++- >>> 1 file changed, 3 insertions(+), 1 deletion(-) >>> >>> diff --git a/oauth2.el b/oauth2.el >>> index 3a3e50ad2b..9780ac3a1d 100644 >>> --- a/oauth2.el >>> +++ b/oauth2.el >>> @@ -63,7 +63,9 @@ It returns the code provided by the service." >>> "&response_type=code" >>> "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >>> (if scope (concat "&scope=" (url-hexify-string scope)) "") >>> - (if state (concat "&state=" (url-hexify-string state)) "")))) >>> + (if state (concat "&state=" (url-hexify-string state)) "") >>> + "&access_type=offline" >>> + "&prompt=consent"))) >> >> You do what you say, but perhaps a comment explaining why you have these >> queries might be useful? >> > > Added a comment above to explain the reasons. 1+ >>> (browse-url url) >>> (read-string (concat "Follow the instruction on your default browser, or " >>> "visit:\n" url >>> -- >>> 2.39.2 >>> >>> >>> From 59225412e1d06ae9e165cfde6a4a985cee4fc569 Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Sun, 21 Jul 2024 14:54:08 -0700 >>> Subject: [PATCH 3/6] Encode parameters when requesting access >>> >>> * packages/oauth2/oauth2.el (oauth2-request-access): encode all >>> parameters which may contain characters that breaks URL. >>> --- >>> oauth2.el | 6 +++--- >>> 1 file changed, 3 insertions(+), 3 deletions(-) >>> >>> diff --git a/oauth2.el b/oauth2.el >>> index 9780ac3a1d..b035742fc1 100644 >>> --- a/oauth2.el >>> +++ b/oauth2.el >>> @@ -107,10 +107,10 @@ Return an `oauth2-token' structure." >>> (oauth2-make-access-request >>> token-url >>> (concat >>> - "client_id=" client-id >>> + "client_id=" (url-hexify-string client-id) >>> (when client-secret >>> - (concat "&client_secret=" client-secret)) >>> - "&code=" code >>> + (concat "&client_secret=" (url-hexify-string client-secret))) >>> + "&code=" (url-hexify-string code) >>> "&redirect_uri=" (url-hexify-string (or redirect-uri "urn:ietf:wg:oauth:2.0:oob")) >>> "&grant_type=authorization_code")))) >>> (make-oauth2-token :client-id client-id >>> -- >>> 2.39.2 >> >> 1+ Though one might also consider using something like `url-encode-url' >> to generate the entire URL. >> > > It is indeed much cleaner. Changed accordingly. I am glad to hear that this was possible. >>> >>> >>> From e801af578e63c7e333e668bdfef05e4cf0802582 Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Sun, 28 Jul 2024 03:00:04 -0700 >>> Subject: [PATCH 4/6] Support storing data for multiple accounts of the same >>> provider >>> >>> Currently the plstore id computed by `oauth2-compute-id' only takes >>> `auth-url', `token-url', and `scope' into account, which could be the >>> same for the same provider (e.g. Gmail). This prevents storing >>> information for multiple accounts of the same service for some >>> providers. >>> >>> This patch adds `client-id' to the calculation of plstore id to make >>> sure that it is unique for different accounts of the same provider. >>> >>> It also changes the hash function to sha512 to be more secure. >>> >>> * packages/oauth2/oauth2.el (oauth2-compute-id): add `client-id' as a >>> parameter of `oauth2-compute-id' to ensure unique id amount multiple >>> accounts of the same provider, and change hash function to sha512. >>> --- >>> oauth2.el | 6 +++--- >>> 1 file changed, 3 insertions(+), 3 deletions(-) >>> >>> diff --git a/oauth2.el b/oauth2.el >>> index b035742fc1..035971ac85 100644 >>> --- a/oauth2.el >>> +++ b/oauth2.el >>> @@ -163,17 +163,17 @@ TOKEN should be obtained with `oauth2-request-access'." >>> :group 'oauth2 >>> :type 'file) >>> >>> -(defun oauth2-compute-id (auth-url token-url scope) >>> +(defun oauth2-compute-id (auth-url token-url scope client-id) >>> "Compute an unique id based on URLs. >>> This allows to store the token in an unique way." >>> - (secure-hash 'md5 (concat auth-url token-url scope))) >>> + (secure-hash 'sha512 (concat auth-url token-url scope client-id))) >>> >>> ;;;###autoload >>> (defun oauth2-auth-and-store (auth-url token-url scope client-id client-secret &optional redirect-uri state) >>> "Request access to a resource and store it using `plstore'." >>> ;; We store a MD5 sum of all URL >>> (let* ((plstore (plstore-open oauth2-token-file)) >>> - (id (oauth2-compute-id auth-url token-url scope)) >>> + (id (oauth2-compute-id auth-url token-url scope client-id)) >>> (plist (cdr (plstore-get plstore id)))) >>> ;; Check if we found something matching this access >>> (if plist >>> -- >>> 2.39.2 >> >> LGTM >> >>> >>> From 55417ec61c91f6b4d8e16a0c9933fb178d7bb657 Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Sun, 28 Jul 2024 03:41:20 -0700 >>> Subject: [PATCH 5/6] Add debug messages and provide a switch variable for >>> enabling >>> >>> This helps debugging whether the authorization and refresh requests >>> were successful and inspecting the responses. >>> >>> * packages/oauth2/oauth2.el: add support for debug messages and a >>> switch variable for enabling. >>> --- >>> oauth2.el | 13 +++++++++++++ >>> 1 file changed, 13 insertions(+) >>> >>> diff --git a/oauth2.el b/oauth2.el >>> index 035971ac85..ce7a835100 100644 >>> --- a/oauth2.el >>> +++ b/oauth2.el >>> @@ -40,6 +40,7 @@ >>> (require 'plstore) >>> (require 'json) >>> (require 'url-http) >>> +(require 'pp) >>> >>> (defvar url-http-data) >>> (defvar url-http-method) >>> @@ -53,6 +54,14 @@ >>> :link '(url-link :tag "Savannah" "https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/oauth2") >>> :link '(url-link :tag "ELPA" "https://elpa.gnu.org/packages/oauth2.html")) >>> >>> +(defvar oauth2-debug nil >>> + "Enable debug messages.") >> >> I'd slightly expand this comment in case someone wants to use something >> like `apropos-documentation'. > > Expanded the doc string to explain the purpose. > >> Also, why is this not a user option? >> > > As Robert mentioned in the follow-up mail it was his suggestion (thanks > Robert!) Do let me know if `defcustom' is preferred. No, some packages use user options, others regular variables so I guess it is up to the person with the strongest opinion on the topic to decide (which isn't me in this case). >>> + >>> +(defun oauth2--do-debug (&rest msg) >>> + "Output debug messages when `oauth2-debug' is enabled." >>> + (if oauth2-debug >> >> I'd use `when' here. >> > > Done > >>> + (apply #'message msg))) >> >> ... and perhaps prefix the message with a string like "[oauth2] ". >> > > Done. I used `setcar' but let me know if there is a more elegant way to > do this. No complains from my side, I find it rather elegant! >> I also notice that you manually mention the function issuing the debug >> message below. It is possible to infer this using `backtrace-frame' >> (but that is really something that should be added to the core, instead >> of packages having to re-implement it themselves.) >> > > Added a let-bounded `func-name' by getting function name from > backtrace-frame. And it would be great if there is a built-in macro > like `__PRETTY_FUNCTION__' and `__LINE__'. I know of nothing. >>> + >>> (defun oauth2-request-authorization (auth-url client-id &optional scope state redirect-uri) >>> "Request OAuth authorization at AUTH-URL by launching `browse-url'. >>> CLIENT-ID is the client id provided by the provider. >>> @@ -79,6 +88,8 @@ It returns the code provided by the service." >>> >>> (defun oauth2-make-access-request (url data) >>> "Make an access request to URL using DATA in POST." >>> + (oauth2--do-debug "oauth2-make-access-request: url: %s" url) >>> + (oauth2--do-debug "oauth2-make-access-request: data: %s" data) >>> (let ((url-request-method "POST") >>> (url-request-data data) >>> (url-request-extra-headers >>> @@ -86,6 +97,8 @@ It returns the code provided by the service." >>> (with-current-buffer (url-retrieve-synchronously url) >>> (let ((data (oauth2-request-access-parse))) >>> (kill-buffer (current-buffer)) >>> + (oauth2--do-debug "oauth2-make-access-request: response: %s" >>> + (pp-to-string data)) >> >> Is pp-to-string here really much better than prin1-to-string? Note that >> 'oauth2--do-debug' is a function, so the arguments are always evaluated. >> I have experienced pp being significantly slower than the core print >> functions, so this is something that is worth thinking about IMO. >> > > It's just a personal preference. And it look like the values are mostly > key-value pairs so they are not that hard to read through > `prin1-to-string' so I changed to it. 1+ >>> data)))) >>> >>> (cl-defstruct oauth2-token >>> -- >>> 2.39.2 >>> >>> >>> From e8735da21ac82b0698edad1796ddf4a1b8eb4bb2 Mon Sep 17 00:00:00 2001 >>> From: Xiyue Deng <manphiz@gmail.com> >>> Date: Tue, 30 Jul 2024 03:46:57 -0700 >>> Subject: [PATCH 6/6] Add NEWS file to document the changes to plstore id >>> generation >>> >>> --- >>> NEWS | 23 +++++++++++++++++++++++ >>> 1 file changed, 23 insertions(+) >>> create mode 100644 NEWS >>> >>> diff --git a/NEWS b/NEWS >>> new file mode 100644 >>> index 0000000000..6715a1914a >>> --- /dev/null >>> +++ b/NEWS >>> @@ -0,0 +1,23 @@ >>> +Summary of changes to oauth2.el >>> +------------------------------- >>> + >>> +For changes of 0.16 and older or full changes please check the git >>> +history of the repository of oauth2.el. >>> + >>> +* 0.17 >>> + >>> +** Changes to plstore id generation and needs to reacquire refresh_token >>> + >>> +The generation of plstore id used to include `auth-url', `token-url', >>> +and `scope'. Now `client-id' is also included. This is required to >>> +support multiple accounts of some providers which use the same >>> +`auth-url', `token-url', and `scope' (e.g. Gmail), and hence the >>> +generated plstore id is not unique amount accounts. Adding >>> +`client-id' solves this problem. >>> + >>> +The hash function of calculating the plstore id has also changed from >>> +MD5 to SHA512 to be more secure. >>> + >>> +As a result, users of oauth2.el will need to redo the authentication >>> +process to get a new refresh_token when upgrading from older version >>> +to 0.17. >> >> Perhaps you can add a file local variable to enable outline-mode? >> > > Done. Also improved the wording. > >> All in all the patches look more than fine though, nothing I mentioned >> is pressing. If you don't have the time and the changes are urgent, >> then I can also apply them as such. >> >> Xiyue Deng <manphiz@gmail.com> writes: >> >> [...] >> >>> Friendly ping. >> >> Friendly pong. > > The updated patches are attached. Thanks again! OK, this looks very good. I'll apply the changes and am closing the report. Thanks, -- Philip Kaludercic on peregrine ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-30 7:09 ` Philip Kaludercic @ 2024-08-30 8:32 ` Xiyue Deng 2024-08-30 10:07 ` Philip Kaludercic 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-30 8:32 UTC (permalink / raw) To: Philip Kaludercic; +Cc: 72358-done [-- Attachment #1: Type: text/plain, Size: 456 bytes --] Philip Kaludercic <philipk@posteo.net> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> [..snip..] >> >> The updated patches are attached. Thanks again! > > OK, this looks very good. I'll apply the changes and am closing the > report. > > Thanks, Thanks a lot Philip! One last thing: I'd like to ask for a new release (0.17) with these changes if that's OK (patch attached.) Or let me know if I should create a new bug for that. -- Xiyue Deng [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-packages-oauth2-oauth2.el-update-version-to-0.17.patch --] [-- Type: text/x-diff, Size: 636 bytes --] From e05c2bb372c712b809e31fe317ca993e63384207 Mon Sep 17 00:00:00 2001 From: Xiyue Deng <manphiz@gmail.com> Date: Fri, 30 Aug 2024 01:28:09 -0700 Subject: [PATCH] * packages/oauth2/oauth2.el: update version to 0.17 --- oauth2.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/oauth2.el b/oauth2.el index 9c91bb08a1..87e0c39c5c 100644 --- a/oauth2.el +++ b/oauth2.el @@ -3,7 +3,7 @@ ;; Copyright (C) 2011-2021 Free Software Foundation, Inc ;; Author: Julien Danjou <julien@danjou.info> -;; Version: 0.16 +;; Version: 0.17 ;; Keywords: comm ;; Package-Requires: ((cl-lib "0.5") (nadvice "0.3")) -- 2.39.2 ^ permalink raw reply related [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-30 8:32 ` Xiyue Deng @ 2024-08-30 10:07 ` Philip Kaludercic 2024-08-30 21:13 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Philip Kaludercic @ 2024-08-30 10:07 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358-done Xiyue Deng <manphiz@gmail.com> writes: > Philip Kaludercic <philipk@posteo.net> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> [..snip..] >>> >>> The updated patches are attached. Thanks again! >> >> OK, this looks very good. I'll apply the changes and am closing the >> report. >> >> Thanks, > > Thanks a lot Philip! One last thing: I'd like to ask for a new release > (0.17) with these changes if that's OK (patch attached.) Of course, I forgot to ask if this should trigger a new release or not, that's my bad. > Or let me know > if I should create a new bug for that. Nah, not for something as simple as this. I'll apply it straight away. -- Philip Kaludercic on peregrine ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-30 10:07 ` Philip Kaludercic @ 2024-08-30 21:13 ` Xiyue Deng 2024-09-03 18:08 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-30 21:13 UTC (permalink / raw) To: Philip Kaludercic; +Cc: 72358-done Philip Kaludercic <philipk@posteo.net> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Philip Kaludercic <philipk@posteo.net> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>>> [..snip..] >>>> >>>> The updated patches are attached. Thanks again! >>> >>> OK, this looks very good. I'll apply the changes and am closing the >>> report. >>> >>> Thanks, >> >> Thanks a lot Philip! One last thing: I'd like to ask for a new release >> (0.17) with these changes if that's OK (patch attached.) > > Of course, I forgot to ask if this should trigger a new release or not, > that's my bad. > >> Or let me know >> if I should create a new bug for that. > > Nah, not for something as simple as this. I'll apply it straight away. Thanks very much, Philip! -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-30 21:13 ` Xiyue Deng @ 2024-09-03 18:08 ` Xiyue Deng 0 siblings, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-09-03 18:08 UTC (permalink / raw) To: 72358-done In case anyone is interested, I have filed bug#72992[1] where I posted my code to enable xoauth2 support for nnimap and smtpmail as I promised. I am also trying to gather feedback on how it can be improved. Comments welcome! [1] https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-09/msg00089.html -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
[parent not found: <66a8f323.170a0220.9172c.8e28SMTPIN_ADDED_BROKEN@mx.google.com>]
* bug#72358: 29.4; oauth2.el improvements [not found] ` <66a8f323.170a0220.9172c.8e28SMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-07-30 19:40 ` Xiyue Deng 2024-07-30 21:50 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-07 23:22 ` Xiyue Deng 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-07-30 19:40 UTC (permalink / raw) To: Björn Bidar; +Cc: Robert Pluim, 72358 Björn Bidar <bjorn.bidar@thaodan.de> writes: > Robert Pluim <rpluim@gmail.com> writes: > >> Xiyue> - This will invalidate all existing entries and a user will have to redo >> Xiyue> the authorization process again to get a new refresh token. However, >> Xiyue> I think it's more important to ensure that oauth2.el works correctly >> Xiyue> for multiple accounts of the same provider, or a user may suffer from >> Xiyue> confusion when adding a new account invalidates a previous account. >> >> I donʼt think thatʼs too big a concern. 'modern' authentication flows >> regularly re-prompt, so this will not be too surprising (although >> maybe call it out in the packageʼs NEWS or README). > > In many cases the refreshing of tokens is transparent to the user there > doesn't have to be a re-prompt to refresh the token if the OAuth > provider support it. > Micrsofts OAuth workflow is quite good in this regard as there's a > non-standard error to indicate when the user has to re-authorize the > application. > Actually I am currently having trouble for a few weeks to get my outlook.com email work with MS OAuth2. To avoid some repeated typing, I have documented the issues and steps I have tried in this stackoverflow question[1]. I would great appreciated it if you can shed some lights there > I assume all implementation of OAuth have their quirks. Indeed. [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 19:40 ` Xiyue Deng @ 2024-07-30 21:50 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-07 23:22 ` Xiyue Deng 1 sibling, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 21:50 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Björn Bidar <bjorn.bidar@thaodan.de> writes: > >> Robert Pluim <rpluim@gmail.com> writes: >> >>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>> Xiyue> the authorization process again to get a new refresh token. However, >>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>> Xiyue> confusion when adding a new account invalidates a previous account. >>> >>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>> regularly re-prompt, so this will not be too surprising (although >>> maybe call it out in the packageʼs NEWS or README). >> >> In many cases the refreshing of tokens is transparent to the user there >> doesn't have to be a re-prompt to refresh the token if the OAuth >> provider support it. >> Micrsofts OAuth workflow is quite good in this regard as there's a >> non-standard error to indicate when the user has to re-authorize the >> application. >> > > Actually I am currently having trouble for a few weeks to get my > outlook.com email work with MS OAuth2. To avoid some repeated typing, I > have documented the issues and steps I have tried in this stackoverflow > question[1]. I would great appreciated it if you can shed some lights > there > I remember when adding OAuth support to Sailfish OS we needed to patch our signon to work with the non-standard Microsoft flow. We have this patch on top of the OAuth2 plugin for signond to accept that they send the authentication as a request for you to fetch upon you have to another request with the new url to get authentication code. The patch can be found here: https://github.com/sailfishos/signon-plugin-oauth2/blob/master/rpm/0005-Support-Microsoft-OAuth2-flow.patch I'm not the person who wrote the patch but I fixed the plugin later for Dropbox so that PKCE (RFC7636) isn't used unless the server requests it (response type must be code to request PKCE). PKCE is strongly recommended. The patch above already works kinda that way without requesting PKCE. Read here for further information: https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-auth-code-flow https://datatracker.ietf.org/doc/html/rfc7636 ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 19:40 ` Xiyue Deng 2024-07-30 21:50 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-07 23:22 ` Xiyue Deng 2024-08-08 6:11 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors ` (3 more replies) 1 sibling, 4 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-07 23:22 UTC (permalink / raw) To: Björn Bidar; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Björn Bidar <bjorn.bidar@thaodan.de> writes: > >> Robert Pluim <rpluim@gmail.com> writes: >> >>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>> Xiyue> the authorization process again to get a new refresh token. However, >>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>> Xiyue> confusion when adding a new account invalidates a previous account. >>> >>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>> regularly re-prompt, so this will not be too surprising (although >>> maybe call it out in the packageʼs NEWS or README). >> >> In many cases the refreshing of tokens is transparent to the user there >> doesn't have to be a re-prompt to refresh the token if the OAuth >> provider support it. >> Micrsofts OAuth workflow is quite good in this regard as there's a >> non-standard error to indicate when the user has to re-authorize the >> application. >> > > Actually I am currently having trouble for a few weeks to get my > outlook.com email work with MS OAuth2. To avoid some repeated typing, I > have documented the issues and steps I have tried in this stackoverflow > question[1]. I would great appreciated it if you can shed some lights > there > >> I assume all implementation of OAuth have their quirks. > > Indeed. > > > [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo Just want to report back that after confirming with an MS representative through online chat, outlook.com has actually disabled refreshing access_token through the token endpoint, and users are asked to migrate to Outlook app or compatibles apps (Thunderbird still works). I'm not sure whether this is also the case for organization emails, which may also be disabled by default (or soonish if not already) but can be enabled separately by an org admin. Anyway, I'd suggest people stop wasting your time here and use Gmail (or maybe Yahoo mail) which has decent 3rd party OAuth2 support. Meanwhile I have submitted a request to re-enable this support[1]. [1] https://feedbackportal.microsoft.com/feedback/idea/069f1816-0a55-ef11-b4ad-0022484d3ecc -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-07 23:22 ` Xiyue Deng @ 2024-08-08 6:11 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-08 6:14 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors ` (2 subsequent siblings) 3 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-08 6:11 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Björn Bidar <bjorn.bidar@thaodan.de> writes: >> >>> Robert Pluim <rpluim@gmail.com> writes: >>> >>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>> Xiyue> the authorization process again to get a new refresh token. However, >>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>> >>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>> regularly re-prompt, so this will not be too surprising (although >>>> maybe call it out in the packageʼs NEWS or README). >>> >>> In many cases the refreshing of tokens is transparent to the user there >>> doesn't have to be a re-prompt to refresh the token if the OAuth >>> provider support it. >>> Micrsofts OAuth workflow is quite good in this regard as there's a >>> non-standard error to indicate when the user has to re-authorize the >>> application. >>> >> >> Actually I am currently having trouble for a few weeks to get my >> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >> have documented the issues and steps I have tried in this stackoverflow >> question[1]. I would great appreciated it if you can shed some lights >> there >> >>> I assume all implementation of OAuth have their quirks. >> >> Indeed. >> >> >> [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo > > Just want to report back that after confirming with an MS representative > through online chat, outlook.com has actually disabled refreshing > access_token through the token endpoint, and users are asked to migrate > to Outlook app or compatibles apps (Thunderbird still works). Thank you for notifying me on this I will forward this to my employer. > I'm not sure whether this is also the case for organization emails, which may > also be disabled by default (or soonish if not already) but can be > enabled separately by an org admin. It does depend some domains use whitelist e.g. Tampere University of Applies sciences. Without a specific Emacs GNUs/Caldav/whatever AppID inside Microsoft OAuth2 it will be hard to pass that. > Anyway, I'd suggest people stop > wasting your time here and use Gmail (or maybe Yahoo mail) which has > decent 3rd party OAuth2 support. I don't think that's an option for most user that complain about working OAuth2 support, in most cases it's a work or some other organization account. Another thing I think is very important is to support Nextcloud as it's a FOSS app supporting OAuth2 which quite many users and organizations adopted. > Meanwhile I have submitted a request to re-enable this support[1]. > > [1] https://feedbackportal.microsoft.com/feedback/idea/069f1816-0a55-ef11-b4ad-0022484d3ecc ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-07 23:22 ` Xiyue Deng 2024-08-08 6:11 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-08 6:14 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66b46180.170a0220.1fb02.1d6eSMTPIN_ADDED_BROKEN@mx.google.com> [not found] ` <66b46251.170a0220.f2be9.afeeSMTPIN_ADDED_BROKEN@mx.google.com> 3 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-08 6:14 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Björn Bidar <bjorn.bidar@thaodan.de> writes: >> >>> Robert Pluim <rpluim@gmail.com> writes: >>> >>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>> Xiyue> the authorization process again to get a new refresh token. However, >>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>> >>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>> regularly re-prompt, so this will not be too surprising (although >>>> maybe call it out in the packageʼs NEWS or README). >>> >>> In many cases the refreshing of tokens is transparent to the user there >>> doesn't have to be a re-prompt to refresh the token if the OAuth >>> provider support it. >>> Micrsofts OAuth workflow is quite good in this regard as there's a >>> non-standard error to indicate when the user has to re-authorize the >>> application. >>> >> >> Actually I am currently having trouble for a few weeks to get my >> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >> have documented the issues and steps I have tried in this stackoverflow >> question[1]. I would great appreciated it if you can shed some lights >> there >> >>> I assume all implementation of OAuth have their quirks. >> >> Indeed. >> >> >> [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo > > Just want to report back that after confirming with an MS representative > through online chat, outlook.com has actually disabled refreshing > access_token through the token endpoint, and users are asked to migrate > to Outlook app or compatibles apps (Thunderbird still works). I'm not > sure whether this is also the case for organization emails, which may > also be disabled by default (or soonish if not already) but can be > enabled separately by an org admin. Anyway, I'd suggest people stop > wasting your time here and use Gmail (or maybe Yahoo mail) which has > decent 3rd party OAuth2 support. Can you link a source about that? ^ permalink raw reply [flat|nested] 49+ messages in thread
[parent not found: <66b46180.170a0220.1fb02.1d6eSMTPIN_ADDED_BROKEN@mx.google.com>]
* bug#72358: 29.4; oauth2.el improvements [not found] ` <66b46180.170a0220.1fb02.1d6eSMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-08-08 8:28 ` Xiyue Deng 2024-08-08 9:17 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-12 13:22 ` Thomas Fitzsimmons 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-08-08 8:28 UTC (permalink / raw) To: Björn Bidar; +Cc: Robert Pluim, 72358 Björn Bidar <bjorn.bidar@thaodan.de> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Björn Bidar <bjorn.bidar@thaodan.de> writes: >>> >>>> Robert Pluim <rpluim@gmail.com> writes: >>>> >>>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>>> Xiyue> the authorization process again to get a new refresh token. However, >>>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>>> >>>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>>> regularly re-prompt, so this will not be too surprising (although >>>>> maybe call it out in the packageʼs NEWS or README). >>>> >>>> In many cases the refreshing of tokens is transparent to the user there >>>> doesn't have to be a re-prompt to refresh the token if the OAuth >>>> provider support it. >>>> Micrsofts OAuth workflow is quite good in this regard as there's a >>>> non-standard error to indicate when the user has to re-authorize the >>>> application. >>>> >>> >>> Actually I am currently having trouble for a few weeks to get my >>> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >>> have documented the issues and steps I have tried in this stackoverflow >>> question[1]. I would great appreciated it if you can shed some lights >>> there >>> >>>> I assume all implementation of OAuth have their quirks. >>> >>> Indeed. >>> >>> >>> [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo >> >> Just want to report back that after confirming with an MS representative >> through online chat, outlook.com has actually disabled refreshing >> access_token through the token endpoint, and users are asked to migrate >> to Outlook app or compatibles apps (Thunderbird still works). > > Thank you for notifying me on this I will forward this to my employer. > >> I'm not sure whether this is also the case for organization emails, which may >> also be disabled by default (or soonish if not already) but can be >> enabled separately by an org admin. > > It does depend some domains use whitelist e.g. Tampere University of > Applies sciences. Without a specific Emacs GNUs/Caldav/whatever AppID > inside Microsoft OAuth2 it will be hard to pass that. > > >> Anyway, I'd suggest people stop >> wasting your time here and use Gmail (or maybe Yahoo mail) which has >> decent 3rd party OAuth2 support. > > I don't think that's an option for most user that complain about working > OAuth2 support, in most cases it's a work or some other organization > account. > > Another thing I think is very important is to support Nextcloud as it's > a FOSS app supporting OAuth2 which quite many users and organizations > adopted. > > Nextcloud sounds interesting. Do you know where I can check for the OAuth2 credentials like client_id and client_secret? >> Meanwhile I have submitted a request to re-enable this support[1]. >> >> [1] https://feedbackportal.microsoft.com/feedback/idea/069f1816-0a55-ef11-b4ad-0022484d3ecc -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-08 8:28 ` Xiyue Deng @ 2024-08-08 9:17 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-12 13:22 ` Thomas Fitzsimmons 1 sibling, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-08 9:17 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Björn Bidar <bjorn.bidar@thaodan.de> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>> Anyway, I'd suggest people stop >>> wasting your time here and use Gmail (or maybe Yahoo mail) which has >>> decent 3rd party OAuth2 support. >> >> I don't think that's an option for most user that complain about working >> OAuth2 support, in most cases it's a work or some other organization >> account. >> >> Another thing I think is very important is to support Nextcloud as it's >> a FOSS app supporting OAuth2 which quite many users and organizations >> adopted. >> >> > > Nextcloud sounds interesting. Do you know where I can check for the > OAuth2 credentials like client_id and client_secret? > You can find them in the administrator settings the Nextcloud server. Here is there documentation: https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/oauth2.html ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-08 8:28 ` Xiyue Deng 2024-08-08 9:17 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-12 13:22 ` Thomas Fitzsimmons 2024-08-12 16:26 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 1 sibling, 1 reply; 49+ messages in thread From: Thomas Fitzsimmons @ 2024-08-12 13:22 UTC (permalink / raw) To: Xiyue Deng; +Cc: Björn Bidar, Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Björn Bidar <bjorn.bidar@thaodan.de> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>>> Björn Bidar <bjorn.bidar@thaodan.de> writes: >>>> >>>>> Robert Pluim <rpluim@gmail.com> writes: >>>>> >>>>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>>>> Xiyue> the authorization process again to get a new refresh token. However, >>>>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>>>> >>>>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>>>> regularly re-prompt, so this will not be too surprising (although >>>>>> maybe call it out in the packageʼs NEWS or README). >>>>> >>>>> In many cases the refreshing of tokens is transparent to the user there >>>>> doesn't have to be a re-prompt to refresh the token if the OAuth >>>>> provider support it. >>>>> Micrsofts OAuth workflow is quite good in this regard as there's a >>>>> non-standard error to indicate when the user has to re-authorize the >>>>> application. >>>>> >>>> >>>> Actually I am currently having trouble for a few weeks to get my >>>> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >>>> have documented the issues and steps I have tried in this stackoverflow >>>> question[1]. I would great appreciated it if you can shed some lights >>>> there >>>> >>>>> I assume all implementation of OAuth have their quirks. >>>> >>>> Indeed. >>>> >>>> >>>> [1] >>>> https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo >>> >>> Just want to report back that after confirming with an MS representative >>> through online chat, outlook.com has actually disabled refreshing >>> access_token through the token endpoint, and users are asked to migrate >>> to Outlook app or compatibles apps (Thunderbird still works). >> >> Thank you for notifying me on this I will forward this to my employer. >> >>> I'm not sure whether this is also the case for organization emails, which may >>> also be disabled by default (or soonish if not already) but can be >>> enabled separately by an org admin. >> >> It does depend some domains use whitelist e.g. Tampere University of >> Applies sciences. Without a specific Emacs GNUs/Caldav/whatever AppID >> inside Microsoft OAuth2 it will be hard to pass that. >> >> >>> Anyway, I'd suggest people stop >>> wasting your time here and use Gmail (or maybe Yahoo mail) which has >>> decent 3rd party OAuth2 support. >> >> I don't think that's an option for most user that complain about working >> OAuth2 support, in most cases it's a work or some other organization >> account. >> >> Another thing I think is very important is to support Nextcloud as it's >> a FOSS app supporting OAuth2 which quite many users and organizations >> adopted. >> >> > > Nextcloud sounds interesting. Do you know where I can check for the > OAuth2 credentials like client_id and client_secret? sourcehut [1] provides a Free Software OAuth2 flow, and it has the benefit of not requiring JavaScript (even FOSS JavaScript) anywhere in the process. I wrote url-http-oauth-demo.el [2] as a complete "worked" example demonstrating its use with url-http-oauth.el. Thomas 1. https://sourcehut.org/ 2. https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/url-http-oauth-demo.el?h=externals/url-http-oauth ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-12 13:22 ` Thomas Fitzsimmons @ 2024-08-12 16:26 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-12 16:26 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Robert Pluim, 72358, Xiyue Deng Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Björn Bidar <bjorn.bidar@thaodan.de> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>>> Xiyue Deng <manphiz@gmail.com> writes: >>>> >>>>> Björn Bidar <bjorn.bidar@thaodan.de> writes: >>>>> >>>>>> Robert Pluim <rpluim@gmail.com> writes: >>>>>> >>>>>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>>>>> Xiyue> the authorization process again to get a new refresh token. However, >>>>>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>>>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>>>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>>>>> >>>>>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>>>>> regularly re-prompt, so this will not be too surprising (although >>>>>>> maybe call it out in the packageʼs NEWS or README). >>>>>> >>>>>> In many cases the refreshing of tokens is transparent to the user there >>>>>> doesn't have to be a re-prompt to refresh the token if the OAuth >>>>>> provider support it. >>>>>> Micrsofts OAuth workflow is quite good in this regard as there's a >>>>>> non-standard error to indicate when the user has to re-authorize the >>>>>> application. >>>>>> >>>>> >>>>> Actually I am currently having trouble for a few weeks to get my >>>>> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >>>>> have documented the issues and steps I have tried in this stackoverflow >>>>> question[1]. I would great appreciated it if you can shed some lights >>>>> there >>>>> >>>>>> I assume all implementation of OAuth have their quirks. >>>>> >>>>> Indeed. >>>>> >>>>> >>>>> [1] >>>>> https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo >>>> >>>> Just want to report back that after confirming with an MS representative >>>> through online chat, outlook.com has actually disabled refreshing >>>> access_token through the token endpoint, and users are asked to migrate >>>> to Outlook app or compatibles apps (Thunderbird still works). >>> >>> Thank you for notifying me on this I will forward this to my employer. >>> >>>> I'm not sure whether this is also the case for organization emails, which may >>>> also be disabled by default (or soonish if not already) but can be >>>> enabled separately by an org admin. >>> >>> It does depend some domains use whitelist e.g. Tampere University of >>> Applies sciences. Without a specific Emacs GNUs/Caldav/whatever AppID >>> inside Microsoft OAuth2 it will be hard to pass that. >>> >>> >>>> Anyway, I'd suggest people stop >>>> wasting your time here and use Gmail (or maybe Yahoo mail) which has >>>> decent 3rd party OAuth2 support. >>> >>> I don't think that's an option for most user that complain about working >>> OAuth2 support, in most cases it's a work or some other organization >>> account. >>> >>> Another thing I think is very important is to support Nextcloud as it's >>> a FOSS app supporting OAuth2 which quite many users and organizations >>> adopted. >>> >>> >> >> Nextcloud sounds interesting. Do you know where I can check for the >> OAuth2 credentials like client_id and client_secret? > > sourcehut [1] provides a Free Software OAuth2 flow, and it has the > benefit of not requiring JavaScript (even FOSS JavaScript) anywhere in > the process. I wrote url-http-oauth-demo.el [2] as a complete "worked" > example demonstrating its use with url-http-oauth.el. Would that provide OAuth2 for providers that require a login through their webinterface, such as Nextcloud Login, without a browser? Most platforms such as Android, KDE or Sailfish OS use a browser for OAuth2 login to login, authorize and then forward the token to the OS/app. > Thomas > > 1. https://sourcehut.org/ > 2. https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/url-http-oauth-demo.el?h=externals/url-http-oauth ^ permalink raw reply [flat|nested] 49+ messages in thread
[parent not found: <66b46251.170a0220.f2be9.afeeSMTPIN_ADDED_BROKEN@mx.google.com>]
* bug#72358: 29.4; oauth2.el improvements [not found] ` <66b46251.170a0220.f2be9.afeeSMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-08-08 8:29 ` Xiyue Deng 2024-08-08 9:31 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-08 8:29 UTC (permalink / raw) To: Björn Bidar; +Cc: Robert Pluim, 72358 Björn Bidar <bjorn.bidar@thaodan.de> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Björn Bidar <bjorn.bidar@thaodan.de> writes: >>> >>>> Robert Pluim <rpluim@gmail.com> writes: >>>> >>>>> Xiyue> - This will invalidate all existing entries and a user will have to redo >>>>> Xiyue> the authorization process again to get a new refresh token. However, >>>>> Xiyue> I think it's more important to ensure that oauth2.el works correctly >>>>> Xiyue> for multiple accounts of the same provider, or a user may suffer from >>>>> Xiyue> confusion when adding a new account invalidates a previous account. >>>>> >>>>> I donʼt think thatʼs too big a concern. 'modern' authentication flows >>>>> regularly re-prompt, so this will not be too surprising (although >>>>> maybe call it out in the packageʼs NEWS or README). >>>> >>>> In many cases the refreshing of tokens is transparent to the user there >>>> doesn't have to be a re-prompt to refresh the token if the OAuth >>>> provider support it. >>>> Micrsofts OAuth workflow is quite good in this regard as there's a >>>> non-standard error to indicate when the user has to re-authorize the >>>> application. >>>> >>> >>> Actually I am currently having trouble for a few weeks to get my >>> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >>> have documented the issues and steps I have tried in this stackoverflow >>> question[1]. I would great appreciated it if you can shed some lights >>> there >>> >>>> I assume all implementation of OAuth have their quirks. >>> >>> Indeed. >>> >>> >>> [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo >> >> Just want to report back that after confirming with an MS representative >> through online chat, outlook.com has actually disabled refreshing >> access_token through the token endpoint, and users are asked to migrate >> to Outlook app or compatibles apps (Thunderbird still works). I'm not >> sure whether this is also the case for organization emails, which may >> also be disabled by default (or soonish if not already) but can be >> enabled separately by an org admin. Anyway, I'd suggest people stop >> wasting your time here and use Gmail (or maybe Yahoo mail) which has >> decent 3rd party OAuth2 support. > > Can you link a source about that? Unfortunately the MS representative didn't provide any link for this. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-08 8:29 ` Xiyue Deng @ 2024-08-08 9:31 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-08 9:31 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: >>>>> In many cases the refreshing of tokens is transparent to the user there >>>>> doesn't have to be a re-prompt to refresh the token if the OAuth >>>>> provider support it. >>>>> Micrsofts OAuth workflow is quite good in this regard as there's a >>>>> non-standard error to indicate when the user has to re-authorize the >>>>> application. >>>>> >>>> >>>> Actually I am currently having trouble for a few weeks to get my >>>> outlook.com email work with MS OAuth2. To avoid some repeated typing, I >>>> have documented the issues and steps I have tried in this stackoverflow >>>> question[1]. I would great appreciated it if you can shed some lights >>>> there >>>> >>>>> I assume all implementation of OAuth have their quirks. >>>> >>>> Indeed. >>>> >>>> >>>> [1] https://stackoverflow.com/questions/78787763/getting-aadsts65001-error-invalid-grant-when-trying-to-refresh-access-token-fo >>> >>> Just want to report back that after confirming with an MS representative >>> through online chat, outlook.com has actually disabled refreshing >>> access_token through the token endpoint, and users are asked to migrate >>> to Outlook app or compatibles apps (Thunderbird still works). I'm not >>> sure whether this is also the case for organization emails, which may >>> also be disabled by default (or soonish if not already) but can be >>> enabled separately by an org admin. Anyway, I'd suggest people stop >>> wasting your time here and use Gmail (or maybe Yahoo mail) which has >>> decent 3rd party OAuth2 support. >> >> Can you link a source about that? > > Unfortunately the MS representative didn't provide any link for this. OK that is to bad, if possible I would ask for an official response. Microsoft counts as a gate keeper, at least officially in the EU, they should not be able to act like this. A workaround would be to request a manual token refresh by authorizing the application all over again. I will update you on anything new I have if I get new information that I can post. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-29 21:25 bug#72358: 29.4; oauth2.el improvements Xiyue Deng 2024-07-30 7:46 ` Robert Pluim @ 2024-07-30 14:08 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-07-30 14:39 ` Robert Pluim [not found] ` <66a8f3d6.050a0220.8facb.d530SMTPIN_ADDED_BROKEN@mx.google.com> 2024-07-31 23:53 ` Andrew Cohen 3 siblings, 1 reply; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 14:08 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 Xiyue Deng <manphiz@gmail.com> writes: > The fourth patch may need a bit of background: oauth2.el (optionally) > uses plstore to save authentication data for future reuse, and the > plstore id for an account is computed using a combination of `auth-url', > `token-url', and `scope'. However, this combination of data doesn't > guarantee uniqueness for accounts for a same provider, e.g. for Gmail, > the three parameters are the same for different accounts, and hence > storing a second account information will override the first one. Would it make sense to plug OAuth2.el into auth-source to store the authentication token safely inside an existing credential storage? Various applications already do so when using the native credential storages such as Freedesktop.org or the macOS keyring. ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 14:08 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 14:39 ` Robert Pluim 2024-07-30 19:44 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Robert Pluim @ 2024-07-30 14:39 UTC (permalink / raw) To: Xiyue Deng; +Cc: Björn Bidar, 72358 >>>>> On Tue, 30 Jul 2024 17:08:21 +0300, Björn Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said: Björn> Xiyue Deng <manphiz@gmail.com> writes: >> The fourth patch may need a bit of background: oauth2.el (optionally) >> uses plstore to save authentication data for future reuse, and the >> plstore id for an account is computed using a combination of `auth-url', >> `token-url', and `scope'. However, this combination of data doesn't >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >> the three parameters are the same for different accounts, and hence >> storing a second account information will override the first one. Björn> Would it make sense to plug OAuth2.el into auth-source to store the Björn> authentication token safely inside an existing credential storage? Björn> Various applications already do so when using the native credential Björn> storages such as Freedesktop.org or the macOS keyring. Yes. In fact thereʼs the auth-source-xoauth2 package that does that. And oauth2 can already store stuff using plstore, so Iʼm sure it can be extended to use auth-source. Robert -- ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 14:39 ` Robert Pluim @ 2024-07-30 19:44 ` Xiyue Deng 2024-08-01 18:49 ` Thomas Fitzsimmons 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-07-30 19:44 UTC (permalink / raw) To: Robert Pluim; +Cc: Björn Bidar, 72358 Robert Pluim <rpluim@gmail.com> writes: >>>>>> On Tue, 30 Jul 2024 17:08:21 +0300, Björn Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said: > > Björn> Xiyue Deng <manphiz@gmail.com> writes: > >> The fourth patch may need a bit of background: oauth2.el (optionally) > >> uses plstore to save authentication data for future reuse, and the > >> plstore id for an account is computed using a combination of `auth-url', > >> `token-url', and `scope'. However, this combination of data doesn't > >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, > >> the three parameters are the same for different accounts, and hence > >> storing a second account information will override the first one. > > Björn> Would it make sense to plug OAuth2.el into auth-source to store the > Björn> authentication token safely inside an existing credential storage? > > Björn> Various applications already do so when using the native credential > Björn> storages such as Freedesktop.org or the macOS keyring. > > Yes. In fact thereʼs the auth-source-xoauth2 package that does > that. And oauth2 can already store stuff using plstore, so Iʼm sure it > can be extended to use auth-source. > auth-source-xoauth2 doesn't actually use auth-source (e.g. ~/.authinfo.gpg) to store the data it needs, but use a custom file storing an ELisp hash table to store the client-id, client-secret, etc. It does advice the authentication code to use the calculated token. > > Robert -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 19:44 ` Xiyue Deng @ 2024-08-01 18:49 ` Thomas Fitzsimmons 2024-08-02 8:09 ` Xiyue Deng 0 siblings, 1 reply; 49+ messages in thread From: Thomas Fitzsimmons @ 2024-08-01 18:49 UTC (permalink / raw) To: Xiyue Deng; +Cc: Björn Bidar, Robert Pluim, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Robert Pluim <rpluim@gmail.com> writes: > >>>>>>> On Tue, 30 Jul 2024 17:08:21 +0300, Björn Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said: >> >> Björn> Xiyue Deng <manphiz@gmail.com> writes: >> >> The fourth patch may need a bit of background: oauth2.el (optionally) >> >> uses plstore to save authentication data for future reuse, and the >> >> plstore id for an account is computed using a combination of `auth-url', >> >> `token-url', and `scope'. However, this combination of data doesn't >> >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >> >> the three parameters are the same for different accounts, and hence >> >> storing a second account information will override the first one. >> >> Björn> Would it make sense to plug OAuth2.el into auth-source to store the >> Björn> authentication token safely inside an existing credential storage? >> >> Björn> Various applications already do so when using the native credential >> Björn> storages such as Freedesktop.org or the macOS keyring. >> >> Yes. In fact thereʼs the auth-source-xoauth2 package that does >> that. And oauth2 can already store stuff using plstore, so Iʼm sure it >> can be extended to use auth-source. >> > > auth-source-xoauth2 doesn't actually use auth-source > (e.g. ~/.authinfo.gpg) to store the data it needs, but use a custom file > storing an ELisp hash table to store the client-id, client-secret, etc. > It does advice the authentication code to use the calculated token. I have not seen it mentioned in this thread yet, so here goes: my url-http-oauth package in GNU ELPA supports storing credentials in ~/.authinfo.gpg and refreshing them. It would be nice if your OAuth2 work could get feature parity with it, then I could delete my package; feel free to copy any code that makes sense. (I do not use url-http-oauth anymore, but I felt the need to write it when I was using Excorporate and OAuth.) Ideally you could get the result (and the xoauth2 support for IMAP and SMTP) accepted in Emacs core. (Then, extremely ideally, the FSF could work out legal agreements with the various OAuth providers to get Emacs registered as an OAuth application, like, e.g., Thunderbird.) Thomas ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-01 18:49 ` Thomas Fitzsimmons @ 2024-08-02 8:09 ` Xiyue Deng 2024-08-02 14:43 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 1 reply; 49+ messages in thread From: Xiyue Deng @ 2024-08-02 8:09 UTC (permalink / raw) To: Thomas Fitzsimmons; +Cc: Björn Bidar, Robert Pluim, 72358 Hi Thomas, Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Robert Pluim <rpluim@gmail.com> writes: >> >>>>>>>> On Tue, 30 Jul 2024 17:08:21 +0300, Björn Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said: >>> >>> Björn> Xiyue Deng <manphiz@gmail.com> writes: >>> >> The fourth patch may need a bit of background: oauth2.el (optionally) >>> >> uses plstore to save authentication data for future reuse, and the >>> >> plstore id for an account is computed using a combination of `auth-url', >>> >> `token-url', and `scope'. However, this combination of data doesn't >>> >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >>> >> the three parameters are the same for different accounts, and hence >>> >> storing a second account information will override the first one. >>> >>> Björn> Would it make sense to plug OAuth2.el into auth-source to store the >>> Björn> authentication token safely inside an existing credential storage? >>> >>> Björn> Various applications already do so when using the native credential >>> Björn> storages such as Freedesktop.org or the macOS keyring. >>> >>> Yes. In fact thereʼs the auth-source-xoauth2 package that does >>> that. And oauth2 can already store stuff using plstore, so Iʼm sure it >>> can be extended to use auth-source. >>> >> >> auth-source-xoauth2 doesn't actually use auth-source >> (e.g. ~/.authinfo.gpg) to store the data it needs, but use a custom file >> storing an ELisp hash table to store the client-id, client-secret, etc. >> It does advice the authentication code to use the calculated token. > > I have not seen it mentioned in this thread yet, so here goes: my > url-http-oauth package in GNU ELPA supports storing credentials in > ~/.authinfo.gpg and refreshing them. It would be nice if your OAuth2 > work could get feature parity with it, then I could delete my package; > feel free to copy any code that makes sense. (I do not use > url-http-oauth anymore, but I felt the need to write it when I was using > Excorporate and OAuth.) > Thanks for working on url-http-oauth! I think it adds credential management using auth-source, e.g. prompt for client-id and client-secret and store them, which my other addon (that I'll post next as it depends on the changes I made here) didn't do. Ideally this should be handled transparently by all auth-source backends and say Gnus when you add a new account, but IIUC currently the JSON backend doesn't support creation, which I'm using for ease to read and modify. > Ideally you could get the result (and the xoauth2 support for IMAP and > SMTP) accepted in Emacs core. > That would be great! My other addon uses advice, but it would definitely be better to be integrated in core (which already has partial support) > (Then, extremely ideally, the FSF could work out legal agreements with > the various OAuth providers to get Emacs registered as an OAuth > application, like, e.g., Thunderbird.) > That would be the best for the end user. Imagine a Gnus user could just add a new account and on launch Gnus the default browser will open the login page (or be prompted an URL to visit), which then normally handles all the login shenanigans (2FA, authenticator, etc.) and viola, you're logged in. > Thomas -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-08-02 8:09 ` Xiyue Deng @ 2024-08-02 14:43 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-08-02 14:43 UTC (permalink / raw) To: Xiyue Deng; +Cc: Robert Pluim, Thomas Fitzsimmons, 72358 Xiyue Deng <manphiz@gmail.com> writes: > Hi Thomas, > > Thomas Fitzsimmons <fitzsim@fitzsim.org> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> Robert Pluim <rpluim@gmail.com> writes: >>> >>>>>>>>> On Tue, 30 Jul 2024 17:08:21 +0300, Björn Bidar via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said: >>>> >>>> Björn> Xiyue Deng <manphiz@gmail.com> writes: >>>> >> The fourth patch may need a bit of background: oauth2.el (optionally) >>>> >> uses plstore to save authentication data for future reuse, and the >>>> >> plstore id for an account is computed using a combination of `auth-url', >>>> >> `token-url', and `scope'. However, this combination of data doesn't >>>> >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >>>> >> the three parameters are the same for different accounts, and hence >>>> >> storing a second account information will override the first one. >>>> >>>> Björn> Would it make sense to plug OAuth2.el into auth-source to store the >>>> Björn> authentication token safely inside an existing credential storage? >>>> >>>> Björn> Various applications already do so when using the native credential >>>> Björn> storages such as Freedesktop.org or the macOS keyring. >>>> >>>> Yes. In fact thereʼs the auth-source-xoauth2 package that does >>>> that. And oauth2 can already store stuff using plstore, so Iʼm sure it >>>> can be extended to use auth-source. >>>> >>> >>> auth-source-xoauth2 doesn't actually use auth-source >>> (e.g. ~/.authinfo.gpg) to store the data it needs, but use a custom file >>> storing an ELisp hash table to store the client-id, client-secret, etc. >>> It does advice the authentication code to use the calculated token. >> >> I have not seen it mentioned in this thread yet, so here goes: my >> url-http-oauth package in GNU ELPA supports storing credentials in >> ~/.authinfo.gpg and refreshing them. It would be nice if your OAuth2 >> work could get feature parity with it, then I could delete my package; >> feel free to copy any code that makes sense. (I do not use >> url-http-oauth anymore, but I felt the need to write it when I was using >> Excorporate and OAuth.) >> > > Thanks for working on url-http-oauth! I think it adds credential > management using auth-source, e.g. prompt for client-id and > client-secret and store them, which my other addon (that I'll post next > as it depends on the changes I made here) didn't do. Ideally this > should be handled transparently by all auth-source backends and say Gnus > when you add a new account, but IIUC currently the JSON backend doesn't > support creation, which I'm using for ease to read and modify. I think depending on the authentication storage it would make sense to provide a custom setting with a sane default that allows the user to configure a path inside the storage to store their credentials. These could then have templates for the hostname, user, port etc just like the regular search parameters that `auth-sources-search' supports. >> Ideally you could get the result (and the xoauth2 support for IMAP and >> SMTP) accepted in Emacs core. >> > > That would be great! My other addon uses advice, but it would > definitely be better to be integrated in core (which already has partial > support) > From Gnus point of view I think it would make it easier to support auth-source as then Oauth support would come for free as then the credentials can be retrieved just for basic authentication. Some kind of hook or trigger to call renewal functions would be needed though so that tokens can be refreshed transparently or manually if the user needs to re-authenticate. >> (Then, extremely ideally, the FSF could work out legal agreements with >> the various OAuth providers to get Emacs registered as an OAuth >> application, like, e.g., Thunderbird.) >> > > That would be the best for the end user. Imagine a Gnus user could just > add a new account and on launch Gnus the default browser will open the > login page (or be prompted an URL to visit), which then normally handles > all the login shenanigans (2FA, authenticator, etc.) and viola, you're > logged in. > To allow seamless authentication the FSF could make things much smoother by registering Emacs with common providers, working with FSFE together in this regard could help too (there are other countries than the U.S.). The issue I see is that registering Emacs is not enough since Emacs is in this context merely a runtime for these modes that want to use OAuth. The scope of the permissions they need varies, so their names etc. ^ permalink raw reply [flat|nested] 49+ messages in thread
[parent not found: <66a8f3d6.050a0220.8facb.d530SMTPIN_ADDED_BROKEN@mx.google.com>]
* bug#72358: 29.4; oauth2.el improvements [not found] ` <66a8f3d6.050a0220.8facb.d530SMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-07-30 19:41 ` Xiyue Deng 2024-07-30 21:51 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66a96079.170a0220.1522dd.3e68SMTPIN_ADDED_BROKEN@mx.google.com> 0 siblings, 2 replies; 49+ messages in thread From: Xiyue Deng @ 2024-07-30 19:41 UTC (permalink / raw) To: Björn Bidar; +Cc: 72358 Björn Bidar <bjorn.bidar@thaodan.de> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> The fourth patch may need a bit of background: oauth2.el (optionally) >> uses plstore to save authentication data for future reuse, and the >> plstore id for an account is computed using a combination of `auth-url', >> `token-url', and `scope'. However, this combination of data doesn't >> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >> the three parameters are the same for different accounts, and hence >> storing a second account information will override the first one. > > Would it make sense to plug OAuth2.el into auth-source to store the > authentication token safely inside an existing credential storage? > > Various applications already do so when using the native credential > storages such as Freedesktop.org or the macOS keyring. As I mentioned to Robert, I do have another addon to do exactly this, though through an awkward advice. Would be great if auth-source can make use of oauth2.el and handle that more gracefully. I'll file another bug to explore options once this one is done. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-30 19:41 ` Xiyue Deng @ 2024-07-30 21:51 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66a96079.170a0220.1522dd.3e68SMTPIN_ADDED_BROKEN@mx.google.com> 1 sibling, 0 replies; 49+ messages in thread From: Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-07-30 21:51 UTC (permalink / raw) To: Xiyue Deng; +Cc: 72358 Xiyue Deng <manphiz@gmail.com> writes: > Björn Bidar <bjorn.bidar@thaodan.de> writes: > >> Xiyue Deng <manphiz@gmail.com> writes: >> >>> The fourth patch may need a bit of background: oauth2.el (optionally) >>> uses plstore to save authentication data for future reuse, and the >>> plstore id for an account is computed using a combination of `auth-url', >>> `token-url', and `scope'. However, this combination of data doesn't >>> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >>> the three parameters are the same for different accounts, and hence >>> storing a second account information will override the first one. >> >> Would it make sense to plug OAuth2.el into auth-source to store the >> authentication token safely inside an existing credential storage? >> >> Various applications already do so when using the native credential >> storages such as Freedesktop.org or the macOS keyring. > > As I mentioned to Robert, I do have another addon to do exactly this, > though through an awkward advice. Would be great if auth-source can > make use of oauth2.el and handle that more gracefully. I'll file > another bug to explore options once this one is done. Care to post this advice? It's not an optimal solution but better than nothing in the interim. ^ permalink raw reply [flat|nested] 49+ messages in thread
[parent not found: <66a96079.170a0220.1522dd.3e68SMTPIN_ADDED_BROKEN@mx.google.com>]
* bug#72358: 29.4; oauth2.el improvements [not found] ` <66a96079.170a0220.1522dd.3e68SMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-07-31 7:43 ` Xiyue Deng 0 siblings, 0 replies; 49+ messages in thread From: Xiyue Deng @ 2024-07-31 7:43 UTC (permalink / raw) To: Björn Bidar; +Cc: 72358 Björn Bidar <bjorn.bidar@thaodan.de> writes: > Xiyue Deng <manphiz@gmail.com> writes: > >> Björn Bidar <bjorn.bidar@thaodan.de> writes: >> >>> Xiyue Deng <manphiz@gmail.com> writes: >>> >>>> The fourth patch may need a bit of background: oauth2.el (optionally) >>>> uses plstore to save authentication data for future reuse, and the >>>> plstore id for an account is computed using a combination of `auth-url', >>>> `token-url', and `scope'. However, this combination of data doesn't >>>> guarantee uniqueness for accounts for a same provider, e.g. for Gmail, >>>> the three parameters are the same for different accounts, and hence >>>> storing a second account information will override the first one. >>> >>> Would it make sense to plug OAuth2.el into auth-source to store the >>> authentication token safely inside an existing credential storage? >>> >>> Various applications already do so when using the native credential >>> storages such as Freedesktop.org or the macOS keyring. >> >> As I mentioned to Robert, I do have another addon to do exactly this, >> though through an awkward advice. Would be great if auth-source can >> make use of oauth2.el and handle that more gracefully. I'll file >> another bug to explore options once this one is done. > > Care to post this advice? It's not an optimal solution but better than > nothing in the interim. It currently depends on this modified oauth2.el. Once the patches are accepted I'll post it in a separate bug. -- Xiyue Deng ^ permalink raw reply [flat|nested] 49+ messages in thread
* bug#72358: 29.4; oauth2.el improvements 2024-07-29 21:25 bug#72358: 29.4; oauth2.el improvements Xiyue Deng ` (2 preceding siblings ...) [not found] ` <66a8f3d6.050a0220.8facb.d530SMTPIN_ADDED_BROKEN@mx.google.com> @ 2024-07-31 23:53 ` Andrew Cohen 3 siblings, 0 replies; 49+ messages in thread From: Andrew Cohen @ 2024-07-31 23:53 UTC (permalink / raw) To: 72358 [-- Attachment #1: Type: text/plain, Size: 1166 bytes --] I have been using the existing oauth2.el and auth-source.el to use both gmail and outlook (through my university) with oauth2 for several years now (I posted a bit about it some time ago on the devel list). I didn't need to change much to get it to work so I thought as long as the changes by Xiyue are being considered (all of which look good to me) I would chime in. I'm happy to provide more info about my setup and usage if anyone is interested. Firstly, I note that I have gmail working fine without the change in patchset 2 (although I see nothing wrong with the change, I wonder why it isn't necessary for me but is for Xiyue). Secondly, there is one other important change that I have been using which should probably be added to oauth2.el (I communicated the change to Julien a long time ago, but he said he is no longer actively maintaining oauth2.el): in refreshing the token the access-response is ignored (as is the response-error). The access-response contains information about the token expiration so its needed in order to control when to fetch a new token. The simple patch below stores the access-response in the appropriate slot in the token: [-- Attachment #2: store access-response on refresh --] [-- Type: application/octet-stream, Size: 3101 bytes --] diff --git a/oauth2.el b/oauth2.el index 7da9702..e460b01 100644 --- a/oauth2.el +++ b/oauth2.el @@ -119,28 +119,35 @@ Return an `oauth2-token' structure." (defun oauth2-refresh-access (token) "Refresh OAuth access TOKEN. TOKEN should be obtained with `oauth2-request-access'." - (setf (oauth2-token-access-token token) - (cdr (assoc 'access_token - (oauth2-make-access-request - (oauth2-token-token-url token) - (concat "client_id=" (oauth2-token-client-id token) - (when (oauth2-token-client-secret token) - (concat "&client_secret=" (oauth2-token-client-secret token))) - "&refresh_token=" (oauth2-token-refresh-token token) - "&grant_type=refresh_token"))))) - ;; If the token has a plstore, update it - (let ((plstore (oauth2-token-plstore token))) - (when plstore - (plstore-put plstore (oauth2-token-plstore-id token) - nil `(:access-token - ,(oauth2-token-access-token token) - :refresh-token - ,(oauth2-token-refresh-token token) - :access-response - ,(oauth2-token-access-response token) - )) - (plstore-save plstore))) - token) + (let ((response (oauth2-make-access-request + (oauth2-token-token-url token) + (concat "client_id=" (oauth2-token-client-id token) + (when (oauth2-token-client-secret token) + (concat "&client_secret=" (oauth2-token-client-secret token))) + "&refresh_token=" (oauth2-token-refresh-token token) + "&grant_type=refresh_token")))) + (if-let ((response-error (cdr (assoc 'error response)))) + (message "oauth2 token refresh error: %s" response-error) + (setf (oauth2-token-access-token token) + (cdr (assoc 'access_token response))) + (when-let ((refresh_token (cdr (assoc 'refresh_token response)))) + (setf (oauth2-token-refresh-token token) refresh_token)) + (setf (oauth2-token-access-response token) + (assoc-delete-all 'refresh_token + (assoc-delete-all 'access_token response))) + ;; If the token has a plstore, update it + (let ((plstore (oauth2-token-plstore token))) + (when plstore + (plstore-put plstore (oauth2-token-plstore-id token) + nil `(:access-token + ,(oauth2-token-access-token token) + :refresh-token + ,(oauth2-token-refresh-token token) + :access-response + ,(oauth2-token-access-response token) + )) + (plstore-save plstore)))) + token)) ;;;###autoload (defun oauth2-auth (auth-url token-url client-id client-secret &optional scope state redirect-uri) [-- Attachment #3: Type: text/plain, Size: 2542 bytes --] Lastly, a brief description of how to get things to work with auth-source and existing code (subject to the change I mentioned above): auth-source entries using the plstore backend allow the secret to be a function (which is passed the whole entry plist as an argument). All that is needed then is a function that returns the access token (which is then used in gnus and smtpmail, both of which already work properly with an oauth2 access-token). A simple function to check the expiration time and fetch a new access-token if necessary (and update the new token and expiration information) and then return the access-token is what I use. So I used auth-source to create plstore entries for gmail and outlook containing the oauth2 tokens, and set the secret to the following function (defun gnus-refresh-access (plist) "Return an oauth2 access-token for PLIST. If the current token has expired, fetch, save, and return a new one." (cl-destructuring-bind (&key user host port token last-update (expires_in (alist-get 'expires_in (oauth2-token-access-response token))) (create-args (list :type 'plstore :create '(:encrypted (token client-secret-sav) :unencrypted (auth-url scope redirect-uri last-update smtp-auth)))) &allow-other-keys) plist (unless (and (numberp expires_in) (numberp last-update) (< (float-time) (+ last-update expires_in))) (message "Getting new token for %s at %s:%s" user host port) (setq plist (plist-put plist :secret 'gnus-refresh-access)) (setq plist (plist-put plist :last-update (truncate (float-time)))) ;; get a new token and update the plist (setq plist (plist-put plist :token (oauth2-refresh-access token))) ;; update auth-source---if something in the plist has changed ;; then no entry will be found during the search, and the ;; create flag will be honored. (apply #'auth-source-search (append plist create-args))) ;; return the access token (oauth2-token-access-token (plist-get plist :token)))) By the way, I let auth-source handle the plstore rather than oauth2.el. It seemed simpler to have only one of them managing the store rather than both. By the by the way, there are some important bugs in auth-source.el that I have fixed in my personal tree (and a few that I haven't). I'll post about them in a separate bug report at some point. Best, Andy -- Andrew Cohen ^ permalink raw reply related [flat|nested] 49+ messages in thread
end of thread, other threads:[~2024-09-03 18:08 UTC | newest] Thread overview: 49+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-07-29 21:25 bug#72358: 29.4; oauth2.el improvements Xiyue Deng 2024-07-30 7:46 ` Robert Pluim 2024-07-30 14:05 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-07-30 19:37 ` Xiyue Deng 2024-07-31 8:54 ` Robert Pluim 2024-07-31 11:13 ` Xiyue Deng 2024-08-02 8:15 ` Xiyue Deng 2024-08-02 8:38 ` Robert Pluim 2024-08-03 0:04 ` Xiyue Deng 2024-08-03 5:52 ` Eli Zaretskii 2024-08-03 9:26 ` Xiyue Deng 2024-08-13 22:03 ` Xiyue Deng 2024-08-14 5:28 ` Eli Zaretskii 2024-08-14 8:23 ` Xiyue Deng 2024-08-14 8:40 ` Xiyue Deng 2024-08-14 9:13 ` Eli Zaretskii 2024-08-21 18:22 ` Xiyue Deng 2024-08-21 19:42 ` Philip Kaludercic 2024-08-21 22:11 ` Xiyue Deng 2024-08-29 6:58 ` Xiyue Deng 2024-08-29 14:14 ` Philip Kaludercic 2024-08-29 15:18 ` Robert Pluim 2024-08-29 23:54 ` Xiyue Deng 2024-08-30 7:09 ` Philip Kaludercic 2024-08-30 8:32 ` Xiyue Deng 2024-08-30 10:07 ` Philip Kaludercic 2024-08-30 21:13 ` Xiyue Deng 2024-09-03 18:08 ` Xiyue Deng [not found] ` <66a8f323.170a0220.9172c.8e28SMTPIN_ADDED_BROKEN@mx.google.com> 2024-07-30 19:40 ` Xiyue Deng 2024-07-30 21:50 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-07 23:22 ` Xiyue Deng 2024-08-08 6:11 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-08 6:14 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66b46180.170a0220.1fb02.1d6eSMTPIN_ADDED_BROKEN@mx.google.com> 2024-08-08 8:28 ` Xiyue Deng 2024-08-08 9:17 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-08-12 13:22 ` Thomas Fitzsimmons 2024-08-12 16:26 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66b46251.170a0220.f2be9.afeeSMTPIN_ADDED_BROKEN@mx.google.com> 2024-08-08 8:29 ` Xiyue Deng 2024-08-08 9:31 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-07-30 14:08 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors 2024-07-30 14:39 ` Robert Pluim 2024-07-30 19:44 ` Xiyue Deng 2024-08-01 18:49 ` Thomas Fitzsimmons 2024-08-02 8:09 ` Xiyue Deng 2024-08-02 14:43 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66a8f3d6.050a0220.8facb.d530SMTPIN_ADDED_BROKEN@mx.google.com> 2024-07-30 19:41 ` Xiyue Deng 2024-07-30 21:51 ` Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors [not found] ` <66a96079.170a0220.1522dd.3e68SMTPIN_ADDED_BROKEN@mx.google.com> 2024-07-31 7:43 ` Xiyue Deng 2024-07-31 23:53 ` Andrew Cohen
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.