From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrew Cohen Newsgroups: gmane.emacs.devel Subject: Re: oauth2 support for Emacs email clients Date: Wed, 11 Aug 2021 08:43:21 +0800 Organization: Hong Kong University of Science and Technology Message-ID: <87fsvgolp2.fsf@ust.hk> References: <52589.36892.953561.24840@gargle.gargle.HOWL> <39093.96315.985670.24841@gargle.gargle.HOWL> <87o8acpwqe.fsf@ust.hk> <8735rhfjoi.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16017"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: "Roland Winkler" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Aug 11 02:44:36 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mDcMF-0003zA-8p for ged-emacs-devel@m.gmane-mx.org; Wed, 11 Aug 2021 02:44:36 +0200 Original-Received: from localhost ([::1]:37136 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDcME-0002m0-1D for ged-emacs-devel@m.gmane-mx.org; Tue, 10 Aug 2021 20:44:34 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:52334) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDcLH-0001pR-Gd for emacs-devel@gnu.org; Tue, 10 Aug 2021 20:43:35 -0400 Original-Received: from mail-eopbgr1410095.outbound.protection.outlook.com ([40.107.141.95]:3232 helo=JPN01-OS2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDcLE-0008NT-CP; Tue, 10 Aug 2021 20:43:35 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wlx5fql6sbhvCNNVZUX+95xMsh9Sj5r2Glv9WQsgIcxLpz6qqwarMawRsC1FvKJ8iFR9ihWT9ynAJwZ6mxD0dzOlFmM+G3PmEb/wB+dwo3kMC04tauGsWzUz1ql/WKvBzaAwutKxEMDfUEKVn0JijJw4tXwfmL/Kwd6izUHcAwyOUAK3ENkDtiQpAbTErPP9uTdd500NyD3yDPNnX25deVjgjuahSbm+YLcKNaT1PwL52pJRMopuk0H3qbMv/q4tzvKz2oWwlLRAp9c6RpvNN1OjOTQuj5iSygtKloUbK0vD6MxZDagHdrhwWx51wtIkhnJBUA+kx1eftLffFG3Bnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PyNXzmRJ/7ztEYp8b15DM4yiGHrxQnBLl3S9dY33QK8=; b=igRUQ3SqOJfeoKkUyZ/WNs8wnbCRVfKdFlZ0wzdtMcxDEVzonvxcRjUVLG4sux+YaI/DvWxRUJ5ec7lGcadkE6zr/euBSdqGd0/NKj8HhP3ZTUvPWXq5NZOlFsKJsbVeP6r3wkSqmi9w8KJEU3sk4qo7+Fxvh1yYBWwR+9nMKJ9x8B/BfWsJuVPQGp52mVufyMugLZmytj8OKzJsouQaCGfQ4LZByq+2yKdZBiAPWtsJXAg+MN87M4qL2HnYEoiKu+xZoSbBmGoeWPmVc4JhuMFx2AGo7Dvc+O+2y8WwZFmAgMv/R8ChjMbXDZTpWAVtLkEN/YMehZCo9NwqEgD0Ng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ust.hk; dmarc=pass action=none header.from=ust.hk; dkim=pass header.d=ust.hk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ust.hk; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PyNXzmRJ/7ztEYp8b15DM4yiGHrxQnBLl3S9dY33QK8=; b=KKkJhs4m9AELXh0KRAN5XmZjPPtshbJuFYL6hmwYL2bGHldwmxvuO7XgcochWKfZ8OVkJVHhR3ARA3RJj9b9EbKEqKxs3SMB0i4zwIpNNohMmqKD6u41CUV8LbC3EpRlUhZnjXCd5nWBZ4phReLWILV8QDMIvRRo/L14JmJU08Nh2ITq53L8M1ysiB+/T2QTXeg8N30jdOvtGOwBwi9LDloW1gwGH8Ny9F+YB90hCpxiR8CR3AIX5g+rqYZ3VsLWrB/Wt6p/19IIHxhRG8dd8yApUc2LFNFyY/JWesUCHQQNscSmKLNoj/++UdlgvmqmMkfGFNA7Gz3mJf0mXx3ETA== Authentication-Results: gnu.org; dkim=none (message not signed) header.d=none;gnu.org; dmarc=none action=none header.from=ust.hk; Original-Received: from TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM (2603:1096:404:8039::16) by TYBP286MB0479.JPNP286.PROD.OUTLOOK.COM (2603:1096:404:802d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.14; Wed, 11 Aug 2021 00:43:24 +0000 Original-Received: from TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM ([fe80::455a:6595:5a00:6de2]) by TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM ([fe80::455a:6595:5a00:6de2%9]) with mapi id 15.20.4394.023; Wed, 11 Aug 2021 00:43:24 +0000 In-Reply-To: <8735rhfjoi.fsf@gnu.org> (Roland Winkler's message of "Tue, 10 Aug 2021 09:39:25 -0500") X-ClientProxiedBy: HK2PR02CA0184.apcprd02.prod.outlook.com (2603:1096:201:21::20) To TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM (2603:1096:404:8039::16) X-MS-Exchange-MessageSentRepresentingType: 1 Original-Received: from hanan (193.176.211.29) by HK2PR02CA0184.apcprd02.prod.outlook.com (2603:1096:201:21::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.15 via Frontend Transport; Wed, 11 Aug 2021 00:43:23 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 602cd4f7-13b8-4888-004b-08d95c610691 X-MS-TrafficTypeDiagnostic: TYBP286MB0479: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BfD4UAH3i1TXwBKKNw1oaXUPuFdreTtJjEPBPgkXqFntQaFZjoxAxstfXUnkIk3jTPoYsCfbRk4OrVa2ldveBpKo3ke/4FIX+dVy0wyAOwg6x1+IhFh+NwHS/crUhCL9R4j3us9wpClEIejGCLOTPl+9rAxm980Qhh1MlbkrvAQy/oySsDb1a3IOcVarNCcPJaNtiZxpqeTozv3M2qgpW+muA2+wT730rNUMX2NIxS+s77GaZEjy6teTTgBDOXGuVqZNxGlUZqqpRqwOHBQWZZHUKtp5/SQo8uMQ3P5+/6ArteTIKesaNg9MAqsy8NPavMqkJuidSLy5E2lfCD8xB4TLa3uZNxZ+Fs9VeSb6WnOGy2MVP+s1ZgCIRjy862Gc2cBLtmRUgpHgS4jE+MW58Jf98NRkqAOWjjT5z2N/TvF7Gsx0N4C16UFda4dJuBOShUdicVcvYO9kxSVqNhE1E/cuxeGTBMtChEwdjmyavP6YVcC6j9AbAyRfqBk/B9vR/6ldoFz4vxqxe20VmGybSoIINMhktsOnsf0hrUQw8Jf+tebWVRFgHgD1Yt1wj4bd8B7JQJnOzxV04qr8qs68xPkUIoHzVuETGedwX70bDkOrDpquWBIuGHqBSBP+2mYj+8k9aPWx2AfqwX3HpZn80w== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(4636009)(366004)(36756003)(26005)(66946007)(66476007)(8676002)(2906002)(450100002)(86362001)(66556008)(186003)(6486002)(2616005)(956004)(508600001)(316002)(36916002)(5660300002)(38100700002)(6496006)(83380400001)(6916009)(786003)(8936002)(4326008); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?UL3btosI0h7J4h6WsLjlF38MQyPnzmhTcI596mcTkRAMaHCIo7GTEB8lXTW+?= =?us-ascii?Q?uuoUcS/CSbOc5QWi3ifDgc8A8qIRDvWfchGnYP0sBdU50MtJ71j5GxXdDgRV?= =?us-ascii?Q?s2WlkmHj1ycrjYqokFrok1WYkIcR7H/W25haD2+AGh9kjH1tL3Q5ckn6T15v?= =?us-ascii?Q?Ne5/VoYTCAu7U6OWkM6CpXN+q41drcpHSBS4DbD4rWzqc16MVoS3koNS3YbE?= =?us-ascii?Q?TK6L8hXHJRuvTGKlg1Eo0ykPpYq53yGgzjq4qnXhDOVSVdCQ1L1oSzUYYNPc?= =?us-ascii?Q?XGSA5htFkZHtLVbN/7tjuUEXC2DnIBR7yDXWsPzFt8XkrT1DRvRoFLJIn3+u?= =?us-ascii?Q?QlRtKSzqSOgn2gsXFMrLOfUo+oqJhUwei42gxauP9yakBYk09/DdZec4rjpO?= =?us-ascii?Q?9NKPMTWYn2BFEz0uQSuwSsz/mu5LT2l7jHMaE7+ZEeJ6MvcHtlnXHhDQ/J1y?= =?us-ascii?Q?T9+COKpQFO2Yhh8FDCwdWiGI3bTnvIGBJlxonGNW7nxTHQbjNwkJKN+Jb71h?= =?us-ascii?Q?5wkqjSopyhpFYnUF7m17s3Qo19/FPNKhwPYorvZdHZSYyXGxTf5eiPKyRhbi?= =?us-ascii?Q?pJOGrlEQpITsW3R8jbrJfetkjCMz1/tc3PoBlYMZWNR95/oIMBJM9if4L15X?= =?us-ascii?Q?X9 X-OriginatorOrg: ust.hk X-MS-Exchange-CrossTenant-Network-Message-Id: 602cd4f7-13b8-4888-004b-08d95c610691 X-MS-Exchange-CrossTenant-AuthSource: TYAP286MB0282.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2021 00:43:24.4063 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: c917f3e2-9322-4926-9bb3-daca730413ca X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t4QjBoemQO5qhUdlybJ2WuyyjMmnw3fHrYUSXSb6Nq+FHPneX+zW8D+ZOO2MFpGF X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYBP286MB0479 Received-SPF: pass client-ip=40.107.141.95; envelope-from=acohen@ust.hk; helo=JPN01-OS2-obe.outbound.protection.outlook.com X-Spam_score_int: -1 X-Spam_score: -0.2 X-Spam_bar: / X-Spam_report: (-0.2 / 5.0 requ) BAYES_40=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:272303 Archived-At: >>>>> "RW" == Roland Winkler writes: [...] RW> I just want to confirm I have read your posts with great RW> interest, thank you. I am still in the process of digesting RW> them. These topics (including emacs tools like plstore / RW> auth-store / auth-source) have not been my area of expertise. RW> So I am still trying to understand better how to put these RW> things together so that I get what I want / need. I hope I can provide useful info. (I keep accidentally typing "auth-store" instead of "auth-source" so I think that's one less topic to worry about :)) RW> In the meanwhile one question: RW> With your setup, what happens if you need to restart emacs? Are RW> you storing the oauth2 tokens on disk so that you need not RW> re-authenticate (as long as the tokens themselves remain valid, RW> which is 30 days for my organizations)? Yes, the oauth2 credentials are stored on disk. Everything works as usual upon restarting emacs: gnus uses auth-source to retrieve the appropriate credentials from the on-disk storage and the existing refresh-token is used to automatically obtain a new access-token (a password is requested to decrypt the on-disk storage if the cached value has timed out, but otherwise no action by the user is needed). auth-source is a great way to store and retrieve credentials for accessing services (and other things too). It is used by gnus, smtpmail (and I think rmail) and probably lots of other service-based things. A typical entry includes host, user, port, secret (i.e. password) and whatever other things you like. auth-source has several different storage backends (netrc, plstore, json, secrets, and several more) and a simple search mechanism that spans different backends to retrieve appropriate authentication data. For my oauth2 imap and smtp access I simply created appropriate entries (each of which includes a token structure as defined in oauth2.el) using the plstore backend (so the entry is stored on disk with the sensitive parts encrypted). Aside from the slightly funky structure of the auth-store entry (including an ouath2 token) this works exactly as any other auth-store entry to using imap or smtp---that was the reason for setting it up this way, so it looked like any other SASL to the user and didn't require a new package (just some minor tweaks to existing code). Aside: I have another minor change to push to auth-source to make this work. The plstore backend in auth-source only encrypts the :secret entry, but I want to keep the client-secret and the oauth2 token structure in the entry encrypted as well. This required some minor tweaking to auth-source to allow specifying which parts of the entry are encrypted and which are unencrypted. Patch below (which I will push myself in awhile). diff --git a/lisp/auth-source.el b/lisp/auth-source.el index 6919738398..faddbdee1a 100644 --- a/lisp/auth-source.el +++ b/lisp/auth-source.el @@ -2120,11 +2120,17 @@ auth-source-plstore-create (base-secret '(secret)) ;; we know (because of an assertion in auth-source-search) that the ;; :create parameter is either t or a list (which includes nil) - (create-extra (if (eq t create) nil create)) + (create-secret-extra (plist-get create :encrypted)) + (create-extra (if (eq t create) nil + (or + (append + (plist-get create :unencrypted) create-secret-extra) + create))) (current-data (car (auth-source-search :max 1 :host host :port port))) (required (append base-required create-extra)) + (required-secret (append base-secret create-secret-extra)) ;; `valist' is an alist valist ;; `artificial' will be returned if no creation is needed Best, Andy -- Andrew Cohen Director, HKUST Jockey Club Institute for Advanced Study Lam Woo Foundation Professor and Chair Professor of Physics The Hong Kong University of Science and Technology