From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrew Cohen Newsgroups: gmane.emacs.bugs Subject: bug#72992: 29.4; towards xoauth2 support in Emacs Date: Thu, 19 Sep 2024 17:06:06 +0800 Message-ID: <87zfo4au81.fsf@ust.hk> References: <87h6ayfo87.fsf_-_@debian-hx90.lan> <877cb8oihg.fsf@debian-hx90.lan> <878qvocjkz.fsf@ust.hk> <87ldzom4rz.fsf@debian-hx90.lan> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="28397"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Ted Zlatanov , Philip Kaludercic , 72992@debbugs.gnu.org, Stefan Kangas To: Xiyue Deng Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Sep 19 11:07:04 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1srD7v-0007EL-5Z for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 19 Sep 2024 11:07:04 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1srD7g-0001CM-Nv; Thu, 19 Sep 2024 05:06:49 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1srD7e-0001C3-6b for bug-gnu-emacs@gnu.org; Thu, 19 Sep 2024 05:06:46 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1srD7d-0003im-Tc for bug-gnu-emacs@gnu.org; Thu, 19 Sep 2024 05:06:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=zJWR7E/6Iyl8xqvdFiagGt6zzRyOwc4fTIZeL+9Pmts=; b=GdpbSL08OZ6QuLQdHBCcdkC144yEc5jPWGTCV6DCW0+BUciCZrPbUmWyDsbpBwfuZgxlnTDtQBLVTix0Tbyu/oNa4WcwDQF92d/oGo4zIO77SSReP3HOs/Neo8/rLWuUl9puvgMIBGZ+uP7csQgHtk5Zbv39f6jzIma4D+zjByQqmmMV77/D5UMys1qTXxXHvYoo1FVsjabV6CSNWFxKxMib0pM5fPu+oiJkPKs7Cb9L8WRVOMzmQFgmO/ONLVFzsDl7DgcJHCWGsUeNucQhXaDXvsDjCdqi83o7l1dG+rwyDzRwsCXBS+vNcC3vaVLkDx1TYHCZZDMi9NhF5144+Q==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1srD7u-0001SL-1u for bug-gnu-emacs@gnu.org; Thu, 19 Sep 2024 05:07:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Andrew Cohen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 19 Sep 2024 09:07:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72992 X-GNU-PR-Package: emacs Original-Received: via spool by 72992-submit@debbugs.gnu.org id=B72992.17267368035566 (code B ref 72992); Thu, 19 Sep 2024 09:07:02 +0000 Original-Received: (at 72992) by debbugs.gnu.org; 19 Sep 2024 09:06:43 +0000 Original-Received: from localhost ([127.0.0.1]:59796 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1srD7a-0001Ri-G3 for submit@debbugs.gnu.org; Thu, 19 Sep 2024 05:06:42 -0400 Original-Received: from mail-japanwestazon11020138.outbound.protection.outlook.com ([52.101.228.138]:53347 helo=OS0P286CU011.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1srD7X-0001RP-ME for 72992@debbugs.gnu.org; Thu, 19 Sep 2024 05:06:41 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XWYqqC4yIRVvoOHlyxGEUAzj1TojvVu7oRU6mPhnXaUv11VqXuH5nFU5c1c14B6ro4+E4kzNjHXKHJ0T3q3yDGjIlY1sNRhsUglaaNQXVVh3GlBF1eN6nnTeMAaIt5DC8GHKMksJMoqzMZYgm9k6kj8JbrNe9UnH5zC+rV2nWJla7i6W804pxOKG3qoftZz2MT7c/sJ+8omD1dy+bLAHPiTANM8Du7ztcWnRCgov2kpZxsOD2M49FS7rADcc14HyR5ZloGsDFsD/L6BeQNJz0agBKSzH4jztT1JRIvDHaA9DYCmLlSxebU6XaGrA8gSJBmWg5og7AXzMGeFdmwh80A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zJWR7E/6Iyl8xqvdFiagGt6zzRyOwc4fTIZeL+9Pmts=; b=tjV1MAEKt6QeUzIpKVZXPTh5egLP7+9JR2FCi+LLIzJUPpQ8ZZao2mtzazHqOPgRrkFoxHSo+7yUuOfD/UiJI8jarAieqzszsybjovx7allQVfpCudUE2h2Uj2SABXfW5KMBLNBKXzr4LQ78OvOz+s2ikigLYxP4J2eK43kgHQKlk68O3DdRaGLVOOH6SDh3/P7aq3bZ7pO7oyr3suWjV6m5D7MzKhtt+XAAUyeNEQ1zpqWT5QHLcPLg0M+61H0mVV4bt3I0pZRowJzbQVfC8D03tk7H1L4OsU72m1iS/nEpCEyG5NjDmUvRYNUmF8oQTB3BJ4lkGkJdfgj1gZsM3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ust.hk; dmarc=pass action=none header.from=ust.hk; dkim=pass header.d=ust.hk; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ust.hk; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zJWR7E/6Iyl8xqvdFiagGt6zzRyOwc4fTIZeL+9Pmts=; b=PTDoS3R/SfVUuKX4Ji54nu31SE/H+UV0De/GGtafTLS+7btfq3Tg8TSPawYA9L/jyzYbSbMRbUI8XF9UtrBgjV7F+HnFBSyMY0vZfAcVuCl2QljkDEi5wZ/oYNuc7dCtEiQj7jFFtkZv58EMRzo4Rys2tSJ9f0eHzerjVojnzvXa/RTjiKBUrcm0RWTGbb8qfI7dU+XZTw5slJXMdyXDTvpohHNE83LsO2cUHkBKC3eBCI7UKypXYtvHZns/Vb4zv5rfVS3NYzKZopPE3AYNAMHKiApLlAiGkXD6nEIQoAnAlP3XizcGk9Pp8EuPs7vW2fQfzyjdbIGXu60GH3tu4g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ust.hk; Original-Received: from OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:167::9) by TYRP286MB4342.JPNP286.PROD.OUTLOOK.COM (2603:1096:405:138::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7962.24; Thu, 19 Sep 2024 09:06:14 +0000 Original-Received: from OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM ([fe80::e097:a79c:e231:14c9]) by OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM ([fe80::e097:a79c:e231:14c9%5]) with mapi id 15.20.7982.012; Thu, 19 Sep 2024 09:06:14 +0000 In-Reply-To: <87ldzom4rz.fsf@debian-hx90.lan> X-ClientProxiedBy: KL1PR0401CA0027.apcprd04.prod.outlook.com (2603:1096:820:e::14) To OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM (2603:1096:604:167::9) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: OSZP286MB1870:EE_|TYRP286MB4342:EE_ X-MS-Office365-Filtering-Correlation-Id: dcb1e9ac-4984-4e7f-688f-08dcd88a5040 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|366016; X-Microsoft-Antispam-Message-Info: KelvXi+MQQXP+rJxWi+DXEnXgPZpVBF+OUeylj8IPHsVKWwKnyJEjaehwJrwQ2gPtsI7YbyfO8Ch3hdedXdbAm5VvdgvNK6ors/8/pJiImQb9qKKD4SyItZ+cuSWwoCu6qQi8L2BTjnw88GUdbzbG2NdM6A/I1LAtVXAbtOTpUgWSEo/Cs/33BLWgJUzV6bQna0LNKmaK/2V6hMcsoZTnIg861bLjN92dCWmESCZXI82mTRWCQFS4PaiuoZqigm1Nfn7glM69ZgHytjo6OBWQWXih6lXS4wDgrHdTfznvdc7e/aSx3gJcq4lb8yftOKPJMzshTVd2SXkPzqKadQZBeSJpvQImqisDJtEemdSlKXeuUMQeFU5IC2ICeta/0gxo3NhBlWpeAEGA92hovTf9o3ReBYad2WBaBmgKQ/lBvj8ZBYVhmcUDMTGCboHJKxcliqeGx0iS7BdTv6fWVpDGvT5nBOglaTXC08nqNZc5+LFXmizQOHkhlcE8lFA4DJCtbH0UWPwFdFSlnEImcK/aUTQj+5fhwHYHuhFyujAwdDsT5vKOTjczu1+8ZY2ASHcQSSTdX1k8q9V03yoS/nhKqWMrEzXhvbCbtyHIKIBGJrTmyK4gbUUz7p0ACu6W1bJHZ7SMTAT79m20FzeBE+Uuiyfy5DkEVTiNLwunEPyQNjb0knApIaKN1fGfzatzfLaMOd5M9aMMdEKe0IONDjtOauvn34OrHhpSuHAxW5NVJtiQs4ujq6lQKnUcX/NH FFLLisQR2sVH/MTtapkAk4bprzxWG4yejv+BaKlznlBAah1LIeM2AWM7+gJRihX61Mlius4et23Ki9zoLq9jegFhcdZ6Oe3yUz X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SaB1Yo3r/F/YsLUqZMTyDL7H8tCooZ+g4ZYPq6h1VGzwlip1byy0/VMvGJDdDKjNgW24B1wsaV0gX+ErsNJc/Up3kvGDjDKoesnKVvE5zHyGLbWTBf2KVzpgME/huMQsrmyIkxbLRu9aA104Kd+5qnco2HpR13h057typpEHz+s9l7j8o3YLKHGpWOZ9j2jcH33anfr4a4AtMoMsMTTih8Q/PVcz6t0YS+v/Dl9LaNbMcYI/5MDOrX0mPc1ZyECBJzZv1mnuoUK1kOo1E94EC5OI/kWeHMZUxexVNOztzErAt7FvT18DlwDdKygvyaELKJMdMiKE5GTykzu5Vs73m+VjjMe8uruD+CIYAx7tkMWU6oRkyPm7S3am86ZRNNwqcfM95rdbwTy4Hs4Aqrm+TvckqGtvZPyoHjsZ9iJiqk8O43jk0XhBvCqTcOemiSJZMRlr717GCZXhgFkFdE8gCOyFxW3dt+7tllOKvAdw+kmIBcNaR0c3j57uGNPKjQ5VdV5FzAnl53OoAPwz3+emQ23BZL1kLJH3MO+V0Mb5brNzUH+lvgQpLk+ktQ2/lBsIDB+Hhi5/QwfN16vKjn6aNfgkY1SlH4z4lam2vE2VoJ+SaMkeSF/ybd/7RaSB4Q2lXY+e8VW7UMKX74AWQAGDMG3nSigWjoTlM7yb0XK3q2uWlHsnjK4L0UWPcPC/km67Au/V4IoOaUo0LRaxxuStaRMDJPlLqu0mCfQCqApaXCVxyoLPlV8NLBUiFe DrzyUHlXo/oV4aumgWwp/CwjMkWI95/RYWKCeLm32I2v5bhOcB3chxEG+ZKqLnva5Q2iiwCViTGl9+WaF3mQgQT5dqLAdFZuvZ X-OriginatorOrg: ust.hk X-MS-Exchange-CrossTenant-Network-Message-Id: dcb1e9ac-4984-4e7f-688f-08dcd88a5040 X-MS-Exchange-CrossTenant-AuthSource: OSZP286MB1870.JPNP286.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2024 09:06:14.4533 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: c917f3e2-9322-4926-9bb3-daca730413ca X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CAMthYoVJSEnVmHZpSaMW07CuuIyWjOUmxuY5nYNEp5VvBSrXybMrAQSsBCHcZNg X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYRP286MB4342 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:292037 Archived-At: >>>>> "XD" == Xiyue Deng writes: XD> Hi Andrew, Andrew Cohen writes: >>>>>>> "XD" == Xiyue Deng writes: >> [...] XD> The basic support is actually in the Emacs core already, XD> e.g. for Gnus nnimap[2] and smtpmail[3]. However, this assumes XD> one to put the access_token in place of `:secret' in the XD> auth-source file as Emacs uses password as the access_token in XD> both places. However, access_token expires quite frequently XD> (e.g. about 1 hour for Gmail) and without refreshing it XD> automatically it is practically impossible to use conveniently. XD> Hence the propose hack and the following suggestion. >> >> >> This isn't actually true. When I added the support many years >> ago, I updated auth-source so that the :secret field can be a >> function, and this is how you should be using the current xoauth >> support. XD> Thanks for pointing this out! I found the place where `:secret' XD> is handled as a function[1]. However, this requires a user to XD> implement the oauth2 logic oneself, which I'm afraid is a bit XD> too low-level and error-prone. (Actually, can I actually put a XD> lisp function in auth-source.gpg?) I don't think you have to do anything low level, and I don't think there is anything error prone here; you can use the functions from oauth themselves (oauth2.el can create its own plstores, but I prefer to use auth-source.el to manage the stores). The only things needed are a call to oauth2-refresh-access to get a new token, and then oauth2-token-access-token to return the new access token. The function I wrote computes the refresh time to decide when to create a new token. This logic could easily be put into oauth2 instead. And yes, you can put the lisp function in auth-source.gpg (this is what I do). By the way there are some significant bugs in auth-source.el which I have fixed in my personal tree but haven't yet pushed. I have so little time for emacs at the moment, but I'll try to get around to it. And there is one major deficiency in auth-source.el that I want to deal with: obfuscation of the :secret. When Ted originally wrote auth-source.el he wrapped the :secret in a closure so that the secret itself wasn't visible in memory. At the time he did this, closures weren't fully part of emacs, and their implementation at the time didn't expose the contents of the closure in bytecode. But the current official implementation does, so this obfuscation trick no longer works. I want to remove it since it no longer works and might lead to confusion. XD> Maybe auth-source source can host a helper function that checks XD> if `:secret' is not set and xaouth2 is preferred (e.g. `:auth' XD> is `xoauth2') and all required credentials are available it will XD> get the access_token and put it `:secret' (or basically my hacky XD> advice :) I think this isn't the right way to go. Currently xoauth2 is one of several supported SASL methods. The logic is supposed to be to try them in a certain order, but this hasn't worked properly for some time. Nobody has noticed since almost everyone uses only the basic method. In gnus there has always been a server variable, nnimap-authenticator, that chooses the preferred sasl method, which is how the current support for xaouth2 is designed to work. I think this is the right way to handle this (rather than relying on some specific form of the auth-source entry) but it would be good to fix the logic in nnimap.el to allow multiple methods to be tried. [...] XD> P.S. Is your set up mentioned in Bug#72358 still working for XD> outlook.com emails? After reaching out to an MS representative XD> they mentioned that token refresh was disabled[3] for XD> outlook.com so I just gave up. Maybe it still works for Outlook XD> Org emails? Yes, it still works perfectly. I suspect that the information they gave you isn't fully accurate :) -- Andrew Cohen