From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id Mu2RIuzPpGAXoQAAgWs5BA (envelope-from ) for ; Wed, 19 May 2021 10:44:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uHe3HezPpGARPgAAB5/wlQ (envelope-from ) for ; Wed, 19 May 2021 08:44:28 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E1DCC22705 for ; Wed, 19 May 2021 10:44:27 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id E3CCD2AA67; Wed, 19 May 2021 04:44:23 -0400 (EDT) Received: from lahtoruutu.iki.fi (lahtoruutu.iki.fi [IPv6:2a0b:5c81:1c1::37]) by mail.notmuchmail.org (Postfix) with ESMTPS id BBB652A9A9 for ; Wed, 19 May 2021 04:44:20 -0400 (EDT) Received: from guru.guru-group.fi (unknown [IPv6:2a02:2380:1:9:5054:ff:feb7:a4bc]) (using TLSv1.2 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: too) by lahtoruutu.iki.fi (Postfix) with ESMTPSA id 55EF81B002B6; Wed, 19 May 2021 11:44:15 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1621413855; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=its3j7OqNpeQ/VTVrT6vED+H3YWCGLx1tbuFRSe7TGY=; b=heWwms6KdVHI5prediwSE5OE78tV9XhCdTeZR27HDD1GROcgP3O/OUW2d6IiXVw3+hwoaH 9C6vWVTjQXroEvy39M3J4h3v7TswJiavNrmGe3xby2AMDKZ0lk8cVSggpDgKHz099x0BIQ OZYFNffD1aexIZ9hDEIKbk4BqtWn+cmAQ/V4Fl8RtITcNALIE5SHdpF+87S4xFIhipQHsF dB82dC3SlVdpihvnfeoY1gliPzQ+AIp4eePWI3ibYiLYeDQ61wts/BZMH4G2pxdmTh0bUW V360NNJDN1zBulJZJE2MVvVT6OKuRtDFDQ8/RSsEFDJjac7Ju3RssiZtBVHkQg== From: Tomi Ollila To: Felipe Contreras Subject: Re: [PATCH v3] test: replace notmuch_passwd_sanitize() with _libconfig_sanitize() In-Reply-To: References: <20210518055443.21964-1-tomi.ollila@iki.fi> User-Agent: Notmuch/0.32+21~gfdddd4c (https://notmuchmail.org) Emacs/27.1 X-Face: HhBM'cA~ MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1621413855; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=its3j7OqNpeQ/VTVrT6vED+H3YWCGLx1tbuFRSe7TGY=; b=gxHFWPTuwJbbXtpfaudSo1bhOxny9smAHjAGk3H9IzuAi2zU1eV8ti1jjRILrJraZWC1A5 A9s6gd8AqIg6yvC6fr4zIECeKrE/ansmIKAavm1rs133QVCn/O2flw5DH73e5YqeKclWYD qubo9yrfyHR1D1DT9ybdzkwRDkcCnbu9tBdmCmD0Ss4qoJCYD4UoWkaN1uls0nKMRZRxrK JgIi8FJKNb1ZPVD8kNA3vazfneqoqaWoXKKG+klNZdNUoVWEvGQXNcxBYQmLFf3cGXFyoh ljE+XZNA963lFMMnOdTB24LUNGcJxOotOEauXEJiEe8HtWQQmzsqhJoMK+OLIw== ARC-Authentication-Results: i=1; ORIGINATING; auth=pass smtp.auth=too smtp.mailfrom=tomi.ollila@iki.fi ARC-Seal: i=1; s=lahtoruutu; d=iki.fi; t=1621413855; a=rsa-sha256; cv=none; b=hR2s+k1z70KY2QRjT8Jw5+V6JgQto6liDyh8AV81pk+VlOMMR2AzKYEmLccQ5uZGbAYjec Tvb0KlM3qg5DVqvIKrfWg1YS+z/IJaHL2ik5yeUZWgju575LFslPiZP/A55vCOopTBabbI FrQOPHZts9A3qf+kj4ILauyhZE8l7Heal81D9rBc2EFIN+Z+gnMKv46aoBTzlvrkuo8o+P UWwUvSg3nApXk0B/8xREljSPhtmJ58aCGogaYTiXiMjt3YJMfQXH8+XverdM5okFJ5xs+y k/6ZYG7zWjLXRFYDYmNLO38U/qNX8+l487hyGtvD1hg54LnXd6yWYWgGSWmNTg== Message-ID-Hash: EA24HCI4WP5GVUZJ2YAVP5UMR2NVLTUU X-Message-ID-Hash: EA24HCI4WP5GVUZJ2YAVP5UMR2NVLTUU X-MailFrom: tomi.ollila@iki.fi X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: "notmuch@notmuchmail.org" X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1621413868; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=ylFaAuyG09NL20dn8Cp/uo41UR0ly8+uVACiOfj3Bn0=; b=S/LH5MC8w7gT6UYuBrYpsCmfQDtvZvLBd7Ia1t376qFWMUERQ+Dbj/FppqlhsjLCV58NCe +F6lMUZv6KEkvFP6A6pRbDBuWGL1+bHzpNCk2dPKsRQho/6YedgUUHIC1LsqQt93c6gprP aweBm9IRCSJ9eh4njQl6g60MC8AahHk/TwXJCDk6/YGeU/tT8srwmtnc43PiG0IECJ/27/ OekusHE3ADZVwja9iSEyDp7gjdkFDi809UpB8CbhNr/hZM4eSJnqBpRKRq9mqtLGwYf//n 3mgGXsspihUdqUAAkyPF13FEXE6DoQZSDyW0oHmg9vpUCc44X2BcOXJeN+x1Fg== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1621413868; a=rsa-sha256; cv=fail; b=XzMtPz/+Z7TUPETDNmBxX0OjmwJ+ylitYS5WOqChxCJmDPpKCz/aL2r2ZQVeBp0xLb4+23 tAY70vEaOOmaGhUpe+IBjMZ6CnQlpvA78sPwbNkmHuNvKZBrS+YS5CzKtxkRRjXbBcvkvB pQ+jEErQQzmPxccJNndh3hHXQ3i1myfy/iyZdrTZGwEjkWbxLV2MFcBqWUPAD2eIwuaQaM slRtZ8nj547ZmQRCGk5Hg14PTof8//g2MH14VJ1x/rQpy1imFPyY9SbcJlrvpel05qfWgI PZciQUQwGZwnXPO4MhX+rm6xWaRb0XfRQP5Cx2c8vz50gFM4Rr9yFzdeLfexGQ== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=heWwms6K; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 0.93 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=heWwms6K; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: E1DCC22705 X-Spam-Score: 0.93 X-Migadu-Scanner: scn0.migadu.com X-TUID: /vzwp1AcNHDz On Wed, May 19 2021, Felipe Contreras wrote: > On Tue, May 18, 2021 at 12:55 AM Tomi Ollila wrote: >> >> notmuch_passwd_sanitize() in test-lib.sh is too generic, it cannot >> work in many cases... >> >> The more specific version _libconfig_sanitize() replaces it in >> T590-libconfig.sh and the code that uses it is modified to output >> the keys (ascending numbers printed in hex) so the sanitizer knows >> what to sanitize in which lines... >> >> "@" + fqdn -> "@FQDN" replacement is used as fqdn could -- >> in theory -- be substring of 'USERNAME'. >> >> 'user -> 'USER_FULL_NAME replacement to work in cases where user >> is empty -- as only first ' is replaced that works as expected. >> >> In addition to ".(none)" now also ".localdomain" is filtered from >> USERNAME@FQDN. >> --- >> >> Changes to [v2]: >> >> * work in cases of empty user (e.g. in passwd gecos field) >> * replace only 1st match; e.g. fqdn could contain substring of user >> >> v2: id:20210517193315.11343-1-tomi.ollila@iki.fi >> v1: id:20210502181535.31292-1-tomi.ollila@iki.fi >> >> When tried w/ one replacement and w/o sq usage and emptied gecos, got >> >> . @@ -9,5 +9,5 @@ >> . 7: 'true' >> . 8: 'USERNAME@FQDN' >> . 9: 'NULL' >> . -a: 'USER_FULL_NAME' >> . +USER_FULL_NAMEa: '' >> >> test/T590-libconfig.sh | 97 +++++++++++++++++++++++++----------------- >> test/test-lib.sh | 20 --------- >> 2 files changed, 59 insertions(+), 58 deletions(-) >> >> diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh >> index 745e1bb4..42cbe6e0 100755 >> --- a/test/T590-libconfig.sh >> +++ b/test/T590-libconfig.sh >> @@ -5,6 +5,26 @@ test_description="library config API" >> >> add_email_corpus >> >> +_libconfig_sanitize() { >> + ${NOTMUCH_PYTHON} -c ' >> +import os, sys, pwd, socket > > Why not use a heredoc? > > python <<-EOF > .. > EOF tldr: I'll post change to use heredoc. Probably my bias against heredoc's when there are alternatives -- although this is much more tolerable than cat <;) ) While testing this option I looked (once again) how dash and bash do heredocs (in linux) (just to update my knowledge): dash creates pipe and dup2's fd[0] to 0 (that makes stdin not seekable) bash clones subprocess; in subprocess it creates temporary file, writes data there, closes it, opens it for reading, unlinks it from fs (could be problematic on windows), dup2()'s it to stdin, closes the dupped fd and finally execve's python zsh works like bash (i.e. these 2 provide seekable stdin) $ (strace -f -ofile zsh heredoc-test.sh) Tomi > >> +pw = pwd.getpwuid(os.getuid()) >> +user = pw.pw_name >> +name = pw.pw_gecos.partition(",")[0] >> +fqdn = socket.getaddrinfo(socket.gethostname(), 0, 0, >> + socket.SOCK_STREAM, 0, socket.AI_CANONNAME)[0][3] >> +for l in sys.stdin: >> + if l[:3] == "8: ": >> + l = l.replace(user, "USERNAME", 1).replace("@" + fqdn, "@FQDN", 1) >> + l = l.replace(".(none)", "", 1).replace(".localdomain", "", 1) >> + elif l[:3] == "a: ": >> + sq = chr(39) # single quote >> + l = l.replace(sq + name, sq + "USER_FULL_NAME", 1) > > Then we can simply do: > > l.replace("'" + name, "'USER_FULL_NAME", 1) > > The rest looks fine to me. > > -- > Felipe Contreras