From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ONEiHli+pGCqggAAgWs5BA (envelope-from ) for ; Wed, 19 May 2021 09:29:28 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id kFlmGVi+pGCsMQAA1q6Kng (envelope-from ) for ; Wed, 19 May 2021 07:29:28 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [IPv6:2607:5300:201:3100::1657]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 83F5821722 for ; Wed, 19 May 2021 09:29:27 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id C88602AA01; Wed, 19 May 2021 03:29:23 -0400 (EDT) Received: from mail-wm1-x32a.google.com (mail-wm1-x32a.google.com [IPv6:2a00:1450:4864:20::32a]) by mail.notmuchmail.org (Postfix) with ESMTPS id 9B6692A9E5 for ; Wed, 19 May 2021 03:29:20 -0400 (EDT) Received: by mail-wm1-x32a.google.com with SMTP id o127so6681625wmo.4 for ; Wed, 19 May 2021 00:29:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yOaF4cRD1L9miZy5W9m1QlOiy0oOm7kkkhetZlOMF9w=; b=hNhdAezMG1nTmFkIlJzkQ8YCmD23Qq0c6btXTQkeF4TMb2Ym86BF1xb6uIb62re6ig NAiKMK4NhEpuSMdXnziPLxPOhAlQwE2zyDUmEBJi/7HwgJ+tU0z21xXgJRA798J3Kq1d n6i8kiCjYYPurdiBlDl7GoLkrooa1VNXqccI1ggOMdBugRAX0RD3SThBYQTjWU3YqTRu k/m9isLKYfwH11V0XhEYpE+CoQhH8gf45fRi2MAcM5JOTJKrnpQJuLO8XsbPxt43JxR7 6WABKIlMlWbUHQNOvgEAv/3dAM/OqQiOcR9zBxYWTLh1dyIzp/u+X/Q4Fah9yiP+85lV KfgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yOaF4cRD1L9miZy5W9m1QlOiy0oOm7kkkhetZlOMF9w=; b=rQDR1X2wB4PXry+GQBcQnWXwdLnD79vXwAmBJh91kvYlEdOZzTWOuYhiWvYMfuB2Gg 4uYqksGlpNzFZWyYq6UYqGYCvtTLels14hEaP4YKOAxy0ncuB9g1JvDo/kvU+ohijqWn nS6SiJqMtLrFjyLoduYYLXwkzbNaasalPm6GeEIEvmFKJJBFplrrEl35R7wvkJwH3nQP u4eYagl+PvGLYHQhF93xyzRVv5siOjvvRZ7BFHVK3ME7d0YvjjwQ6Yum1oM+FJTpb8pU QWV5kfs1GlyuzNIWna7WnlcxFcm3EDJZIK5VRCKDaLl4vuk+jAVcYgZJXbBbTxdK/0pH yAFg== X-Gm-Message-State: AOAM533eNyFepx5nCHJwHzgM0aDduSR/HJZhcTIShJ6Ggiq7VvkW7enf Hp+VPKNhRdPWS80XyBlNZXOjiKOHPBEtH9UEHWEzU4q3o/9lsw== X-Google-Smtp-Source: ABdhPJxVbKI1cEPMOORSly+jhjPCiarHWnV1OQi4rQPaRQA4hGjJmhybY8hIptq3ib/uXjXqMMNGKhcsHrN7VoqPslM= X-Received: by 2002:a1c:20d2:: with SMTP id g201mr9627010wmg.48.1621409359432; Wed, 19 May 2021 00:29:19 -0700 (PDT) MIME-Version: 1.0 References: <20210518055443.21964-1-tomi.ollila@iki.fi> In-Reply-To: <20210518055443.21964-1-tomi.ollila@iki.fi> From: Felipe Contreras Date: Wed, 19 May 2021 02:29:08 -0500 Message-ID: Subject: Re: [PATCH v3] test: replace notmuch_passwd_sanitize() with _libconfig_sanitize() To: Tomi Ollila Message-ID-Hash: 3ZWMRWSRLABI3AYSU3OQCPIVHWBZWLE3 X-Message-ID-Hash: 3ZWMRWSRLABI3AYSU3OQCPIVHWBZWLE3 X-MailFrom: felipe.contreras@gmail.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: "notmuch@notmuchmail.org" X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1621409367; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=d/+jFVx//YY1HPYhIAzifkmsq8IeBd2zF07+0CyYaXE=; b=G8HTjehx8je48QP41PakKkgidlwP8PqUGujLIFXIMk80yMEUbOAolEQ3xyiuDc75gjGQQb ZA6SzIlFElp9TtRwwuhHiyuXfptT47nhFfvyfXS0pDHX/FpT++Za6/fUA35TD7vQVmjbxc 7A8kS9PUCxQmSyT4JftFTIKzRH8/R0rnt59gudcD1KEvXavQRgM0gEYaJRJwAiNxyFXXmb PtT7OWXXV5QnTh3VbP3w3jZhkcIwHZfjhlRq1RDNMCo2j6pQ6cZgKrcvISaYfAb+sSV5uH jkqKEQzSjAIcldhawC+NseL09FZzLx01NoR8WoCrTnwrEpD6z+tbGbtIUX5GGA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1621409367; a=rsa-sha256; cv=none; b=s0RIl25qhzhST0k00XHCXFbj2mVGshuegaazEdZ9yAeRWJl+YHxqMyMgdiTlBsMeduy0OW vF26wS8OjJecSqHlGk73EecHSgdx3Dk92b/vs+DIU0+XAcg0oXjOxftvgBLG1yo8xk0ekJ EhsFCUEmd2HiKGrn6q02FoNgTYAVRLnVhfJHa8t+4HkgY9lD9wwUahtYerCf1B8kU8gx8M yi7ng485qua5Cci8llRR0SYS8TRbnjvoMuNIMQPSKREE9JtQlazjNkjRA7G57fETmGMxDE 2osTCVofZ0Ba+0SbGYD9TefXM4xfImHElxM8EII/DuvQX/Q8p09mS3YqQfVZvg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20161025 header.b=hNhdAezM; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: -0.97 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20161025 header.b=hNhdAezM; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 2607:5300:201:3100::1657 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 83F5821722 X-Spam-Score: -0.97 X-Migadu-Scanner: scn0.migadu.com X-TUID: rkQPT5Hp04Yq On Tue, May 18, 2021 at 12:55 AM Tomi Ollila wrote: > > notmuch_passwd_sanitize() in test-lib.sh is too generic, it cannot > work in many cases... > > The more specific version _libconfig_sanitize() replaces it in > T590-libconfig.sh and the code that uses it is modified to output > the keys (ascending numbers printed in hex) so the sanitizer knows > what to sanitize in which lines... > > "@" + fqdn -> "@FQDN" replacement is used as fqdn could -- > in theory -- be substring of 'USERNAME'. > > 'user -> 'USER_FULL_NAME replacement to work in cases where user > is empty -- as only first ' is replaced that works as expected. > > In addition to ".(none)" now also ".localdomain" is filtered from > USERNAME@FQDN. > --- > > Changes to [v2]: > > * work in cases of empty user (e.g. in passwd gecos field) > * replace only 1st match; e.g. fqdn could contain substring of user > > v2: id:20210517193315.11343-1-tomi.ollila@iki.fi > v1: id:20210502181535.31292-1-tomi.ollila@iki.fi > > When tried w/ one replacement and w/o sq usage and emptied gecos, got > > . @@ -9,5 +9,5 @@ > . 7: 'true' > . 8: 'USERNAME@FQDN' > . 9: 'NULL' > . -a: 'USER_FULL_NAME' > . +USER_FULL_NAMEa: '' > > test/T590-libconfig.sh | 97 +++++++++++++++++++++++++----------------- > test/test-lib.sh | 20 --------- > 2 files changed, 59 insertions(+), 58 deletions(-) > > diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh > index 745e1bb4..42cbe6e0 100755 > --- a/test/T590-libconfig.sh > +++ b/test/T590-libconfig.sh > @@ -5,6 +5,26 @@ test_description="library config API" > > add_email_corpus > > +_libconfig_sanitize() { > + ${NOTMUCH_PYTHON} -c ' > +import os, sys, pwd, socket Why not use a heredoc? python <<-EOF .. EOF > +pw = pwd.getpwuid(os.getuid()) > +user = pw.pw_name > +name = pw.pw_gecos.partition(",")[0] > +fqdn = socket.getaddrinfo(socket.gethostname(), 0, 0, > + socket.SOCK_STREAM, 0, socket.AI_CANONNAME)[0][3] > +for l in sys.stdin: > + if l[:3] == "8: ": > + l = l.replace(user, "USERNAME", 1).replace("@" + fqdn, "@FQDN", 1) > + l = l.replace(".(none)", "", 1).replace(".localdomain", "", 1) > + elif l[:3] == "a: ": > + sq = chr(39) # single quote > + l = l.replace(sq + name, sq + "USER_FULL_NAME", 1) Then we can simply do: l.replace("'" + name, "'USER_FULL_NAME", 1) The rest looks fine to me. -- Felipe Contreras