From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id sMmuE9uTQ2N8iQAAbAwnHQ (envelope-from ) for ; Mon, 10 Oct 2022 05:39:07 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 8PqrE9uTQ2OFZwEA9RJhRA (envelope-from ) for ; Mon, 10 Oct 2022 05:39:07 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C4C8F19C9F for ; Mon, 10 Oct 2022 05:39:06 +0200 (CEST) Received: from localhost ([::1]:33830 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ohjdB-0007Id-IJ for larch@yhetil.org; Sun, 09 Oct 2022 23:39:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58854) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ohjd8-0007IN-CB for bug-guix@gnu.org; Sun, 09 Oct 2022 23:39:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46195) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ohjd8-0008Aa-0p for bug-guix@gnu.org; Sun, 09 Oct 2022 23:39:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ohjd7-0007No-PU for bug-guix@gnu.org; Sun, 09 Oct 2022 23:39:01 -0400 Subject: bug#30265: Fish embeds store file names in UCS-4/UTF-32 literal strings Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-To: bug-guix@gnu.org Resent-Date: Mon, 10 Oct 2022 03:39:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: cc-closed 30265 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mark H Weaver Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , John Soo , 30265-done@debbugs.gnu.org Mail-Followup-To: 30265@debbugs.gnu.org, maxim.cournoyer@gmail.com, meiyo.peng@gmail.com Received: via spool by 30265-done@debbugs.gnu.org id=D30265.166537313328363 (code D ref 30265); Mon, 10 Oct 2022 03:39:01 +0000 Received: (at 30265-done) by debbugs.gnu.org; 10 Oct 2022 03:38:53 +0000 Received: from localhost ([127.0.0.1]:45271 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohjcy-0007NN-Hd for submit@debbugs.gnu.org; Sun, 09 Oct 2022 23:38:52 -0400 Received: from mail-qk1-f176.google.com ([209.85.222.176]:35748) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohjcw-0007NB-Fv for 30265-done@debbugs.gnu.org; Sun, 09 Oct 2022 23:38:50 -0400 Received: by mail-qk1-f176.google.com with SMTP id x25so2410503qki.2 for <30265-done@debbugs.gnu.org>; Sun, 09 Oct 2022 20:38:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=UaJqpwyPHP9ag+p6nisKxYqt/OdZhwKfLAYjvMnHJlg=; b=QWXOBVM49zV7KO4aRoQqiq58BSM1pLU4VTDBfDI5CAEsWe0l1sxWCRlO8UsPujYLun WWDXvzbsEReZf5/bnZIivlAy2U6o2f61+jNe3EjOXMLV0FlR8oBZR8pYngBt0TreRrjX WbegrDDUhsmjQA5dhI/R58C5QmvSIY3lb7Rw5n+biZOTuq2Bl08EvSxfaMn3u+xiz/nA Cu6B8UzlHsRHV6Gm/fcZ2JWLKy1arGj9jrHx5bfCMR/Ye39J+UM+uFEaD0adY4mHmBru On33aasucljhdcu1NV2BnUEpZibAW4HAvRrn3gEgF8dTbpX2wZgB2Q0Agyc9osm+LWg2 +jGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:user-agent:message-id:in-reply-to:date:references :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UaJqpwyPHP9ag+p6nisKxYqt/OdZhwKfLAYjvMnHJlg=; b=0FMcLmYRBXNlbp2dexAN6/gARPhk9PoOLgvw0mvZLkn3y02GeC84oFNC9ktzm+jG3h FwQo1ssYULkacOKpls+4RrEJVfUGNMjVCTgqQGrxJ9Tw/4zB2JHz0K75xxTE5F0sioYD E8YvF4ozy+ycKp5h1aeSF0DlTRMq1TbmKpmz52giWkxs6Xwx0KMaL7sk9InP+bM+AsPa LgZuj0z0AyeCuNuZgoAaOsQVtPgovcWxrXKazHwXMGv27YmVSTwvzxouvB4u6YYKH15+ QEj5Af2d3zaABwvhFWZ1yG8EPKZHxwMIwuB9YjWcU6cgUOEVWUZjG83QmVUa8+AVeHfe gsmw== X-Gm-Message-State: ACrzQf0Po0fW3vXF6ysajbW2lYTZ3ZDJmAg5U8iw5XSa+HiDtv+BclmP jrawPirQKwGnksnZW/dTSvTckc6p5Yo= X-Google-Smtp-Source: AMsMyM5CEZzGSg02ykXVDNpANlISewW/5xNqRPr+fMzVqYxzvohlVY6xnN1V7jbMJfSdF3Ym3EvFTQ== X-Received: by 2002:a05:620a:208d:b0:6e9:d161:a9a8 with SMTP id e13-20020a05620a208d00b006e9d161a9a8mr8903620qka.650.1665373124820; Sun, 09 Oct 2022 20:38:44 -0700 (PDT) Received: from hurd (dsl-150-38.b2b2c.ca. [66.158.150.38]) by smtp.gmail.com with ESMTPSA id o15-20020a05620a2a0f00b006ce40fbb8f6sm9297729qkp.21.2022.10.09.20.38.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 09 Oct 2022 20:38:44 -0700 (PDT) From: Maxim Cournoyer References: <87inbnpsx9.fsf@gmail.com> <8B84EF65-4CCC-4CAF-A6E9-BF6F5566D11E@asu.edu> <871qrj5sj9.fsf@gmail.com> <87czb3uza5.fsf@netris.org> Date: Sun, 09 Oct 2022 23:38:43 -0400 In-Reply-To: <87czb3uza5.fsf@netris.org> (Mark H. Weaver's message of "Fri, 07 Oct 2022 16:57:43 -0400") Message-ID: <87fsfw2vr0.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1665373146; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-to:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=UaJqpwyPHP9ag+p6nisKxYqt/OdZhwKfLAYjvMnHJlg=; b=JA4OpVFiz+u3mqeGgHCc3ITXIcX65rjYWwn9iCQo7MkOsc0cktn8xks+jrrHvcXyOLwgqE bQ5Tn/IoWGagCIMfJaIv85QVV4Zf7VXCHYvz3V2Ykychq+MkwjgyQwR8TQ1E5nfASNOBaj +5qSWf6tnIo+xlmLM+fVF0LjeRiAVhp+GtIJqS5dp10vIu/xus0YG/pR1K0K+DF1aE/pB3 YPtymp/a0YZenxEHhn62rubYBpl88IxRsE5GJwZvwTpmlGV3M6j+8HZGHFw8jdY99lCOk0 iwZt03orYzxhgypdabSFDZGKXCCfEYMXjNpoCsvno9Jb0L1ht7nxiENk9sZc9Q== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1665373146; a=rsa-sha256; cv=none; b=HV+aMpNAbq+D7WHPYB7kyEJmqUrv0/7puSz0SalhTqPgQJNN6SaswC+gTwIvh3gwDicxbw yGQ3m3V6lcdY68DlO67n7N5U0FxjJ/RdR8Cma68vXsBoSwa38p1M9wTRlHYUHb8HeFkMtm UEsPK3VjxW4CjZ+8PXb1xGhXD7W884lMIZc7K2MziU4TtsFhXw+jfU3LeIC0HXwMtGOz61 rNrqTrg1ptP6XWJA8+HrOUJ7xd7fn21vwhV7v8Rib6X2qxjcGWHi1YCWUsp726+WS8zAcB W/ameU3aGJai0ZLh5eEwkfgE0bX6KDQGbVopaRDG4Ug6Ta0fhp1O+8m4DmFUkQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=QWXOBVM4; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 7.62 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=QWXOBVM4; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: C4C8F19C9F X-Spam-Score: 7.62 X-Migadu-Scanner: scn1.migadu.com X-TUID: 6V0+QkIoa3Yh Hi, Mark H Weaver writes: > John Soo writes: >> I looked into it and I think a patch to fish might be required but I >> got buried in other work. > > Note that commit 1bab9b9f17256a9e4f45f5b0cceb8b52e0a1b1ed (April 2021) > added support in our grafting code to find and rewrite UTF-16 and UTF-32 > store references. That might have mitigated or even eliminated the > adverse effects of this bug. > > However, the Guix daemon's reference scanner still does not detect > UTF-16/32 references. This could be a problem if some store item is > reachable *only* via UTF-16/32 store references, because "guix gc" might > delete it even though it is still needed. > > However, if it is the case that every referenced store item is > represented in ASCII or UTF-8 at least once, everything should work. > Therefore, an easy workaround would be to add another phase that simply > creates a file in the output(s) that contains ASCII or UTF-8 references > to any needed store items. Working with what I see (the fish build outputs results), the only UCS-4 references (either big or small endian) it registered to the store via multi-byte encoded strings are: --8<---------------cut here---------------start------------->8--- $ strings -e L /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin/fish* | grep /gnu /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/share/doc/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/share/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/etc/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin strings -e B /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin/fish* | grep /gnu /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/share/doc/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/share/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/etc/fish /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin /gnu/store/qfy1rxm1vzd68y9jvcvq4zzz0cnbla8i-fish-3.5.1/bin --8<---------------cut here---------------end--------------->8--- No UCS-2 references are detected via 'strings'. Thanks for having shared the history and background. Closing. -- Maxim