From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Subject: bug#35350: Some compile output still leaks through with --verbosity=1 Date: Sat, 04 May 2019 11:33:51 +0200 Message-ID: <87r29e5zsw.fsf@gnu.org> References: <87mukkfd2j.fsf@netris.org> <87r29v2jz2.fsf@gnu.org> <87ftq9silk.fsf@netris.org> <87imv5jai5.fsf@gnu.org> <87k1fgh9c0.fsf@netris.org> <874l6jh0bx.fsf@gnu.org> <87imuvme7g.fsf@netris.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([209.51.188.92]:52816) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hMr4U-0003cW-M4 for bug-guix@gnu.org; Sat, 04 May 2019 05:35:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hMr4S-0006pr-PA for bug-guix@gnu.org; Sat, 04 May 2019 05:35:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36845) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hMr4S-0006pT-6W for bug-guix@gnu.org; Sat, 04 May 2019 05:35:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hMr4Q-0001n4-Ck for bug-guix@gnu.org; Sat, 04 May 2019 05:35:03 -0400 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87imuvme7g.fsf@netris.org> (Mark H. Weaver's message of "Tue, 30 Apr 2019 16:26:32 -0400") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Mark H Weaver Cc: 35350@debbugs.gnu.org Hi Mark, Mark H Weaver skribis: > Ludovic Court=C3=A8s writes: [...] >> So there are two things. To fix the issue you reported (build output >> that goes through), I think we must simply turn off UTF-8 decoding from >> =E2=80=98process-stderr=E2=80=99 and leave that entirely to =E2=80=98bui= ld-event-output-port=E2=80=99. > > Can we assume that UTF-8 is the appropriate encoding for > (current-build-output-port)? My interpretation of the Guix manual entry > for 'current-build-output-port' suggests that the answer should be "no". What goes to =E2=80=98current-build-output-port=E2=80=99 comes from builds = processes. It=E2=80=99s usually UTF-8 but it can be anything, including binary garbage, which should be gracefully handled. That=E2=80=99s why =E2=80=98process-stderr=E2=80=99 currently uses =E2=80= =98read-maybe-utf8-string=E2=80=99. > Also, in your previous message you wrote: > > The problem is the first layer of UTF-8 decoding that happens in > =E2=80=98process-stderr=E2=80=99, in the =E2=80=98%stderr-next=E2=80=99= case. We would need to > disable it, but only if the build output port is > =E2=80=98build-event-output-port=E2=80=99 (i.e., it=E2=80=99s capable o= f interpreting > =E2=80=9Cmultiplexed build output=E2=80=9D correctly.) > > It sounds like you're suggesting that 'process-stderr' should look to > see if (current-build-output-port) is a 'build-event-output-port', and > in that case it should use binary I/O primitives to write raw binary > data to it, otherwise it should use text I/O primitives and write > characters to it. Do I understand correctly? Yes. (Actually, rather than guessing if (current-build-output-port) is a =E2=80=98build-event-output-port=E2=80=99, there could be a fluid to ask = for the use of raw binary primitives.) > IMO, it would be cleaner to treat 'build-event-output-port' uniformly, > and specifically as a textual port of unknown encoding. (You mean =E2=80=98current-build-output-port=E2=80=99, right?) I think you=E2=80=99re right. I=E2=80=99m not yet entirely sure what the i= mplications are. There=E2=80=99s a couple of tests in tests/store.scm for UTF-8 interpretation that describe behavior that I think we should preserve. > I would suggest changing 'build-event-output-port' to create an R6RS > custom *textual* output port, so that it wouldn't have to worry about > encodings at all, and it would only be given whole characters. > Internally, it would be doing exactly what you suggest above, but those > details would be encapsulated within the custom textual port. > > However, I don't think we can use Guile's current implementation of R6RS > custom textual output ports, which are currently built on Guile's legacy > soft ports, which I suspect have a similar bug with multibyte characters > sometimes being split (see 'soft_port_write' in vports.c). > > Having said all of this, my suggestions would ultimately entail having > two separate places along the stderr pipeline where 'utf8->string!' > would be used, and maybe that's too much until we have a more optimized > C implementation of it. Yeah it looks like we don=E2=80=99t yet have custom textual output ports th= at we could rely on, do we? I support your work to add that in Guile proper! Thanks, Ludo=E2=80=99.