bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Eli Zaretskii]

When an async native-compilation is running, some warning messages
emitted by the compilation are truncated.  Example:

  Warning (comp): seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of Disable showing Disable logging

Note the truncation after "(as of".

Why does this happen? can it be fixed?

In GNU Emacs 28.0.50 (build 1076, i686-pc-mingw32)
 of 2021-03-09 built on HOME-C4E4A596F7
Repository revision: 40d8f83e53ba64355035da78967c994d09a7802d
Repository branch: master
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int --with-modules
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XPM
ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win
w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads w32notify
w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 56671 11176)
 (symbols 48 7803 1)
 (strings 16 21563 1999)
 (string-bytes 1 626919)
 (vectors 16 13076)
 (vector-slots 8 172294 12605)
 (floats 8 23 106)
 (intervals 40 263 114)
 (buffers 888 10))

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> When an async native-compilation is running, some warning messages
> emitted by the compilation are truncated.  Example:
>
>   Warning (comp): seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of Disable showing Disable logging
>
> Note the truncation after "(as of".
>
> Why does this happen? can it be fixed?

ATM we capture each Warning (or error) with the following regexp:
"^.*+?\\(?:Error\\|Warning\\): .*$"

So I expect we match the full line (if is a multi line diagnostic we'll
loose what's following).

The matched string is passed directly to `display-warning'.  I don't
know if we have same machinery cutting lines that are considered too
long.

Thanks

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47024@debbugs.gnu.org
> Date: Tue, 09 Mar 2021 20:37:57 +0000
> 
> >   Warning (comp): seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of Disable showing Disable logging
> >
> > Note the truncation after "(as of".
> >
> > Why does this happen? can it be fixed?
> 
> ATM we capture each Warning (or error) with the following regexp:
> "^.*+?\\(?:Error\\|Warning\\): .*$"
> 
> So I expect we match the full line (if is a multi line diagnostic we'll
> loose what's following).

The message comes from a sub-process, right?  Could it be that we
match prematurely, when only part of the text was received?

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47024@debbugs.gnu.org
>> Date: Tue, 09 Mar 2021 20:37:57 +0000
>> 
>> >   Warning (comp): seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of Disable showing Disable logging
>> >
>> > Note the truncation after "(as of".
>> >
>> > Why does this happen? can it be fixed?
>> 
>> ATM we capture each Warning (or error) with the following regexp:
>> "^.*+?\\(?:Error\\|Warning\\): .*$"
>> 
>> So I expect we match the full line (if is a multi line diagnostic we'll
>> loose what's following).
>
> The message comes from a sub-process, right?

Yes

> Could it be that we
> match prematurely, when only part of the text was received?

That's possible, just before using the regexp we call
`accept-process-output' on the process, I thought this was sufficient to
handle the case but I'm no expert into this area.

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47024@debbugs.gnu.org
> Date: Wed, 10 Mar 2021 06:50:15 +0000
> 
> > The message comes from a sub-process, right?
> 
> Yes
> 
> > Could it be that we
> > match prematurely, when only part of the text was received?
> 
> That's possible, just before using the regexp we call
> `accept-process-output' on the process, I thought this was sufficient to
> handle the case but I'm no expert into this area.

I think I see the problem: it's nothing as complicated as that.

Here's what the original warning looks like:

 seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
     27.1); use `seq-contains-p' instead.

There's an actual newline at the end of the first line, because by
default we _fill_ the text.

So I think the solution should be to bind (in the sub-process which
performs the actual compilation) warning-fill-column to a large
number.

Is there a similar issue with error messages?

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47024@debbugs.gnu.org
>> Date: Wed, 10 Mar 2021 06:50:15 +0000
>> 
>> > The message comes from a sub-process, right?
>> 
>> Yes
>> 
>> > Could it be that we
>> > match prematurely, when only part of the text was received?
>> 
>> That's possible, just before using the regexp we call
>> `accept-process-output' on the process, I thought this was sufficient to
>> handle the case but I'm no expert into this area.
>
> I think I see the problem: it's nothing as complicated as that.
>
> Here's what the original warning looks like:
>
>  seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
>      27.1); use `seq-contains-p' instead.
>
> There's an actual newline at the end of the first line, because by
> default we _fill_ the text.
>
> So I think the solution should be to bind (in the sub-process which
> performs the actual compilation) warning-fill-column to a large
> number.

Nice, is 256 a reasonable number?

> Is there a similar issue with error messages?

Yes, we suffer from the same issue if the Error is emitted with new
lines.

Thanks

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: 47024@debbugs.gnu.org
> Date: Wed, 10 Mar 2021 13:12:59 +0000
> 
> >  seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
> >      27.1); use `seq-contains-p' instead.
> >
> > There's an actual newline at the end of the first line, because by
> > default we _fill_ the text.
> >
> > So I think the solution should be to bind (in the sub-process which
> > performs the actual compilation) warning-fill-column to a large
> > number.
> 
> Nice, is 256 a reasonable number?

How about most-positive-fixnum?

> > Is there a similar issue with error messages?
> 
> Yes, we suffer from the same issue if the Error is emitted with new
> lines.

But is there a similar variable to bind in that case? fill-column,
perhaps?

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: 47024@debbugs.gnu.org
>> Date: Wed, 10 Mar 2021 13:12:59 +0000
>> 
>> >  seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
>> >      27.1); use `seq-contains-p' instead.
>> >
>> > There's an actual newline at the end of the first line, because by
>> > default we _fill_ the text.
>> >
>> > So I think the solution should be to bind (in the sub-process which
>> > performs the actual compilation) warning-fill-column to a large
>> > number.
>> 
>> Nice, is 256 a reasonable number?
>
> How about most-positive-fixnum?

Sounds good, fe1c081c38 does that.

>> > Is there a similar issue with error messages?
>> 
>> Yes, we suffer from the same issue if the Error is emitted with new
>> lines.
>
> But is there a similar variable to bind in that case? fill-column,
> perhaps?

Dunno, I'll make some tests if we don't get a precise suggestion.

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: 47024@debbugs.gnu.org
>>> Date: Wed, 10 Mar 2021 13:12:59 +0000
>>>
>>> >  seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
>>> >      27.1); use `seq-contains-p' instead.
>>> >
>>> > There's an actual newline at the end of the first line, because by
>>> > default we _fill_ the text.
>>> >
>>> > So I think the solution should be to bind (in the sub-process which
>>> > performs the actual compilation) warning-fill-column to a large
>>> > number.
>>>
>>> Nice, is 256 a reasonable number?
>>
>> How about most-positive-fixnum?
>
> Sounds good, fe1c081c38 does that.
>
>>> > Is there a similar issue with error messages?
>>>
>>> Yes, we suffer from the same issue if the Error is emitted with new
>>> lines.
>>
>> But is there a similar variable to bind in that case? fill-column,
>> perhaps?
>
> Dunno, I'll make some tests if we don't get a precise suggestion.

With 0144764d1d error reporting appears to be working for me.

I had no problems with line truncations but I had to tweak the way we
are printing them (before we emitted always the full backtrace) so now
we can parse them and report to the user correctly.

My testcase is to signal and error with a very long message within an
`eval-when-compile' and I see it now reported entirely in the
"*Async-native-compile-log*".

Please let me know if you have some other suggestion on how to test
this.

Thanks

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Andrea Corallo via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Andrea Corallo <akrl@sdf.org> writes:

> Andrea Corallo via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs@gnu.org> writes:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: Andrea Corallo <akrl@sdf.org>
>>>> Cc: 47024@debbugs.gnu.org
>>>> Date: Wed, 10 Mar 2021 13:12:59 +0000
>>>>
>>>> >  seq.el:396:16: Warning: `seq-contains' is an obsolete generic function (as of
>>>> >      27.1); use `seq-contains-p' instead.
>>>> >
>>>> > There's an actual newline at the end of the first line, because by
>>>> > default we _fill_ the text.
>>>> >
>>>> > So I think the solution should be to bind (in the sub-process which
>>>> > performs the actual compilation) warning-fill-column to a large
>>>> > number.
>>>>
>>>> Nice, is 256 a reasonable number?
>>>
>>> How about most-positive-fixnum?
>>
>> Sounds good, fe1c081c38 does that.
>>
>>>> > Is there a similar issue with error messages?
>>>>
>>>> Yes, we suffer from the same issue if the Error is emitted with new
>>>> lines.
>>>
>>> But is there a similar variable to bind in that case? fill-column,
>>> perhaps?
>>
>> Dunno, I'll make some tests if we don't get a precise suggestion.
>
> With 0144764d1d error reporting appears to be working for me.
>
> I had no problems with line truncations but I had to tweak the way we
> are printing them (before we emitted always the full backtrace) so now
> we can parse them and report to the user correctly.
>
> My testcase is to signal and error with a very long message within an
> `eval-when-compile' and I see it now reported entirely in the
> "*Async-native-compile-log*".
>
> Please let me know if you have some other suggestion on how to test
> this.

Hi Eli,

is there anything left to be done for this bug?

TIA

  Andrea

bug#47024: 28.0.50; [feature/native-comp] Warnings from async compilations truncated

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: eliz@gnu.org
> Date: Fri, 19 Mar 2021 14:16:18 +0000
> 
> is there anything left to be done for this bug?

No, I think we are done here, thanks.  Closing.