bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

[Thomas Klausner]

Hi!

Since September 8, I can't compile emacs 25.0.50 from git.
It fails with:

In toplevel form:
leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "�<9E>�<8B>/�<96>��<97>/�<80>� 信", 10

I'm on NetBSD-7.99.22/amd64 with all dependencies from pkgsrc.

If this is the wrong way to report this or if you need more details,
please let me know!

Thanks,
 Thomas

bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

[Eli Zaretskii]

> Date: Mon, 14 Sep 2015 14:40:34 +0200
> From: Thomas Klausner <tk@giga.or.at>
> 
> Since September 8, I can't compile emacs 25.0.50 from git.
> It fails with:
> 
> In toplevel form:
> leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "�<9E>�<8B>/�<96>��<97>/�<80>� 信", 10

I have no problem compiling that file.

Did you try "make bootstrap"?

bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

[Thomas Klausner]

On Mon, Sep 14, 2015 at 03:47:22PM +0300, Eli Zaretskii wrote:
> > Date: Mon, 14 Sep 2015 14:40:34 +0200
> > From: Thomas Klausner <tk@giga.or.at>
> > 
> > Since September 8, I can't compile emacs 25.0.50 from git.
> > It fails with:
> > 
> > In toplevel form:
> > leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "�<9E>�<8B>/�<96>��<97>/�<80>� 信", 10
> 
> I have no problem compiling that file.
> 
> Did you try "make bootstrap"?

I build from a fresh git checkout.

I just tried again, I can reproduce it in a pbulk (basically, building
lots of programs in a chroot), but not when manually building in the
same environment. Weird, I don't understand this. It's always after
printing the "20000" line for that file.

Do you have any ideas what could cause this?
 Thomas

bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

[Eli Zaretskii]

> Date: Mon, 14 Sep 2015 17:58:29 +0200
> From: Thomas Klausner <tk@giga.or.at>
> Cc: 21474@debbugs.gnu.org
> 
> > I have no problem compiling that file.
> > 
> > Did you try "make bootstrap"?
> 
> I build from a fresh git checkout.
> 
> I just tried again, I can reproduce it in a pbulk (basically, building
> lots of programs in a chroot), but not when manually building in the
> same environment. Weird, I don't understand this. It's always after
> printing the "20000" line for that file.
> 
> Do you have any ideas what could cause this?

Sorry, no.

bug#21474: emacs-25.0.50: compilation error in leim: Args out of range

[Thomas Klausner]

I haven't seen this for a few days now, so whatever it was, it seems
gone now. You can close his bug report. Thank you!
 Thomas

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Rob Browning]

For what it's worth, I see a similar error here in a Debian unstable
sbuild chroot, i.e.:

    ELC      leim/quail/TONEPY.elc
    ELC      leim/quail/ZIRANMA.elc
    ELC      leim/quail/ZOZY.elc
  20000 entries
    ELC      leim/quail/arabic.elc
    ELC      leim/quail/croatian.elc
    ELC      leim/quail/cyril-jis.elc

  In toplevel form:
  leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "神事/新字/進 信", 10
  Makefile:272: recipe for target 'leim/ja-dic/ja-dic.elc' failed
  make[4]: *** [leim/ja-dic/ja-dic.elc] Error 1
  make[4]: *** Waiting for unfinished jobs....
  make[4]: Leaving directory '/<<BUILDDIR>>/emacs25-25.1+1/debian/build-x/lisp'
  Makefile:295: recipe for target 'compile-main' failed
  make[3]: *** [compile-main] Error 2
  make[3]: Leaving directory '/<<BUILDDIR>>/emacs25-25.1+1/debian/build-x/lisp'
  Makefile:387: recipe for target 'lisp' failed
  make[2]: *** [lisp] Error 2
  make[2]: Leaving directory '/<<BUILDDIR>>/emacs25-25.1+1/debian/build-x'
  debian/rules:360: recipe for target 'override_dh_auto_build' failed
  make[1]: *** [override_dh_auto_build] Error 2
  make[1]: Leaving directory '/<<BUILDDIR>>/emacs25-25.1+1'
  debian/rules:225: recipe for target 'build' failed
  make: *** [build] Error 2
  dpkg-buildpackage: error: debian/rules build gave error exit status 2

I'd be happy to try to diagnose this further, if there's anything you'd
like me to try.  Though it's not easy to trigger.

And I actually I hit this while trying to track down another
intermittent failure (that may or may not also only happen in a chroot),
and which may also cause Emacs 25 to be omitted from the next Debian
stable release if I can't fix it soon:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842728

I was planning to raise that issue on emacs-devel after I'd tried to
reproduce it myself a bit more.  I haven't yet, but someone else (today)
was just able to reproduce it in a similar environment.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Noam Postavsky]

On Fri, Nov 25, 2016 at 5:23 PM, Rob Browning <rlb@defaultvalue.org> wrote:
>
> For what it's worth, I see a similar error here in a Debian unstable
> sbuild chroot, i.e.:
>
>     ELC      leim/quail/TONEPY.elc
>     ELC      leim/quail/ZIRANMA.elc
>     ELC      leim/quail/ZOZY.elc
>   20000 entries
>     ELC      leim/quail/arabic.elc
>     ELC      leim/quail/croatian.elc
>     ELC      leim/quail/cyril-jis.elc
>
>   In toplevel form:
>   leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "神事/新字/進 信", 10
[...]
>
> I'd be happy to try to diagnose this further, if there's anything you'd
> like me to try.  Though it's not easy to trigger.
>
> And I actually I hit this while trying to track down another
> intermittent failure (that may or may not also only happen in a chroot),
> and which may also cause Emacs 25 to be omitted from the next Debian
> stable release if I can't fix it soon:
>
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842728

Those errors look similar to
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24358#123, does
configuring with REL_ALLOC=no help?

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Rob Browning]

Rob Browning <rlb@defaultvalue.org> writes:

> For what it's worth, I see a similar error here in a Debian unstable
> sbuild chroot, i.e.:

Oh, and apologies -- of course I should have said, this was for an
sbuild of the Debian 25.1+1-2 tree.

I'll probably try to reproduce it with the unadulterated upstream 25.1
tree next.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Fri, 25 Nov 2016 17:34:24 -0500
> Cc: 21474@debbugs.gnu.org
> 
> >   In toplevel form:
> >   leim/ja-dic/ja-dic.el:16979:1:Error: Args out of range: "神事/新字/進 信", 10
> [...]
> >
> > I'd be happy to try to diagnose this further, if there's anything you'd
> > like me to try.  Though it's not easy to trigger.
> >
> > And I actually I hit this while trying to track down another
> > intermittent failure (that may or may not also only happen in a chroot),
> > and which may also cause Emacs 25 to be omitted from the next Debian
> > stable release if I can't fix it soon:
> >
> >   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=842728
> 
> Those errors look similar to
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24358#123, does
> configuring with REL_ALLOC=no help?

But those problems should have been solved, even without disabling
REL_ALLOC, if Rob is building the current emacs-25 branch.  Right?

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Eli Zaretskii]

> From: Rob Browning <rlb@defaultvalue.org>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Fri, 25 Nov 2016 16:45:09 -0600
> 
> Oh, and apologies -- of course I should have said, this was for an
> sbuild of the Debian 25.1+1-2 tree.

What does this mean?  Is that the emacs-25 branch of the Emacs Git
repository?  Or is that something else?  If the latter, please tell
more details about its relation to the upstream sources, otherwise we
have no idea whether the problem you see was already fixed or not.  I
think it was already fixed, but that's just a guess.

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Rob Browning]

Eli Zaretskii <eliz@gnu.org> writes:

> But those problems should have been solved, even without disabling
> REL_ALLOC, if Rob is building the current emacs-25 branch.  Right?

Well, Debian's on 25.1 (very few additional patches so far), so not the
tip of emacs-25.  But thanks to both of your help, and after reading
some of the related bugs, I looked over the changes since 25.1, and saw
what I think are the commits in question, commits that appear relevant
to this problem, and maybe to the other one.

But backing up -- with respect to Debian's version, the current Debian
package (25.1+1-2) should be the deb/emacs25/v/25.1+1-2 tag here:

  https://anonscm.debian.org/cgit/users/rlb/emacs.git/tag/?h=deb/emacs25/v/25.1%2b1-2

Which is effectively the 25.1 tree, after making adjustments for
Debian's DFSG issue, and applying these patches:

  https://anonscm.debian.org/cgit/users/rlb/emacs.git/tree/debian/patches?h=deb/emacs25/v/25.1%2b1-2

(Those patches are both in that directory, *and* applied to the working
tree you get if you check out that tag.)

That tree is the one that's been intermittently crashing, and I think
the only patch it has from upstream after 25.1 is the nopie fix.

In any case, after poking around in the newer emacs-25 commits (as
mentioned above), it seemed like applying these allocation related
patches might help:

  b6998ea * src/regex.h (re_match_object): Improve commentary.
  ad66b3f Fix handling of allocation in regex matching
  f6134bb Port to GCC 6.2.1 + --enable-gcc-warnings
  c2a1792 * src/regex.c (re_search_2): Make new code safe for -Wjump-misses-init.
  ee04aed Fix handling of buffer relocation in regex.c functions
  7bb5c4f Port --enable-gcc-warnings to bleeding-edge glibc
  1047496 Another fix for using pointer to buffer text
  96ac0c3 Yet another fix for using pointers into buffer text
  32827b3 Default REL_ALLOC to 'no'
  fee4cef Revert fixes to allocation of regex matching
  43986d1 Inhibit buffer relocation during regex searches
  0221b7a Mark relocation workarounds with REL_ALLOC

And indeed, with those, I've yet to see a crash, but as we know, they're
intermittent.  To help increase confidence, two others who have been
able to produce the crash are probably also going to test that tree
soon.

Of course, if y'all have some other way you'd like to see this handled,
I'd be more than happy to try to accommodate.

Thanks again
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Noam Postavsky]

On Sat, Nov 26, 2016 at 4:17 PM, Rob Browning <rlb@defaultvalue.org> wrote:
>
> In any case, after poking around in the newer emacs-25 commits (as
> mentioned above), it seemed like applying these allocation related
> patches might help:
>
>   b6998ea * src/regex.h (re_match_object): Improve commentary.
>   ad66b3f Fix handling of allocation in regex matching
>   f6134bb Port to GCC 6.2.1 + --enable-gcc-warnings
>   c2a1792 * src/regex.c (re_search_2): Make new code safe for -Wjump-misses-init.
>   ee04aed Fix handling of buffer relocation in regex.c functions
>   7bb5c4f Port --enable-gcc-warnings to bleeding-edge glibc
>   1047496 Another fix for using pointer to buffer text
>   96ac0c3 Yet another fix for using pointers into buffer text
>   32827b3 Default REL_ALLOC to 'no'
>   fee4cef Revert fixes to allocation of regex matching
>   43986d1 Inhibit buffer relocation during regex searches
>   0221b7a Mark relocation workarounds with REL_ALLOC

You could probably trim a bit, e.g. just the "Default REL_ALLOC to
'no'" by itself should work, or a minimal set with REL_ALLOC=yes
should be listed at
https://lists.gnu.org/archive/html/emacs-devel/2016-10/msg00971.html.

>
> And indeed, with those, I've yet to see a crash, but as we know, they're
> intermittent.  To help increase confidence, two others who have been
> able to produce the crash are probably also going to test that tree
> soon.

Doing  'make extraclean && ./configure && make' seemed to be fairly
reliable way of reproducing it (if configured with GTK).

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Rob Browning]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> You could probably trim a bit, e.g. just the "Default REL_ALLOC to
> 'no'" by itself should work, or a minimal set with REL_ALLOC=yes
> should be listed at
> https://lists.gnu.org/archive/html/emacs-devel/2016-10/msg00971.html.

OK, well I'm happy to try to follow upstream preferences here if there
are any.

I'd imagined the set I listed might make any future cherry-picks easier
since it hopefully leaves the tree much like current upstream for those
files, but it could certainly be possible that minimizing the changes is
preferable.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Noam Postavsky]

On Sat, Nov 26, 2016 at 6:15 PM, Rob Browning <rlb@defaultvalue.org> wrote:
> Noam Postavsky <npostavs@users.sourceforge.net> writes:
>
>> You could probably trim a bit, e.g. just the "Default REL_ALLOC to
>> 'no'" by itself should work, or a minimal set with REL_ALLOC=yes
>> should be listed at
>> https://lists.gnu.org/archive/html/emacs-devel/2016-10/msg00971.html.
>
> OK, well I'm happy to try to follow upstream preferences here if there
> are any.
>
> I'd imagined the set I listed might make any future cherry-picks easier
> since it hopefully leaves the tree much like current upstream for those
> files, but it could certainly be possible that minimizing the changes is
> preferable.

Current upstream will be released as 25.2 fairly soon, so there
shouldn't be too many future cherry-picks anyway. I thought maybe on
your end it would be easier to have a smaller patch set, or even just
configure with REL_ALLOC=no.

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Rob Browning]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> Current upstream will be released as 25.2 fairly soon, so there
> shouldn't be too many future cherry-picks anyway. I thought maybe on
> your end it would be easier to have a smaller patch set, or even just
> configure with REL_ALLOC=no.

OK, well I think we may have until around the end of December (with
respect to making the next Debian stable release), so if 25.2 comes out
at least a bit before then, I can try to get it included.

And just to make sure I understand, you're saying that we might be OK in
the short term if I just export REL_ALLOC=no for the build?

If so, and assuming that fixes both of the intermittent failures, that
probably would be preferable for now.  At the moment, we'd just like to
upload a fix to prevent emacs25 from being removed from unstable in a
week or two.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

bug#21474: emacs-25.0.50: compilation error in leim: Args out of

[Noam Postavsky]

On Sat, Nov 26, 2016 at 10:16 PM, Rob Browning <rlb@defaultvalue.org> wrote:
>
> And just to make sure I understand, you're saying that we might be OK in
> the short term if I just export REL_ALLOC=no for the build?

For ./configure, yes.

Essentially the problem is that recent changes in glibc caused the
default configuration to use REL_ALLOC=yes, but since it hadn't been
used in a long time, several bugs had crept in. All the patches
(except 32827b3 "Default REL_ALLOC to 'no'", of course) are fixing
things in the REL_ALLOC=yes case.