From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.bugs Subject: bug#36447: 27.0.50; New "Unknown keyword" errors Date: Fri, 5 Jul 2019 09:09:13 +0000 Message-ID: References: <875zon7x0a.fsf@web.de> <8336jqgbhp.fsf@gnu.org> <87h886eoke.fsf@web.de> <87d0iu54d1.fsf@gmail.com> <87k1d14djr.fsf@web.de> <87h884fo0i.fsf@web.de> <85d0is5ry1.fsf@gmail.com> <87lfxdgs1k.fsf@web.de> <83y31capj1.fsf@gnu.org> <83tvc0anwi.fsf@gnu.org> <83r274an61.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="000000000000ebb7d3058ceb7431" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="266221"; mail-complaints-to="usenet@blaine.gmane.org" Cc: michael_heerdegen@web.de, npostavs@gmail.com, 36447@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 05 11:11:33 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hjKFf-001796-Th for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jul 2019 11:11:32 +0200 Original-Received: from localhost ([::1]:50802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjKFe-0002EB-SE for geb-bug-gnu-emacs@m.gmane.org; Fri, 05 Jul 2019 05:11:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:36791) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hjKFD-0002Dw-G3 for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 05:11:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hjKFC-0007oF-8f for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 05:11:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:43538) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hjKFC-0007ny-1X for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 05:11:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hjKFB-0005Ql-QL for bug-gnu-emacs@gnu.org; Fri, 05 Jul 2019 05:11:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Jul 2019 09:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36447 X-GNU-PR-Package: emacs Original-Received: via spool by 36447-submit@debbugs.gnu.org id=B36447.156231780120790 (code B ref 36447); Fri, 05 Jul 2019 09:11:01 +0000 Original-Received: (at 36447) by debbugs.gnu.org; 5 Jul 2019 09:10:01 +0000 Original-Received: from localhost ([127.0.0.1]:52359 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjKEC-0005P4-WD for submit@debbugs.gnu.org; Fri, 05 Jul 2019 05:10:01 -0400 Original-Received: from mail-ot1-f46.google.com ([209.85.210.46]:33859) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hjKE8-0005On-0y for 36447@debbugs.gnu.org; Fri, 05 Jul 2019 05:09:59 -0400 Original-Received: by mail-ot1-f46.google.com with SMTP id n5so8423130otk.1 for <36447@debbugs.gnu.org>; Fri, 05 Jul 2019 02:09:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kpawt80bX5TqtKrLuo66qu6GZbuvilOUrQ7uoPHtcx0=; b=ku5RUe13y1uLQKC7kqvKPFlyizmtZcfsO8VG/oTUynlZRJEvMzy11yRWsyxktBpr04 O5GaQDDRDqAoDrlCmxBmzch+qOfW9+Exekvt6dGr071hOZkeQw3ZByvTatI32ea7ecDU nRokroBLu5OD7FPDz8VLgiN7zmgcO3rnwHAUtJTEj7DjjDlN8J1gyq1uVsQuB21aZNs/ B1syOVrmqmrtKIHR+qZuyCBi0YLbRkrG4GOzdRa9zHF4oWsB2Q1Gy2ZH/sl1YA9eSkJo X0XvnLIQhwFp/Jre3DbwFMsXGGj7uBggMqXCvf+4fFjsP/rB+zgoNGsdvU8wSIuksfX/ KBQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kpawt80bX5TqtKrLuo66qu6GZbuvilOUrQ7uoPHtcx0=; b=hWNwZK97pB9yetP853wKme9GGKI4Rdf+p8tT1+k2NBcsj70EL8nongbRC3oDowFHOk 19cbFTLGIOmZdI3CNH477xG2HsfGFokGoMXIODJy5A4ZMrvCTchWY5tkbqkYtqDx820L l/rN0yErnHpPOSUsmoCzg9kKqbuO+2tWuGP9HtE3H4XfK7x54ZN9uaJIXhhLx341G4lR N30nPK67x3nFjCRiIH9dY8cZOuHFyb4NWD1HNz4+2W3yHn65CmKl3lrj/HZXGEy0VttG Wne09nCXMJvO5QTWiz6rKynrAouXmyHhJWtK8s/PGz2SlalSbHE6zElbpw0WEyV4VyW8 5ptA== X-Gm-Message-State: APjAAAXYAhoqH0jNQsmOtITXKgiqOKDAAbhepgkD1uQQS3UbxqlPppW5 assDOjC7I6Atty++eh9H1puBJScgDryXAKtrMwQ= X-Google-Smtp-Source: APXvYqzjmRdznUcS7Yg2GzqDTLD2Kg020xDVP8MjXuYmyvEkMM3ZBS9WqqlwPmbkb26ahYK2ZE0DCgj+kAbxbQWAnIM= X-Received: by 2002:a9d:7284:: with SMTP id t4mr2169947otj.154.1562317790329; Fri, 05 Jul 2019 02:09:50 -0700 (PDT) In-Reply-To: <83r274an61.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162112 Archived-At: --000000000000ebb7d3058ceb7431 Content-Type: text/plain; charset="UTF-8" On Fri, Jul 5, 2019 at 8:41 AM Eli Zaretskii wrote: > > > From: Pip Cet > > Date: Fri, 5 Jul 2019 08:36:57 +0000 > > Cc: michael_heerdegen@web.de, npostavs@gmail.com, 36447@debbugs.gnu.org > > > > > > I don't think we can sensibly add tests for this bug, but the fix I > > > > posted earlier still seems valid to me. > > > > > > Sorry, I'm not tracking this part of the discussion, as it lost me > > > long ago. > > > > What's the best way of getting this fixed? > > Sorry, I don't think I know what "this bug" is about, The bug: Building emacs with "-O0 -g3 -ggdb" on current Linux will result in binaries that sometimes, depending on the precise compiler version used, will fail weirdly if you evaluate this emacs -Q recipe line by line: (custom-handle-keyword nil :group nil nil) (y-or-n-p "prompt") (custom-handle-keyword nil :group nil nil) The error produced will be "unknown keyword :group", which is nonsensical as :group is indeed a valid keyword. The analysis: It's not the byte code, which is fine and looks like this: byte code for custom-handle-keyword: doc: For customization option SYMBOL, handle KEYWORD with VALUE. ... args: (arg1 arg2 arg3 arg4) 0 varref purify-flag 1 goto-if-nil 1 4 constant purecopy 5 stack-ref 2 6 call 1 7 stack-set 2 9:1 stack-ref 2 10 constant 11 switch 12 goto 9 ... 52:9 constant error 53 constant "Unknown keyword %s" 54 stack-ref 4 55 call 2 56 return Note that the code uses a jump table, which is a hash table mapping keys to integers for the "switch" op. This is where hash tables come in. We can inspect the hash table `custom-handle-keyword' uses by evaluating (aref (aref (symbol-function #'custom-handle-keyword) 2) 2) The hash table prints fine. But investigating its C in-memory representation, we find that the hash collision chains, stored in the ->next vector, are corrupted. It turns out that this is because hash_table_rehash was called on a different hash table which had the same ->next vector, but different contents. That's the problem I fixed: for reasons explained below, we sometimes see two hash tables with the same ->next vector, then try to rehash both of them, obtaining different results. Last caller wins, first caller gets the corruption (each hash table is rehashed at most once). The reasons are this: when a hash table is purecopied, its ->next vector is purecopied, which merges it with another, similar, hash table's ->next vector if purify-flag is a (third) hash table. The vectors are compared using `equal', but the pure copies are actually `eq'. This worked fine with the old dumper, because we never modified pure storage. However, with the current pdumper code, we have to do that. The (disappointingly trivial) fix: call copy-sequence on h->next before rehashing the table. This will make h->next impure, which is good since we're going to modify it. While we're there, do the same for the other vectors used in the hash table representation, except for h->key_and_value, which we need not touch. > and how is the issue with hash tables relevant. The bytecode is executing incorrectly because it relies on a purecopied hash table, which is effectively part of the compiled function. The hash table has become corrupted. --000000000000ebb7d3058ceb7431 Content-Type: text/x-patch; charset="US-ASCII"; name="0002-Don-t-alter-shared-structure-in-dumped-purecopied-ha.patch" Content-Disposition: attachment; filename="0002-Don-t-alter-shared-structure-in-dumped-purecopied-ha.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jxpvmcpm0 RnJvbSA3MDQxOWY2MzBmNjA5MTljODY0NWExMGFlZWY4ZDI5OWY1MDk4ZmY1IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBXZWQs IDMgSnVsIDIwMTkgMTE6NDg6MjIgKzAwMDAKU3ViamVjdDogW1BBVENIIDIvMl0gRG9uJ3QgYWx0 ZXIgc2hhcmVkIHN0cnVjdHVyZSBpbiBkdW1wZWQgcHVyZWNvcGllZCBoYXNoCiB0YWJsZXMuCgoq IHNyYy9mbnMuYyAoaGFzaF90YWJsZV9yZWhhc2gpOiBNYWtlIHN1cmUgd2UncmUgb3BlcmF0aW5n IG9uCmZyZXNoIGNvcGllcyBvZiAtPm5leHQsIC0+aW5kZXgsIC0+aGFzaC4KLS0tCiBzcmMvZm5z LmMgfCA2ICsrKysrKwogMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdp dCBhL3NyYy9mbnMuYyBiL3NyYy9mbnMuYwppbmRleCAyZmMwMDBhN2Y0Li40NGQyZGU1MjNhIDEw MDY0NAotLS0gYS9zcmMvZm5zLmMKKysrIGIvc3JjL2Zucy5jCkBAIC00MjIzLDYgKzQyMjMsMTIg QEAgaGFzaF90YWJsZV9yZWhhc2ggKHN0cnVjdCBMaXNwX0hhc2hfVGFibGUgKmgpCiB7CiAgIHB0 cmRpZmZfdCBzaXplID0gSEFTSF9UQUJMRV9TSVpFIChoKTsKIAorICAvKiBUaGVzZSBzdHJ1Y3R1 cmVzIG1heSBoYXZlIGJlZW4gcHVyZWNvcGllZCBhbmQgc2hhcmVkCisgICAgIChidWcjMzY0NDcp LiAgKi8KKyAgaC0+bmV4dCA9IEZjb3B5X3NlcXVlbmNlIChoLT5uZXh0KTsKKyAgaC0+aW5kZXgg PSBGY29weV9zZXF1ZW5jZSAoaC0+aW5kZXgpOworICBoLT5oYXNoID0gRmNvcHlfc2VxdWVuY2Ug KGgtPmhhc2gpOworCiAgIC8qIFJlY29tcHV0ZSB0aGUgYWN0dWFsIGhhc2ggY29kZXMgZm9yIGVh Y2ggZW50cnkgaW4gdGhlIHRhYmxlLgogICAgICBPcmRlciBpcyBzdGlsbCBpbnZhbGlkLiAgKi8K ICAgZm9yIChwdHJkaWZmX3QgaSA9IDA7IGkgPCBzaXplOyArK2kpCi0tIAoyLjIwLjEKCg== --000000000000ebb7d3058ceb7431--