From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.bugs Subject: bug#36447: 27.0.50; New "Unknown keyword" errors Date: Wed, 3 Jul 2019 11:57:52 +0000 Message-ID: References: <875zon7x0a.fsf@web.de> <8336jqgbhp.fsf@gnu.org> <87h886eoke.fsf@web.de> <87d0iu54d1.fsf@gmail.com> <87k1d14djr.fsf@web.de> <87h884fo0i.fsf@web.de> <85d0is5ry1.fsf@gmail.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="000000000000b0f333058cc59446" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="17426"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Michael Heerdegen , 36447@debbugs.gnu.org To: Noam Postavsky Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jul 03 13:59:12 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hidup-0004Mc-Cs for geb-bug-gnu-emacs@m.gmane.org; Wed, 03 Jul 2019 13:59:11 +0200 Original-Received: from localhost ([::1]:35174 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiduo-0005Kj-6G for geb-bug-gnu-emacs@m.gmane.org; Wed, 03 Jul 2019 07:59:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46426) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hiduj-0005KU-R8 for bug-gnu-emacs@gnu.org; Wed, 03 Jul 2019 07:59:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hidui-0002H0-DL for bug-gnu-emacs@gnu.org; Wed, 03 Jul 2019 07:59:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39687) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hidug-0002FY-I0 for bug-gnu-emacs@gnu.org; Wed, 03 Jul 2019 07:59:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hidug-0005WL-E6 for bug-gnu-emacs@gnu.org; Wed, 03 Jul 2019 07:59:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Pip Cet Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 03 Jul 2019 11:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36447 X-GNU-PR-Package: emacs Original-Received: via spool by 36447-submit@debbugs.gnu.org id=B36447.156215512221195 (code B ref 36447); Wed, 03 Jul 2019 11:59:02 +0000 Original-Received: (at 36447) by debbugs.gnu.org; 3 Jul 2019 11:58:42 +0000 Original-Received: from localhost ([127.0.0.1]:48508 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiduM-0005Vm-BU for submit@debbugs.gnu.org; Wed, 03 Jul 2019 07:58:42 -0400 Original-Received: from mail-oi1-f176.google.com ([209.85.167.176]:40515) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hiduK-0005VZ-AT for 36447@debbugs.gnu.org; Wed, 03 Jul 2019 07:58:40 -0400 Original-Received: by mail-oi1-f176.google.com with SMTP id w196so1812990oie.7 for <36447@debbugs.gnu.org>; Wed, 03 Jul 2019 04:58:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mywJ75uQhBXf9WTvlJVUD4FV0gR5TJbOb5Uwz61NVfc=; b=hkEE/Ot4nv2KujJ+1zJKAdIvVns23l+4N9bGT2dgc6HYDKal9z1D1QpyLWks4J7cOh 45g7QHEKxu+dl0JfXu4Y+E0zkpSzQwi4yiIfJ1H0z8a2sas9jUkRi+Ycv0YMcRl/8WdX Lh7EjRi0JXltrufJTYHb2fuKUfXzMZD4vBMSo6IYctkgC9e2zYh6pQMNGG/INxRXH8vb XLlVdTHpiN1TA8pl4sqIgynvOyMgO7rkXxbrm8efsYDZi06gWz8wzyQjXGOreknGRLfO qE8LEwabzFPNGDdBbQZekA/4yfvcnlB8iMGiTfALDHSv2hsvKq95+HosobZ0NZ8tYKil Efnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mywJ75uQhBXf9WTvlJVUD4FV0gR5TJbOb5Uwz61NVfc=; b=uDZecVLK4ptG2WGn8d4p1ufbbWiHT899qrvHfMksfNGdSfDSmq+r+KRNkR3exUCWxO pfHAgGBLLqyD4dn7lKD9IgNppiTySz6cqp46cn2c+txvIy9+o217RxTb6bKoUTzfLvrJ XhqwesX8y4xEtF3sqjReV+7frCLqQR5r/DqYt8p83SlWJBPxbHIaxXeFvA84eqL2vveq +VdVfKMbkZdYgDC73G1kRulQlPNuj28zHCmCWd0gV4lSWMb0/Tch3NtQ8c/X7G7jLO04 QoRGQSsizEl2hfe6MQbXhVeonSM5TTfSwmTRdNGX68YkPVlRbXaCqsRob9HMvoCFT3W5 58pg== X-Gm-Message-State: APjAAAUN9odoDlkuQrPnbgPQMGGmMUiFwzwgGy73htxWwSABOfCOHoaa HS49Y4pEjVjrlbhIPQDAdAzq6ZVSiRQVjxVZQEk= X-Google-Smtp-Source: APXvYqxti0ZQH9eZo3ysr02KcE5CQMI5AJHHwGOceWPKXvBKJv2mz6+RjfVKZ5tfnOqDYZ9zIPcNOc7zjarMWgvX/nY= X-Received: by 2002:aca:4790:: with SMTP id u138mr3263471oia.44.1562155114612; Wed, 03 Jul 2019 04:58:34 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162021 Archived-At: --000000000000b0f333058cc59446 Content-Type: text/plain; charset="UTF-8" On Tue, Jul 2, 2019 at 10:50 PM Pip Cet wrote: > > The compiler translates repeated `eq' in a cond like that into a hash > > and jump. See byte-compile-cond-use-jump-table. > > I think I found the problem. It's a bit tricky. I'm attaching a patch which modifies standard Emacs in what should be harmless ways, but demonstrates the bug. As far as I can tell, this is a serious issue that will pop up seemingly at random. We really should fix it. However, I thought hash_table_rehash was called more often than it actually is, so the fix should be simple: just copy-sequence a hash table's ->next, ->hash, and ->index vectors before rehashing. (->key_and_value isn't actually modified, so it's safe to share between (read-only) hash tables. What's happening is this: Purecopied hash tables share structure, particularly the ->next vector, with similar but different purecopied hash tables. Some purecopied hash tables dumped with pdumper are rehashed upon first access after loading the dump (in the example, this is the tables which map 'dummy-symbol to t), while others are not (the tables with just fixnums as keys). If a hash table is rehashed, it's ->next vector will change to reflect the changed hash (of 'dummy-symbol); however, the non-rehashed table that shares the ->next vector will be confused: its key_and_value array will stay the same, and valid, but the ->next vector will no longer match it. In practice, this means (gethash valid-key hash-table) will return nil even though the key is valid. While the attached patch appears to work, I would prefer simply dumping hash tables with nil values for ->next, ->hash, and ->index, and rebuilding the entire hash table upon first access. This would also allow us to switch to secure randomized hashes should the need arise. --000000000000b0f333058cc59446 Content-Type: text/x-patch; charset="US-ASCII"; name="0001-Test-bug-36447.patch" Content-Disposition: attachment; filename="0001-Test-bug-36447.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jxn6ridi0 RnJvbSA4ZmU0NjRkYmIwMzBhNzIyY2ExNzNjNDQyOTA2NDgyZWYyMTBmNDZlIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBXZWQs IDMgSnVsIDIwMTkgMTE6Mjc6NTAgKzAwMDAKU3ViamVjdDogW1BBVENIIDEvMl0gVGVzdCBidWcj MzY0NDcKCi0tLQogbGlzcC9jdXN0b20uZWwgfCAxMCArKysrKysrKysrCiBzcmMvcHVyZXNpemUu aCB8ICAyICstCiB0ZXN0LWN1c3RvbS5lbCB8IDEyICsrKysrKysrKysrKwogMyBmaWxlcyBjaGFu Z2VkLCAyMyBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCiBjcmVhdGUgbW9kZSAxMDA2NDQg dGVzdC1jdXN0b20uZWwKCmRpZmYgLS1naXQgYS9saXNwL2N1c3RvbS5lbCBiL2xpc3AvY3VzdG9t LmVsCmluZGV4IDczNjQ2MGZlYzcuLmM0OTMyMWI4MjkgMTAwNjQ0Ci0tLSBhL2xpc3AvY3VzdG9t LmVsCisrKyBiL2xpc3AvY3VzdG9tLmVsCkBAIC0zNSw2ICszNSwxNiBAQAogCiAocmVxdWlyZSAn d2lkZ2V0KQogCisoZGVmdmFyIGhhc2gtdGFibGVzIChtYWtlLXZlY3RvciAyNTYgbmlsKSkKKwor KGRvdGltZXMgKGkgMjU2KQorICAobGV0ICgoaHQgKG1ha2UtaGFzaC10YWJsZSA6cHVyZWNvcHkg dCA6c2l6ZSAyIDp0ZXN0ICdlcSkpKQorICAgICh3aGVuICg9ICglIGkgMykgMCkKKyAgICAgIChw dXRoYXNoICdkdW1teS1zeW1ib2wgdCBodCkpCisgICAgKGRvdGltZXMgKGogMTYpCisgICAgICAo cHV0aGFzaCAobG9nYW5kIGkgKGxzaCAxIGopKSB0IGh0KSkKKyAgICAoYXNldCBoYXNoLXRhYmxl cyBpIChwdXJlY29weSBodCkpKSkKKwogKGRlZnZhciBjdXN0b20tZGVmaW5lLWhvb2sgbmlsCiAg IDs7IEN1c3RvbWl6ZSBpbmZvcm1hdGlvbiBmb3IgdGhpcyBvcHRpb24gaXMgaW4gYGN1cy1lZGl0 LmVsJy4KICAgIkhvb2sgY2FsbGVkIGFmdGVyIGRlZmluaW5nIGVhY2ggY3VzdG9taXplIG9wdGlv bi4iKQpkaWZmIC0tZ2l0IGEvc3JjL3B1cmVzaXplLmggYi9zcmMvcHVyZXNpemUuaAppbmRleCBm NWZhZDhiNDJiLi5kMjllM2U4MGRmIDEwMDY0NAotLS0gYS9zcmMvcHVyZXNpemUuaAorKysgYi9z cmMvcHVyZXNpemUuaApAQCAtNDcsNyArNDcsNyBAQCAjZGVmaW5lIFNJVEVMT0FEX1BVUkVTSVpF X0VYVFJBIDAKICNlbmRpZgogCiAjaWZuZGVmIEJBU0VfUFVSRVNJWkUKLSNkZWZpbmUgQkFTRV9Q VVJFU0laRSAoMjAwMDAwMCArIFNZU1RFTV9QVVJFU0laRV9FWFRSQSArIFNJVEVMT0FEX1BVUkVT SVpFX0VYVFJBKQorI2RlZmluZSBCQVNFX1BVUkVTSVpFICg4MDAwMDAwMCArIFNZU1RFTV9QVVJF U0laRV9FWFRSQSArIFNJVEVMT0FEX1BVUkVTSVpFX0VYVFJBKQogI2VuZGlmCiAKIC8qIEluY3Jl YXNlIEJBU0VfUFVSRVNJWkUgYnkgYSByYXRpbyBkZXBlbmRpbmcgb24gdGhlIG1hY2hpbmUncyB3 b3JkIHNpemUuICAqLwpkaWZmIC0tZ2l0IGEvdGVzdC1jdXN0b20uZWwgYi90ZXN0LWN1c3RvbS5l bApuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwLi5iODcxZDMwY2ZhCi0tLSAv ZGV2L251bGwKKysrIGIvdGVzdC1jdXN0b20uZWwKQEAgLTAsMCArMSwxMiBAQAorKGRvdGltZXMg KGkgMjU2KQorICAobGV0ICgoaHQgKGFyZWYgaGFzaC10YWJsZXMgaSkpKQorICAgIChkb3RpbWVz IChqIDE2KQorICAgICAgKHVubGVzcyAoZXEgKC89IDAgKGxvZ2FuZCAobHNoIDEgaikgaSkpCisg ICAgICAgICAgICAgICAgICAoZ2V0aGFzaCAobHNoIDEgaikgaHQpKQorICAgICAgICAoZXJyb3Ig Imhhc2ggdGFibGUgY29ycnVwdGlvbiBhdCB0YWJsZSAlUywgYml0ICVTIiBpIGopKSkpKQorCiso YXJlZiBoYXNoLXRhYmxlcyAxNykKKzs7ICNzKGhhc2gtdGFibGUgc2l6ZSAzIHRlc3QgZXEgcmVo YXNoLXNpemUgMS41IHJlaGFzaC10aHJlc2hvbGQgMC44MTI1IHB1cmVjb3B5IHQgZGF0YSAoMSB0 IDAgdCAxNiB0KSkKKworKGdldGhhc2ggMSAoYXJlZiBoYXNoLXRhYmxlcyAxNykpCis7OyBuaWwK LS0gCjIuMjAuMQoK --000000000000b0f333058cc59446 Content-Type: text/x-patch; charset="US-ASCII"; name="0002-Don-t-alter-shared-structure-in-dumped-purecopied-ha.patch" Content-Disposition: attachment; filename="0002-Don-t-alter-shared-structure-in-dumped-purecopied-ha.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jxn6rlog1 RnJvbSA3MDQxOWY2MzBmNjA5MTljODY0NWExMGFlZWY4ZDI5OWY1MDk4ZmY1IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBQaXAgQ2V0IDxwaXBjZXRAZ21haWwuY29tPgpEYXRlOiBXZWQs IDMgSnVsIDIwMTkgMTE6NDg6MjIgKzAwMDAKU3ViamVjdDogW1BBVENIIDIvMl0gRG9uJ3QgYWx0 ZXIgc2hhcmVkIHN0cnVjdHVyZSBpbiBkdW1wZWQgcHVyZWNvcGllZCBoYXNoCiB0YWJsZXMuCgoq IHNyYy9mbnMuYyAoaGFzaF90YWJsZV9yZWhhc2gpOiBNYWtlIHN1cmUgd2UncmUgb3BlcmF0aW5n IG9uCmZyZXNoIGNvcGllcyBvZiAtPm5leHQsIC0+aW5kZXgsIC0+aGFzaC4KLS0tCiBzcmMvZm5z LmMgfCA2ICsrKysrKwogMSBmaWxlIGNoYW5nZWQsIDYgaW5zZXJ0aW9ucygrKQoKZGlmZiAtLWdp dCBhL3NyYy9mbnMuYyBiL3NyYy9mbnMuYwppbmRleCAyZmMwMDBhN2Y0Li40NGQyZGU1MjNhIDEw MDY0NAotLS0gYS9zcmMvZm5zLmMKKysrIGIvc3JjL2Zucy5jCkBAIC00MjIzLDYgKzQyMjMsMTIg QEAgaGFzaF90YWJsZV9yZWhhc2ggKHN0cnVjdCBMaXNwX0hhc2hfVGFibGUgKmgpCiB7CiAgIHB0 cmRpZmZfdCBzaXplID0gSEFTSF9UQUJMRV9TSVpFIChoKTsKIAorICAvKiBUaGVzZSBzdHJ1Y3R1 cmVzIG1heSBoYXZlIGJlZW4gcHVyZWNvcGllZCBhbmQgc2hhcmVkCisgICAgIChidWcjMzY0NDcp LiAgKi8KKyAgaC0+bmV4dCA9IEZjb3B5X3NlcXVlbmNlIChoLT5uZXh0KTsKKyAgaC0+aW5kZXgg PSBGY29weV9zZXF1ZW5jZSAoaC0+aW5kZXgpOworICBoLT5oYXNoID0gRmNvcHlfc2VxdWVuY2Ug KGgtPmhhc2gpOworCiAgIC8qIFJlY29tcHV0ZSB0aGUgYWN0dWFsIGhhc2ggY29kZXMgZm9yIGVh Y2ggZW50cnkgaW4gdGhlIHRhYmxlLgogICAgICBPcmRlciBpcyBzdGlsbCBpbnZhbGlkLiAgKi8K ICAgZm9yIChwdHJkaWZmX3QgaSA9IDA7IGkgPCBzaXplOyArK2kpCi0tIAoyLjIwLjEKCg== --000000000000b0f333058cc59446--