From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Robert Pluim Newsgroups: gmane.emacs.bugs Subject: bug#38407: 27.0.50; infinite loop with display of large file without newlines Date: Wed, 04 Dec 2019 10:15:55 +0100 Message-ID: References: <39c498717f8958e7fdc408d4da51d378@webmail.orcon.net.nz> <24031.28277.201123.531348@cochabamba.vanoostrum.org> <83r21sowx1.fsf@gnu.org> <24032.4698.256238.87458@cochabamba.vanoostrum.org> <83k17jq1ch.fsf@gnu.org> <24032.17845.921546.629745@cochabamba.vanoostrum.org> <83fti7p349.fsf@gnu.org> <4B1ABCA7-A69C-4251-8EBD-A11654A92642@vanoostrum.org> <83v9r2o4z9.fsf@gnu.org> <24035.27244.755074.180653@cochabamba.vanoostrum.org> <83lfrwlz2w.fsf@gnu.org> <83o8wpjsx5.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="127054"; mail-complaints-to="usenet@blaine.gmane.org" Cc: psainty@orcon.net.nz, pieter@vanoostrum.org, 38407@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Dec 04 10:29:50 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1icQyj-000WuU-OY for geb-bug-gnu-emacs@m.gmane.org; Wed, 04 Dec 2019 10:29:49 +0100 Original-Received: from localhost ([::1]:36086 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1icQyh-0004rp-Kh for geb-bug-gnu-emacs@m.gmane.org; Wed, 04 Dec 2019 04:29:47 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56811) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1icQmS-0001fs-TZ for bug-gnu-emacs@gnu.org; Wed, 04 Dec 2019 04:17:11 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1icQmN-0005m3-J0 for bug-gnu-emacs@gnu.org; Wed, 04 Dec 2019 04:17:05 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:35464) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1icQmN-0005lU-F4 for bug-gnu-emacs@gnu.org; Wed, 04 Dec 2019 04:17:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1icQmM-0000rE-AH for bug-gnu-emacs@gnu.org; Wed, 04 Dec 2019 04:17:02 -0500 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Robert Pluim Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 04 Dec 2019 09:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38407 X-GNU-PR-Package: emacs Original-Received: via spool by 38407-submit@debbugs.gnu.org id=B38407.15754509703154 (code B ref 38407); Wed, 04 Dec 2019 09:17:02 +0000 Original-Received: (at 38407) by debbugs.gnu.org; 4 Dec 2019 09:16:10 +0000 Original-Received: from localhost ([127.0.0.1]:41437 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1icQlW-0000oo-2Z for submit@debbugs.gnu.org; Wed, 04 Dec 2019 04:16:10 -0500 Original-Received: from mail-wr1-f68.google.com ([209.85.221.68]:44668) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1icQlR-0000nh-7n for 38407@debbugs.gnu.org; Wed, 04 Dec 2019 04:16:07 -0500 Original-Received: by mail-wr1-f68.google.com with SMTP id q10so7531708wrm.11 for <38407@debbugs.gnu.org>; Wed, 04 Dec 2019 01:16:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:message-id:mime-version :content-transfer-encoding; bh=VJPGbxi7KMun302kNeHFw1V2JOCoLme00qnBhMcUpcs=; b=CxTNKs7OrgGKzwO8VsfdBp2tt65Mf1+dBQDcvPTP7LYSDWs9VJr2nH0CnNiPt1Aoqr OEiP0yZjbwvzecDKG4Lj2xnryHp0pR4v7xLs0f8AFWUOgH6OPt2cmtZbagCzZgAZf3hW IicfunzNJfBEOHwUJaGY1IjKf4+gWDvhLIQFHnwatHgLmhWE4lLCWx8qnxzfDpdPacgw E3wgeVrlsh+14HPElyEnt+aCwUA50qnceLKaVzHhzqNVoghIEQ4PYNyY7su/DRvUTTJ/ JgTIfWdLRG9aI0vnRyf5e875nrvHyBOtS1wSrFsC1BQg1z2hBTQ5sevlFa2RZgRoprBn 86kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:message-id :mime-version:content-transfer-encoding; bh=VJPGbxi7KMun302kNeHFw1V2JOCoLme00qnBhMcUpcs=; b=Wy6h8mvsKFou8baOGoBvIdaON2o1v0uon5AHmzP7DuhxScJ4/rIQQ2j6o5bgWd/OEn D2Ra7jxYUg4+GUgYXUp4A2yKg0P83+wUGTuNzhqPTNILu/R67+FHchK1GL+Df7Ofc7Pk FLh1NraiEPmXw1WHGRPD1qUJOv/JohLczkeVPJuSoB/cb9ciilV4WGhokylmGc2QL3kz NS5Qe/dhMpdUveUzG6ZK0M4yubchwe9cnw86iHfkeOlCfW02H1jgRjyWk+kb/J0gjK+W Ej9yhCdqTVey7J9mqY1tpkoEJKuxMbIuFZed5MZ2RmRQ4+7iloqGNiuofR9lTBqlp/H9 SbIA== X-Gm-Message-State: APjAAAWkkNd/9FTiX/hO9fV7K99WNBd6XVJivijHtVCH4k78lGaD57RA 6faOj+xvcg3KEg3FVVPMv+8C7hc+ X-Google-Smtp-Source: APXvYqzyEHzi3zAAOiV5zibIP7EiSoOgK6KMg17of/GVhXWuIP/dlcon5Kut1iFol85jxSMebal8tA== X-Received: by 2002:a5d:5345:: with SMTP id t5mr2962482wrv.0.1575450956798; Wed, 04 Dec 2019 01:15:56 -0800 (PST) Original-Received: from rpluim-mac ([149.5.228.1]) by smtp.gmail.com with ESMTPSA id u10sm5829316wmd.1.2019.12.04.01.15.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Dec 2019 01:15:56 -0800 (PST) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:172828 Archived-At: >>>>> On Tue, 03 Dec 2019 18:05:42 +0200, Eli Zaretskii said: Eli> Yes, I was wondering about that myself. But we need more details = to Eli> understand better what, if anything, can be done about this. Eli> First, which part of SAVE_IT causes this? I'm guessing it's this Eli> part: Eli> #define SAVE_IT(ITCOPY, ITORIG, CACHE) \ Eli> do { \ Eli> if (CACHE) \ Eli> bidi_unshelve_cache (CACHE, true); \ Eli> ITCOPY =3D ITORIG; \ Eli> CACHE =3D bidi_shelve_cache (); \ <<<<<<<<<<<< Eli> } while (false) Yes, it=CA=BCs bidi_shelve_cache Eli> If that is true, then I think the offending part of bidi_shelve_ca= che Eli> is this: Eli> alloc =3D (bidi_shelve_header_size Eli> + bidi_cache_idx * sizeof (struct bidi_it)); Eli> databuf =3D xmalloc (alloc); Eli> bidi_cache_total_alloc +=3D alloc; Eli> memcpy (databuf, &bidi_cache_idx, sizeof (bidi_cache_idx)); Eli> memcpy (databuf + sizeof (bidi_cache_idx), = <<<<<<< Eli> bidi_cache, bidi_cache_idx * sizeof (struct bidi_it)); <<<<<<< Eli> memcpy (databuf + sizeof (bidi_cache_idx) Eli> + bidi_cache_idx * sizeof (struct bidi_it), Eli> bidi_cache_start_stack, sizeof (bidi_cache_start_stack)); Eli> And if this guess is also true, then I think the problem is that Eli> databuf + sizeof (bidi_cache_idx) is unaligned on 64-bit systems, Eli> since bidi_cache_idx is an int. The '_unaligned_' bit of that memmove function name does not mean that=CA=BCs it=CA=BCs doing unoptimized unaligned copies: it means it accep= ts unaligned pointers, and aligns them as necessary to enable fast copying. Anyway, I made bidi_cache_idx an intptr_t, and it made no difference. I think that misalignment is vastly dwarfed by the following: (gdb) l bidi_shelve_cache 973 { 974 unsigned char *databuf; 975 ptrdiff_t alloc; 976 977 /* Empty cache. */ 978 if (bidi_cache_idx =3D=3D 0) 979 return NULL; 980 981 alloc =3D (bidi_shelve_header_size 982 + bidi_cache_idx * sizeof (struct bidi_it)); (gdb) b 980 Breakpoint 3 at 0x4b66aa: file bidi.c, line 981. (gdb) commands Type commands for breakpoint(s) 3, one per line. End with a line saying just "end". >p bidi_cache_idx >p bidi_cache_idx * sizeof(struct bidi_it) >end Thread 1 "emacs" hit Breakpoint 3, bidi_shelve_cache () at bidi.c:981 981 alloc =3D (bidi_shelve_header_size $25 =3D 30860 $26 =3D 71842080 which means Emacs is copying 70MB of data every time bidi_shelve_cache is called, and it=CA=BCs called *a lot* in this scenario. Could we not do this shelving by pointer-swapping or similar rather than copying? Robert