From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.devel Subject: Re: Questionable code in handling of wordend in the regexp engine in regex-emacs.c Date: Sat, 2 Mar 2019 11:16:40 +0000 Message-ID: <20190302111640.GA21061@ACM> References: <20190301111018.GA5674@ACM> <83bm2uiu6x.fsf@gnu.org> <20190301141448.GC5674@ACM> <834l8mirj9.fsf@gnu.org> <20190301145856.GE5674@ACM> <83zhqeh8ds.fsf@gnu.org> <20190301163824.GF5674@ACM> <20190301191607.GG5674@ACM> <83woligzmu.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="170932"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.10.1 (2018-07-13) Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sat Mar 02 12:22:55 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1h02jH-000iKm-HD for ged-emacs-devel@m.gmane.org; Sat, 02 Mar 2019 12:22:55 +0100 Original-Received: from localhost ([127.0.0.1]:52812 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h02jG-0004di-HJ for ged-emacs-devel@m.gmane.org; Sat, 02 Mar 2019 06:22:54 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:38592) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h02iJ-0004bz-8G for emacs-devel@gnu.org; Sat, 02 Mar 2019 06:21:56 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h02iH-0002JC-UN for emacs-devel@gnu.org; Sat, 02 Mar 2019 06:21:55 -0500 Original-Received: from colin.muc.de ([193.149.48.1]:32201 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1h02iH-0002Gr-IU for emacs-devel@gnu.org; Sat, 02 Mar 2019 06:21:53 -0500 Original-Received: (qmail 36751 invoked by uid 3782); 2 Mar 2019 11:21:52 -0000 Original-Received: from acm.muc.de (p4FE150E7.dip0.t-ipconnect.de [79.225.80.231]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 02 Mar 2019 12:21:50 +0100 Original-Received: (qmail 21144 invoked by uid 1000); 2 Mar 2019 11:16:40 -0000 Content-Disposition: inline In-Reply-To: <83woligzmu.fsf@gnu.org> X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-Received-From: 193.149.48.1 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:233759 Archived-At: Hello, Eli. On Fri, Mar 01, 2019 at 21:31:37 +0200, Eli Zaretskii wrote: > > Date: Fri, 1 Mar 2019 19:16:07 +0000 > > Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org > > From: Alan Mackenzie > > + /* Check bytepos is not in the middle of a character. */ > > + eassert (bytepos >= BUF_Z_BYTE (b) > > + || CHAR_HEAD_P (BUF_FETCH_BYTE (b, bytepos))); > LGTM. > > After configuring with --enable-checking and building, I tried make > > check. The tests errored out with this bytepos check three times. In: > > src/coding-tests.log > > lisp/epg-tests.log > > lisp/emacs-lisp/package-tests.log > > . Quite possibly there is just one bug here, but there might be two or > > three. I think it would be best to track it/them down before committing > > the change to marker.c. > I agree. Let me know if you need help with that. On the coding tests, we fail in one of them when bytepos == 1, and the first byte in the buffer is 0xa4. The Lisp call stack at the time is Lisp Backtrace: "decode-coding-region" (0xc35b5cb0) "progn" (0xc35b5d68) "unwind-protect" (0xc35b5ea8) "save-current-buffer" (0xc35b6018) "let" (0xc35b6208) 0xd985d350 Lisp type 3 "ert--run-test-internal" (0xc35b69b0) "ert-run-test" (0xc35b6ed8) "ert-run-or-rerun-test" (0xc35b7410) "ert-run-tests" (0xc35b7938) "ert-run-tests-interactively" (0xc35b7f30) "funcall-interactively" (0xc35b7f28) "call-interactively" (0xc35b8310) "command-execute" (0xc35b8838) "execute-extended-command" (0xc35b8f20) "funcall-interactively" (0xc35b8f18) "call-interactively" (0xc35b9300) "command-execute" (0xc35b9808) , and the first few lines of the C backtrace are: (gdb) backtrace #0 terminate_due_to_signal (sig=6, backtrace_limit=2147483647) at emacs.c:370 #1 0x000055a4d8603595 in die (msg=0x55a4d87637a8 "bytepos >= BUF_Z_BYTE (b) || CHAR_HEAD_P (BUF_FETCH_BYTE (b, bytepos))", file=0x55a4d8763728 "marker.c", line=327) at alloc.c:7442 #2 0x000055a4d85b42cf in buf_bytepos_to_charpos (b=0x55a4da051710, bytepos=1) at marker.c:326 #3 0x000055a4d85ab93f in move_gap_both (charpos=1, bytepos=1) at insdel.c:92 #4 0x000055a4d84ca5d9 in decode_coding_object (coding=0x7ffcc35b5950, src_object=XIL(0x55a4da051715), from=1, from_byte=1, to=5, to_byte=5, dst_object=XIL(0x55a4da051715)) at coding.c:8072 #5 0x000055a4d84cff9f in code_convert_region (start=make_number(1), end=make_number(5), coding_system=XIL(0x2a1906036aa0), dst_object=XIL(0x55a4da051715), encodep=false, norecord=false) at coding.c:9371 #6 0x000055a4d84d005a in Fdecode_coding_region (start=make_number(1), end=make_number(5), coding_system=XIL(0x2a1906036aa0), destination=XIL(0)) at coding.c:9401 #7 0x000055a4d86370e1 in eval_sub (form=XIL(0x55a4d981ded3)) at eval.c:2325 . More precisely, the first few bytes of the buffer are: a4 a2 0d 0a 00 00 00 00 , and I suspect the buffer is just 4 bytes long (but I'm too lazy to check it properly). I have a suspicion that the CHAR_HEAD_P test isn't valid here. I'm guessing that we're converting an external coding, something like an MS-DOS 8-bit coding, to internal UTF-8-like coding, so we can't use CHAR_HEAD_P. In fact, is buf_bytepos_to_charpos the Right Thing to use here? -- Alan Mackenzie (Nuremberg, Germany).