bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

[Wasim Abbas]

[-- Attachment #1: Type: text/plain, Size: 362 bytes --]

For a very specific c++ header file (attached rorrhi_types_test.hpp) emacs is hanging in
font-lock-default-fontify-region(xxx xxx nil) consistently. Just
removing a few characters from the begining of the file or removing some
doxygen comments solves the problem. I tried a few versions including
current master etc. Emacs 27.1.91 doesn't have the issue.


[-- Attachment #2: rorrhi_types_test.hpp --]
[-- Type: application/octet-stream, Size: 4858 bytes --]

// Roar Source Code
// Wasim Abbas
// http://www.waZim.com
// Copyright (c) 2008-2021
//
// Permission is hereby granted, free of charge, to any person obtaining
// a copy of this software and associated documentation files (the 'Software'),
// to deal in the Software without restriction, including without limitation
// the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the Software
// is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
// OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
// IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
// CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
// TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
// OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
//
// Version: 1.0.0

#pragma once

#include "something_random.hpp"
#include <cassert>
#include <cstddef>
#include <functional>
#include <memory>
#include <stdint.h>
#include <tuple>
#include <unordered_map>
#include <utility>
#include <vector>

namespace rhi
{
// template <bool>
// class HasPositions
// {};

// template <>
// class HasPositions<true>
// {
//   public:
//	ror::Vector3f m_positions;
// };

// template <class _type, bool>
// class HasPositionsSOA
// {};

// // This is probably much better for copying later on
// template <class _type>
// class HasPositionsSOA<_type, true>
// {
//   public:
//	std::vector<_type> m_positions;
// };

/**
 * Describes how many buffers are created for static geometry data and with what layout
 * This should only be used for static data that won't be touched later for anything else
 * use Buffer directly. Ideally this will try to create
 *     1 buffer for positions,
 *     1 for all other vertex attributes,
 *     1 for index buffers,
 *     1 for instanced data
 */
class ROAR_ENGINE_ITEM BuffersPack
{
  public:
	FORCE_INLINE BuffersPack(const BuffersPack &a_other)     = delete;                    //! Copy constructor
	FORCE_INLINE BuffersPack(BuffersPack &&a_other) noexcept = delete;                    //! Move constructor
	FORCE_INLINE BuffersPack &operator=(const BuffersPack &a_other) = delete;             //! Copy assignment operator
	FORCE_INLINE BuffersPack &operator=(BuffersPack &&a_other) noexcept = delete;         //! Move assignment operator
	FORCE_INLINE virtual ~BuffersPack() noexcept                        = default;        //! Destructor

	declare_translation_unit_vtable();

	FORCE_INLINE BuffersPack()
	{
		std::shared_ptr<ror::BuffersFormat> buffers_format = ror::get_buffer_pack();

		this->m_buffers = buffers_format->m_buffer_packs[buffers_format->m_current_format].m_buffers;        // Copy the buffers over

		uint32_t i = 0;
		for (const auto &bfr : this->m_buffers)
		{
			const auto semantics = bfr.semantics();

			for (const auto &sem : semantics)
			{
				this->m_attribute_indices.emplace(sem.first, i);        // Creating mapping here, If ShaderSemantic is defined for multiple buffers, behaviour is undefined
			}
			i++;
		}
	}

	FORCE_INLINE uint32_t attribute_buffer_index(ShaderSemantic a_semantic) const
	{
		return this->m_attribute_indices.at(a_semantic);
	}

	/**
	 * Returns a pair with buffer index and the offsets in that buffer where the data is copied
	 */
	FORCE_INLINE std::pair<uint64_t, uint64_t> upload_data(ShaderSemantic a_semantic, uint8_t &a_data, uint64_t a_bytes)
	{
		uint64_t index = this->attribute_buffer_index(a_semantic);
		return std::make_pair(index, this->m_buffers[index].upload(a_data, a_bytes));
	}

  private:

	std::vector<Buffer>                          m_buffers;                   //! All buffers created for different type data
	std::unordered_map<ShaderSemantic, uint32_t> m_attribute_indices;         //! All indices for any of the ShaderSemantic type

	// uint32_t                                     m_position_attribute_index{0};            //! All positions buffer (VBOs)
	// uint32_t                                     m_non_position_attribute_index{0};        //! All non-positions buffer (VBOs)
	// uint32_t                                     m_index_attribute_index{0};               //! All index buffer (IBOs)
	// uint32_t                                     m_instance_attribute_index{0};            //! All instanced data buffer (VBOs)
};

define_translation_unit_vtable(BuffersPack);

/**
* Description of a geometry vertex
* Prefered to be used via VertexDescriptor not directly
*/
class ROAR_ENGINE_ITEM VertexAttribute final
{
 
};
}

[-- Attachment #3: Type: text/plain, Size: 3187 bytes --]





In GNU Emacs 28.0.50 (build 2, x86_64-apple-darwin19.6.0, NS appkit-1894.60 Version 10.15.7 (Build 19H524))
of 2021-03-12 built on Wasims-iMac-Pro.local
Windowing system distributor 'Apple', version 10.3.1894
System Description:  Mac OS X 10.15.7

Configured using:
'configure --enable-locallisppath=/usr/local/share/emacs/site-lisp
--infodir=/usr/local/Cellar/emacs-head@28/28.0.50_1/share/info/emacs
--prefix=/usr/local/Cellar/emacs-head@28/28.0.50_1 --without-x
--without-dbus --with-imagemagick --with-modules --with-gnutls
--with-rsvg --with-xml2 --with-ns --disable-ns-self-contained'

Configured features:
ACL GIF GLIB GMP GNUTLS IMAGEMAGICK JPEG JSON LCMS2 LIBXML2 MODULES
NOTIFY KQUEUE NS PDUMPER PNG RSVG THREADS TIFF TOOLKIT_SCROLL_BARS XIM
ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//l

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git
diff-mode easy-mmode vc-dispatcher cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs cl-loaddefs cl-lib
iso-transl tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/ns-win ns-win ucs-normalize mule-util
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads kqueue
cocoa ns lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 77492 7028)
(symbols 48 9542 1)
(strings 32 26136 2260)
(string-bytes 1 978378)
(vectors 16 16918)
(vector-slots 8 222274 13643)
(floats 8 24 47)
(intervals 56 477 0)
(buffers 992 12))

bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

[Wasim Abbas]

Forgot to mention to reproduce it, you need to scroll up and down for this to kick in. I have seen this happening with shortcuts too but usually scrolling up and down quickly with mouse will trigger it.

Also did some more digging and git bisect found the first bad commit:

92c56300c317c9e5573dca787a2cf20f777b3179 is the first bad commit
commit 92c56300c317c9e5573dca787a2cf20f777b3179
Author: Alan Mackenzie <acm@muc.de>
Date:   Tue Dec 15 12:09:47 2020 +0000

    CC Mode: Optimize for scrolling large buffers containing few braces

    This fixes bug #25706.  It particularly pertains to .h files which contain
    only macro definitions.  Many of these files are to be found, for example, in
    the driver sections of the Linux kernel.

    * lisp/progmodes/cc-engine.el (c-beginning-of-statement-1, c-on-identifier)
    (c-syntactic-skip-backward, c-find-decl-prefix-search, c-find-decl-spots)
    (c-forward-name, c-back-over-list-of-member-inits)
    (c-back-over-member-initializers, c-looking-at-inexpr-block)
    (c-guess-basic-syntax): Give search limits to, or amend existing ones to
    c-backward-syntactic-ws, c-forward-syntactic-ws, c-backward-token-2,
    c-beginning-of-statement-1.
    (c-determine-limit-no-macro): New function.
    (c-determine-limit-get-base): Remove unlimted invocation of
    c-backward-syntactic-ws.
    (c-determine-limit): Exclude movement between two different macros.  Use new
    function c-determine-limit-no-macro.
    (c-back-over-list-of-member-inits): New parameter `limit'.

    * lisp/progmodes/cc-fonts.el (c-font-lock-complex-decl-prepare)
    (c-font-lock-declarations, c-font-lock-c++-using): Give search limits to, or
    amend existing ones to c-backward-syntactic-ws, c-beginning-of-decl-1.

    * lisp/progmodes/cc-mode.el (c-unfind-coalesced-tokens, c-before-changer)
    (c-fl-decl-end): Give search limits to, or amend existing ones to
    c-backward-syntactic-ws, c-forward-syntactic-ws, skip-chars-backward,
    skip-chars-forward.

 lisp/progmodes/cc-engine.el | 282 ++++++++++++++++++++++++++++----------------
 lisp/progmodes/cc-fonts.el  |  16 ++-
 lisp/progmodes/cc-mode.el   |  33 ++++--
 3 files changed, 212 insertions(+), 119 deletions(-)

bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

[Lars Ingebrigtsen]

Wasim Abbas <abbas.wasim@gmail.com> writes:

> For a very specific c++ header file (attached rorrhi_types_test.hpp)
> emacs is hanging in
> font-lock-default-fontify-region(xxx xxx nil) consistently. Just
> removing a few characters from the begining of the file or removing some
> doxygen comments solves the problem. I tried a few versions including
> current master etc. Emacs 27.1.91 doesn't have the issue.

Wasim Abbas <abbas.wasim@gmail.com> writes:

> Forgot to mention to reproduce it, you need to scroll up and down for this to kick in. I have seen this happening with shortcuts too but usually scrolling up and down quickly with mouse will trigger it.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I'm unable to reproduce this problem in Emacs 29.  Do you still see this
problem in recent Emacs versions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

[Wasim Abbas]

I can’t reproduce it anymore either in 28.1.

//Wasim

> On 25 Jun 2022, at 13:02, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Wasim Abbas <abbas.wasim@gmail.com> writes:
> 
>> For a very specific c++ header file (attached rorrhi_types_test.hpp)
>> emacs is hanging in
>> font-lock-default-fontify-region(xxx xxx nil) consistently. Just
>> removing a few characters from the begining of the file or removing some
>> doxygen comments solves the problem. I tried a few versions including
>> current master etc. Emacs 27.1.91 doesn't have the issue.
> 
> Wasim Abbas <abbas.wasim@gmail.com> writes:
> 
>> Forgot to mention to reproduce it, you need to scroll up and down for this to kick in. I have seen this happening with shortcuts too but usually scrolling up and down quickly with mouse will trigger it.
> 
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
> 
> I'm unable to reproduce this problem in Emacs 29.  Do you still see this
> problem in recent Emacs versions?
> 
> -- 
> (domestic pets only, the antidote for overdose, milk.)
>   bloggy blog: http://lars.ingebrigtsen.no

bug#47267: 28.0.50; Emacs hangs in font-lock-default-fontify-region

[Lars Ingebrigtsen]

Wasim Abbas <abbas.wasim@gmail.com> writes:

> I can’t reproduce it anymore either in 28.1.

Thanks for checking; I'm closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no