From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id yPsVK+uwl2HnpgAAgWs5BA (envelope-from ) for ; Fri, 19 Nov 2021 15:12:59 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 6Pu9Juuwl2FZCwAAB5/wlQ (envelope-from ) for ; Fri, 19 Nov 2021 14:12:59 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 60C64C88A for ; Fri, 19 Nov 2021 15:12:59 +0100 (CET) Received: from localhost ([::1]:46648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mo4dO-0005kl-Er for larch@yhetil.org; Fri, 19 Nov 2021 09:12:58 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mo4bj-0005OI-Di for emacs-orgmode@gnu.org; Fri, 19 Nov 2021 09:11:15 -0500 Received: from [2a00:1450:4864:20::135] (port=37587 helo=mail-lf1-x135.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mo4bg-0004O8-UZ for emacs-orgmode@gnu.org; Fri, 19 Nov 2021 09:11:14 -0500 Received: by mail-lf1-x135.google.com with SMTP id c32so44085434lfv.4 for ; Fri, 19 Nov 2021 06:11:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:date:message-id:mime-version; bh=B4yB4cMTSJma2UO5ZD2hatrRR92e+VFy2aKvZKAelY8=; b=XUkkJh+p2EMkUpXBUbEIz8Qfb61Wqcr465Ejx+0iwKyXa0BdgcvHHu9lcyRw8t4644 BhbPKFteFA/QYwXlnVjt8qAhNDy8WBZOdQuE0oGz2gzmrmfRAgOOKA/wSFTyFvkN+yUR LHoq7PPfvgfU8eEc7DEVqtQKjxk/5zwj8gVi3+ITSicl8/9mHSJs1Vj1+GvaUscg/FWf hSQk6SRyvO0M/9k3FwX7qFHgakht+idjvnmH/4wFUTCe6kCcsi3BaYlGbz1O4NMp9wQs 6hs7O3GisEraSr+cgalnWHLLZGx3OYL+ocazbnXJe74XN4HiScE1Fuo2QCeAk2OCAqex k/qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:mime-version; bh=B4yB4cMTSJma2UO5ZD2hatrRR92e+VFy2aKvZKAelY8=; b=vMTl7dGzdzx7+xW65YdoC950wWT0vGYekVKSmFfH1JNDc19LRfQBpVKiu77v01yvGI KRkGbUL0fWrlEWGukYqW1d7Kj9Bo2WwtQ/84zoa9D0m8SDLja2PHfsSBzC8aonn/sxuq 7uIe1lKuA76t9s8j53L9ZKRlGN6V4U1keYqP9I5dpqYI6M8RDlSlSWsHdobByh7LDJhB /lxdL38tOFVNFfabhgDKeVvTLSOcfO2WfPSHCCxiwv+N8hnLepXRh0wfXb4/m0XIQTD7 QvTAzde2rxdomZfRU9twiGHNRgMVM1l615v9a2PyIyX5TGJSgfckqpPV0Dqr0ncPZh67 QzZg== X-Gm-Message-State: AOAM530S42P9PDLmGW41plH4YgRp5kWgSG1ECjVsRUeIBUr25Q/+IQju 0QztKdR7pm9PNl+KYZW993YDGT2I2ZA41Q== X-Google-Smtp-Source: ABdhPJySIkZUvq3iK0HhnYJ3PaaJGB4egIVGa0/hcsA9xPSj0Wlu3N4jaDqisIAtNgQZhxR7/ZJSow== X-Received: by 2002:a05:6512:b84:: with SMTP id b4mr33151092lfv.472.1637331070480; Fri, 19 Nov 2021 06:11:10 -0800 (PST) Received: from localhost ([91.210.107.150]) by smtp.gmail.com with ESMTPSA id u23sm375418lfo.35.2021.11.19.06.11.09 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 19 Nov 2021 06:11:10 -0800 (PST) From: Ihor Radchenko To: emacs-orgmode@gnu.org Subject: [DISCUSSION] Refactoring fontification system Date: Fri, 19 Nov 2021 22:12:35 +0800 Message-ID: <87ee7c9quk.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::135 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::135; envelope-from=yantar92@gmail.com; helo=mail-lf1-x135.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, PDS_HP_HELO_NORDNS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637331179; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=B4yB4cMTSJma2UO5ZD2hatrRR92e+VFy2aKvZKAelY8=; b=dbFVjJtPdZuVtN0Hwz+ggc5YjrEVtHdY5yFHWBEsyIDastBnNDTJ9T06+oP8LWwi1lvsQM MdhmYiWLkmklyKuMcrxkgTfTrCo6nUN36BQMiUA7H8ZRVNsT9N5szwi8U4JAfcGHsnNAkD idK3LUL+Gsb0M+22VMgyOWiPBstRN3Ub3DvsF/DZS1Ge567AoYshGdyeXzB3SfgmAvY5ta liWfJjWlbItBkEK7R/iDFGfg5AxPC6IblgFEYWMQmMgfWfchgbivfJfqXxp4+D9aaU1IMU vTYIJaoPGff7mGSuI89/7C5VKoVoQK3uUDh8va7NyLLG3RDnUoOGRcPCNHgsUg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637331179; a=rsa-sha256; cv=none; b=ucghzxBcGtt+ciNzCg+x6+aUoCV0FlHldeVli4raBkmQRydQfARlmsMQwbmEKusgr7tUHr T4dRzt7yDLD3CbnvQBFU4EbNM6cc9B8lcoahWWwSgKfOMA8fI5Uo4+x9M4D60KTikM0L+3 EVNAOgYei+HTSkSdRCP1OJj77f0kA0bTRL6nRAlkNrvkJc8PUN53OxnGtSkkHnOKzXDwyu sfbLhLZ16PweES855lnAfbu7YXTfWA708G4jS80Msf0MUIL4N4o65sF9Anr/jA3TMyDunP 3t/Fta0f+vL/vF8d01pm+D8LimMmS0zrV73DmTzoneRZBWrP4ruhVhjt17k1Iw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XUkkJh+p; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -3.57 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XUkkJh+p; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 60C64C88A X-Spam-Score: -3.57 X-Migadu-Scanner: scn1.migadu.com X-TUID: KUOtB6J0PX6d Dear all, Recently, there have been multiple issues related to incorrect fontification: - https://list.orgmode.org/orgmode/23707.20428.546749.44853@frac.u-strasbg.fr/ - https://list.orgmode.org/orgmode/87fsujp7mc.fsf@web.de/ - https://list.orgmode.org/orgmode/87czvqxdn9.fsf@gmail.com/ - https://list.orgmode.org/8735nsv9qo.fsf@nicolasgoaziou.fr/T/#me1c44b6e493dd280cca4f042b833c24749ae4fe0 These issues keep appearing because our current fontification code is based on regexps and only approximates the actual Org grammar elements. It is largely a legacy from the times when org-element parser was not a thing. Maybe it is a time to upgrade the fontification according to our state-of-art parser? Instead of fontifying elements individually via regexps, we can leverage org-element-map, org-element-parse-buffer, org-element-cache, and jit-lock-mode. Each type of Org element/object can be assigned with a fontification function accepting a single argument - the element datum. Also, the fontification code can be move to a separate library. WDYT? Best, Ihor