Re: "Final" version of tty child frames

[Paul Eggert <eggert@cs.ucla.edu>]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

On 2024-10-22 07:40, Eli Zaretskii wrote:
> Paul, should we make line_hash_code return 'size_t' instead (and
> change the data type of the variables in 'scrolling' accordingly)?  Or
> maybe we should simply mask off high bits of the face_cache's pointer,
> leaving only the low 32 bits, before adding it to the hash?

Either approach should be correct. The attached (untested and 
uninstalled) patch does the equivalent of the latter, which is simpler. 
I don't know whether the more-complicated one would perform better.

This patch also fixes a glitch in that one should cast pointers to 
uintptr_t or to intptr_t, not to ptrdiff_t. The difference can matter on 
unusual platforms like CheriBSD where uintptr_t is wider than ptrdiff_t.

[-- Attachment #2: 0001-Fix-UB-in-line_hash_code.patch --]
[-- Type: text/x-patch, Size: 1157 bytes --]

From f624661eec854f09b28076b8f1bf80b1ed326475 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 22 Oct 2024 12:18:14 -0700
Subject: [PATCH] Fix UB in line_hash_code

* src/dispnew.c (line_hash_code): Avoid undefined behavior on
integer overflow.
---
 src/dispnew.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/dispnew.c b/src/dispnew.c
index 200ffaaca21..1ece9cc1d45 100644
--- a/src/dispnew.c
+++ b/src/dispnew.c
@@ -1175,12 +1175,12 @@ line_hash_code (struct frame *f, struct glyph_row *row)
       while (glyph < end)
 	{
 	  int c = glyph->u.ch;
-	  int face_id = glyph->face_id;
+	  unsigned int face_id = glyph->face_id;
 	  /* Struct frame can move with igc, and so on.  But we need
 	     something that takes different frames into account.  Use the
 	     face_cache pointer for that which is malloc'd.  */
 	  if (glyph->frame && glyph->frame != f)
-	    face_id += (ptrdiff_t) glyph->frame->face_cache;
+	    face_id += (uintptr_t) glyph->frame->face_cache;
 	  if (FRAME_MUST_WRITE_SPACES (f))
 	    c -= SPACEGLYPH;
 	  hash = (((hash << 4) + (hash >> 24)) & 0x0fffffff) + c;
-- 
2.43.0