Two questions about overlays

Two questions about overlays

[dalanicolai]

[-- Attachment #1.1: Type: text/plain, Size: 3268 bytes --]

I have two short questions about overlays. To clarify, let me start
with a short motivation.

I have been working on some alternative 'display engine' (I hope I am
using a somewhat correct term here).  Of course, I would prefer to
make functions reusable if possible (currently, image-dired, doc-view,
pdf-tools, and dvju all 'implemented their own' display engines). A
demonstration can be found in the animated gif for the 'universal
document reader' here <https://github.com/dalanicolai/doc-tools> (I would
prefer to reuse/contribute to
doc-view/tools, but it turns out to be much easier to start from
scratch and, if possible, port things over. That simultaneously cleans
up that code).

The 'essence' of that document reader is a very simple 'overlay grid',
which I like to provide as a 'reusable function' (the idea is very
simple and an example is provided in the following file . To try it,
you can simply load the code and show images from a directory by doing
M-x scrap-dir-images. I am not trying to replace image-dired. The
function is just an easy way to implement an example. Also the name
scrap comes from 'scrapbook', but I guess I could simply call it
overlay-grid. Let me know if you have some idea about the naming).

I have been using the overlay grid already in image-roll
<https://github.com/dalanicolai/image-roll.el>, but later I
'discovered' a much simpler way to implement the logic for
'dynamically updating' of visible overlays (i.e. pages, for 'normal
page sizes' we can not display all images of a document at
simultaneously), by simply using the 'overlays-in' function in a
'clever' (or actually quite straightforward) way (instead of
reimplementing that same functionality in lisp).

So now here are the questions:

- when displaying multiple pages in columns, I would like to use the
overlays-in function to determine which overlays should display
images.  So I am creating a full 'book roll' by giving the overlays a
size via the 'space' display property
<https://www.gnu.org/software/emacs/manual/html_node/elisp/Specified-Space.html>,
after which I use overlays-in
to determine which overlays are actually visible. However, after it
takes some time for the 'space' display property to take effect, so I
am manuall adding a 'sit-for' with some reasonable delay
time. However, I would like to ask if there is someone has an idea for
a 'better' mechanism to wait until/detect if the 'overlay expansion'
has finished.

If the explanation is not clear then please load the following file
and do 'M-x scrap-dir-images' on a directory that contains enough
images to not fit all on a single screen. It will print the number of
overlays found via 'overlays-in' directly after 'displaying the
images' (here by assigning the image as display property instead of
space). You will find it prints all overlays in the buffer (instead of
only the ones on screen). To find what I expect it to print now
(again) do 'M-: (overlays-in (window-start) (window-end))'.

My second question is about the function 'overlay-recenter' I don't
really understand its docstring. What kind of 'overlay lookup' would
go faster? What is 'overlay-lookup' anyway?

I hope that some experts here could clear things up in a few words (as
usually is the case :)

Thanks!

[-- Attachment #1.2: Type: text/html, Size: 3714 bytes --]

[-- Attachment #2: scrap-mode.el --]
[-- Type: text/x-emacs-lisp, Size: 3931 bytes --]

;;; image-scroll-mode.el --- Image and document display engine  -*- lexical-binding: t; -*-

;; Copyright (C) 2023  Daniel Nicolai

;; Author: Daniel Nicolai <dalanicolai@gmail.com>
;; Keywords: tools

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;;

;;; Code:
(defvar scrap-incompatible-modes '(visual-line-mode
                                   global-hl-line-mode))

(defvar-local scrap-overlays nil)
(defvar-local scrap-columns nil)
(defvar-local scrap-next-function #'scrap-next)

(defsubst scrap-current-overlay ()
  (car (overlays-at (point))))

(defsubst scrap-get-overlay-n ()
  (overlay-get (scrap-current-overlay) 'n))

(defun scrap-dir-images (dir)
  (interactive
   (list (read-file-name "Select document: " nil nil t nil #'file-directory-p)))
  (pop-to-buffer (get-buffer-create dir))
  (setq buffer-file-name dir)
  (let ((images (seq-filter #'image-supported-file-p (directory-files dir)))
	(max-w (window-text-width nil t))
	;; (max-h 200)
	)
    (scrap-create-overlays (length images) 1)
    (seq-do-indexed (lambda (im n)
		      (overlay-put (nth n scrap-overlays)
				   'display (create-image (concat (file-name-as-directory dir) im) nil nil
							  :max-width max-w)))
		    images))
  (print (length (overlays-in (window-start) (window-end)))))


(defun scrap-create-overlays (number
                              &optional columns hspace vspace
                              &rest overlay-props)
  (dolist (m scrap-incompatible-modes)
    (funcall m -1))
  (toggle-truncate-lines 1) ; also disables visual-mode
  (setq scrap-columns columns)
  (let (overlays)
    (dotimes (i number)
      (let* ((n (1+ i))
             (o (make-overlay
                 (point)
                 (progn (insert " ") (point))
                 (unless (= (% columns 3) 0)
		   (insert (make-string (or hspace 1) (string-to-char " ")))))))
        (when (and (= (% n (or columns 1)) 0)
                   (not (= n number)))
          (insert "\n")
          (insert (make-string (or vspace 1) (string-to-char "\n"))))
        (overlay-put o 'n n)
        (dotimes (j (/ (length overlay-props) 2))
          (let ((m (* j 2)))
            (overlay-put o (nth m overlay-props) (nth (+ m 1) overlay-props))))
        (push o overlays)))
    (goto-char (point-min))
    (setq scrap-overlays (nreverse overlays))
    (scrap-mode)))

(defun scrap-next (n &optional previous)
  (let* ((current (1- (scrap-get-overlay-n)))
         (next (nth (+ current (if previous (- n) n)) scrap-overlays)))
    (goto-char (overlay-start next))
    (overlay-recenter (point))))

(defun scrap-scroll-next (n)
  (interactive "p")
  (funcall scrap-next-function n))

(defun scrap-scroll-previous (n)
  (interactive "p")
  (funcall scrap-next-function n t))

(defun scrap-scroll-next-line (n)
  (interactive "p")
  (funcall scrap-next-function (* n scrap-columns)))

(defun scrap-scroll-previous-line (n)
  (interactive "p")
  (funcall scrap-next-function (* n scrap-columns) t))

(define-minor-mode scrap-mode "Display images in a grid."
  :lighter "Scrap"
  :keymap `((,(kbd "C-n") . scrap-scroll-next)
            (,(kbd "C-p") . scrap-scroll-previous)
            (,(kbd "C-f") . scrap-scroll-next-line)
            (,(kbd "C-b") . scrap-scroll-previous-line)))

(provide 'image-scroll-mode)
;;; image-scroll-mode.el ends here

Re: Two questions about overlays

[Eli Zaretskii]

> From: dalanicolai <dalanicolai@gmail.com>
> Date: Tue, 21 Feb 2023 13:46:56 +0100
> 
> So now here are the questions:
> 
> - when displaying multiple pages in columns, I would like to use the
> overlays-in function to determine which overlays should display
> images.  So I am creating a full 'book roll' by giving the overlays a
> size via the 'space' display property, after which I use overlays-in
> to determine which overlays are actually visible. However, after it
> takes some time for the 'space' display property to take effect, so I
> am manuall adding a 'sit-for' with some reasonable delay
> time. However, I would like to ask if there is someone has an idea for
> a 'better' mechanism to wait until/detect if the 'overlay expansion'
> has finished.
> 
> If the explanation is not clear then please load the following file
> and do 'M-x scrap-dir-images' on a directory that contains enough
> images to not fit all on a single screen. It will print the number of
> overlays found via 'overlays-in' directly after 'displaying the
> images' (here by assigning the image as display property instead of
> space). You will find it prints all overlays in the buffer (instead of
> only the ones on screen). To find what I expect it to print now
> (again) do 'M-: (overlays-in (window-start) (window-end))'.

I did all that, and I still don't understand the question.  In
particular, your code doesn't call sit-for, so I'm unsure what exactly
is the problem you are asking about here.

> My second question is about the function 'overlay-recenter' I don't
> really understand its docstring. What kind of 'overlay lookup' would
> go faster? What is 'overlay-lookup' anyway?

Looking up overlays that are relevant to a particular buffer position.

But this function is a no-op in Emacs 29 and later, since the overlays
were reimplemented in a way that makes it unnecessary to "center" the
list of overlays.  So you can forget about that and ignore this
function.

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 3692 bytes --]

In this example code, I am simply displaying all images at once,
because I am assuming that the images in the directory are not too
many and that they are small (which is not a very reasonable
assumption, but this is just my personal 'test' function).

Indeed, the example does not call sit-for, because it should show that
the printed number of overlays, is the number of all images in the
directory (instead of only the number of images currently on the
screen, i.e. within '(overlays-in (window-start) (window-end))' ).

For example, I have a directory with 108 images, when I run 'M-x
scrap-dir-images' it print 108, although it should print the number of
images on screen i.e. by '(overlays-in (window-start) (window-end))',
which value is what 'scrap-dir-images' prints.  However, the images
are normal 'foto' size, so that I only see two images on screen, and
indeed doing 'M-: (overlays-in (window-start) (window-end))' manually
now prints 2.  If I had added the '(sit-for)' in the example
code, then I would have 'fixed' the problem, and 'scrap-dir-images'
would have printed 2 immediately because the display property
('expansion of the overlays') got enough time to take effect.

I hope this clears things up, but of course I would be happy to try
another explanation (e.g. sending by adding an animated gif).

But this function is a no-op in Emacs 29 and later, since the overlays
> were reimplemented in a way that makes it unnecessary to "center" the
> list of overlays.  So you can forget about that and ignore this
> function.
>

Thanks, that is indeed handy to know.

On Tue, 21 Feb 2023 at 14:21, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: dalanicolai <dalanicolai@gmail.com>
> > Date: Tue, 21 Feb 2023 13:46:56 +0100
> >
> > So now here are the questions:
> >
> > - when displaying multiple pages in columns, I would like to use the
> > overlays-in function to determine which overlays should display
> > images.  So I am creating a full 'book roll' by giving the overlays a
> > size via the 'space' display property, after which I use overlays-in
> > to determine which overlays are actually visible. However, after it
> > takes some time for the 'space' display property to take effect, so I
> > am manuall adding a 'sit-for' with some reasonable delay
> > time. However, I would like to ask if there is someone has an idea for
> > a 'better' mechanism to wait until/detect if the 'overlay expansion'
> > has finished.
> >
> > If the explanation is not clear then please load the following file
> > and do 'M-x scrap-dir-images' on a directory that contains enough
> > images to not fit all on a single screen. It will print the number of
> > overlays found via 'overlays-in' directly after 'displaying the
> > images' (here by assigning the image as display property instead of
> > space). You will find it prints all overlays in the buffer (instead of
> > only the ones on screen). To find what I expect it to print now
> > (again) do 'M-: (overlays-in (window-start) (window-end))'.
>
> I did all that, and I still don't understand the question.  In
> particular, your code doesn't call sit-for, so I'm unsure what exactly
> is the problem you are asking about here.
>
> > My second question is about the function 'overlay-recenter' I don't
> > really understand its docstring. What kind of 'overlay lookup' would
> > go faster? What is 'overlay-lookup' anyway?
>
> Looking up overlays that are relevant to a particular buffer position.
>
> But this function is a no-op in Emacs 29 and later, since the overlays
> were reimplemented in a way that makes it unnecessary to "center" the
> list of overlays.  So you can forget about that and ignore this
> function.
>

[-- Attachment #2: Type: text/html, Size: 4736 bytes --]

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 4611 bytes --]

Let me, additionally, very briefly explain what the code is doing, and
what I am expecting (as an attempt to make it immediately clear what I
mean).

So the code is first simply creating (for my dir) 108 overlays.  Then
in 'scrap-dir-images' are made to display the images from the
directory. Immediately, after the
'(seq-do-indexed (lambda (im n) (overlay-put ... 'display ...)'
the function print the value of '(overlays-in (window-start) (window-end))',
which return 108. However, I expect it to print 2, as the overlays should
have the size of the images now (and I see only two images simultaneously on
screen). Indeed when doing 'M-: (overlays-in (window-start) (window-end))'
immediately after creating the 'image gallery', it returns 2.

On Tue, 21 Feb 2023 at 14:39, dalanicolai <dalanicolai@gmail.com> wrote:

> In this example code, I am simply displaying all images at once,
> because I am assuming that the images in the directory are not too
> many and that they are small (which is not a very reasonable
> assumption, but this is just my personal 'test' function).
>
> Indeed, the example does not call sit-for, because it should show that
> the printed number of overlays, is the number of all images in the
> directory (instead of only the number of images currently on the
> screen, i.e. within '(overlays-in (window-start) (window-end))' ).
>
> For example, I have a directory with 108 images, when I run 'M-x
> scrap-dir-images' it print 108, although it should print the number of
> images on screen i.e. by '(overlays-in (window-start) (window-end))',
> which value is what 'scrap-dir-images' prints.  However, the images
> are normal 'foto' size, so that I only see two images on screen, and
> indeed doing 'M-: (overlays-in (window-start) (window-end))' manually
> now prints 2.  If I had added the '(sit-for)' in the example
> code, then I would have 'fixed' the problem, and 'scrap-dir-images'
> would have printed 2 immediately because the display property
> ('expansion of the overlays') got enough time to take effect.
>
> I hope this clears things up, but of course I would be happy to try
> another explanation (e.g. sending by adding an animated gif).
>
> But this function is a no-op in Emacs 29 and later, since the overlays
>> were reimplemented in a way that makes it unnecessary to "center" the
>> list of overlays.  So you can forget about that and ignore this
>> function.
>>
>
> Thanks, that is indeed handy to know.
>
> On Tue, 21 Feb 2023 at 14:21, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: dalanicolai <dalanicolai@gmail.com>
>> > Date: Tue, 21 Feb 2023 13:46:56 +0100
>> >
>> > So now here are the questions:
>> >
>> > - when displaying multiple pages in columns, I would like to use the
>> > overlays-in function to determine which overlays should display
>> > images.  So I am creating a full 'book roll' by giving the overlays a
>> > size via the 'space' display property, after which I use overlays-in
>> > to determine which overlays are actually visible. However, after it
>> > takes some time for the 'space' display property to take effect, so I
>> > am manuall adding a 'sit-for' with some reasonable delay
>> > time. However, I would like to ask if there is someone has an idea for
>> > a 'better' mechanism to wait until/detect if the 'overlay expansion'
>> > has finished.
>> >
>> > If the explanation is not clear then please load the following file
>> > and do 'M-x scrap-dir-images' on a directory that contains enough
>> > images to not fit all on a single screen. It will print the number of
>> > overlays found via 'overlays-in' directly after 'displaying the
>> > images' (here by assigning the image as display property instead of
>> > space). You will find it prints all overlays in the buffer (instead of
>> > only the ones on screen). To find what I expect it to print now
>> > (again) do 'M-: (overlays-in (window-start) (window-end))'.
>>
>> I did all that, and I still don't understand the question.  In
>> particular, your code doesn't call sit-for, so I'm unsure what exactly
>> is the problem you are asking about here.
>>
>> > My second question is about the function 'overlay-recenter' I don't
>> > really understand its docstring. What kind of 'overlay lookup' would
>> > go faster? What is 'overlay-lookup' anyway?
>>
>> Looking up overlays that are relevant to a particular buffer position.
>>
>> But this function is a no-op in Emacs 29 and later, since the overlays
>> were reimplemented in a way that makes it unnecessary to "center" the
>> list of overlays.  So you can forget about that and ignore this
>> function.
>>
>

[-- Attachment #2: Type: text/html, Size: 5949 bytes --]

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 4915 bytes --]

B.t.w. I am using Emacs 29 (and 30), and the overlay-recenter is still
there.
Maybe it would be handy to remove it then?

On Tue, 21 Feb 2023 at 14:50, dalanicolai <dalanicolai@gmail.com> wrote:

> Let me, additionally, very briefly explain what the code is doing, and
> what I am expecting (as an attempt to make it immediately clear what I
> mean).
>
> So the code is first simply creating (for my dir) 108 overlays.  Then
> in 'scrap-dir-images' are made to display the images from the
> directory. Immediately, after the
> '(seq-do-indexed (lambda (im n) (overlay-put ... 'display ...)'
> the function print the value of '(overlays-in (window-start)
> (window-end))',
> which return 108. However, I expect it to print 2, as the overlays should
> have the size of the images now (and I see only two images simultaneously
> on
> screen). Indeed when doing 'M-: (overlays-in (window-start) (window-end))'
> immediately after creating the 'image gallery', it returns 2.
>
> On Tue, 21 Feb 2023 at 14:39, dalanicolai <dalanicolai@gmail.com> wrote:
>
>> In this example code, I am simply displaying all images at once,
>> because I am assuming that the images in the directory are not too
>> many and that they are small (which is not a very reasonable
>> assumption, but this is just my personal 'test' function).
>>
>> Indeed, the example does not call sit-for, because it should show that
>> the printed number of overlays, is the number of all images in the
>> directory (instead of only the number of images currently on the
>> screen, i.e. within '(overlays-in (window-start) (window-end))' ).
>>
>> For example, I have a directory with 108 images, when I run 'M-x
>> scrap-dir-images' it print 108, although it should print the number of
>> images on screen i.e. by '(overlays-in (window-start) (window-end))',
>> which value is what 'scrap-dir-images' prints.  However, the images
>> are normal 'foto' size, so that I only see two images on screen, and
>> indeed doing 'M-: (overlays-in (window-start) (window-end))' manually
>> now prints 2.  If I had added the '(sit-for)' in the example
>> code, then I would have 'fixed' the problem, and 'scrap-dir-images'
>> would have printed 2 immediately because the display property
>> ('expansion of the overlays') got enough time to take effect.
>>
>> I hope this clears things up, but of course I would be happy to try
>> another explanation (e.g. sending by adding an animated gif).
>>
>> But this function is a no-op in Emacs 29 and later, since the overlays
>>> were reimplemented in a way that makes it unnecessary to "center" the
>>> list of overlays.  So you can forget about that and ignore this
>>> function.
>>>
>>
>> Thanks, that is indeed handy to know.
>>
>> On Tue, 21 Feb 2023 at 14:21, Eli Zaretskii <eliz@gnu.org> wrote:
>>
>>> > From: dalanicolai <dalanicolai@gmail.com>
>>> > Date: Tue, 21 Feb 2023 13:46:56 +0100
>>> >
>>> > So now here are the questions:
>>> >
>>> > - when displaying multiple pages in columns, I would like to use the
>>> > overlays-in function to determine which overlays should display
>>> > images.  So I am creating a full 'book roll' by giving the overlays a
>>> > size via the 'space' display property, after which I use overlays-in
>>> > to determine which overlays are actually visible. However, after it
>>> > takes some time for the 'space' display property to take effect, so I
>>> > am manuall adding a 'sit-for' with some reasonable delay
>>> > time. However, I would like to ask if there is someone has an idea for
>>> > a 'better' mechanism to wait until/detect if the 'overlay expansion'
>>> > has finished.
>>> >
>>> > If the explanation is not clear then please load the following file
>>> > and do 'M-x scrap-dir-images' on a directory that contains enough
>>> > images to not fit all on a single screen. It will print the number of
>>> > overlays found via 'overlays-in' directly after 'displaying the
>>> > images' (here by assigning the image as display property instead of
>>> > space). You will find it prints all overlays in the buffer (instead of
>>> > only the ones on screen). To find what I expect it to print now
>>> > (again) do 'M-: (overlays-in (window-start) (window-end))'.
>>>
>>> I did all that, and I still don't understand the question.  In
>>> particular, your code doesn't call sit-for, so I'm unsure what exactly
>>> is the problem you are asking about here.
>>>
>>> > My second question is about the function 'overlay-recenter' I don't
>>> > really understand its docstring. What kind of 'overlay lookup' would
>>> > go faster? What is 'overlay-lookup' anyway?
>>>
>>> Looking up overlays that are relevant to a particular buffer position.
>>>
>>> But this function is a no-op in Emacs 29 and later, since the overlays
>>> were reimplemented in a way that makes it unnecessary to "center" the
>>> list of overlays.  So you can forget about that and ignore this
>>> function.
>>>
>>

[-- Attachment #2: Type: text/html, Size: 6467 bytes --]

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 6211 bytes --]

>
> In this example code, I am simply displaying all images at once,
> because I am assuming that the images in the directory are not too
> many and that they are small (which is not a very reasonable
> assumption, but this is just my personal 'test' function).


This is exactly why I want to display the images dynamically. Because
for a large directory with large images, displaying all images at once
would quickly use up all memory. So, what I am doing for example in
doc-scroll
<https://github.com/dalanicolai/doc-tools/blob/main/doc-scroll.el> (the
'engine' for doc-tools <https://github.com/dalanicolai/doc-tools>, in that
file you can find the
'sit-for'), is to first assign all overlays the size of the images by
using the 'space' display property (which uses only very minimal
memory). Then, using, `(overlays-in (window-start) (window-end))`, I
can determine which images should be displayed. When scrolling, I am
again using this function to determine which images to update (which
to display and which to undisplay). In this way, I can list a huge
amount of large images without running out of memory.



On Tue, 21 Feb 2023 at 14:57, dalanicolai <dalanicolai@gmail.com> wrote:

> B.t.w. I am using Emacs 29 (and 30), and the overlay-recenter is still
> there.
> Maybe it would be handy to remove it then?
>
> On Tue, 21 Feb 2023 at 14:50, dalanicolai <dalanicolai@gmail.com> wrote:
>
>> Let me, additionally, very briefly explain what the code is doing, and
>> what I am expecting (as an attempt to make it immediately clear what I
>> mean).
>>
>> So the code is first simply creating (for my dir) 108 overlays.  Then
>> in 'scrap-dir-images' are made to display the images from the
>> directory. Immediately, after the
>> '(seq-do-indexed (lambda (im n) (overlay-put ... 'display ...)'
>> the function print the value of '(overlays-in (window-start)
>> (window-end))',
>> which return 108. However, I expect it to print 2, as the overlays should
>> have the size of the images now (and I see only two images simultaneously
>> on
>> screen). Indeed when doing 'M-: (overlays-in (window-start) (window-end))'
>> immediately after creating the 'image gallery', it returns 2.
>>
>> On Tue, 21 Feb 2023 at 14:39, dalanicolai <dalanicolai@gmail.com> wrote:
>>
>>> In this example code, I am simply displaying all images at once,
>>> because I am assuming that the images in the directory are not too
>>> many and that they are small (which is not a very reasonable
>>> assumption, but this is just my personal 'test' function).
>>>
>>> Indeed, the example does not call sit-for, because it should show that
>>> the printed number of overlays, is the number of all images in the
>>> directory (instead of only the number of images currently on the
>>> screen, i.e. within '(overlays-in (window-start) (window-end))' ).
>>>
>>> For example, I have a directory with 108 images, when I run 'M-x
>>> scrap-dir-images' it print 108, although it should print the number of
>>> images on screen i.e. by '(overlays-in (window-start) (window-end))',
>>> which value is what 'scrap-dir-images' prints.  However, the images
>>> are normal 'foto' size, so that I only see two images on screen, and
>>> indeed doing 'M-: (overlays-in (window-start) (window-end))' manually
>>> now prints 2.  If I had added the '(sit-for)' in the example
>>> code, then I would have 'fixed' the problem, and 'scrap-dir-images'
>>> would have printed 2 immediately because the display property
>>> ('expansion of the overlays') got enough time to take effect.
>>>
>>> I hope this clears things up, but of course I would be happy to try
>>> another explanation (e.g. sending by adding an animated gif).
>>>
>>> But this function is a no-op in Emacs 29 and later, since the overlays
>>>> were reimplemented in a way that makes it unnecessary to "center" the
>>>> list of overlays.  So you can forget about that and ignore this
>>>> function.
>>>>
>>>
>>> Thanks, that is indeed handy to know.
>>>
>>> On Tue, 21 Feb 2023 at 14:21, Eli Zaretskii <eliz@gnu.org> wrote:
>>>
>>>> > From: dalanicolai <dalanicolai@gmail.com>
>>>> > Date: Tue, 21 Feb 2023 13:46:56 +0100
>>>> >
>>>> > So now here are the questions:
>>>> >
>>>> > - when displaying multiple pages in columns, I would like to use the
>>>> > overlays-in function to determine which overlays should display
>>>> > images.  So I am creating a full 'book roll' by giving the overlays a
>>>> > size via the 'space' display property, after which I use overlays-in
>>>> > to determine which overlays are actually visible. However, after it
>>>> > takes some time for the 'space' display property to take effect, so I
>>>> > am manuall adding a 'sit-for' with some reasonable delay
>>>> > time. However, I would like to ask if there is someone has an idea for
>>>> > a 'better' mechanism to wait until/detect if the 'overlay expansion'
>>>> > has finished.
>>>> >
>>>> > If the explanation is not clear then please load the following file
>>>> > and do 'M-x scrap-dir-images' on a directory that contains enough
>>>> > images to not fit all on a single screen. It will print the number of
>>>> > overlays found via 'overlays-in' directly after 'displaying the
>>>> > images' (here by assigning the image as display property instead of
>>>> > space). You will find it prints all overlays in the buffer (instead of
>>>> > only the ones on screen). To find what I expect it to print now
>>>> > (again) do 'M-: (overlays-in (window-start) (window-end))'.
>>>>
>>>> I did all that, and I still don't understand the question.  In
>>>> particular, your code doesn't call sit-for, so I'm unsure what exactly
>>>> is the problem you are asking about here.
>>>>
>>>> > My second question is about the function 'overlay-recenter' I don't
>>>> > really understand its docstring. What kind of 'overlay lookup' would
>>>> > go faster? What is 'overlay-lookup' anyway?
>>>>
>>>> Looking up overlays that are relevant to a particular buffer position.
>>>>
>>>> But this function is a no-op in Emacs 29 and later, since the overlays
>>>> were reimplemented in a way that makes it unnecessary to "center" the
>>>> list of overlays.  So you can forget about that and ignore this
>>>> function.
>>>>
>>>

[-- Attachment #2: Type: text/html, Size: 8247 bytes --]

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1.1: Type: text/plain, Size: 5965 bytes --]

Okay, finally, (I should have done this immediately), here is the best
and most brief explanation: So load the code of the attached UPDATED
scrap-mode.el. Then do `M-x scrap-expand-overlays`. It will print 1000, but
I expect it to print the number of overlays you are actually seeing on
screen, because the function calls

(print (length (overlays-in (window-start) (window-end)))))

after adding the 'space' display properties/expanding the overlays. So
although the display properties have been added, they seem to not have
taken effect at the time

(print (length (overlays-in (window-start) (window-end)))))

is evaluated. If you uncomment the `sit-for` in
`scrap-expand-overlays`, then you will see it prints a different
(i.e. the correct/expected) number.

Another way to get the expected number, is to now do

M-: (length (overlays-in (window-start) (window-end)))

On Tue, 21 Feb 2023 at 14:57, dalanicolai <dalanicolai@gmail.com> wrote:

> B.t.w. I am using Emacs 29 (and 30), and the overlay-recenter is still
> there.
> Maybe it would be handy to remove it then?
>
> On Tue, 21 Feb 2023 at 14:50, dalanicolai <dalanicolai@gmail.com> wrote:
>
>> Let me, additionally, very briefly explain what the code is doing, and
>> what I am expecting (as an attempt to make it immediately clear what I
>> mean).
>>
>> So the code is first simply creating (for my dir) 108 overlays.  Then
>> in 'scrap-dir-images' are made to display the images from the
>> directory. Immediately, after the
>> '(seq-do-indexed (lambda (im n) (overlay-put ... 'display ...)'
>> the function print the value of '(overlays-in (window-start)
>> (window-end))',
>> which return 108. However, I expect it to print 2, as the overlays should
>> have the size of the images now (and I see only two images simultaneously
>> on
>> screen). Indeed when doing 'M-: (overlays-in (window-start) (window-end))'
>> immediately after creating the 'image gallery', it returns 2.
>>
>> On Tue, 21 Feb 2023 at 14:39, dalanicolai <dalanicolai@gmail.com> wrote:
>>
>>> In this example code, I am simply displaying all images at once,
>>> because I am assuming that the images in the directory are not too
>>> many and that they are small (which is not a very reasonable
>>> assumption, but this is just my personal 'test' function).
>>>
>>> Indeed, the example does not call sit-for, because it should show that
>>> the printed number of overlays, is the number of all images in the
>>> directory (instead of only the number of images currently on the
>>> screen, i.e. within '(overlays-in (window-start) (window-end))' ).
>>>
>>> For example, I have a directory with 108 images, when I run 'M-x
>>> scrap-dir-images' it print 108, although it should print the number of
>>> images on screen i.e. by '(overlays-in (window-start) (window-end))',
>>> which value is what 'scrap-dir-images' prints.  However, the images
>>> are normal 'foto' size, so that I only see two images on screen, and
>>> indeed doing 'M-: (overlays-in (window-start) (window-end))' manually
>>> now prints 2.  If I had added the '(sit-for)' in the example
>>> code, then I would have 'fixed' the problem, and 'scrap-dir-images'
>>> would have printed 2 immediately because the display property
>>> ('expansion of the overlays') got enough time to take effect.
>>>
>>> I hope this clears things up, but of course I would be happy to try
>>> another explanation (e.g. sending by adding an animated gif).
>>>
>>> But this function is a no-op in Emacs 29 and later, since the overlays
>>>> were reimplemented in a way that makes it unnecessary to "center" the
>>>> list of overlays.  So you can forget about that and ignore this
>>>> function.
>>>>
>>>
>>> Thanks, that is indeed handy to know.
>>>
>>> On Tue, 21 Feb 2023 at 14:21, Eli Zaretskii <eliz@gnu.org> wrote:
>>>
>>>> > From: dalanicolai <dalanicolai@gmail.com>
>>>> > Date: Tue, 21 Feb 2023 13:46:56 +0100
>>>> >
>>>> > So now here are the questions:
>>>> >
>>>> > - when displaying multiple pages in columns, I would like to use the
>>>> > overlays-in function to determine which overlays should display
>>>> > images.  So I am creating a full 'book roll' by giving the overlays a
>>>> > size via the 'space' display property, after which I use overlays-in
>>>> > to determine which overlays are actually visible. However, after it
>>>> > takes some time for the 'space' display property to take effect, so I
>>>> > am manuall adding a 'sit-for' with some reasonable delay
>>>> > time. However, I would like to ask if there is someone has an idea for
>>>> > a 'better' mechanism to wait until/detect if the 'overlay expansion'
>>>> > has finished.
>>>> >
>>>> > If the explanation is not clear then please load the following file
>>>> > and do 'M-x scrap-dir-images' on a directory that contains enough
>>>> > images to not fit all on a single screen. It will print the number of
>>>> > overlays found via 'overlays-in' directly after 'displaying the
>>>> > images' (here by assigning the image as display property instead of
>>>> > space). You will find it prints all overlays in the buffer (instead of
>>>> > only the ones on screen). To find what I expect it to print now
>>>> > (again) do 'M-: (overlays-in (window-start) (window-end))'.
>>>>
>>>> I did all that, and I still don't understand the question.  In
>>>> particular, your code doesn't call sit-for, so I'm unsure what exactly
>>>> is the problem you are asking about here.
>>>>
>>>> > My second question is about the function 'overlay-recenter' I don't
>>>> > really understand its docstring. What kind of 'overlay lookup' would
>>>> > go faster? What is 'overlay-lookup' anyway?
>>>>
>>>> Looking up overlays that are relevant to a particular buffer position.
>>>>
>>>> But this function is a no-op in Emacs 29 and later, since the overlays
>>>> were reimplemented in a way that makes it unnecessary to "center" the
>>>> list of overlays.  So you can forget about that and ignore this
>>>> function.
>>>>
>>>

[-- Attachment #1.2: Type: text/html, Size: 7783 bytes --]

[-- Attachment #2: scrap-mode.el --]
[-- Type: text/x-emacs-lisp, Size: 4337 bytes --]

;;; image-scroll-mode.el --- Image and document display engine  -*- lexical-binding: t; -*-

;; Copyright (C) 2023  Daniel Nicolai

;; Author: Daniel Nicolai <dalanicolai@gmail.com>
;; Keywords: tools

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;;

;;; Code:
(defvar scrap-incompatible-modes '(visual-line-mode
                                   global-hl-line-mode))

(defvar-local scrap-overlays nil)
(defvar-local scrap-columns nil)
(defvar-local scrap-next-function #'scrap-next)

(defsubst scrap-current-overlay ()
  (car (overlays-at (point))))

(defsubst scrap-get-overlay-n ()
  (overlay-get (scrap-current-overlay) 'n))

(defun scrap-dir-images (dir)
  (interactive
   (list (read-file-name "Select document: " nil nil t nil #'file-directory-p)))
  (pop-to-buffer (get-buffer-create dir))
  (setq buffer-file-name dir)
  (let ((images (seq-filter #'image-supported-file-p (directory-files dir)))
	(max-w (window-text-width nil t))
	;; (max-h 200)
	)
    (scrap-create-overlays (length images) 1)
    (seq-do-indexed (lambda (im n)
		      (overlay-put (nth n scrap-overlays)
				   'display (create-image (concat (file-name-as-directory dir) im) nil nil
							  :max-width max-w)))
		    images))
  (print (length (overlays-in (window-start) (window-end)))))

(defun scrap-expand-overlays ()
  (interactive)
  (let ((number-of-overlays 1000))
    (pop-to-buffer "*scrap-example*")
    (scrap-create-overlays number-of-overlays 1)
    (dolist (o scrap-overlays)
      (overlay-put o 'display '(space . (:width (100) :height (140))))
      (overlay-put o 'face '(:background "gray"))))
  ;; (sit-for 0.05)
  (print (length (overlays-in (window-start) (window-end)))))

(defun scrap-create-overlays (number
                              &optional columns hspace vspace
                              &rest overlay-props)
  (dolist (m scrap-incompatible-modes)
    (funcall m -1))
  (toggle-truncate-lines 1) ; also disables visual-mode
  (setq scrap-columns columns)
  (let (overlays)
    (dotimes (i number)
      (let* ((n (1+ i))
             (o (make-overlay
                 (point)
                 (progn (insert " ") (point))
                 (unless (= (% columns 3) 0)
		   (insert (make-string (or hspace 1) (string-to-char " ")))))))
        (when (and (= (% n (or columns 1)) 0)
                   (not (= n number)))
          (insert "\n")
          (insert (make-string (or vspace 1) (string-to-char "\n"))))
        (overlay-put o 'n n)
        (dotimes (j (/ (length overlay-props) 2))
          (let ((m (* j 2)))
            (overlay-put o (nth m overlay-props) (nth (+ m 1) overlay-props))))
        (push o overlays)))
    (goto-char (point-min))
    (setq scrap-overlays (nreverse overlays))
    (scrap-mode)))

(defun scrap-next (n &optional previous)
  (let* ((current (1- (scrap-get-overlay-n)))
         (next (nth (+ current (if previous (- n) n)) scrap-overlays)))
    (goto-char (overlay-start next))
    (overlay-recenter (point))))

(defun scrap-scroll-next (n)
  (interactive "p")
  (funcall scrap-next-function n))

(defun scrap-scroll-previous (n)
  (interactive "p")
  (funcall scrap-next-function n t))

(defun scrap-scroll-next-line (n)
  (interactive "p")
  (funcall scrap-next-function (* n scrap-columns)))

(defun scrap-scroll-previous-line (n)
  (interactive "p")
  (funcall scrap-next-function (* n scrap-columns) t))

(define-minor-mode scrap-mode "Display images in a grid."
  :lighter "Scrap"
  :keymap `((,(kbd "C-n") . scrap-scroll-next)
            (,(kbd "C-p") . scrap-scroll-previous)
            (,(kbd "C-f") . scrap-scroll-next-line)
            (,(kbd "C-b") . scrap-scroll-previous-line)))

(provide 'image-scroll-mode)
;;; image-scroll-mode.el ends here

Re: Two questions about overlays

[Eli Zaretskii]

> From: dalanicolai <dalanicolai@gmail.com>
> Date: Tue, 21 Feb 2023 14:39:36 +0100
> Cc: emacs-devel@gnu.org
> 
> In this example code, I am simply displaying all images at once,
> because I am assuming that the images in the directory are not too
> many and that they are small (which is not a very reasonable
> assumption, but this is just my personal 'test' function).
> 
> Indeed, the example does not call sit-for, because it should show that
> the printed number of overlays, is the number of all images in the
> directory (instead of only the number of images currently on the
> screen, i.e. within '(overlays-in (window-start) (window-end))' ).

You assume that window-start and window-end will reliably tell you the
beginning and end of the window in the middle of a Lisp program?
Their doc strings explicitly say that the values are updated by
redisplay.  window-end is specifically documented to be accurate only
after redisplay ends.  I believe this is why you need to call sit-for:
that function triggers redisplay.

> B.t.w. I am using Emacs 29 (and 30), and the overlay-recenter is still there.
> Maybe it would be handy to remove it then?

We don't want to remove it because that could break some third-party
packages.  So we made the function do nothing instead.

Re: Two questions about overlays

[dalanicolai]

[-- Attachment #1: Type: text/plain, Size: 1934 bytes --]

>
> You assume that window-start and window-end will reliably tell you the
> beginning and end of the window in the middle of a Lisp program?
> Their doc strings explicitly say that the values are updated by
> redisplay.  window-end is specifically documented to be accurate only
> after redisplay ends.  I believe this is why you need to call sit-for:
> that function triggers redisplay.


Ah okay, I had not seen/noted that (part of the) docstring. Well, this
clears things up (now for both questions).

Thanks again for your help!

On Tue, 21 Feb 2023 at 16:03, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: dalanicolai <dalanicolai@gmail.com>
> > Date: Tue, 21 Feb 2023 14:39:36 +0100
> > Cc: emacs-devel@gnu.org
> >
> > In this example code, I am simply displaying all images at once,
> > because I am assuming that the images in the directory are not too
> > many and that they are small (which is not a very reasonable
> > assumption, but this is just my personal 'test' function).
> >
> > Indeed, the example does not call sit-for, because it should show that
> > the printed number of overlays, is the number of all images in the
> > directory (instead of only the number of images currently on the
> > screen, i.e. within '(overlays-in (window-start) (window-end))' ).
>
> You assume that window-start and window-end will reliably tell you the
> beginning and end of the window in the middle of a Lisp program?
> Their doc strings explicitly say that the values are updated by
> redisplay.  window-end is specifically documented to be accurate only
> after redisplay ends.  I believe this is why you need to call sit-for:
> that function triggers redisplay.
>
> > B.t.w. I am using Emacs 29 (and 30), and the overlay-recenter is still
> there.
> > Maybe it would be handy to remove it then?
>
> We don't want to remove it because that could break some third-party
> packages.  So we made the function do nothing instead.
>

[-- Attachment #2: Type: text/html, Size: 2683 bytes --]