all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Should `get-file-buffer' be implemented in Elisp?
@ 2023-07-16 18:40 Ihor Radchenko
  2023-07-16 18:51 ` Eli Zaretskii
  0 siblings, 1 reply; 10+ messages in thread
From: Ihor Radchenko @ 2023-07-16 18:40 UTC (permalink / raw)
  To: emacs-devel

Hi,

AFAIU, we generally aim to move C functions to Elisp.
And `get-file-buffer' does not appear to have anything that prevents it
from being implemented in Elisp.

So, should it? Are there any other known functions that need to be
converted?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 18:40 Should `get-file-buffer' be implemented in Elisp? Ihor Radchenko
@ 2023-07-16 18:51 ` Eli Zaretskii
  2023-07-16 19:05   ` Ihor Radchenko
  0 siblings, 1 reply; 10+ messages in thread
From: Eli Zaretskii @ 2023-07-16 18:51 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-devel

> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Sun, 16 Jul 2023 18:40:12 +0000
> 
> AFAIU, we generally aim to move C functions to Elisp.

Only when there's a good reason to do so.

> And `get-file-buffer' does not appear to have anything that prevents it
> from being implemented in Elisp.
> 
> So, should it? Are there any other known functions that need to be
> converted?

Let's not make changes without a good reason, okay?



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 18:51 ` Eli Zaretskii
@ 2023-07-16 19:05   ` Ihor Radchenko
  2023-07-16 19:10     ` Eli Zaretskii
  0 siblings, 1 reply; 10+ messages in thread
From: Ihor Radchenko @ 2023-07-16 19:05 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> AFAIU, we generally aim to move C functions to Elisp.
>
> Only when there's a good reason to do so.

Interesting.
Then, my memory is failing me. I recall being told on emacs-devel that
the general direction is to have as many things as possible in Elisp and
reduce the C core.

I stay corrected.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 19:05   ` Ihor Radchenko
@ 2023-07-16 19:10     ` Eli Zaretskii
  2023-07-16 20:19       ` Ihor Radchenko
  0 siblings, 1 reply; 10+ messages in thread
From: Eli Zaretskii @ 2023-07-16 19:10 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: emacs-devel

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Sun, 16 Jul 2023 19:05:00 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> AFAIU, we generally aim to move C functions to Elisp.
> >
> > Only when there's a good reason to do so.
> 
> Interesting.
> Then, my memory is failing me. I recall being told on emacs-devel that
> the general direction is to have as many things as possible in Elisp and
> reduce the C core.
> 
> I stay corrected.

No need to stand corrected: the key is the interpretation of "as many
as possible" -- it is not as simplistic as perhaps it could be.

Basically, we have enough good reasons to rock the boat and make
changes which could as side effects introduce bugs and subtle behavior
change; let's not risk that when we don't have such a good reason.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 19:10     ` Eli Zaretskii
@ 2023-07-16 20:19       ` Ihor Radchenko
  2023-07-16 23:28         ` Po Lu
  2023-07-17  5:06         ` tomas
  0 siblings, 2 replies; 10+ messages in thread
From: Ihor Radchenko @ 2023-07-16 20:19 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> I stay corrected.
>
> No need to stand corrected: the key is the interpretation of "as many
> as possible" -- it is not as simplistic as perhaps it could be.
>
> Basically, we have enough good reasons to rock the boat and make
> changes which could as side effects introduce bugs and subtle behavior
> change; let's not risk that when we don't have such a good reason.

Ok.

My original motivation about `get-file-buffer' was an observation that
searching alist might be sub-optimal.
I now have (length (buffer-list)) ;=> 578
and I though that it might be a good idea to keep track of
buffer name -> buffer associations in a hash table.
In particular, when using `get-buffer-create', and the buffer name does
not exist, Emacs has to scan all the buffer list, fail, and go ahead
creating the buffer. Hash table would help significantly in such
scenarios. And in general lookup as well as hash tables start to
outperform alists just at 20-30 elements [1]

But hacking C code and adding yet another internal variable looked like
not justified.

[1] https://flandrew.srht.site/listful/embracing-emacs-lisp-hash-tables-introducing-xht.html

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 20:19       ` Ihor Radchenko
@ 2023-07-16 23:28         ` Po Lu
  2023-07-17  5:06         ` tomas
  1 sibling, 0 replies; 10+ messages in thread
From: Po Lu @ 2023-07-16 23:28 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Eli Zaretskii, emacs-devel

Ihor Radchenko <yantar92@posteo.net> writes:

> My original motivation about `get-file-buffer' was an observation that
> searching alist might be sub-optimal.

Let's wait for this problem to demonstrate itself in practice before
looking for drastic solutions.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-16 20:19       ` Ihor Radchenko
  2023-07-16 23:28         ` Po Lu
@ 2023-07-17  5:06         ` tomas
  2023-07-17  9:28           ` Ihor Radchenko
  1 sibling, 1 reply; 10+ messages in thread
From: tomas @ 2023-07-17  5:06 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Eli Zaretskii, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]

On Sun, Jul 16, 2023 at 08:19:10PM +0000, Ihor Radchenko wrote:

[...]

> scenarios. And in general lookup as well as hash tables start to
> outperform alists just at 20-30 elements [1]

For me, it's more like 45-ish:

(let* ((size 50) ; alist, hash size
       (runs 10000) ; benchmark runs
       (alist '())
       (hash (make-hash-table :test 'eq :size 1549))
       (keys (make-vector size nil))
       (count size))
  ;; setup
  (while (> count 0)
    (setq count (1- count))
    (let ((sym (intern (format "SYM%04d%03d" (random 10000) count))))
      (aset keys count sym)
      (setq alist (cons (cons sym "foo") alist))
      (puthash sym "foo" hash)))
  ;; run
  (cons
   (benchmark runs '(assq (aref keys (random size)) alist))
   (benchmark runs '(gethash (aref keys (random size)) hash))))

And even at 100, the absolute time is rather negligible. Somewhere
near 5000 buffers my (not very fast) machine reaches the 100ms
area for alists and lingers in the 2ms area for hashes (that would
be where I'd start worrying).

How many people around here have regularly 1000 buffers?

Those simple alists are surprisingly fast (I, too have often
the reflex to reach for hash tables, I know).

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-17  5:06         ` tomas
@ 2023-07-17  9:28           ` Ihor Radchenko
  2023-07-17 13:11             ` Eli Zaretskii
  2023-07-17 17:48             ` tomas
  0 siblings, 2 replies; 10+ messages in thread
From: Ihor Radchenko @ 2023-07-17  9:28 UTC (permalink / raw)
  To: tomas; +Cc: Eli Zaretskii, emacs-devel

<tomas@tuxteam.de> writes:

> On Sun, Jul 16, 2023 at 08:19:10PM +0000, Ihor Radchenko wrote:
>
> [...]
>
>> scenarios. And in general lookup as well as hash tables start to
>> outperform alists just at 20-30 elements [1]
>
> For me, it's more like 45-ish:

Right. The link I provided actually also mentions similar number.

> How many people around here have regularly 1000 buffers?

Well. Check out https://github.com/org-roam/org-roam
People routinely deal with thousands of buffers there and complain about
that.

I also did a benchmark for opening several thousands of files in Emacs
and the performance was quite disappointing. (Though, admittedly, not
because of `get-file-buffer').

> And even at 100, the absolute time is rather negligible. Somewhere
> near 5000 buffers my (not very fast) machine reaches the 100ms
> area for alists and lingers in the 2ms area for hashes (that would
> be where I'd start worrying).

100ms is fast when you do it once. But `get-file-buffer' may be called
rather frequently by `with-temp-buffer'. 

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-17  9:28           ` Ihor Radchenko
@ 2023-07-17 13:11             ` Eli Zaretskii
  2023-07-17 17:48             ` tomas
  1 sibling, 0 replies; 10+ messages in thread
From: Eli Zaretskii @ 2023-07-17 13:11 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: tomas, emacs-devel

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Mon, 17 Jul 2023 09:28:50 +0000
> 
> <tomas@tuxteam.de> writes:
> 
> > On Sun, Jul 16, 2023 at 08:19:10PM +0000, Ihor Radchenko wrote:
> >
> > [...]
> >
> >> scenarios. And in general lookup as well as hash tables start to
> >> outperform alists just at 20-30 elements [1]
> >
> > For me, it's more like 45-ish:
> 
> Right. The link I provided actually also mentions similar number.
> 
> > How many people around here have regularly 1000 buffers?
> 
> Well. Check out https://github.com/org-roam/org-roam
> People routinely deal with thousands of buffers there and complain about
> that.
> 
> I also did a benchmark for opening several thousands of files in Emacs
> and the performance was quite disappointing. (Though, admittedly, not
> because of `get-file-buffer').
> 
> > And even at 100, the absolute time is rather negligible. Somewhere
> > near 5000 buffers my (not very fast) machine reaches the 100ms
> > area for alists and lingers in the 2ms area for hashes (that would
> > be where I'd start worrying).
> 
> 100ms is fast when you do it once. But `get-file-buffer' may be called
> rather frequently by `with-temp-buffer'. 

Performance improvement when there are many buffers is indeed a worthy
goal, but if we want to make get-file-buffer as fast as possible,
leaving it in C makes more sense.  We can call hash-table functions
from C as well; moreover, we can call them in a more efficient way,
bypassing validity checks that primitives often perform on their
arguments.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Should `get-file-buffer' be implemented in Elisp?
  2023-07-17  9:28           ` Ihor Radchenko
  2023-07-17 13:11             ` Eli Zaretskii
@ 2023-07-17 17:48             ` tomas
  1 sibling, 0 replies; 10+ messages in thread
From: tomas @ 2023-07-17 17:48 UTC (permalink / raw)
  To: Ihor Radchenko; +Cc: Eli Zaretskii, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

On Mon, Jul 17, 2023 at 09:28:50AM +0000, Ihor Radchenko wrote:
> <tomas@tuxteam.de> writes:
> 
> > On Sun, Jul 16, 2023 at 08:19:10PM +0000, Ihor Radchenko wrote:
> >
> > [...]
> >
> >> scenarios. And in general lookup as well as hash tables start to
> >> outperform alists just at 20-30 elements [1]
> >
> > For me, it's more like 45-ish:
> 
> Right. The link I provided actually also mentions similar number.
> 
> > How many people around here have regularly 1000 buffers?
> 
> Well. Check out https://github.com/org-roam/org-roam
> People routinely deal with thousands of buffers there and complain about
> that.

Oh, I see. The exciting thing about Emacs is that it has so many
different usages :-)

(I seldom surpass 10 or 12 buffers, FWIW).

> I also did a benchmark for opening several thousands of files in Emacs
> and the performance was quite disappointing. (Though, admittedly, not
> because of `get-file-buffer').

Yes, it would be interesting to know which part the lookup plays
in the grand scheme of things.

> > And even at 100, the absolute time is rather negligible. Somewhere
> > near 5000 buffers my (not very fast) machine reaches the 100ms
> > area for alists and lingers in the 2ms area for hashes (that would
> > be where I'd start worrying).
> 
> 100ms is fast when you do it once. But `get-file-buffer' may be called
> rather frequently by `with-temp-buffer'. 

Yes. I don't know about the original benchmark you quoted, but mine
is extremely "synthetic" and therefore full of lies: the alist gets
allocated in a tight loop and might show an unusual locality of
reference, as far as lists go. OTOH, if you are shuffling heavy
data structures in between lookups, the bigger hash structure might
compete for low level caches. On the third hand, the bad locality
of a "naturally grown" list might play havoc with TLB. And so on.

The only real measure would be to measure how much of the perceived
latency in org-roam is actually attributable to the lookup :-)

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2023-07-17 17:48 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-07-16 18:40 Should `get-file-buffer' be implemented in Elisp? Ihor Radchenko
2023-07-16 18:51 ` Eli Zaretskii
2023-07-16 19:05   ` Ihor Radchenko
2023-07-16 19:10     ` Eli Zaretskii
2023-07-16 20:19       ` Ihor Radchenko
2023-07-16 23:28         ` Po Lu
2023-07-17  5:06         ` tomas
2023-07-17  9:28           ` Ihor Radchenko
2023-07-17 13:11             ` Eli Zaretskii
2023-07-17 17:48             ` tomas

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.