One goal in my package is it to display PDF pages quickly. 

There are 2 factors delaying this:  The actual rendering in the back-end (via poppler) and the loading of the image by Emacs.  

In order to limit this timespan, pages are speculative pre-rendered and pre-loaded. E.g. if page n is currently displayed, the user probably wants to view page n+1 next. This is implemented by filling  a LRU cache while Emacs is idle.

Note how this conflicts with a least-recently-displayed eviction strategy.

Of course this can be solved in Lisp, just not very elegantly.  I spare you the details.

Another extension point, which would help managing the image-cache in my case, would be the ability to flush images based on their data.  Note, this is currenly only supported for file based images.

Thanks for taking the time,

Andreas