Re: Emacs on Android OOMs with pdf-tools. (And an Android howto.)

[Lockywolf Laptop <lockywolf@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 9300 bytes --]


(my answer to Po Lu is below)

To Eli:

>if the OOM mechanism of the OS kicks in while Emacs
>still thinks it has more memory to use, it means something is wrong
>with how the available memory is being reported to Emacs.  Emacs
>should have sensed that it is approaching the limit and signaled the
>memory-full signal before that happens.  I think this warrants a bug
>report, with a reproducer.

Emacs doesn't actually crash or doesn't get killed because of OOM.
Rather it produces a message of something like "Memory exhausted, run
C-x C-s, and restart Emacs". Something does get broken at this point,
as, for example, not all faces get rendered, but overall Emacs works.

I am not sure how how bad this is.

>Try playing with the value of image-cache-eviction-delay.

I ended up doing like this in init.el:

#+begin_src elisp
(setf pdf-cache-image-limit 0)
(setq-default pdf-view-display-size 'fit-height)
(setq-default pdf-view-max-image-width 2000)
(setq-default pdf-cache-image-inihibit t)
(advice-add 'pdf-view-redisplay :before #'clear-image-cache)
(advice-add 'pdf-view-redisplay :after #'clear-image-cache)
#+end_src

This works. It's a bit slow, but I can read a whole book of 400 pages
with no issues. Each page generates about 100 Mb of cache, but since
it's evicted after rendering, overall the memory consumption is not that
high, about 16 Mb of image cache.

>But how did you get to so many images? what images are they and what
>did you do in Emacs to have them all loaded?  Emacs is not meant to be
>used with so many images on a device that has such severe memory
>limitations.

Well, this thread is titled "using pdf-tools on Android 6" :).

This is an eBook reader Onyx Boox Max 2, with an eInk display. I am
using it because it has a reflective display that is so-so-so much
better for my strained eyes that I am ready to suffer quite some
hardships to make it applicable to many of my daily tasks.

So I was trying to use pdf-tools to read pdfs on it, with Emacs. Its
native PDF reader works quite okay, the problem is that it's Android
software, and not aimed at power users, so its integration options are
limited. I have written quite a lot about this tool in the howto.

pdf-tools is somehow smarter than doc-view-mode. Doc-view-mode, as far
as I understand, generates a png file with ghostscript out of each page
of the pdf file, whereas pdf-tools uses poppler, and renders much more
than just an image. In particular, it allows for pdf annotations, and
can be integrated with auctex, et cetera.

I have not managed to make pdf-annotating features of pdf-tools work on
this eBook, Emacs just hangs if I try to touch the screen with a pen,
and can be unfrozen with C-g. But so far I didn't really need it, as I
seldom annotate PDFs using PDF built-in annotation features. I prefer to
pass a pdf through mathpix (or some other de-texifying OCR, or even
better, just ask the author to send me the source), and write
my annotations right into the resulting TeX code.


Po Lu <luangruo@yahoo.com> writes:

> Lockywolf <lockywolf@gmail.com> writes:
>
>> Date: Thu, 22 Aug 2024 08:12:44 +0800
>>
>> Thanks everyone for the memory-report command, I hadn't known about it
>> before this case.
>>
>>>How much memory does your device provide?
>>
>> The device has 2 Gb of RAM, of which about 1.5 Gb are ordinarily used by
>> the system and built-in annoyances.
>> I am trying to _only_ use Emacs on this device, so it should have about
>> 500 Mb left.
>>
>> But I am pretty sure it is not actually the problem of not enough
>> memory, as memory-report is clearly showing that something is wrong.
>>
>> The two reports, before and after the OOM message, are below.
>> The concerning bit seems to be: "256 MiB  Total Image Cache Size"
>> 256 Mb is a significant number for this device. Below I am also
>> pasting memory-report with doc-view-mode, which seems to "only" generate
>> 160 Mb of image-cache.
>>
>> Is it possible to tell Emacs to not generate so much cache?
>
> Please try Eli's advice, but also please run `adb logcat' (from a PC)
> after Emacs signals that memory has been exhausted, and verify that no
> line reading "Possible out of memory error" exists which does not
> immediately precede a java.lang.OutOfMemoryError, since Emacs assumes by
> default that any unknown Java exception is an out-of-memory error.

To Po Lu:

Adb logcat produces the following message:

08-22 14:05:09.144 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.144 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.149 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.160 20974 20987 I art     : Alloc partial concurrent mark sweep GC freed 383(124KB) AllocSpace objects, 21(336KB) LOS objects, 8% free, 175MB/191MB, paused 219us total 11.322ms
08-22 14:05:09.160 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.171 20974 20987 I art     : Alloc concurrent mark sweep GC freed 141(6KB) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 205us total 10.038ms
08-22 14:05:09.171 20974 20987 I art     : Forcing collection of SoftReferences for 23MB allocation
08-22 14:05:09.171 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.181 20974 20987 I art     : Alloc concurrent mark sweep GC freed 5(176B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 214us total 9.850ms
08-22 14:05:09.181 20974 20987 W art     : Throwing OutOfMemoryError "Failed to allocate a 24687948 byte allocation with 16777056 free bytes and 16MB until OOM"
08-22 14:05:09.181 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.181 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.184 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.194 20974 20987 I art     : Alloc partial concurrent mark sweep GC freed 6(192B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 207us total 10.227ms
08-22 14:05:09.194 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.204 20974 20987 I art     : Alloc concurrent mark sweep GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 198us total 9.826ms
08-22 14:05:09.204 20974 20987 I art     : Forcing collection of SoftReferences for 23MB allocation
08-22 14:05:09.204 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:09.214 20974 20987 I art     : Alloc concurrent mark sweep GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 207us total 9.940ms
08-22 14:05:09.214 20974 20987 W art     : Throwing OutOfMemoryError "Failed to allocate a 24687948 byte allocation with 16777056 free bytes and 16MB until OOM"
08-22 14:05:10.767 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.767 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.771 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.781 20974 20987 I art     : Alloc partial concurrent mark sweep GC freed 43(1712B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 214us total 10.370ms
08-22 14:05:10.781 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.791 20974 20987 I art     : Alloc concurrent mark sweep GC freed 2(64B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 195us total 9.755ms
08-22 14:05:10.791 20974 20987 I art     : Forcing collection of SoftReferences for 23MB allocation
08-22 14:05:10.791 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.801 20974 20987 I art     : Alloc concurrent mark sweep GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 199us total 9.804ms
08-22 14:05:10.801 20974 20987 W art     : Throwing OutOfMemoryError "Failed to allocate a 24687948 byte allocation with 16777056 free bytes and 16MB until OOM"
08-22 14:05:10.801 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.801 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.804 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.814 20974 20987 I art     : Alloc partial concurrent mark sweep GC freed 6(192B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 206us total 10.249ms
08-22 14:05:10.814 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.825 20974 20987 I art     : Alloc concurrent mark sweep GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 200us total 9.893ms
08-22 14:05:10.825 20974 20987 I art     : Forcing collection of SoftReferences for 23MB allocation
08-22 14:05:10.825 20974 20987 I art     : Starting a blocking GC Alloc
08-22 14:05:10.835 20974 20987 I art     : Alloc concurrent mark sweep GC freed 3(96B) AllocSpace objects, 0(0B) LOS objects, 8% free, 175MB/191MB, paused 196us total 9.771ms
08-22 14:05:10.835 20974 20987 W art     : Throwing OutOfMemoryError "Failed to allocate a 24687948 byte allocation with 16777056 free bytes and 16MB until OOM"

I do not know whether this is helpful.


-- 
Your sincerely,
Vladimir Nikishkin (MiEr, lockywolf)
(Laptop)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]