>> Here is a preliminary patch
>
> Thanks.  I only quickly read the patch, and have one comment: the
> hard-coded left-margin should depend on the value returned by
> current-bidi-paragraph-direction, because text that is displayed
> right-to-left should have the icons on the right margin, not left
> margin.

To not create a new mirrored image file for RTL arrow required a change in
icons--create to support ':rotation 180' for the existing image file:

  (define-icon outline-close-rtl-in-margins outline-close
    '((image "outline-close.svg" "outline-close.pbm"
             :height 10 :ascent center :rotation 180))
    "Icon used for buttons for closing a section in outline buffers."
    :version "29.1"
    :help-echo "Close this section")

Then the right margin could be used with: