[BR] Clipboard support on macOS and Org 9.7

[BR] Clipboard support on macOS and Org 9.7

[Juergen Fenn]

Dear all,

over at the Org mode mailinglist, Ihor and Visuwesh asked me to post a
bug report to emacs-devel about how Emacs handles the macOS clipboard. I
ran into troubles with a new feature that was introduced with Org 9.7
for attaching images to Org documents from the clipboard. We found out
that this does not work as expected under macOS. Please see our thread
starting at:

https://lists.gnu.org/archive/html/emacs-orgmode/2024-06/msg00014.html

I am sorry it took me a moment to respond to their request because (1)
in the meanwhile I had to upgrade from macOS Ventura to Sonoma and and
re-compile Emacs, and (2) there were some more unexpected technical and
real-life matters. So, here we go.

You might like to know that upgrading to Sonoma meant that I had to
re-compile my Emacs 29.3 because the one I had built under Ventura would
not run any more on the new platform. I managed to re-compile Emacs from
the 29.3 tarball I had used on Ventura, but I realised that I cannot
compile Emacs under Sonoma with gnutls. Although the latest gnutls is
installed on my machine (I even re-installed gnutls over Homebrew) and
though this posed no problem under Ventura, I had to use =./configure
--with-gnutls=ifavailable=. Perhaps there were some changes to the Apple
Command Line Tools, too, that I re-installed for Sonoma. I can install
new packages with package.el, but Emacs now keeps telling me that
=Package tls is deprecated=.

Anyway, I have tested the clipboard bug with Org 9.7.4 on GNU Emacs 29.3
(build 1, aarch64-apple-darwin23.5.0, NS appkit-2487.60 Version 14.5
(Build 23F79)) of 2024-06-09. Here is what I can reproduce:

- Open the file manager Finder and select a local jpg file.
- Press Command-c to copy the file to the clipboard.
- Switch to an Org buffer in Emacs.
- M-x yank-media.

It turns out that any file, regardless of its real format, gets attached
with a TIFF file ending.

M-x yank-media-type yields:

(1/3) Yank type:
clipboard:image/tiff
clipboard:STRING
clipboard:text/plain

Ihor found out that MIME type support in macOS indeed is incomplete:

https://lists.gnu.org/archive/html/emacs-orgmode/2024-06/msg00063.html

So, you might like to have a look into this.

My observations were confirmed by Christopher, another user on macOS, I
have included in CC.

Many thanks in advance.

Regards,
Jürgen.

Re: [BR] Clipboard support on macOS and Org 9.7

[Eli Zaretskii]

> Date: Sun, 16 Jun 2024 23:22:14 +0200
> From: Juergen Fenn <jfenn@gmx.net>
> Cc: Ihor Radchenko <yantar92@posteo.net>, Visuwesh <visuweshm@gmail.com>,
>  "Christopher M. Miles" <numbchild@gmail.com>
> 
> over at the Org mode mailinglist, Ihor and Visuwesh asked me to post a
> bug report to emacs-devel about how Emacs handles the macOS clipboard.

Bug reports should be sent to the Emacs bug tracker, preferably by
using report-emacs-bug.  If Ihor indeed told you to post this here, I
think he made a mistake.

> Anyway, I have tested the clipboard bug with Org 9.7.4 on GNU Emacs 29.3
> (build 1, aarch64-apple-darwin23.5.0, NS appkit-2487.60 Version 14.5
> (Build 23F79)) of 2024-06-09. Here is what I can reproduce:
> 
> - Open the file manager Finder and select a local jpg file.
> - Press Command-c to copy the file to the clipboard.
> - Switch to an Org buffer in Emacs.
> - M-x yank-media.
> 
> It turns out that any file, regardless of its real format, gets attached
> with a TIFF file ending.
> 
> M-x yank-media-type yields:
> 
> (1/3) Yank type:
> clipboard:image/tiff
> clipboard:STRING
> clipboard:text/plain
> 
> Ihor found out that MIME type support in macOS indeed is incomplete:
> 
> https://lists.gnu.org/archive/html/emacs-orgmode/2024-06/msg00063.html
> 
> So, you might like to have a look into this.

If you can try the latest master branch of the Emacs Git repository,
perhaps this is now improved (not sure).

In any case, this will need a developer who knows about selections on
macOS.  Can anyone look into this, please?

Re: [BR] Clipboard support on macOS and Org 9.7

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> Bug reports should be sent to the Emacs bug tracker, preferably by
> using report-emacs-bug

Yup.
Should we continue discussion here or should we move to debbugs?

>> Ihor found out that MIME type support in macOS indeed is incomplete:
>> 
>> https://lists.gnu.org/archive/html/emacs-orgmode/2024-06/msg00063.html
>> 
>> So, you might like to have a look into this.
>
> If you can try the latest master branch of the Emacs Git repository,
> perhaps this is now improved (not sure).
>
> In any case, this will need a developer who knows about selections on
> macOS.  Can anyone look into this, please?

The patch for macOS support was discussed in
https://yhetil.org/emacs-devel/YZ18W18vybG7Cb%2FH@idiocy.org/

I am CCing Alan Third, the author of the patch.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BR] Clipboard support on macOS and Org 9.7

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Juergen Fenn <jfenn@gmx.net>, emacs-devel@gnu.org, visuweshm@gmail.com,
>  numbchild@gmail.com
> Date: Mon, 17 Jun 2024 11:15:07 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Bug reports should be sent to the Emacs bug tracker, preferably by
> > using report-emacs-bug
> 
> Yup.
> Should we continue discussion here or should we move to debbugs?

I'd prefer a bug report with all the details, TIA.

Re: [BR] Clipboard support on macOS and Org 9.7

[Juergen Fenn]

Am 17.06.24 um 14:07 Uhr schrieb Eli Zaretskii:
> I'd prefer a bug report with all the details, TIA.

Eli, I'd be happy to give you all the information you need in order to
improve the NextStep/macOS port, but I am unsure about how to do that. I
am used to reporting bugs on bug trackers such as Mozilla's Bugzilla and
Wikimedia's Phabricator, while you seem to prefer an email-based
workflow, don't you?

Which details do you lack, and what do I have to do in order to provide
them to you? When I posted to the list yesterday I thought it would be
enough to link to the debugging we already did on the Org mailing list
the other day, but it looks like i was wrong.

Thanks and best regards,
Jürgen.

Re: [BR] Clipboard support on macOS and Org 9.7

[Visuwesh]

On 18 June 2024 04:00:06 GMT+05:30, Juergen Fenn <jfenn@gmx.net> wrote:
>
>
>Am 17.06.24 um 14:07 Uhr schrieb Eli Zaretskii:
>> I'd prefer a bug report with all the details, TIA.
>
>Eli, I'd be happy to give you all the information you need in order to
>improve the NextStep/macOS port, but I am unsure about how to do that. I
>am used to reporting bugs on bug trackers such as Mozilla's Bugzilla and
>Wikimedia's Phabricator, while you seem to prefer an email-based
>workflow, don't you?
>
>Which details do you lack, and what do I have to do in order to provide
>them to you? When I posted to the list yesterday I thought it would be
>enough to link to the debugging we already did on the Org mailing list
>the other day, but it looks like i was wrong.

M-x report-emacs-bug RET and include all the necessary details.  I think everything that you sent so far and the latest message in org ML by JD Smith should be good.  I think Eli would prefer if you include a summary of what has been discussed already in the org ML with links to the messages to know it in greater detail.

Thanks.

>Thanks and best regards,
>Jürgen.

Re: [BR] Clipboard support on macOS and Org 9.7

[Eli Zaretskii]

> Date: Tue, 18 Jun 2024 10:03:56 +0530
> From: Visuwesh <visuweshm@gmail.com>
> CC: alan@idiocy.org, emacs-devel@gnu.org, numbchild@gmail.com
> 
> On 18 June 2024 04:00:06 GMT+05:30, Juergen Fenn <jfenn@gmx.net> wrote:
> >
> >
> >Am 17.06.24 um 14:07 Uhr schrieb Eli Zaretskii:
> >> I'd prefer a bug report with all the details, TIA.
> >
> >Eli, I'd be happy to give you all the information you need in order to
> >improve the NextStep/macOS port, but I am unsure about how to do that. I
> >am used to reporting bugs on bug trackers such as Mozilla's Bugzilla and
> >Wikimedia's Phabricator, while you seem to prefer an email-based
> >workflow, don't you?
> >
> >Which details do you lack, and what do I have to do in order to provide
> >them to you? When I posted to the list yesterday I thought it would be
> >enough to link to the debugging we already did on the Org mailing list
> >the other day, but it looks like i was wrong.
> 
> M-x report-emacs-bug RET and include all the necessary details.  I think everything that you sent so far and the latest message in org ML by JD Smith should be good.  I think Eli would prefer if you include a summary of what has been discussed already in the org ML with links to the messages to know it in greater detail.

Yes, thanks.

Re: [BR] Clipboard support on macOS and Org 9.7

[Robert Pluim]

>>>>> On Sun, 16 Jun 2024 23:22:14 +0200, Juergen Fenn <jfenn@gmx.net> said:
    Juergen> You might like to know that upgrading to Sonoma meant that I had to
    Juergen> re-compile my Emacs 29.3 because the one I had built under Ventura would
    Juergen> not run any more on the new platform. I managed to re-compile Emacs from
    Juergen> the 29.3 tarball I had used on Ventura, but I realised that I cannot
    Juergen> compile Emacs under Sonoma with gnutls. Although the latest gnutls is
    Juergen> installed on my machine (I even re-installed gnutls over Homebrew) and
    Juergen> though this posed no problem under Ventura, I had to use =./configure
    Juergen> --with-gnutls=ifavailable=. Perhaps there were some changes to the Apple
    Juergen> Command Line Tools, too, that I re-installed for Sonoma. I can install
    Juergen> new packages with package.el, but Emacs now keeps telling me that
    Juergen> =Package tls is deprecated=.

That means youʼre using the cli tools, not the GnuTLS library. Perhaps
you need to have pkg-config installed as well in order for configure
to find the includes and the library.

    Juergen> Anyway, I have tested the clipboard bug with Org 9.7.4 on GNU Emacs 29.3
    Juergen> (build 1, aarch64-apple-darwin23.5.0, NS appkit-2487.60 Version 14.5
    Juergen> (Build 23F79)) of 2024-06-09. Here is what I can reproduce:

    Juergen> - Open the file manager Finder and select a local jpg file.
    Juergen> - Press Command-c to copy the file to the clipboard.
    Juergen> - Switch to an Org buffer in Emacs.
    Juergen> - M-x yank-media.

    Juergen> It turns out that any file, regardless of its real format, gets attached
    Juergen> with a TIFF file ending.

    Juergen> M-x yank-media-type yields:

    Juergen> (1/3) Yank type:
    Juergen> clipboard:image/tiff
    Juergen> clipboard:STRING
    Juergen> clipboard:text/plain

Yep

    Juergen> Ihor found out that MIME type support in macOS indeed is incomplete:

    Juergen> https://lists.gnu.org/archive/html/emacs-orgmode/2024-06/msg00063.html

'ns_get_foreign_selection' queries the clipboard to see what types are
available, and those are the ones it reports. Based on a quick google,
if we were to say "give me the data as JPEG anyway", thereʼs a chance
it might work, but of course the source might actually be TIFF, or PNG.

I canʼt find an official API to query the macOS clipboard for which
types it supports, but then again Appleʼs docs site and I rarely get
along.

Robert
-- 

Re: [BR] Clipboard support on macOS and Org 9.7

[Juergen Fenn]

Am 17.06.24 um 14:05 Uhr schrieb Robert Pluim:
> That means youʼre using the cli tools, not the GnuTLS library. Perhaps
> you need to have pkg-config installed as well in order for configure
> to find the includes and the library.

I had no problems compiling Emacs under Ventura with the Command Line
Tools. GnuTLS was installed even then. Just configure -- make -- make
install. I did that for 29.1 to 29.3. The process seems to change with
each macOS version (and the next one is just around the corner). It got
much more complicated under Sonoma because now it does no more suffice
to deactive Gatekeeper, you also have to unpack the tarball on the
command line (i.e., not in Finder), otherwise the Emacs you compile from
the source later won't run on your platform. That's funny, isn't it?

Such questions get very few answers on StackExchange, if any.

What do I do with pkg-config? Will it just work out of the box after
installing it with Homebrew, or do I have to call it with the configure
script?

Thanks and best regards,
Jürgen.

Re: [BR] Clipboard support on macOS and Org 9.7

[Robert Pluim]

>>>>> On Tue, 18 Jun 2024 00:25:16 +0200, Juergen Fenn <jfenn@gmx.net> said:

    Juergen> Am 17.06.24 um 14:05 Uhr schrieb Robert Pluim:
    >> That means youʼre using the cli tools, not the GnuTLS library. Perhaps
    >> you need to have pkg-config installed as well in order for configure
    >> to find the includes and the library.

    Juergen> I had no problems compiling Emacs under Ventura with the Command Line
    Juergen> Tools. GnuTLS was installed even then. Just configure -- make -- make
    Juergen> install. I did that for 29.1 to 29.3. The process seems to change with
    Juergen> each macOS version (and the next one is just around the corner). It got
    Juergen> much more complicated under Sonoma because now it does no more suffice
    Juergen> to deactive Gatekeeper, you also have to unpack the tarball on the
    Juergen> command line (i.e., not in Finder), otherwise the Emacs you compile from
    Juergen> the source later won't run on your platform. That's funny, isn't it?

The way I compile emacs hasnʼt changed across 2 different HW platforms
and at least 3 different macOS versions. But then again I build from a
git checkout, not tarballs.

    Juergen> Such questions get very few answers on StackExchange, if any.

    Juergen> What do I do with pkg-config? Will it just work out of the box after
    Juergen> installing it with Homebrew, or do I have to call it with the configure
    Juergen> script?

Just install it. The emacs configure script uses it to query packages
such as GnuTLS for include and library locations. You can check
configure.log to see if itʼs being used successfully.

Robert
-- 

Re: [BR] Clipboard support on macOS and Org 9.7

[Juergen Fenn]

Am 18.06.24 um 09:37 Uhr schrieb Robert Pluim:
>     Juergen> Such questions get very few answers on StackExchange, if any.
>
>     Juergen> What do I do with pkg-config? Will it just work out of the box after
>     Juergen> installing it with Homebrew, or do I have to call it with the configure
>     Juergen> script?
>
> Just install it. The emacs configure script uses it to query packages
> such as GnuTLS for include and library locations. You can check
> configure.log to see if itʼs being used successfully.

You were right. pkg-config did the trick. Newly released Emacs 29.4
compiles alright with GnuTLS under macOS Sonona. Thanks, again, for your
advice.

My bug report on Org Mode follows next.

I'm sorry for my late reply. I'm afraid, I felt too busy during the last
week …

Regards,
Jürgen.