emacsformacos: cannot access mac folders: Documents/Downloads

emacsformacos: cannot access mac folders: Documents/Downloads

[Uwe Brauer]

Hi 

This is sort of absurd, using the compiled emacsformacox I cannot access
the Mac specific folders such as Documents, Downloads, while I can
access the files inside these folders.

However this does not happen with the version I compiled from master.

What is the reason for this behaviour and how can I solve it?

Regards

Uwe Brauer 

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Emanuel Berg via Users list for the GNU Emacs text editor]

Uwe Brauer wrote:

> This is sort of absurd, using the compiled emacsformacox
> I cannot access the Mac specific folders such as Documents,
> Downloads, while I can access the files inside
> these folders.

Heh, Mac OS seems to give you a lot of problems :)

It shouldn't, right? Because Emacs is the same everywhere.

errr

-- 
underground experts united
https://dataswamp.org/~incal

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Daniel Martín]

Uwe Brauer <oub@mat.ucm.es> writes:

> Hi 
>
> This is sort of absurd, using the compiled emacsformacox I cannot access
> the Mac specific folders such as Documents, Downloads, while I can
> access the files inside these folders.
>
> However this does not happen with the version I compiled from master.
>
> What is the reason for this behaviour and how can I solve it?
>

The reason is a security protection in recent versions of macOS where
you need to grant explicit permission to applications if you want them
to access your personal folders.  Go to System Preferences, Security &
Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
there.

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Eli Zaretskii]

> From: Daniel Martín <mardani29@yahoo.es>
> Date: Sat, 27 Mar 2021 03:25:11 +0100
> 
> The reason is a security protection in recent versions of macOS where
> you need to grant explicit permission to applications if you want them
> to access your personal folders.  Go to System Preferences, Security &
> Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
> macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
> there.

Should this information be in the Emacs user manual?

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:

>> From: Daniel Martín <mardani29@yahoo.es>
>> Date: Sat, 27 Mar 2021 03:25:11 +0100
>> 
>> The reason is a security protection in recent versions of macOS where
>> you need to grant explicit permission to applications if you want them
>> to access your personal folders.  Go to System Preferences, Security &
>> Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
>> macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
>> there.

> Should this information be in the Emacs user manual?


Not sure, if you compile from master then this problem does not occur it
seems to be more of a emacsformacosx thing

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Pankaj Jangid]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>> "EZ" == Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Daniel Martín <mardani29@yahoo.es>
>>> Date: Sat, 27 Mar 2021 03:25:11 +0100
>>> 
>>> The reason is a security protection in recent versions of macOS where
>>> you need to grant explicit permission to applications if you want them
>>> to access your personal folders.  Go to System Preferences, Security &
>>> Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
>>> macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
>>> there.
>
>> Should this information be in the Emacs user manual?
>
>
> Not sure, if you compile from master then this problem does not occur it
> seems to be more of a emacsformacosx thing

This problem used to occur, about 6-months ago, every time I built a new
binary. Have we changed something in Emacs to circumvent that? At least
I don’t remember such a change.

On my side, I have changed how I access Emacs. Earlier I used to start
from macos terminal program. And now I just copy the
‘nextstep/Emacs.app’ to ‘~/Applications/Emacs.app’, and launch from
Launchpad.

As I am writing this, I have tried to build new binary and launched from
command line. Now that is also working without any problem.

All this may be happening because of following:

1. New macos was release in the mean time and it probably has new
   persistent settings for ‘~/Application/Emacs.app’ even if the user
   changes binary.

2. There are more settings for ‘Files and Folders’ with respect to
   ‘Terminal’ program. These new settings may be allowing anything
   launched from terminal.

What should we add to user manual?

Ideal use-case for a user should be to use the ‘nextstep/Emacs.app’
application. Like other applications in macos. But user may have
downloaded pre-built binary from emacsformacosx.com, as the case in
this thread. So,

“Copy the freshly built ‘nextstep/Emacs.app’ to your ‘Applications’
directory. Go to System Preferences, Security & Privacy, Privacy, Full
Disk Access. Allow ‘Emacs.app’ to access your file-system.

Or, if you have downloaded a pre-built binary from emacsformacosx.com
then allow ‘/usr/bin/ruby’ to access your file-system.”

-- 
Regards,
Pankaj Jangid

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Daniel Martín]

Uwe Brauer <oub@mat.ucm.es> writes:

>
> Not sure, if you compile from master then this problem does not occur it
> seems to be more of a emacsformacosx thing

Because of the way this protection feature works and the different ways
you can build and run an Emacs binary for macOS, the casuistic is a bit
complex.  I'll try to summarize it:

If you compile Emacs from the GNU repository on a macOS system, you have
two ways to run the program:

a) You can run "make" and then execute "src/emacs".  In this case, Emacs
will inherit the privacy permissions of your terminal.  If your terminal
does not already have permission to access personal folders, the first
time you visit something in a personal folder from Emacs you will see a
prompt "<Your terminal> would like to access files in your <Personal
folder> folder".

b) Alternatively, you can run "make install" and then double click on
the "nextstep/Emacs.app" bundle, or move it to a more permanent
directory, like /Applications.  This is already the recommended way for
developers to run the NS native port of Emacs, and the procedure is
described in detail in "nextstep/INSTALL" from the Emacs repository.
The first time Emacs needs to visit something in a personal folder it
will prompt you in a similar way as a), but in the prompt it will show
as "Emacs" making the request and not your terminal program, provided
that you executed Emacs from the graphical user interface (for example,
from Launchpad).

If you prefer to use the version packaged from emacsformacosx.org, or
similarly you installed Emacs using a popular package manager like
Homebrew, there's a crucial difference: Emacs.app/Contents/MacOS/Emacs
is *not* a binary, but a Ruby script.  Why do the packagers use a custom
launcher written in Ruby?  It's a convenience for users so that a single
download can contain Emacs binaries for all supported macOS versions.
The Ruby launcher is maintained separately at
https://github.com/caldwell/build-emacs

This extra Ruby launcher is a nuisance for granting access to personal
folders, because a prompt won't appear and you'll need to grant full
disk access to "/usr/bin/ruby" manually.  There is an open bug report in
the "build-emacs" project to improve this:
https://github.com/caldwell/build-emacs/issues/94

I'm happy to contribute this explanation to the Emacs manuals if you
want (perhaps to the FAQ?), so that users can understand why they can't
access their personal folders in Emacs without resorting to do an online
search which in most cases do not offer the full explanation.

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 3419 bytes --]

>>> "DM" == Daniel Martín <mardani29@yahoo.es> writes:

> Uwe Brauer <oub@mat.ucm.es> writes:
>> 
>> Not sure, if you compile from master then this problem does not occur it
>> seems to be more of a emacsformacosx thing

> Because of the way this protection feature works and the different ways
> you can build and run an Emacs binary for macOS, the casuistic is a bit
> complex.  I'll try to summarize it:

> If you compile Emacs from the GNU repository on a macOS system, you have
> two ways to run the program:

> a) You can run "make" and then execute "src/emacs".  In this case, Emacs
> will inherit the privacy permissions of your terminal.  If your terminal
> does not already have permission to access personal folders, the first
> time you visit something in a personal folder from Emacs you will see a
> prompt "<Your terminal> would like to access files in your <Personal
folder> folder".

That was I am doing at the moment and face some problems with my init
files, problems I don't face in emacsformacos. Since I was unable to
have my favorite lucid option compiled, I am not sure whether it would
be better to switch to emacsforosox (I will not use the Mac most likely
not for any development but more as backup solution of sorts)



> b) Alternatively, you can run "make install" and then double click on
> the "nextstep/Emacs.app" bundle, or move it to a more permanent
> directory, like /Applications.  This is already the recommended way for
> developers to run the NS native port of Emacs, and the procedure is
> described in detail in "nextstep/INSTALL" from the Emacs repository.
> The first time Emacs needs to visit something in a personal folder it
> will prompt you in a similar way as a), but in the prompt it will show
> as "Emacs" making the request and not your terminal program, provided
> that you executed Emacs from the graphical user interface (for example,
> from Launchpad).


Thanks for pointing this out.

> If you prefer to use the version packaged from emacsformacosx.org, or
> similarly you installed Emacs using a popular package manager like
> Homebrew, there's a crucial difference: Emacs.app/Contents/MacOS/Emacs
> is *not* a binary, but a Ruby script.  Why do the packagers use a custom
> launcher written in Ruby?  It's a convenience for users so that a single
> download can contain Emacs binaries for all supported macOS versions.
> The Ruby launcher is maintained separately at
> https://github.com/caldwell/build-emacs

> This extra Ruby launcher is a nuisance for granting access to personal
> folders, because a prompt won't appear and you'll need to grant full
> disk access to "/usr/bin/ruby" manually.  There is an open bug report in
> the "build-emacs" project to improve this:
> https://github.com/caldwell/build-emacs/issues/94

> I'm happy to contribute this explanation to the Emacs manuals if you
> want (perhaps to the FAQ?), so that users can understand why they can't
> access their personal folders in Emacs without resorting to do an online
> search which in most cases do not offer the full explanation.

First of all thanks for the detailed explanation which was extremely
helpful.

As far as the FAQ is concerned, I think that is a very good idea, may I
suggest also to include the information of how to add the rugby script
(as you did in a different email)?

Regards

Uwe 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

>>> "DM" == Daniel Martín <mardani29@yahoo.es> writes:

> Uwe Brauer <oub@mat.ucm.es> writes:
>> 
>> Not sure, if you compile from master then this problem does not occur it
>> seems to be more of a emacsformacosx thing

Hi 

I realised that I did not answer your meail 
> Because of the way this protection feature works and the different ways
> you can build and run an Emacs binary for macOS, the casuistic is a bit
> complex.  I'll try to summarize it:

> If you compile Emacs from the GNU repository on a macOS system, you have
> two ways to run the program:

> a) You can run "make" and then execute "src/emacs".  In this case, Emacs
> will inherit the privacy permissions of your terminal.  If your terminal
> does not already have permission to access personal folders, the first
> time you visit something in a personal folder from Emacs you will see a
> prompt "<Your terminal> would like to access files in your <Personal folder> folder".

Ok, I do this only for test purposes

> b) Alternatively, you can run "make install" and then double click on
> the "nextstep/Emacs.app" bundle, or move it to a more permanent
> directory, like /Applications.  This is already the recommended way for
> developers to run the NS native port of Emacs, and the procedure is
> described in detail in "nextstep/INSTALL" from the Emacs repository.
> The first time Emacs needs to visit something in a personal folder it
> will prompt you in a similar way as a), but in the prompt it will show
> as "Emacs" making the request and not your terminal program, provided
> that you executed Emacs from the graphical user interface (for example,
> from Launchpad).

Hm, I see, but why can't I have the good old unix-like way and install
it say in /usr/local or as I prefer now /opt/emacs28 this way I could
test say a stable version like emacs27 and the current master emacs28

I think I will rise this question in the dev emacs list.

Regards

Uwe 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 960 bytes --]

>>> "DM" == Daniel Martín <mardani29@yahoo.es> writes:

> Uwe Brauer <oub@mat.ucm.es> writes:
>> Hi 
>> 
>> This is sort of absurd, using the compiled emacsformacox I cannot access
>> the Mac specific folders such as Documents, Downloads, while I can
>> access the files inside these folders.
>> 
>> However this does not happen with the version I compiled from master.
>> 
>> What is the reason for this behaviour and how can I solve it?
>> 

> The reason is a security protection in recent versions of macOS where
> you need to grant explicit permission to applications if you want them
> to access your personal folders.  Go to System Preferences, Security &
> Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
> macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
> there.

Thanks, but precisely to add /usr/bin/ruby seem almost impossible the
interface does not allow you to add a simple string.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: emacsformacos: cannot access mac folders: Documents/Downloads

[Daniel Martín]

Uwe Brauer <oub@mat.ucm.es> writes:

>
>> The reason is a security protection in recent versions of macOS where
>> you need to grant explicit permission to applications if you want them
>> to access your personal folders.  Go to System Preferences, Security &
>> Privacy, Privacy, Full Disk Access.  Note that the Emacs launcher on
>> macOS is actually a Ruby script, so you'd need to add /usr/bin/ruby
>> there.
>
> Thanks, but precisely to add /usr/bin/ruby seem almost impossible the
> interface does not allow you to add a simple string.

Click on the plus sign ('+') and then press Shift+Command+G.  A prompt
will appear where you can type "/usr/bin/ruby" to go to that location.