bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Thank you for the recent addition of `trusted-content-p'. Is there a
possibility to use `trusted-content-p' in buffers which are not backed
by a file? I use Flymake in *scratch* or similar buffers and it seems
that this won't continue to work given that `trusted-content-p' needs a
`buffer-file-truename'.

My suggestion would be to replace `trusted-files' by a
`trusted-buffer-function' which is a predicate function or a list of
functions. The functions could then check a custom list of trusted files
or a custom list of trusted buffers.

Alternatively offer `trusted-files', `trusted-buffers' and
`trusted-buffer-function`? `trusted-buffers' could for example rely on
`buffer-match-p`.

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Daniel Mendler <mail@daniel-mendler.de> writes:

> Thank you for the recent addition of `trusted-content-p'. Is there a
> possibility to use `trusted-content-p' in buffers which are not backed
> by a file? I use Flymake in *scratch* or similar buffers and it seems
> that this won't continue to work given that `trusted-content-p' needs a
> `buffer-file-truename'.
>
> My suggestion would be to replace `trusted-files' by a
> `trusted-buffer-function' which is a predicate function or a list of
> functions. The functions could then check a custom list of trusted files
> or a custom list of trusted buffers.
>
> Alternatively offer `trusted-files', `trusted-buffers' and
> `trusted-buffer-function`? `trusted-buffers' could for example rely on
> `buffer-match-p`.

I have also ported back `trusted-content-p' via Compat. I had the plan
to use `trusted-content-p' in external packages which could potentially
perform dangerous operations. This way the new feature can be used to
retroactively improve the safety even of older Emacs installations.

For example in my GNU ELPA Corfu package the plan was to check
`(trusted-content-p)' when starting auto completion. To be clear - Corfu
is safe by default, since auto completion is disabled by default.
However many people enable auto completion unconditionally in all
buffers.

Now with the limitation of `trusted-content-p' to file-backed buffers, I
cannot do this, since otherwise auto completion would be lost for
example in *scratch* buffers. Each package could invent its own trust
mechanism or alternatively one could limit the `trusted-content-p' check
to only file-backed buffers. Both alternatives would be worse than going
through the `trusted-content-p' standard mechanism.

Therefore by making the `trusted-content-p' mechanism too limited, we
get less safety than with a more flexible mechanism. Nevertheless I
would avoid creating a complex mechanism given that the mechanism is
supposed to be part of Emacs 30. The simplest approach I can think of
this this `trusted-buffer-function', a hook called by
`run-hook-with-args-until-success'. Later on trust functions can be
provided and added to the hook list. The trust functions could check
file lists, buffer lists, regexps etc. Users can also write their own
predicate functions.

In any case, I am happy to help providing patches.

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eli Zaretskii]

> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 15 Dec 2024 11:16:17 +0100
> From:  Daniel Mendler via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Daniel Mendler <mail@daniel-mendler.de> writes:
> 
> > Thank you for the recent addition of `trusted-content-p'. Is there a
> > possibility to use `trusted-content-p' in buffers which are not backed
> > by a file? I use Flymake in *scratch* or similar buffers and it seems
> > that this won't continue to work given that `trusted-content-p' needs a
> > `buffer-file-truename'.
> >
> > My suggestion would be to replace `trusted-files' by a
> > `trusted-buffer-function' which is a predicate function or a list of
> > functions. The functions could then check a custom list of trusted files
> > or a custom list of trusted buffers.
> >
> > Alternatively offer `trusted-files', `trusted-buffers' and
> > `trusted-buffer-function`? `trusted-buffers' could for example rely on
> > `buffer-match-p`.
> 
> I have also ported back `trusted-content-p' via Compat. I had the plan
> to use `trusted-content-p' in external packages which could potentially
> perform dangerous operations. This way the new feature can be used to
> retroactively improve the safety even of older Emacs installations.
> 
> For example in my GNU ELPA Corfu package the plan was to check
> `(trusted-content-p)' when starting auto completion. To be clear - Corfu
> is safe by default, since auto completion is disabled by default.
> However many people enable auto completion unconditionally in all
> buffers.
> 
> Now with the limitation of `trusted-content-p' to file-backed buffers, I
> cannot do this, since otherwise auto completion would be lost for
> example in *scratch* buffers. Each package could invent its own trust
> mechanism or alternatively one could limit the `trusted-content-p' check
> to only file-backed buffers. Both alternatives would be worse than going
> through the `trusted-content-p' standard mechanism.
> 
> Therefore by making the `trusted-content-p' mechanism too limited, we
> get less safety than with a more flexible mechanism. Nevertheless I
> would avoid creating a complex mechanism given that the mechanism is
> supposed to be part of Emacs 30. The simplest approach I can think of
> this this `trusted-buffer-function', a hook called by
> `run-hook-with-args-until-success'. Later on trust functions can be
> provided and added to the hook list. The trust functions could check
> file lists, buffer lists, regexps etc. Users can also write their own
> predicate functions.

What do you envision trusted-buffer-function should do in a buffer
that doesn't visit a file?

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>>  Stefan Kangas <stefankangas@gmail.com>
>> Date: Sun, 15 Dec 2024 11:16:17 +0100
>> From:  Daniel Mendler via "Bug reports for GNU Emacs,
>>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
>> 
>> Daniel Mendler <mail@daniel-mendler.de> writes:
>> 
>> > Thank you for the recent addition of `trusted-content-p'. Is there a
>> > possibility to use `trusted-content-p' in buffers which are not backed
>> > by a file? I use Flymake in *scratch* or similar buffers and it seems
>> > that this won't continue to work given that `trusted-content-p' needs a
>> > `buffer-file-truename'.
>> >
>> > My suggestion would be to replace `trusted-files' by a
>> > `trusted-buffer-function' which is a predicate function or a list of
>> > functions. The functions could then check a custom list of trusted files
>> > or a custom list of trusted buffers.
>> >
>> > Alternatively offer `trusted-files', `trusted-buffers' and
>> > `trusted-buffer-function`? `trusted-buffers' could for example rely on
>> > `buffer-match-p`.
>> 
>> I have also ported back `trusted-content-p' via Compat. I had the plan
>> to use `trusted-content-p' in external packages which could potentially
>> perform dangerous operations. This way the new feature can be used to
>> retroactively improve the safety even of older Emacs installations.
>> 
>> For example in my GNU ELPA Corfu package the plan was to check
>> `(trusted-content-p)' when starting auto completion. To be clear - Corfu
>> is safe by default, since auto completion is disabled by default.
>> However many people enable auto completion unconditionally in all
>> buffers.
>> 
>> Now with the limitation of `trusted-content-p' to file-backed buffers, I
>> cannot do this, since otherwise auto completion would be lost for
>> example in *scratch* buffers. Each package could invent its own trust
>> mechanism or alternatively one could limit the `trusted-content-p' check
>> to only file-backed buffers. Both alternatives would be worse than going
>> through the `trusted-content-p' standard mechanism.
>> 
>> Therefore by making the `trusted-content-p' mechanism too limited, we
>> get less safety than with a more flexible mechanism. Nevertheless I
>> would avoid creating a complex mechanism given that the mechanism is
>> supposed to be part of Emacs 30. The simplest approach I can think of
>> this this `trusted-buffer-function', a hook called by
>> `run-hook-with-args-until-success'. Later on trust functions can be
>> provided and added to the hook list. The trust functions could check
>> file lists, buffer lists, regexps etc. Users can also write their own
>> predicate functions.
>
> What do you envision trusted-buffer-function should do in a buffer
> that doesn't visit a file?

`trusted-buffer-function' should be a hook variable, which could be set
to multiple functions, e.g., #'trusted--files-p and
#'trusted--buffers-p. The function `trusted--files-p' would check the
variable `trusted-files' similar to the existing code in the emacs-30
branch.

The function `trusted--buffers-p' could check another variable
`trusted-buffers' which specifies a list of buffer name regexps or
probably even better a `buffer-match-p' condition. This way the user
could specify buffers which they consider safe, for example *scratch*.

In the end it is up to the user how the variables are configured, as is
already the case with `trusted-files'. The user must define which
directories/files/buffers they consider safe.

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eli Zaretskii]

> From: Daniel Mendler <mail@daniel-mendler.de>
> Cc: 74879@debbugs.gnu.org,  monnier@iro.umontreal.ca,  stefankangas@gmail.com
> Date: Sun, 15 Dec 2024 11:56:29 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > What do you envision trusted-buffer-function should do in a buffer
> > that doesn't visit a file?
> 
> `trusted-buffer-function' should be a hook variable, which could be set
> to multiple functions, e.g., #'trusted--files-p and
> #'trusted--buffers-p. The function `trusted--files-p' would check the
> variable `trusted-files' similar to the existing code in the emacs-30
> branch.

I was asking specifically about the non file-visiting buffers.

> The function `trusted--buffers-p' could check another variable
> `trusted-buffers' which specifies a list of buffer name regexps or
> probably even better a `buffer-match-p' condition. This way the user
> could specify buffers which they consider safe, for example *scratch*.

Why would a buffer's name tell _anything_ about whether the user can
trust it?

> In the end it is up to the user how the variables are configured, as is
> already the case with `trusted-files'. The user must define which
> directories/files/buffers they consider safe.

If we wanted to let this completely up to the user, we wouldn't be
introducing this feature, certainly not so close to a release, would
we?

The question is serious: how do we envision this "trust" thing to work
with buffers that don't visit files?  If we are to change the code,
certainly on the emacs-30 branch, we need a solid solution which
provides more safety/security to users.  Adding a variable doesn't
solve a problem, it _adds_ a problem (how to populate the variable).

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> The question is serious: how do we envision this "trust" thing to work
> with buffers that don't visit files?  If we are to change the code,
> certainly on the emacs-30 branch, we need a solid solution which
> provides more safety/security to users.  Adding a variable doesn't
> solve a problem, it _adds_ a problem (how to populate the variable).

Let me try.

If buffer contents is not coming from a file, it must be generated by
some Elisp code. That code may as well set trust status.
For example, *scratch* buffer may have its contents (automatically
generated) marked as trusted by default.

Does it make sense?

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Daniel Mendler <mail@daniel-mendler.de>, 74879@debbugs.gnu.org,
>  monnier@iro.umontreal.ca, stefankangas@gmail.com
> Date: Sun, 15 Dec 2024 11:37:37 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The question is serious: how do we envision this "trust" thing to work
> > with buffers that don't visit files?  If we are to change the code,
> > certainly on the emacs-30 branch, we need a solid solution which
> > provides more safety/security to users.  Adding a variable doesn't
> > solve a problem, it _adds_ a problem (how to populate the variable).
> 
> Let me try.
> 
> If buffer contents is not coming from a file, it must be generated by
> some Elisp code. That code may as well set trust status.
> For example, *scratch* buffer may have its contents (automatically
> generated) marked as trusted by default.
> 
> Does it make sense?

Are you in effect saying that every buffer that doesn't visit a file
should be trusted?  If that's accepted, it doesn't need any function.
And can we really trust arbitrary ELisp code that to set trust?

And what about buffers whose contents came from a network connection?

What about buffers whose contents came from inserting some file or
part thereof, or were generated by processing some file?

What about buffers whose contents came from a program Emacs invoked?

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> If buffer contents is not coming from a file, it must be generated by
>> some Elisp code. That code may as well set trust status.
>> For example, *scratch* buffer may have its contents (automatically
>> generated) marked as trusted by default.
>> 
>> Does it make sense?
>
> Are you in effect saying that every buffer that doesn't visit a file
> should be trusted?

No. The code generating non-file buffers may indicate whether buffer
should be trusted or not.

> ... If that's accepted, it doesn't need any function.
> And can we really trust arbitrary ELisp code that to set trust?

When an arbitrary Elisp code is already running, there is nothing that
can prevent that code from doing anything at all, including, for
example, re-defining `trusted-content-p'. So, discussing whether we can
trust a running Elisp code or not makes no sense in my book. We have to
trust it.

> And what about buffers whose contents came from a network connection?

The code that is putting text received from network connection should be
responsible for marking the buffer appropriately.

> What about buffers whose contents came from inserting some file or
> part thereof, or were generated by processing some file?

Again, the code should be responsible to check things, maybe using some
kind of API function to check whether a given source file should be
trusted or not.

> What about buffers whose contents came from a program Emacs invoked?

Same thing.
I'd say that the codes receiving text contents from network or from a
program should not mark it as trusted.

One alternative might be storing "trust flag" as text property for Emacs
primitives that read file contents, network stream, or program
output. Then, if any part of buffer has "trust flag" set to be not
trusted, the whole buffer should not be considered trusted.

-- 
Ihor Radchenko // yantar92,
Org mode maintainer,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: mail@daniel-mendler.de, 74879@debbugs.gnu.org, monnier@iro.umontreal.ca,
>  stefankangas@gmail.com
> Date: Sun, 15 Dec 2024 12:50:41 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > And can we really trust arbitrary ELisp code that to set trust?
> 
> When an arbitrary Elisp code is already running, there is nothing that
> can prevent that code from doing anything at all, including, for
> example, re-defining `trusted-content-p'. So, discussing whether we can
> trust a running Elisp code or not makes no sense in my book. We have to
> trust it.

"Arbitrary ELisp code" doesn't have to be malicious, just too
trusting.

> > And what about buffers whose contents came from a network connection?
> 
> The code that is putting text received from network connection should be
> responsible for marking the buffer appropriately.

How can that work in practice?  What can that code do to know whether
the stuff can or cannot be trusted?

> > What about buffers whose contents came from inserting some file or
> > part thereof, or were generated by processing some file?
> 
> Again, the code should be responsible to check things, maybe using some
> kind of API function to check whether a given source file should be
> trusted or not.
> 
> > What about buffers whose contents came from a program Emacs invoked?
> 
> Same thing.
> I'd say that the codes receiving text contents from network or from a
> program should not mark it as trusted.

Now we are getting somewhere.

My point is that we should probably not leave this open to some
function, but instead code our own ways of deciding whether a given
buffer is trusted.

> One alternative might be storing "trust flag" as text property for Emacs
> primitives that read file contents, network stream, or program
> output. Then, if any part of buffer has "trust flag" set to be not
> trusted, the whole buffer should not be considered trusted.

My problem is not how NOT to trust, my problem is in which cases to
trust.  Saying that by default such buffers are not trusted is easy --
we already do that, in fact.

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@posteo.net>
>> Cc: Daniel Mendler <mail@daniel-mendler.de>, 74879@debbugs.gnu.org,
>>  monnier@iro.umontreal.ca, stefankangas@gmail.com
>> Date: Sun, 15 Dec 2024 11:37:37 +0000
>>
>> If buffer contents is not coming from a file, it must be generated by
>> some Elisp code. That code may as well set trust status.
>> For example, *scratch* buffer may have its contents (automatically
>> generated) marked as trusted by default.
>>
>> Does it make sense?
>
> Are you in effect saying that every buffer that doesn't visit a file
> should be trusted?  If that's accepted, it doesn't need any function.
> And can we really trust arbitrary ELisp code that to set trust?

I can't speak for Ihor, but I didn't read what he said to mean that
_all_ non-file-visiting buffer should be trusted.

IIUC, the idea is to provide a mechanism that allows marking _some_
non-file-visiting buffers as trusted.  This would be useful in *scratch*
and ielm buffers, for example.

> And what about buffers whose contents came from a network connection?
>
> What about buffers whose contents came from inserting some file or
> part thereof, or were generated by processing some file?
>
> What about buffers whose contents came from a program Emacs invoked?

Such buffers should still be untrusted.

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Dmitry Gutov]

On 15/12/2024 12:16, Daniel Mendler via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> For example in my GNU ELPA Corfu package the plan was to check
> `(trusted-content-p)' when starting auto completion.

Shouldn't that be done in the c-a-p-f function?

>To be clear - Corfu
> is safe by default, since auto completion is disabled by default.
> However many people enable auto completion unconditionally in all
> buffers.

Having completion invoked manually doesn't really ensure that the user 
knows about the odds of it running code from the current file. Some 
languages do that, some don't, and the newbie Lisp users have little 
idea of what macro expansion in completion entails.

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 15/12/2024 12:16, Daniel Mendler via Bug reports for GNU Emacs, the Swiss
> army knife of text editors wrote:
>> For example in my GNU ELPA Corfu package the plan was to check
>> `(trusted-content-p)' when starting auto completion.
>
> Shouldn't that be done in the c-a-p-f function?

Yes, this is a more fine-grained approach. Stefan added a check to the
macroexpansion in Emacs 30 which should make the Elisp Capf safe.

But consider other scenarios like Org-babel or Embark. Org-babel can
execute code blocks and Embark can evaluate Sexps at point. For these
cases it makes sense to check if the buffer is safe before running the
action. However in contrast to auto completion one has to press a
special key to trigger the evaluation.

>>To be clear - Corfu
>> is safe by default, since auto completion is disabled by default.
>> However many people enable auto completion unconditionally in all
>> buffers.
>
> Having completion invoked manually doesn't really ensure that the user knows
> about the odds of it running code from the current file. Some languages do that,
> some don't, and the newbie Lisp users have little idea of what macro expansion
> in completion entails.

That's correct. Nevertheless Eshel specifically mentioned auto
completion in his report. I think that the threshold for auto completion
is a little lower - the user enters normal text and potentially code
execution of in-buffer code happens behind the scenes.

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Thank you for the recent addition of `trusted-content-p'. Is there a
> possibility to use `trusted-content-p' in buffers which are not backed
> by a file? I use Flymake in *scratch* or similar buffers and it seems
> that this won't continue to work given that `trusted-content-p' needs a
> `buffer-file-truename'.

Good question.
We don't really have a good answer yet, AFAIK, in large part because we
don't have enough experience with it.
Off the top of my head, here are some elements relevant to this
discussion, in random order:

- The current setup is a kind of "minimal" change for Emacs-30 because
  it's late in the pretest, so as much as possible we should separate
  the discussion into what's a simple enough solution for Emacs-30 and
  what we should use in the longer term.

- You should be able to get fully-featured Flymake in *scratch*
  with (setq-local trusted-files :all).
  Maybe we should do that when we setup *scratch*?
  Which other non-file buffers would need that?  The minibuffer?

- Trust sucks, so we really should work on better solutions where we
  don't need to rely on trust, such as running code in `bwrap` or other
  kinds of sandboxes.

- I think we do want some kind of hook, with which we can have (for
  instance) `emacs-lisp-mode` tell Emacs to trust the user init file,
  the early-init file, the custom-file, and all the files in
  `load-path`.

- There is overlap with `safe-local-variable-directories`,
  `enable-local-variables` and it would be nice to consolidate (which
  can require delicate timing if we want the major mode to inform which
  content to trust).


- Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Kangas]

Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> - You should be able to get fully-featured Flymake in *scratch*
>   with (setq-local trusted-files :all).
>   Maybe we should do that when we setup *scratch*?
>   Which other non-file buffers would need that?  The minibuffer?

ielm comes to mind.

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> - You should be able to get fully-featured Flymake in *scratch*
>>   with (setq-local trusted-files :all).
>>   Maybe we should do that when we setup *scratch*?
>>   Which other non-file buffers would need that?  The minibuffer?
>
> ielm comes to mind.

Thanks (fixed in `emacs-30`).


        Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Gerd Möllmann]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Thank you for the recent addition of `trusted-content-p'. Is there a
>> possibility to use `trusted-content-p' in buffers which are not backed
>> by a file? I use Flymake in *scratch* or similar buffers and it seems
>> that this won't continue to work given that `trusted-content-p' needs a
>> `buffer-file-truename'.
>
> Good question.
> We don't really have a good answer yet, AFAIK, in large part because we
> don't have enough experience with it.
> Off the top of my head, here are some elements relevant to this
> discussion, in random order:
>
> - The current setup is a kind of "minimal" change for Emacs-30 because
>   it's late in the pretest, so as much as possible we should separate
>   the discussion into what's a simple enough solution for Emacs-30 and
>   what we should use in the longer term.
>
> - You should be able to get fully-featured Flymake in *scratch*
>   with (setq-local trusted-files :all).
>   Maybe we should do that when we setup *scratch*?
>   Which other non-file buffers would need that?  The minibuffer?
>
> - Trust sucks, so we really should work on better solutions where we
>   don't need to rely on trust, such as running code in `bwrap` or other
>   kinds of sandboxes.
>
> - I think we do want some kind of hook, with which we can have (for
>   instance) `emacs-lisp-mode` tell Emacs to trust the user init file,
>   the early-init file, the custom-file, and all the files in
>   `load-path`.
>
> - There is overlap with `safe-local-variable-directories`,
>   `enable-local-variables` and it would be nice to consolidate (which
>   can require delicate timing if we want the major mode to inform which
>   content to trust).
>

Random thought:

- What if a user pastes text from a untrusted source to a trusted buffer?

- Is taint checking relevant in this context?

    https://en.wikipedia.org/wiki/Taint_checking

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Random thought:
>
> - What if a user pastes text from a untrusted source to a trusted buffer?
>
> - Is taint checking relevant in this context?
>
>     https://en.wikipedia.org/wiki/Taint_checking

I'll just repeat that trust sucks.
It's a last recourse and we should work to implement better solutions.


        Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Gerd Möllmann]

Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

>> Random thought:
>>
>> - What if a user pastes text from a untrusted source to a trusted buffer?
>>
>> - Is taint checking relevant in this context?
>>
>>     https://en.wikipedia.org/wiki/Taint_checking
>
> I'll just repeat that trust sucks.
> It's a last recourse and we should work to implement better solutions.

Sure, I didn't mean it as a critique on what's there.

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Thank you for the recent addition of `trusted-content-p'. Is there a
>> possibility to use `trusted-content-p' in buffers which are not backed
>> by a file? I use Flymake in *scratch* or similar buffers and it seems
>> that this won't continue to work given that `trusted-content-p' needs a
>> `buffer-file-truename'.
>
> Good question.
> We don't really have a good answer yet, AFAIK, in large part because we
> don't have enough experience with it.
> Off the top of my head, here are some elements relevant to this
> discussion, in random order:
>
> - The current setup is a kind of "minimal" change for Emacs-30 because
>   it's late in the pretest, so as much as possible we should separate
>   the discussion into what's a simple enough solution for Emacs-30 and
>   what we should use in the longer term.

Agree. As I suggested a simple trusted-buffer-function hook may be the
simplest solution, which is also not limiting and allows us to mark
various buffers as trusted.

> - You should be able to get fully-featured Flymake in *scratch*
>   with (setq-local trusted-files :all).
>   Maybe we should do that when we setup *scratch*?
>   Which other non-file buffers would need that?  The minibuffer?

Given that the trust applies to the given buffer, setting `(setq-local
trusted-files :all)' in this buffer feels odd as a recommended
mechanism. Even more so since the variable is marked as risky local.

> - Trust sucks, so we really should work on better solutions where we
>   don't need to rely on trust, such as running code in `bwrap` or other
>   kinds of sandboxes.

I agree. But what about interactive scenarios like auto completion?
There we may be limited to trust, even if we want sandboxing in other
scenarios. I think trust checking might be helpful in all scenarios
where there is a "low threshold" to invoking code execution or even
unintentionally.

> - I think we do want some kind of hook, with which we can have (for
>   instance) `emacs-lisp-mode` tell Emacs to trust the user init file,
>   the early-init file, the custom-file, and all the files in
>   `load-path`.

You suggest a hook which is executed per buffer? This seems similar to
my proposal of a trusted-buffer-function hook.

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> - The current setup is a kind of "minimal" change for Emacs-30 because
>>   it's late in the pretest, so as much as possible we should separate
>>   the discussion into what's a simple enough solution for Emacs-30 and
>>   what we should use in the longer term.
>
> Agree. As I suggested a simple trusted-buffer-function hook may be the
> simplest solution, which is also not limiting and allows us to mark
> various buffers as trusted.
>
>> - You should be able to get fully-featured Flymake in *scratch*
>>   with (setq-local trusted-files :all).
>>   Maybe we should do that when we setup *scratch*?
>>   Which other non-file buffers would need that?  The minibuffer?
>
> Given that the trust applies to the given buffer, setting `(setq-local
> trusted-files :all)' in this buffer feels odd as
> a recommended mechanism.

I can live with that for now.
It's probably not much worse than

    (add-function :override (local 'trusted-content-function) #'always)

[ BTW, I just renamed the var to `trusted-content`.  ]

> Even more so since the variable is marked as risky local.

"Risky local" refers to file/dir-local settings, not
buffer-local settings.

>> - Trust sucks, so we really should work on better solutions where we
>>   don't need to rely on trust, such as running code in `bwrap` or other
>>   kinds of sandboxes.
> I agree. But what about interactive scenarios like auto completion?

I don't understand the question.

> There we may be limited to trust, even if we want sandboxing in other
> scenarios.

Why?  What's different?

> I think trust checking might be helpful in all scenarios where there
> is a "low threshold" to invoking code execution or
> even unintentionally.

Oh, you mean for code completion we don't want to incur the cost of
spawning a subprocess?  Indeed that can be a reason to fallback to trust.

But note that in the "other kinds of sandboxes" I include things like
labeling macros with some indication about how they can be run safely,
so we can have a version of `elisp--safe-macroexpand-all` which does
something useful even if the buffer's content is not trusted.

>> - I think we do want some kind of hook, with which we can have (for
>>   instance) `emacs-lisp-mode` tell Emacs to trust the user init file,
>>   the early-init file, the custom-file, and all the files in
>>   `load-path`.
>
> You suggest a hook which is executed per buffer? This seems similar to
> my proposal of a `trusted-buffer-function` hook.

Yes, that's exactly what I was referring to.
At first I was thinking of some kind of `trusted-buffer-functions` hook
used with `run-hook-with-args-until-success`, but I think you're right
that it's better to go with `trusted-buffer-function` so we can both add
and remove trust with it.


        Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hi,

Thank you for your work on this mitigation!

Stefan Monnier writes:

> [ BTW, I just renamed the var to `trusted-content`.  ]

Note that now we have both trusted-content and untrusted-content, both
security-related but with somewhat different meanings...  It might be
worth consolidating the two somehow to avoid confusion.


Best,

Eshel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eshel Yaron <me@eshelyaron.com> writes:

> Hi,
>
> Thank you for your work on this mitigation!
>
> Stefan Monnier writes:
>
>> [ BTW, I just renamed the var to `trusted-content`.  ]
>
> Note that now we have both trusted-content and untrusted-content, both
> security-related but with somewhat different meanings...  It might be
> worth consolidating the two somehow to avoid confusion.

Sorry, I now realize that they are already consolidated in the sense
trusted-content-p considers both variables.  org-latex-preview in org.el
still checks for untrusted-content directly though, I wonder if it'd
make sense for it to check for trusted-content-p instead.


Eshel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Sorry, I now realize that they are already consolidated in the sense
> trusted-content-p considers both variables.

It's a limited form of consolidation, but indeed I did try and do a bit
of it.  🙂

> org-latex-preview in org.el still checks for untrusted-content
> directly though, I wonder if it'd make sense for it to check for
> trusted-content-p instead.

I can't remember the details of the `org-latex-preview` situation, so
I'll say tentatively "probably".


        Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> Given that the trust applies to the given buffer, setting `(setq-local
>> trusted-files :all)' in this buffer feels odd as
>> a recommended mechanism.
>
> I can live with that for now.
> It's probably not much worse than
>
>     (add-function :override (local 'trusted-content-function) #'always)
>
> [ BTW, I just renamed the var to `trusted-content`.  ]

I think it is not as good as a central configuration variable where I
configure the trust for buffers or files. Now configuring trust is
scattered across multiple major modes. My proposal was for a global hook
variable which is consulted by `trusted-content-p' and then checks
certain trust list for files or buffers. This way it is easier to check
what we are trusting.

>>> - Trust sucks, so we really should work on better solutions where we
>>>   don't need to rely on trust, such as running code in `bwrap` or other
>>>   kinds of sandboxes.
>> I agree. But what about interactive scenarios like auto completion?
>
> I don't understand the question.

I mean that not all of the scenarios can be run in some sandbox. If we
cannot put all Capfs in a sandbox we could at least prevent auto
completion entirely based on trust.

>> I think trust checking might be helpful in all scenarios where there
>> is a "low threshold" to invoking code execution or
>> even unintentionally.
>
> Oh, you mean for code completion we don't want to incur the cost of
> spawning a subprocess?  Indeed that can be a reason to fallback to trust.

The thought was rather that auto completion may be dangerous in general
and since it is triggered with a low threshold one could prevent auto
completion right away. I think you have a more fine-grained model in mind
where certain macros are trusted and so on.

> But note that in the "other kinds of sandboxes" I include things like
> labeling macros with some indication about how they can be run safely,
> so we can have a version of `elisp--safe-macroexpand-all` which does
> something useful even if the buffer's content is not trusted.
>
>>> - I think we do want some kind of hook, with which we can have (for
>>>   instance) `emacs-lisp-mode` tell Emacs to trust the user init file,
>>>   the early-init file, the custom-file, and all the files in
>>>   `load-path`.
>>
>> You suggest a hook which is executed per buffer? This seems similar to
>> my proposal of a `trusted-buffer-function` hook.
>
> Yes, that's exactly what I was referring to.
> At first I was thinking of some kind of `trusted-buffer-functions` hook
> used with `run-hook-with-args-until-success`, but I think you're right
> that it's better to go with `trusted-buffer-function` so we can both add
> and remove trust with it.

My initial proposal was about a global `trusted-buffer-functions' hook
but I may have not communicated this clearly enough. I have two problems
with the buffer-local approach:

1. I find it difficult to check what we are trusting, since the trust
settings are scattered over multiple files.

2. How can I configure certain buffers to be trusted? One could
configure `trusted-content' at the time of buffer creation. Or is there
a hook which you suggest to use? Right now it seems that the idea is to
set `trusted-content` based on major modes?

Daniel

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> I can live with that for now.
>> It's probably not much worse than
>>
>>     (add-function :override (local 'trusted-content-function) #'always)
>>
>> [ BTW, I just renamed the var to `trusted-content`.  ]
>
> I think it is not as good as a central configuration variable where I
> configure the trust for buffers or files. Now configuring trust is
> scattered across multiple major modes. My proposal was for a global hook
> variable which is consulted by `trusted-content-p' and then checks
> certain trust list for files or buffers. This way it is easier to check
> what we are trusting.

Both `trusted-content-function` and `trusted-content-functions` would be
"normal" hooks in the sense that they are neither specifically global
nor specifically local: the code that adds elements to it get to choose
whether to add it buffer-locally or globally.

For major modes, it makes a lot more sense to add elements locally,
since that avoids having to worry about the performance impact on the
rest of Emacs and things like that.

> The thought was rather that auto completion may be dangerous in general

Maybe I have too narrow a view of completion, but my impression is that
completion is usually safe.  The situation in ELisp where we perform
macro expansion to try and get the set of local variables in scope and
where that macro expansion can end up running local code is rather
unusual, AFAIK.

What kind of situation are you thinking of where completion is unsafe?

> My initial proposal was about a global `trusted-buffer-functions' hook

I don't see any reason to restrict such a hook to be global.

> but I may have not communicated this clearly enough. I have two problems
> with the buffer-local approach:

The `trusted-buffer-function` I sketched would similarly not be
restricted to be buffer-local.  IOW local/global is orthogonal to
whether we go with `trusted-buffer-functions` or
`trusted-buffer-function`.

> I just took another look at the `trusted-content-p' function on the
> emacs-30 branch:
>
> (defun trusted-content-p ()
>   (and (not untrusted-content)
>        buffer-file-truename
>        ...))
>
> There is a check for `buffer-file-truename' which means that the issue
> with non-file-backed buffers remains (Ielm or scratch). Probably (eq
> trusted-content :all) should be checked first?

Duh!  I forgot that the same change toned down the warnings, so I
didn't look at the right place when "testing".  Thanks for the heads up,
I just fixed it in `emacs-30`.

> At the same time one could consolidate the untrusted-content and
> trusted-content variables as Eshel suggested?
>
> (defun trusted-content-p ()
>   (and (not untrusted-content) ;; only for backward compat (deprecated)
>        (not (eq trusted-content :untrusted)) ;; replaces untrusted-content
>        (or (eq trusted-content :all)
>            (and buffer-file-name
>                 ...))))

I could go along with that (not for `emacs-30`, tho).
I'd prefer to get a bit more experience before deciding to do that, tho.


        Stefan

bug#74879: 30.0.92; trusted-content-p and trusted-files cannot be used for non-file buffers

[Daniel Mendler via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Hello Stefan,

I just took another look at the `trusted-content-p' function on the
emacs-30 branch:

(defun trusted-content-p ()
  (and (not untrusted-content)
       buffer-file-truename
       ...))

There is a check for `buffer-file-truename' which means that the issue
with non-file-backed buffers remains (Ielm or scratch). Probably (eq
trusted-content :all) should be checked first?

(defun trusted-content-p ()
  (and (not untrusted-content)
       (or (eq trusted-content :all)
           (and buffer-file-name
                ...))))

At the same time one could consolidate the untrusted-content and
trusted-content variables as Eshel suggested?

(defun trusted-content-p ()
  (and (not untrusted-content) ;; only for backward compat (deprecated)
       (not (eq trusted-content :untrusted)) ;; replaces untrusted-content
       (or (eq trusted-content :all)
           (and buffer-file-name
                ...))))

Daniel