Re: feature/package+vc 04c4c578c7 3/4: Allow for packages to be installed directly from VCS

[Tim Cross <theophilusx@gmail.com>]

Philip Kaludercic <philipk@posteo.net> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>>
>> Far better to just educate users that ANY package they install could
>> contain malicious code.
>
> Is this feasible?  Should this be a (long-)term goal, or will ELPA
> always be a "use at your own risk" kind of thing?
>
> Also, could you clarify what you mean by "formal security review"?

The amount of resources needed to ensure code has no security issues is
immense - well of course you cannot always guarantee no security issues,
but just having sufficient analysis to provide a reasonable level of
confidence is significant. To make it worse, any such analysis needs to
be done after every code update.

While a casual read through of the code may identify many glaring
security problems, it will miss less obvious issues. TO make matters
harder, actively malicious code will be crafted to hide its intent as
much as possible, so is unlikely to be identified just scanning through
the code.

A formal review would likely involve the following

- Formal analysis of the code. More than just a casual read through of
  the code. This would involve reading the code with a focus on security
  and verifying you understood what it did by tracing data flow all the
  way through. This would include any packages/modules the main package
  also uses (if those package versions have also undergone a formal
  security review, you can leverage off that information and would not
  be required to perform such a review on them again. It also involves
  reading assuming there are problems and your trying to find them.

- Use of various networking tools (like wireshark) to identify and
  understand any network traffic. Verify none of that traffic contains
  data which could be considered sensitive.

- Understanding of all data storage used by the system and ensure where
  such storage involves any sensitive data that it is documented and
  storage protection (e.g. hashing or encryption) is implemented or
  easily implemented/turned on.

- Actively attempt to use (abuse) the code for malicious purposes and
  look for possible security flaws. For example, supplying input with
  malicious content such as embedded code, suspect URLs, embedded
  commands etc. 

- Review of the installation and usage documentation to ensure there are
  not recommendations or requirements which would further reduce the
  user's security position.

in addition, there is some housekeeping tasks which would also be
required, including

- Tracking when reviews are done and by who

- Maintaining a registry of identified security issues. This would
  likely involve a high level description, severity ranking (i.e. CVSS),
  date discovered, date maintainers notified and possibly date for
  public release. Might want ot also indicate if issue is known to be
  actively exploited.

- Formal process for handling and managing security issues

What makes this extremely challenging is that the reviews should be
done by people without a vested interest in the package. Package authors
and maintainers are not always the best people to perform such reviews
because they are too close to the code (very similar to using external
testers).

The real challenge here is that this stuff is a hard skill which few
developers have. Finding sufficient numbers of people with these skills
who are able to work on just GNU ELPA packages would be next to
impossible. We simply don't have the numbers necessary.

Consider the commercial 'app stores' such as Apple's, MS's Azure store
or even Google Android store. These companies put significant resources
into trying to ensure the apps in these stores are 'safe' (to varying
degrees), yet we regularly see reports of multiple apps being removed
from these stores for security reasons or about apps which have been in
the store for some time and later found to be malicious. Or for
something perhaps a little closer to GNU ELPA, consider the issues
experienced by NPM and teh supply chain issues which have occurred there
- especially with respect to those cases of trusted and widely used
packages which have been compromised and that compromise was not
detected for a bit of time. The NPM ecosystem is putting a lot of
resources into enforcing better security practices by package
maintainers as this is where many of the failures have occurred.  It
hasn't been the actual package repository (i.e. think GNU ELPA), but the
source repositories (i.e. github, gitlab, sourceHut, random Git host)
which have been compromised. The level of sound security practices
employed by package maintainers varies widely. Some of the things NPM
are doing to try and improve security in this area may well be
informative for GNU ELPA and non-GNU ELPA. For example, sending an email
to package maintainer whenever their package is updated in ELPA,
requiring all source repositories to be protected with 2FA etc. 

The main thing protecting GNU Emacs and the ELPA infrastructure is low
return on investment. Malicious intent is like many things, there has to
be sufficient reward for the effort involved. The 'out run the lion'
principal also factors in here (you don't need to out run the lion, you
just need to not be the slowest person trying to outrun the lion). Right
now, the rewards seem low and there are easier targets out
there. Similar to the fallacy that there are no viruses on Apple or GNU
Linux, it isn't simply that we are more secure, we just don't provide
sufficient return for the effort investment.  

The other reason we should encourage the user to be proactive here and
not have them develop the expectation that all packages in GNU ELPA are
'safe' is because a lot really depends on each individual user and how
they use Emacs and in what environment. If your just using Emacs for
basic code editing at home when working on hobby projects, your risk
profile is likely low. However, if your a leading researcher in a
extremely competitive industry, you might be a much richer target worth
pursuit. If you also use Emacs to read your email, browse the web, as
your window manager, to interact with databases and for accessing remote
sensitive servers/data stores, then perhaps someone might target Emacs
as a way to gain access or exploit your data/system. This level of
targeting of researchers by well resourced industrial espionage groups
(often government backed) has been of sufficient concern that the 'five
eyes' have issued a number of security alerts to various government and
private research organisation over the last few years.