Re: [NonGNU ELPA] New package: llm

Re: [NonGNU ELPA] New package: llm

[chad]

[-- Attachment #1: Type: text/plain, Size: 2796 bytes --]

On Thu, Aug 31, 2023 at 5:06 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> Richard Stallman <rms@gnu.org> writes:
>
> > As for LLMs that run on servers, they are a different issue entirely.
> > They are all SaaSS (Service as a Software Substitute), and SaaSS is
> > always unjust.
> >
> > See https://gnu.org/philosophy/who-does-that-server-really-serve.html
> > for explanation.
>
> I do not fully agree here. [...]
> Thus, for many users (owning less powerful computers) LLMs as a service
> are going to be SaaS, not SaaSS. (Given that the SaaS LLM has free
> licence and users who choose to buy the necessary hardware retain their
> freedom to run the same LLM on their hardware.)
>

 It's a somewhat subtle, gnarly point, and I didn't find a way to express
it as well as Ihor Radchenko here, but I will add: the ability for a free
software-loving user to run their own SaaS is both increasing and
decreasing in ease recently. On the one hand, it's difficult these days to
run a personal email service and not get trapped by the shifting myriad of
overlapping spam/fraud/monopoly `protection' features, at least if you want
to regularly send email to a wide variety of users. On the other hand, it's
increasingly viable to have a hand-held machine that's a tiny fraction of a
space-cadet keyboard running (mostly; binary blobs are a pernicious evil)
free software that easily connects back to one's own free-software
"workstation" for medium and large jobs, even while avoiding "the cloud
trap", as it were.

(Such things have been a long-time hobby/interest of mine, dating back
before my time as a professional programmer. They're still not common, but
they're getting increasingly moreso; native Android support for emacs, as
one example, will likely help.)

For large AI models specifically: there are many users for whom it is not
practical to _actually_ recreate the model from scratch everywhere they
might want to use it. It is important for computing freedom that such
recreations be *possible*, but it will be very limiting to insist that
everyone who wants to use such services actually do so, in a manner that
seems to me to be very similar to not insisting that every potential emacs
user actually compile their own. In this case there's the extra wrinkle
that the actual details of recreating the currently-most-interesting large
language models involves both _gigantic_ amounts of resources and also a
fairly large amount of not-directly-reproducible randomness involved. It
might be worth further consideration.

Just now, re-reading this seems like a topic better suited to
emacs-tangents or even gnu-misc-discuss, so I'm changing the CC there.
Apologies if this causes an accidental fork.

I hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 3471 bytes --]

Re: [NonGNU ELPA] New package: llm

[Ihor Radchenko]

chad <yandros@gmail.com> writes:

> For large AI models specifically: there are many users for whom it is not
> practical to _actually_ recreate the model from scratch everywhere they
> might want to use it. It is important for computing freedom that such
> recreations be *possible*, but it will be very limiting to insist that
> everyone who wants to use such services actually do so, in a manner that
> seems to me to be very similar to not insisting that every potential emacs
> user actually compile their own. In this case there's the extra wrinkle
> that the actual details of recreating the currently-most-interesting large
> language models involves both _gigantic_ amounts of resources and also a
> fairly large amount of not-directly-reproducible randomness involved. It
> might be worth further consideration.

Let me refer to another message by RMS:

    >>   > While I certainly appreciate the effort people are making to produce 
    >>   > LLMs that are more open than OpenAI (a low bar), I'm not sure if 
    >>   > providing several gigabytes of model weights in binary format is really 
    >>   > providing the *source*. It's true that you can still edit these models 
    >>   > in a sense by fine-tuning them, but you could say the same thing about a 
    >>   > project that only provided the generated output from GNU Bison, instead 
    >>   > of the original input to Bison.
    >> 
    >> I don't think that is valid.
    >> Bison processing is very different from training a neural net.
    >> Incremental retraining of a trained neural net
    >> is the same kind of processing as the original training -- except
    >> that you use other data and it produces a neural net
    >> that is trained differently.
    >> 
    >> My conclusiuon is that the trained neural net is effectively a kind of
    >> source code.  So we don't need to demand the "original training data"
    >> as part of a package's source code.  That data does not have to be
    >> free, published, or available.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Ihor Radchenko]

[ Moving this to emacs-tangents ]

Richard Stallman <rms@gnu.org> writes:

>   > > As for LLMs that run on servers, they are a different issue entirely.
>   > > They are all SaaSS (Service as a Software Substitute), and SaaSS is
>   > > always unjust.
>   > >
>   > > See https://gnu.org/philosophy/who-does-that-server-really-serve.html
>   > > for explanation.
>
>   > I do not fully agree here. A number of more powerful LLMs have very
>   > limiting hardware requirements. For example, some LLMs require 64+Gbs of
>   > RAM to run:
>
> That is true, and it is unfortunate.  There may be no practical way
> to run a certain model except for SaaSS.
>
> That does not alter the injustice of SaaSS.  So we should not silence
> our criticism of SaaSS in those cases,

This is a rather theoretical consideration, but, talking about ChatGTP
(owned by OpenAI) specifically, should it even be considered SaaSS?

https://www.gnu.org/philosophy/who-does-that-server-really-serve.html
says:

    Using a joint project's servers isn't SaaSS because the computing
    you do in this way isn't your own. For instance, if you edit pages
    on Wikipedia, you are not doing your own computing; rather, you are
    collaborating in Wikipedia's computing. Wikipedia controls its own
    servers, but organizations as well as individuals encounter the
    problem of SaaSS if they do their computing in someone else's
    server.

Then, ChatGTP is using the user input to train their model:
https://techunwrapped.com/you-can-now-make-chatgpt-not-train-with-your-queries/

    ... what is constant is that the company can use
    our conversations with ChatGPT to train the model. This is not a
    surprise or a secret, the company has always reported it.

There is no doubt that ChatGTP itself is not libre - its model is not
available to public. However, users of the ChatGPT model are technically
providing input that is collaboratively editing that model weights
(training the model further). So, using ChatGTP is a little bit akin
editing Wikipedia pages - collaborating to improve ChatGTP.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Andrew Hyatt]

[-- Attachment #1: Type: text/plain, Size: 2995 bytes --]

On Wed, Sep 6, 2023 at 8:50 AM Ihor Radchenko <yantar92@posteo.net> wrote:

> [ Moving this to emacs-tangents ]
>
> Richard Stallman <rms@gnu.org> writes:
>
> >   > > As for LLMs that run on servers, they are a different issue
> entirely.
> >   > > They are all SaaSS (Service as a Software Substitute), and SaaSS is
> >   > > always unjust.
> >   > >
> >   > > See
> https://gnu.org/philosophy/who-does-that-server-really-serve.html
> >   > > for explanation.
> >
> >   > I do not fully agree here. A number of more powerful LLMs have very
> >   > limiting hardware requirements. For example, some LLMs require
> 64+Gbs of
> >   > RAM to run:
> >
> > That is true, and it is unfortunate.  There may be no practical way
> > to run a certain model except for SaaSS.
> >
> > That does not alter the injustice of SaaSS.  So we should not silence
> > our criticism of SaaSS in those cases,
>
> This is a rather theoretical consideration, but, talking about ChatGTP
> (owned by OpenAI) specifically, should it even be considered SaaSS?
>
> https://www.gnu.org/philosophy/who-does-that-server-really-serve.html
> says:
>
>     Using a joint project's servers isn't SaaSS because the computing
>     you do in this way isn't your own. For instance, if you edit pages
>     on Wikipedia, you are not doing your own computing; rather, you are
>     collaborating in Wikipedia's computing. Wikipedia controls its own
>     servers, but organizations as well as individuals encounter the
>     problem of SaaSS if they do their computing in someone else's
>     server.
>
> Then, ChatGTP is using the user input to train their model:
>
> https://techunwrapped.com/you-can-now-make-chatgpt-not-train-with-your-queries/
>
>     ... what is constant is that the company can use
>     our conversations with ChatGPT to train the model. This is not a
>     surprise or a secret, the company has always reported it.
>
> There is no doubt that ChatGTP itself is not libre - its model is not
> available to public. However, users of the ChatGPT model are technically
> providing input that is collaboratively editing that model weights
> (training the model further). So, using ChatGTP is a little bit akin
> editing Wikipedia pages - collaborating to improve ChatGTP.
>

In addition, you can pay money to train your own model (via fine-tuning) on
top of Open AI's model.  Most other providers also let you do this.  The
model is "yours", and the training is controlled by you with no
restrictions I know of. You can't separate it from the underlying model
(for technical reasons).  I don't know the legal aspects of restrictions on
using your own fine-tuned model, but you still access it via SaaSS.


>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

[-- Attachment #2: Type: text/html, Size: 4347 bytes --]

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In addition, you can pay money to train your own model (via fine-tuning) on
  > top of Open AI's model.  Most other providers also let you do this.  The
  > model is "yours", and the training is controlled by you with no
  > restrictions I know of. You can't separate it from the underlying model
  > (for technical reasons).  I don't know the legal aspects of restrictions on
  > using your own fine-tuned model, but you still access it via SaaSS.

Thanks for informing me about this -- it is interesting.  However, in
regard to freedom and control, it doesn't really change things.

That scheme is the neural network equivalent of making a patch set
with which to modify the standard version of a program.

If you could get your own copy of the program source, apply your patch
set, and run that patched version, you'd have control over that
version.  Under a suitable free license, that would amount to free software.
The situation with TeX is that way.

But what they are doing makes it SaaSS all the way.

It's legitimate to offer a service of virtual servers on which you
can run your choice of system and software.  But systems and software
that you can't extract from those particular virtual servers are not
free software.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Emanuel Berg]

Ihor Radchenko wrote:

>>>> As for LLMs that run on servers, they are a different
>>>> issue entirely. They are all SaaSS (Service as a Software
>>>> Substitute), and SaaSS is always unjust.
>>>>
>>>> See
>>>> https://gnu.org/philosophy/who-does-that-server-really-serve.html
>>>> for explanation.
>>>
>>> I do not fully agree here. A number of more powerful LLMs
>>> have very limiting hardware requirements. For example,
>>> some LLMs require 64+Gbs of RAM to run:
>>
>> That is true, and it is unfortunate. There may be no
>> practical way to run a certain model except for SaaSS.
>>
>> That does not alter the injustice of SaaSS. So we should
>> not silence our criticism of SaaSS in those cases,
>
> This is a rather theoretical consideration, but, talking
> about ChatGTP (owned by OpenAI) specifically, should it even
> be considered SaaSS?

Can't we, i.e. GNU, offer a server where you can run FOSS
programs that offer services?

Wouldn't that be "their" software to people who use it?

They could improve the software like any other FOSS project
and, if necessary even, fork it to also run on the
server alongside.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Lucien Cartier-Tilet]

[-- Attachment #1.1.1: Type: text/plain, Size: 1085 bytes --]

Emanuel Berg <incal@dataswamp.org> writes:

> Can't we, i.e. GNU, offer a server where you can run FOSS
> programs that offer services?
>
> Wouldn't that be "their" software to people who use it?
>
> They could improve the software like any other FOSS project
> and, if necessary even, fork it to also run on the
> server alongside.

This is exactly what a French non-profit organisation (Framasoft [1])
does, they offer multiple services with FOSS software such as:
- Framadate (simple polls),
- Framaform (alternative to Google Forms),
- Framapad (alternative to Google Docs),
- Mobilizon (event plannifier),
And quite a few others, I’ll let you check the list of their services [2].

They are even the main developers of both PeerTube, an AGPL video
service on the Fediverse, and of Mobilizon which I mentioned above and
which is also AGPL.

[1] <https://framasoft.org/en/>
[2] <https://degooglisons-internet.org/en/>

--
Lucien “Phundrak” Cartier-Tilet
<https://phundrak.com> (Français)
<https://phundrak.com/en> (English)
Sent from GNU/Emacs

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Emanuel Berg]

Lucien Cartier-Tilet wrote:

>> Can't we, i.e. GNU, offer a server where you can run FOSS
>> programs that offer services?
>>
>> Wouldn't that be "their" software to people who use it?
>>
>> They could improve the software like any other FOSS project
>> and, if necessary even, fork it to also run on the
>> server alongside.
>
> This is exactly what a French non-profit organisation (Framasoft [1])
> does, they offer multiple services with FOSS software such as:
> - Framadate (simple polls),
> - Framaform (alternative to Google Forms),
> - Framapad (alternative to Google Docs),
> - Mobilizon (event plannifier),
> And quite a few others, I’ll let you check the list of their services [2].
>
> They are even the main developers of both PeerTube, an AGPL
> video service on the Fediverse, and of Mobilizon which
> I mentioned above and which is also AGPL.

Ikr? Good work!

-- 
underground experts united
https://dataswamp.org/~incal

Re: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >     ... what is constant is that the company can use
  >     our conversations with ChatGPT to train the model. This is not a
  >     surprise or a secret, the company has always reported it.

  > There is no doubt that ChatGTP itself is not libre - its model is not
  > available to public. However, users of the ChatGPT model are technically
  > providing input that is collaboratively editing that model weights
  > (training the model further). So, using ChatGTP is a little bit akin
  > editing Wikipedia pages - collaborating to improve ChatGTP.

That is a valid point, at the practical level.  But the differences
are crucial.

1. In Wikipedia, a contributor voluntarily chooses to participate in editing,
Editing participation is separate from consulting the encyclopedia.  This
fits the word "collaborating.

By contrast, a when the develoers of ChatGTP make it learn from the
user, that "contribution" is neither voluntary nor active.  It is more
"being taken advantage of" than "collaborating".

2. Wikipedia is a community project to develop a free/libre work.  (It
is no coincidence that this resembles the GNU Project.)  Morally it
deserves community support, despite some things it handles badly.

By contrast, ChatGTP is neither a community project nor free/libre.
That's perhaps why it arranges to manipulate people into "contributing"
rather than letting them choose.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Collaborative training of Libre LLMs (was: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm))

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

> 1. In Wikipedia, a contributor voluntarily chooses to participate in editing,
> Editing participation is separate from consulting the encyclopedia.  This
> fits the word "collaborating.
>
> By contrast, a when the develoers of ChatGTP make it learn from the
> user, that "contribution" is neither voluntary nor active.  It is more
> "being taken advantage of" than "collaborating".

It is actually voluntary now - according to
https://techunwrapped.com/you-can-now-make-chatgpt-not-train-with-your-queries/,
one can disable or enable training on user queries.
By default, it is enabled though.

> 2. Wikipedia is a community project to develop a free/libre work.  (It
> is no coincidence that this resembles the GNU Project.)  Morally it
> deserves community support, despite some things it handles badly.
>
> By contrast, ChatGTP is neither a community project nor free/libre.
> That's perhaps why it arranges to manipulate people into "contributing"
> rather than letting them choose.

Indeed, they do hold coercive power as people have no choice to copy run
the model independently.

However, I do not care much about OpenAI corporate practices - they are
as bad as we are used to in other bigtech SaaSS companies. What might be
a more interesting question to discuss is actual genuine collaborative
effort training a libre (not ChatGTP) model.

Currently, improving models is rather sequential process. If there is
one publicly available model, anyone can download the weights, train
them locally, and share the results. However, if multiple people take a
single _same_ version of the model and train it, the results, AFAIK,
cannot be combined.

As Andrew mentioned, the approach with "patching" a model is quite
interesting idea - if such "patches" may be combined, we can
get rid of the above concern with collaborative _ethical_ development of
models.

However, if the "patching" technology can only serve a single "patch" +
main model, there is a problem. Improving libre neural networks will
become difficult, unless people utilize collaborative server to
continuously improve a model.

Such collaborative server, similar to ChatGPT, will combine "editing"
(training) and "consulting" together. And, unlike Wikipedia, these
activities are hard to separate.

This raises a moral question about practical ways to improve libre
neural networks without falling into SaaSS practices.

As a practical example, there is https://github.com/khoj-ai/khoj/ Libre
neural network interface in development (it features Emacs support).
They recently started https://khoj.dev/ cloud aiming for people who
cannot afford to run the models locally. This discussion might be one of
the ethical considerations of using such cloud.

I CCed khoj devs.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Collaborative training of Libre LLMs (was: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm))

[Jean Louis]

* Ihor Radchenko <yantar92@posteo.net> [2023-09-09 13:28]:
> > By contrast, ChatGTP is neither a community project nor free/libre.
> > That's perhaps why it arranges to manipulate people into "contributing"
> > rather than letting them choose.
> 
> Indeed, they do hold coercive power as people have no choice to copy run
> the model independently.

There is free software for that type of artificial
intelligence. People do have choice.

Llama, Llama 2, Alpaca, GPT4All, Dolly, Vicuna, etc.

I think that "they do hold coercive power" is out of reality. To find
out if they have coercive power you should find the victim of coercion
and be able to tell name victim. 

The verb coerce has 1 sense (first 1 from tagged texts)p
1. (2) coerce, hale, squeeze, pressure, force -- (to cause to do through pressure or necessity, by physical, moral or intellectual means :"She forced him to take a job in the city"; "He squeezed her for information")

Otherwise it sounds as propaganda. There are too many services online,
nobody need to use them, I see there no coercion.

> However, I do not care much about OpenAI corporate practices - they are
> as bad as we are used to in other bigtech SaaSS companies. What might be
> a more interesting question to discuss is actual genuine collaborative
> effort training a libre (not ChatGTP) model.

Their closed software example is followed by free software. I see that
as positive not "as bad as we are ued to in other bigtech..."

I do not see anything bad here, I see that company offers service and
customers can freely decide to take service, or not. Keeping farms of
servers for that purpose is very expensive. There must be some
exchange between customers and company. Even Wikipedia, and GNU and
free software projects needs funds to continue. 

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: Collaborative training of Libre LLMs (was: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm))

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > However, if the "patching" technology can only serve a single "patch" +
  > main model, there is a problem. Improving libre neural networks will
  > become difficult, unless people utilize collaborative server to
  > continuously improve a model.

  > Such collaborative server, similar to ChatGPT, will combine "editing"
  > (training) and "consulting" together. And, unlike Wikipedia, these
  > activities are hard to separate.

If the users in this "community" can't move their work outside of a
private "collaborative server", they are in effect prisoners of that
server.  Whoever keeps them stuck there will have power, and that will
tempt per to mistreat them with it.

  > This raises a moral question about practical ways to improve libre
  > neural networks without falling into SaaSS practices.

From the example above, I conclude it is crucial that people who use a
particular platform to modify and run the model have the feasible
freedom of copying their modified versions off that platform and onto
any other platform that satisfies the specs needed to run these models.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Collaborative training of Libre LLMs (was: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm))

[Debanjum Singh Solanky]

[-- Attachment #1: Type: text/plain, Size: 1995 bytes --]

>   > However, if the "patching" technology can only serve a single "patch" +
>   > main model, there is a problem. Improving libre neural networks will
>   > become difficult, unless people utilize collaborative server to
>   > continuously improve a model.
>
>   > Such collaborative server, similar to ChatGPT, will combine "editing"
>   > (training) and "consulting" together. And, unlike Wikipedia, these
>   > activities are hard to separate.
>
> If the users in this "community" can't move their work outside of a
> private "collaborative server", they are in effect prisoners of that
> server.  Whoever keeps them stuck there will have power, and that will
> tempt per to mistreat them with it.
>

Versus traditional software, AI systems rely critically on the usage
data generated to improve the original model. Using copyleft licensed
models maybe enough to prevent a server owner from being able
to train a better closed model? This would prevent them from holding
users hostage on their server.



>   > This raises a moral question about practical ways to improve libre
>   > neural networks without falling into SaaSS practices.
>
> From the example above, I conclude it is crucial that people who use a
> particular platform to modify and run the model have the feasible
> freedom of copying their modified versions off that platform and onto
> any other platform that satisfies the specs needed to run these models.
>

Platform portability does not solve for how to improve libre
neural networks in an open, community guided way.

To collaboratively develop better open models we'd need the generated
usage data to be publically shareable. Attempts like open-assistant
(https://open-assistant.io) that share usage data under cc-by-sa maybe
a good enough solution for this. But it'll fall on the server owners
to get explicit user consent and clean sensitive usage data to share
this data publically without liability.

--
Debanjum Singh Solanky
Founder, Khoj (https://khoj.dev/)

[-- Attachment #2: Type: text/html, Size: 3980 bytes --]