From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Debanjum Singh Solanky Newsgroups: gmane.emacs.tangents Subject: Re: Collaborative training of Libre LLMs (was: Is ChatGTP SaaSS? (was: [NonGNU ELPA] New package: llm)) Date: Sat, 9 Sep 2023 19:18:44 -0700 Message-ID: References: <87v8d0iqa5.fsf@posteo.net> <87cyz3vaws.fsf@localhost> <87a5tzsbvl.fsf@localhost> <87fs3n7i98.fsf@localhost> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000ea21360604f7d4bb" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33412"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Ihor Radchenko , emacs-tangents@gnu.org, jporterbugs@gmail.com, ahyatt@gmail.com, team@khoj.dev To: rms@gnu.org Original-X-From: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Sun Sep 10 06:56:56 2023 Return-path: Envelope-to: get-emacs-tangents@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qfCVD-0008VZ-M6 for get-emacs-tangents@m.gmane-mx.org; Sun, 10 Sep 2023 06:56:55 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qfCUn-0001UO-4L; Sun, 10 Sep 2023 00:56:29 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qfA2S-0005eg-Jy for emacs-tangents@gnu.org; Sat, 09 Sep 2023 22:19:04 -0400 Original-Received: from mail-wm1-x32c.google.com ([2a00:1450:4864:20::32c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qfA2M-0004gN-7N for emacs-tangents@gnu.org; Sat, 09 Sep 2023 22:19:04 -0400 Original-Received: by mail-wm1-x32c.google.com with SMTP id 5b1f17b1804b1-401da71b85eso36157105e9.1 for ; Sat, 09 Sep 2023 19:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=khoj.dev; s=google; t=1694312336; x=1694917136; darn=gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QTgogHPPQi+zdG+pWrYl3mNtseFGiyiieTstWyyPfSI=; b=FepXHLMQs82ZHYfp9k7Vwrse8n9nru993yQrri36XwN6LfCaQVjxY8Dq60m4Ye5o5h S/oUsMU+OBc85+c8ktlcS0+jB2JpLUnfKX7E4j4z/vOt24OGcOHm/V/zmGEZ0O+eSOpe 7eD0u47mPvsnqmI7ICopYN+L00xNep5AZgQTuTkLykZOCnEOP+mrbxGPPe7Cp5Vtn+fI k+9ff3qF7Ax+4Oyh950r6Bzj9WPWR3sSvi3zg8iI+cbkZYpur5NqiCEHueqB2NfHyJx6 Q/BtLC6klGvitcu9ztcfwHqt3xJzolz6RFivLrzU55eEjGnLP4yzrguECuKyevrF/2dH QtuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694312336; x=1694917136; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QTgogHPPQi+zdG+pWrYl3mNtseFGiyiieTstWyyPfSI=; b=jOXpzfoGAHYgZkXWk/XYFQWX0GUGttvjfajXfT+Y7Us33dWX30N265g/MfPDrcL0Wf /WfDhraXXhTVQJZmL/uEmF19bzbB7QRyaJPJIuoKFtcZ+hC5rmtJ5YtZ9a9iUfT3epz7 tK2HzKs/4Feva4CtDr9yWY0g84PPKBOWay/5sqX8SmYiVfmvdJT57bGcNbUFgiIBFtma vsYK5yTKbVVO9+uoiKYiMqIjhXnrVtEzr2Io8tzkx8s8J1UrW722Pa500Dv8vbSAypvW Q7vrnvMG2kCgHBM3a7I35/p9T4JDsulsd7np5y4byohIzEqYJXLKSeAnZQnjB+j2JOR4 MLYQ== X-Gm-Message-State: AOJu0YwyUjKHog/JpCKyieo8JmqUyc0r6uVW1D1AZBC4m1JjFEAQBGwc XN9sI00WQMBfxtF84XJyA5c4x4WNbmB8WJnOQl1aLQ== X-Google-Smtp-Source: AGHT+IGKx3y7Okh4MLDaRYsXtoIi+JfIbtwtKA9aPH4yetWPlGbkas0FPdTJ4PGhS2Fi8aWQJbDuvQrEoAL+99gEUAU= X-Received: by 2002:a1c:f70b:0:b0:3fe:22fd:1b23 with SMTP id v11-20020a1cf70b000000b003fe22fd1b23mr5195043wmh.34.1694312335804; Sat, 09 Sep 2023 19:18:55 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2a00:1450:4864:20::32c; envelope-from=debanjum@khoj.dev; helo=mail-wm1-x32c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Sun, 10 Sep 2023 00:56:27 -0400 X-BeenThere: emacs-tangents@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Emacs news and miscellaneous discussions outside the scope of other Emacs mailing lists List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Original-Sender: emacs-tangents-bounces+get-emacs-tangents=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.tangents:1062 Archived-At: --000000000000ea21360604f7d4bb Content-Type: text/plain; charset="UTF-8" > > However, if the "patching" technology can only serve a single "patch" + > > main model, there is a problem. Improving libre neural networks will > > become difficult, unless people utilize collaborative server to > > continuously improve a model. > > > Such collaborative server, similar to ChatGPT, will combine "editing" > > (training) and "consulting" together. And, unlike Wikipedia, these > > activities are hard to separate. > > If the users in this "community" can't move their work outside of a > private "collaborative server", they are in effect prisoners of that > server. Whoever keeps them stuck there will have power, and that will > tempt per to mistreat them with it. > Versus traditional software, AI systems rely critically on the usage data generated to improve the original model. Using copyleft licensed models maybe enough to prevent a server owner from being able to train a better closed model? This would prevent them from holding users hostage on their server. > > This raises a moral question about practical ways to improve libre > > neural networks without falling into SaaSS practices. > > From the example above, I conclude it is crucial that people who use a > particular platform to modify and run the model have the feasible > freedom of copying their modified versions off that platform and onto > any other platform that satisfies the specs needed to run these models. > Platform portability does not solve for how to improve libre neural networks in an open, community guided way. To collaboratively develop better open models we'd need the generated usage data to be publically shareable. Attempts like open-assistant (https://open-assistant.io) that share usage data under cc-by-sa maybe a good enough solution for this. But it'll fall on the server owners to get explicit user consent and clean sensitive usage data to share this data publically without liability. -- Debanjum Singh Solanky Founder, Khoj (https://khoj.dev/) --000000000000ea21360604f7d4bb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
=C2=A0
=C2=A0 > However, if the "patching" technology can only serve = a single "patch" +
=C2=A0 > main model, there is a problem. Improving libre neural networks= will
=C2=A0 > become difficult, unless people utilize collaborative server to=
=C2=A0 > continuously improve a model.

=C2=A0 > Such collaborative server, similar to ChatGPT, will combine &qu= ot;editing"
=C2=A0 > (training) and "consulting" together. And, unlike Wik= ipedia, these
=C2=A0 > activities are hard to separate.

If the users in this "community" can't move their work outsid= e of a
private "collaborative server", they are in effect prisoners of t= hat
server.=C2=A0 Whoever keeps them stuck there will have power, and that will=
tempt per to mistreat them with it.

Versus traditional software, AI systems rely critically= on the usage
data generated to improve t= he original model. Using copyleft licensed
models maybe enough to prevent a server owner from being able
to train a better closed model? This would prevent t= hem from holding
users hostage on their s= erver.

=C2=A0
=C2=A0 > This raises a moral question about practical ways to improve li= bre
=C2=A0 > neural networks without falling into SaaSS practices.

>From the example above, I conclude it is crucial that people who use a
particular platform to modify and run the model have the feasible
freedom of copying their modified versions off that platform and onto
any other platform that satisfies the specs needed to run these models.
=

=
Platform portability does not solve for how to improve libre
neur= al networks in an open, community guided way.

To collaboratively develop = better open models we'd need the generated
usage data to be publically shareable. Attempts like open-assistant
(htt= ps://open-assistant.io) that share usage data under cc-by-sa maybe
a good enough solution for this. But it'l= l fall on the server owners
to get explic= it user consent and clean sensitive usage data to share
this data publically without liability.

--
De= banjum Singh Solanky
Founder, Khoj (https://khoj.dev/)
--000000000000ea21360604f7d4bb--