From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Shane Mulligan Newsgroups: gmane.emacs.devel Subject: Re: Help building Pen.el (GPT for emacs) Date: Sun, 18 Jul 2021 09:02:17 +1200 Message-ID: References: <83im1948mj.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000009371f505c7580535" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27062"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Stefan Kangas , rms@gnu.org, Emacs developers To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jul 17 23:03:25 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m4rT3-0006tG-0Y for ged-emacs-devel@m.gmane-mx.org; Sat, 17 Jul 2021 23:03:25 +0200 Original-Received: from localhost ([::1]:60888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m4rT0-0000hV-T8 for ged-emacs-devel@m.gmane-mx.org; Sat, 17 Jul 2021 17:03:22 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47596) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m4rSC-0008RY-Pb for emacs-devel@gnu.org; Sat, 17 Jul 2021 17:02:36 -0400 Original-Received: from mail-yb1-xb35.google.com ([2607:f8b0:4864:20::b35]:35537) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m4rSA-0001Nx-1x; Sat, 17 Jul 2021 17:02:32 -0400 Original-Received: by mail-yb1-xb35.google.com with SMTP id t186so20727163ybf.2; Sat, 17 Jul 2021 14:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yVvEA7OtjfxuZgyokzXKBlBHT0Id5qJCt6sT3p4DEPQ=; b=e19OEmxZhY3dplgwoHU2vna3puB/Sfg6qOA8HXjDBq+hjBFnFkMq0a/WIC1yK3qGYT wF6jbb2ZdMnBRzPEW+3476cKja4+OpThcuBJD7PQZbV2Vk3mNMgkw0hpPts04uvmNbQN MfYu7nHM+lCD6IAIuShEREsPuF8IaPs+oa0P/HszqARDAtGVYaINxNawLY6Z154w6RyI D9ojEsxrbvtjgIOxHWY28mGG1DyZwcukVKUzqqLnW/4SG2Pzu3eD8y4ORce9v4k/HfJH agMv4o/1AjF+TEzKYK9b4mivX/tFifxozje5fRbAMUr/eNu6QjfhafLHc5afo+wY/S5i yeoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yVvEA7OtjfxuZgyokzXKBlBHT0Id5qJCt6sT3p4DEPQ=; b=WaIaA+9cM/wlTkTV9zVtUhv4WYLnKFYmH8JT511VuoA+l0NyJ3/n2KBE3kvYfjXj6y yhi8EPoIrr2wH5Wv1Jbhk5fdsrM/0yWPATCoc+rhrU5ldAtb1qjnB+evWtKYj4BuFUrZ 8Rh7KJCojzsjLe1DDov8a5J94Ca9VK0v9DQAARFqEc5ZIrKn4eAfOaN1yU1ZOJ+BKZIj 6RzrUaCwc/vDuSkJ1q3nJgoFVAxAAFy84eYUo/R3/LyBfhKpLcbfvQYNnlOyz5FkcOOi 651zHQTWz1QuGiLkWQeQl1ElzwGb1Bk1zS2oNfgUXK+sVVc4XWSJS8t4jOwSJuZmiCRk h/ZA== X-Gm-Message-State: AOAM531RPd65kzEy4PN0PLWJrjpTwaIoeVITQSqhIo9SMp3gpJsiskzh eLIjlVeRjg4CzI1CCiW6E+KwBwN/cuyPZQGjzVGfUwZBuCWMaANrAQ== X-Google-Smtp-Source: ABdhPJw2tC992o0glp5enZvjCXNL2Zu7d2iJBwwhRVcWkD8dvg4OGx010ZY16naBQ9rrfxJkNDNoO82yRHpK0G/qpto= X-Received: by 2002:a25:ed01:: with SMTP id k1mr21079568ybh.74.1626555748150; Sat, 17 Jul 2021 14:02:28 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::b35; envelope-from=mullikine@gmail.com; helo=mail-yb1-xb35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271343 Archived-At: --0000000000009371f505c7580535 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The following is why emacs needs open-source prompts -- ones that don't learn from you or are sold to you - Ones that you write for yourself. - An open-source prompts melpa at the very least! As I tried to describe before, it's a fundamentally new way of programming. An extension of Donald Knuth's literate programming becoming imaginary programming, but being hijacked by microsoft. Microsft GPT is an attack on the innermost workings of emacs -- the text stream. So embracing the OpenSource alternatives from EleutherAI is crucial= . I have said enough. I leave you with this article. https://venturebeat.com/2021/07/16/openai-disbands-its-robotics-research-te= am/ """ OpenAI said recently that GPT-3 is now being used in more than 300 different apps by =E2=80=9Ctens of thousands=E2=80=9D of developers and producing 4.5 billion words per day.) Toward the end of 2020, Microsoft announced that it would exclusively license GPT-3 to develop and deliver AI solutions for customers, as well as creating new products that harness the power of NLG. Microsoft recently announced that GPT-3 will be integrated =E2=80=9Cdeeply=E2=80=9D with Power Apps, its low-code app development platform =E2=80=94 specifically for formula generation. The AI- powered features will allow a user building an ecommerce app, for example, to describe a programming goal using conversational language like =E2=80=9Cfind products where the name starts with =E2=80=98kids.'=E2=80=9D """" On Sat, Jul 17, 2021 at 9:27 PM Shane Mulligan wrote: > Hi Eli, > It's nice to talk again. > > At this stage I am only seeking to inform you of this new technology > which will be transformative to programming and open-source and show you > that we have some quick catching up to do to integrate into emacs so > Microsoft does not have a monopoly on the technology. > > > "I understand that EleutherAI doesn't seem to support > programming at this point, only natural language (is that true?)" > > The existing models which are not optimised to do code, can still do > code well. > > GPT-j is EleutherAI's code model. It's designed as a direct > competitor to codex (the Copilot model) and trained on open-source code. > > The other part of Copilot is the automatic fine-tuning of the model to > enable it to learn your behaviour. > > This would be trickier to distribute as a service open source and > probably isn't necessary, but GPT-j supports it. > > "any number of useful features where it could help." > Name an emacs package and I can explain how GPT will affect that package. > For `dired-git-info-mode`, for instance, a model connected to GPT can > explain what files are for. > > "Name a package and I can name an augmentation." > This is not fantasy. I have many examples. > > I have blogged for this exact purpose, to explain to people what OpenAI > will be working on behind closed doors, to build a version for emacs. > > - https://mullikine.github.io/posts/explainshell-with-gpt-3/ > - https://mullikine.github.io/posts/nlsh-natural-language-shell/ > - https://mullikine.github.io/posts/context-menus-based-on-gpt-3/ > - > https://mullikine.github.io/posts/autocompleting-anything-with-gpt-3-in-e= macs/ > - > https://mullikine.github.io/posts/translating-haskell-to-clojure-with-gpt= -3/ > - > https://mullikine.github.io/posts/a-natural-language-database-using-a-sin= gle-gpt-prompt/ > - https://mullikine.github.io/posts/imaginary-programming-with-gpt-3/ > - > https://mullikine.github.io/posts/creating-a-playground-for-gpt-3-in-emac= s/ > > "How is this different from existing translation servers?" > GPT can replace Google search, Google translate, and many other > services, and GPT can repond to requests with equal time for each > request. It can also be used like stackoverflow to answer questions to > many common problems. > > "Org-brain + GPT =3D a mind map, which automatically generates and > suggests nodes, then lets you talk to a > > chatbot tutor on any weird topic you can think of." > > Does this capability really exist? > > Yes it does I have demonstated it. > > - > https://mullikine.github.io/posts/gpt-3-for-building-mind-maps-with-an-ai= -tutor-for-any-topic/ > > This is on my readme for my GPT project for emacs which supports GPT-3 > and EleutherAI. > > https://github.com/semiosis/pen.el > > At its heart, emacs is an operating system based on a tty, which is a > text stream. > > emacs supports a text-only mode. This makes it ideally suited for > training a LM such as a GPT (Generative Pre-trained Transformer). > > emacs lisp provides a skeleton on which NLP functions can built around. > Ultimately, emacs will become a fractal in the latent space of a future > LM (language model). A graphical editor would not benefit from this > effect until much later on. > > emacs could, if supported, become the vehicle for controllable text > generation, or has the potential to become that, only actually surpassed > when the imaginary programming environment is normal and other > interfaces can be prompted into existence. > > Between then and now we can write prompt functions to help preserve > emacs. > > This is my inspiration for the project. It sounds like science fiction, I > know. > > > On Sat, Jul 17, 2021 at 9:01 PM Eli Zaretskii wrote: > >> > From: Shane Mulligan >> > Date: Sat, 17 Jul 2021 14:36:15 +1200 >> > Cc: Stefan Kangas , Emacs developers < >> emacs-devel@gnu.org> >> > >> > I think the end-goal should be to have a close collaboration with >> EleutherAI, who already have an >> > open-source alternative to the Copilot model. It's called GPT-j. >> > =CE=B5=CE=BB=CE=B5=CF=85=CE=B8=CE=B5=CF=81=CE=AF=CE=B1 is a greek word= that means Freedom. EleutherAI are >> open-sourcing language models. >> > The problem is that there are very few people within EleutherAI using >> emacs and few people who can help. >> >> I'm not sure I understand what features in Emacs this could enable. >> And the references you provided don't seem to answer this question (or >> maybe the answer is buried deeper than I'm prepared to dig at this >> point). I understand that EleutherAI doesn't seem to support >> programming at this point, only natural language (is that true?), but >> that still means there could be any number of useful features where it >> could help. But what are they? The stuff on the EleutherAI site is >> oriented towards people who work in the machine learning domain, not >> to programmers who design applications that could take advantage of >> those capabilities, so it's not easy to understand what these >> capabilities have in store for Emacs. >> >> Thus, description of relevant Emacs features, whether existing or >> imaginary, with enough details for us to be able to discuss that >> intelligently, will be appreciated. I don't think this discussion >> will be meaningful without at least some idea of what we are trying to >> accomplish. >> >> > If you'd please excuse my speculative musings, emacs has 40 years of >> design waiting to be augmented with >> > GPT3 and I believe that emacs is way ahead of the competition. It's a >> gold rush really. >> >> Why do you think Emacs is better fitted to this than other editors? >> It sounds like most of the processing is done server-side, so what >> exactly is the significance of Emacs being the client? >> >> > Name a package and I can name an augmentation. >> >> Is this based on what these services (EleutherAI in particular) can >> do, or are these just unrelated fantasies? We need ideas based on >> capabilities that exist, not on what could exist years from now. AI >> history is chock-full of ideas that didn't work out. >> >> > Take 'erc' and make it the first IRC client to automatically translate >> all messages into any type of dialect -- >> > French, Klingon or Pirate. >> >> How is this different from existing translation servers? >> >> > Company-mode + GPT =3D Copilot. >> >> I don't see how this is true. Copilot is not just generalized >> completion, and AFAIU doesn't fit into the presentation methods used >> by Company. What am I missing? >> >> > Org-roam + GPT =3D A multiversal prose editor ( >> https://github.com/socketteer/loom) >> >> I couldn't understand what that does, looking at the above URL. Any >> details how it works and how it helps the writer? >> >> > Org-brain + GPT =3D a mind map, which automatically generates and >> suggests nodes, then lets you talk to a >> > chatbot tutor on any weird topic you can think of. >> >> Does this capability really exist? >> >> Thanks. >> > --0000000000009371f505c7580535 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The following is why emacs needs open-source prompts -- on= es that don't learn from you or are sold to you

- Ones that you= write for yourself.
- An open-source prompts melpa at the very least!

As I tried to describe before, it's a fundament= ally new way of programming. An extension of Donald Knuth's literate pr= ogramming becoming imaginary programming, but being hijacked by microsoft.<= /div>

Microsft=C2=A0GPT is an attack on the innermost wo= rkings of emacs -- the text stream. So embracing the OpenSource alternative= s from EleutherAI is crucial.

I have said enough. = I leave you with this article.

https://venturebeat.c= om/2021/07/16/openai-disbands-its-robotics-research-team/

"= ""
OpenAI said recently that GPT-3 is now being
used in mor= e than 300 different apps by =E2=80=9Ctens
of thousands=E2=80=9D of deve= lopers and producing 4.5
billion words per day.) Toward the end of
20= 20, Microsoft announced that it would
exclusively license GPT-3 to devel= op and
deliver AI solutions for customers, as well as
creating new pr= oducts that harness the power
of NLG.

Microsoft recently announce= d that GPT-3 will
be integrated =E2=80=9Cdeeply=E2=80=9D with Power Apps= , its
low-code app development platform =E2=80=94
specifically for fo= rmula generation. The AI-
powered features will allow a user building an=
ecommerce app, for example, to describe a
programming goal using con= versational language
like =E2=80=9Cfind products where the name starts w= ith
=E2=80=98kids.'=E2=80=9D
""""


On Sat, Jul 17, 2021 at 9:27 PM Shane Mulligan <mullikine@gmail.com> wrote:
<= div>Hi Eli,
It's nice to talk again.

At this stage I am only = seeking to inform you of this new technology
which will be transformativ= e to programming and open-source and show you
that we have some quick ca= tching up to do to integrate into emacs so
Microsoft does not have a mon= opoly on the technology.


"I understand that EleutherAI does= n't seem to support
programming at this point, only natural language= (is that true?)"

The existing models which are not optimised t= o do code, can still do
code well.

GPT-j is EleutherAI's code= model. It's designed as a direct
competitor to codex (the Copilot m= odel) and trained on open-source code.

The other part of Copilot is = the automatic fine-tuning of the model to
enable it to learn your behavi= our.

This would be trickier to distribute as a service open source a= nd
probably isn't necessary, but GPT-j supports it.

"any= number of useful features where it could help."
Name an emacs pack= age and I can explain how GPT will affect that package.
For `dired-git-i= nfo-mode`, for instance, a model connected to GPT can
explain what files= are for.

"Name a package and I can name an augmentation."=
This is not fantasy. I have many examples.

I have blogged for th= is exact purpose, to explain to people what OpenAI
will be working on be= hind closed doors, to build a version for emacs.

- htt= ps://mullikine.github.io/posts/explainshell-with-gpt-3/
- https://mullikine.github.io/posts/nlsh-natural-language-shell/<= br>- https://mullikine.github.io/posts/context-menus-bas= ed-on-gpt-3/
- https://mullikine.g= ithub.io/posts/autocompleting-anything-with-gpt-3-in-emacs/
- https://mullikine.github.io/posts/translating-has= kell-to-clojure-with-gpt-3/
- https://mullikine.github.io/posts/a-natural-language-database-using-a= -single-gpt-prompt/
- https://mullikine.githu= b.io/posts/imaginary-programming-with-gpt-3/
- https://mullikine.github.io/posts/creating-a-playground-for-gpt= -3-in-emacs/

"How is this different from existing translati= on servers?"
GPT can replace Google search, Google translate, and m= any other
services, and GPT can repond to requests with equal time for e= ach
request. It can also be used like stackoverflow to answer questions = to
many common problems.

"Org-brain + GPT =3D a mind map, wh= ich automatically generates and
suggests nodes, then lets you talk to a<= br>> chatbot tutor on any weird topic you can think of."

Doe= s this capability really exist?

Yes it does I have demonstated it.
- https://mullikine= .github.io/posts/gpt-3-for-building-mind-maps-with-an-ai-tutor-for-any-topi= c/

This is on my readme for my GPT project for emacs which suppo= rts GPT-3
and EleutherAI.

https://github.com/semiosis/pen.el

At = its heart, emacs is an operating system based on a tty, which is a
text = stream.

emacs supports a text-only mode. This makes it ideally suite= d for
training a LM such as a GPT (Generative Pre-trained Transformer).<= br>
emacs lisp provides a skeleton on which NLP functions can built arou= nd.
Ultimately, emacs will become a fractal in the latent space of a fut= ure
LM (language model). A graphical editor would not benefit from this<= br>effect until much later on.

emacs could, if supported, become the= vehicle for controllable text
generation, or has the potential to becom= e that, only actually surpassed
when the imaginary programming environme= nt is normal and other
interfaces can be prompted into existence.
Between then and now we can write prompt functions to help preserve
ema= cs.

This is my inspiration for the project. It sounds like science f= iction, I know.


On Sat, Jul 17, 2021 at 9:01 = PM Eli Zaretskii <eliz= @gnu.org> wrote:
> From: Shane Mulligan <mullikine@gmail.com>
> Date: Sat, 17 Jul 2021 14:36:15 +1200
> Cc: Stefan Kangas <stefan@marxist.se>, Emacs developers <emacs-devel@gnu.org>
>
> I think the end-goal should be to have a close collaboration with Eleu= therAI, who already have an
> open-source alternative to the Copilot model. It's called GPT-j. > =CE=B5=CE=BB=CE=B5=CF=85=CE=B8=CE=B5=CF=81=CE=AF=CE=B1 is a greek word= that means Freedom. EleutherAI are open-sourcing language models.
> The problem is that there are very few people within EleutherAI using = emacs and few people who can help.

I'm not sure I understand what features in Emacs this could enable.
And the references you provided don't seem to answer this question (or<= br> maybe the answer is buried deeper than I'm prepared to dig at this
point).=C2=A0 I understand that EleutherAI doesn't seem to support
programming at this point, only natural language (is that true?), but
that still means there could be any number of useful features where it
could help.=C2=A0 But what are they?=C2=A0 The stuff on the EleutherAI site= is
oriented towards people who work in the machine learning domain, not
to programmers who design applications that could take advantage of
those capabilities, so it's not easy to understand what these
capabilities have in store for Emacs.

Thus, description of relevant Emacs features, whether existing or
imaginary, with enough details for us to be able to discuss that
intelligently, will be appreciated.=C2=A0 I don't think this discussion=
will be meaningful without at least some idea of what we are trying to
accomplish.

> If you'd please excuse my speculative musings, emacs has 40 years = of design waiting to be augmented with
> GPT3 and I believe that emacs is way ahead of the competition. It'= s a gold rush really.

Why do you think Emacs is better fitted to this than other editors?
It sounds like most of the processing is done server-side, so what
exactly is the significance of Emacs being the client?

> Name a package and I can name an augmentation.

Is this based on what these services (EleutherAI in particular) can
do, or are these just unrelated fantasies?=C2=A0 We need ideas based on
capabilities that exist, not on what could exist years from now.=C2=A0 AI history is chock-full of ideas that didn't work out.

> Take 'erc' and make it the first IRC client to automatically t= ranslate all messages into any type of dialect --
> French, Klingon or Pirate.

How is this different from existing translation servers?

> Company-mode + GPT =3D Copilot.

I don't see how this is true.=C2=A0 Copilot is not just generalized
completion, and AFAIU doesn't fit into the presentation methods used by Company.=C2=A0 What am I missing?

> Org-roam + GPT =3D A multiversal prose editor (https://github= .com/socketteer/loom)

I couldn't understand what that does, looking at the above URL.=C2=A0 A= ny
details how it works and how it helps the writer?

> Org-brain + GPT =3D a mind map, which automatically generates and sugg= ests nodes, then lets you talk to a
> chatbot tutor on any weird topic you can think of.

Does this capability really exist?

Thanks.
--0000000000009371f505c7580535--