From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Shane Mulligan Newsgroups: gmane.emacs.devel Subject: Re: Help building Pen.el (GPT for emacs) Date: Sat, 17 Jul 2021 21:27:17 +1200 Message-ID: References: <83im1948mj.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000001cc1bf05c74e5096" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="26654"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Stefan Kangas , rms@gnu.org, Emacs developers To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jul 17 11:28:49 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1m4gcq-0006me-FR for ged-emacs-devel@m.gmane-mx.org; Sat, 17 Jul 2021 11:28:48 +0200 Original-Received: from localhost ([::1]:39630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m4gcp-0001lJ-H3 for ged-emacs-devel@m.gmane-mx.org; Sat, 17 Jul 2021 05:28:47 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59638) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m4gbd-0000Wh-LX for emacs-devel@gnu.org; Sat, 17 Jul 2021 05:27:33 -0400 Original-Received: from mail-yb1-xb35.google.com ([2607:f8b0:4864:20::b35]:44933) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m4gba-0004QA-V9; Sat, 17 Jul 2021 05:27:33 -0400 Original-Received: by mail-yb1-xb35.google.com with SMTP id g19so18902639ybe.11; Sat, 17 Jul 2021 02:27:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=adKIRNp8xhVi61FRkxZGD+0l2wcQx7INiyWjgAQxebU=; b=PXK3+EC3wrjXJVXYQashXksXbHj+0BD0/ltWW5Ou6OMLHydC8ayxHPbVf+m5kAvhoq +GcWF2kTXuHUQH/PFuOnXvu4ANGFU8WAw+sN0aYS3LQtTnYucXxzeG/rY7uKqbwwgLJT IwUZgwq5PoT205xAgekdjJAzulWyFt7B7mBHdxTvb0CvyesN100Zi4Cd7/5rnTm+5M3y A4pEB2GmcRVOoDm0EstarhHJiHbhEqBhNQDXSpKER+F9qPL0Od+zW81QyZhJF+7btV7o BlsFIWamjX+rxd4XLjmthr/HG4nk1c8l6FYLMMTTZ8XTpkvD4Pp5dz/Sey/Ja9oCjrjM YW3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=adKIRNp8xhVi61FRkxZGD+0l2wcQx7INiyWjgAQxebU=; b=XwElIjqfPcZSa721yunx9l6X9X4xViPS/X8qvD7fnGlRt/tGWKILbWZyC3gg3BdUbk haqtdLvy44Mdxn+6oyds5NX7w7PEyPiBvjrGTchYiOfI2EpDve3mD00lbpXvmjHeRf+c qe7N9sASdv+bXdACIzYlAcE2Rm5ViIW4WMkcHCPRzJybmOtXUT/L+iyVwDf7S5j89pi1 0xWwKbMj60GbVChAnKiBoyTjVBrGnLssVGCLaxlJOjAf0p/+pGT18NFypwfwkREYehZ0 WBrSwQZSF72bgjMGXGcfZSLynM2ANcGHLJOmfrZNgBbFoDzh3/D/yKhEjDsJgoTbpIN8 zAAw== X-Gm-Message-State: AOAM530NCqcN9BlFfdZXVOZT0wsbWoDyR7nIy+NEfUkU/iRGdR3SWNgF 4uAFN0xIoj9WVBFpz3qSklf0b4lwyUnhjDK91SkuSdJokQ4LAyqlhw== X-Google-Smtp-Source: ABdhPJx0bTT3vtltwA5lbviA98Jdb1XBPhoCjyUOkfcF4Bq1+8ajwqLboAKUnZyPdv1WqnXTSkp0antKo1leUJBN+5Y= X-Received: by 2002:a25:ed01:: with SMTP id k1mr18174860ybh.74.1626514048990; Sat, 17 Jul 2021 02:27:28 -0700 (PDT) In-Reply-To: <83im1948mj.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::b35; envelope-from=mullikine@gmail.com; helo=mail-yb1-xb35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:271334 Archived-At: --0000000000001cc1bf05c74e5096 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Eli, It's nice to talk again. At this stage I am only seeking to inform you of this new technology which will be transformative to programming and open-source and show you that we have some quick catching up to do to integrate into emacs so Microsoft does not have a monopoly on the technology. "I understand that EleutherAI doesn't seem to support programming at this point, only natural language (is that true?)" The existing models which are not optimised to do code, can still do code well. GPT-j is EleutherAI's code model. It's designed as a direct competitor to codex (the Copilot model) and trained on open-source code. The other part of Copilot is the automatic fine-tuning of the model to enable it to learn your behaviour. This would be trickier to distribute as a service open source and probably isn't necessary, but GPT-j supports it. "any number of useful features where it could help." Name an emacs package and I can explain how GPT will affect that package. For `dired-git-info-mode`, for instance, a model connected to GPT can explain what files are for. "Name a package and I can name an augmentation." This is not fantasy. I have many examples. I have blogged for this exact purpose, to explain to people what OpenAI will be working on behind closed doors, to build a version for emacs. - https://mullikine.github.io/posts/explainshell-with-gpt-3/ - https://mullikine.github.io/posts/nlsh-natural-language-shell/ - https://mullikine.github.io/posts/context-menus-based-on-gpt-3/ - https://mullikine.github.io/posts/autocompleting-anything-with-gpt-3-in-ema= cs/ - https://mullikine.github.io/posts/translating-haskell-to-clojure-with-gpt-3= / - https://mullikine.github.io/posts/a-natural-language-database-using-a-singl= e-gpt-prompt/ - https://mullikine.github.io/posts/imaginary-programming-with-gpt-3/ - https://mullikine.github.io/posts/creating-a-playground-for-gpt-3-in-emacs/ "How is this different from existing translation servers?" GPT can replace Google search, Google translate, and many other services, and GPT can repond to requests with equal time for each request. It can also be used like stackoverflow to answer questions to many common problems. "Org-brain + GPT =3D a mind map, which automatically generates and suggests nodes, then lets you talk to a > chatbot tutor on any weird topic you can think of." Does this capability really exist? Yes it does I have demonstated it. - https://mullikine.github.io/posts/gpt-3-for-building-mind-maps-with-an-ai-t= utor-for-any-topic/ This is on my readme for my GPT project for emacs which supports GPT-3 and EleutherAI. https://github.com/semiosis/pen.el At its heart, emacs is an operating system based on a tty, which is a text stream. emacs supports a text-only mode. This makes it ideally suited for training a LM such as a GPT (Generative Pre-trained Transformer). emacs lisp provides a skeleton on which NLP functions can built around. Ultimately, emacs will become a fractal in the latent space of a future LM (language model). A graphical editor would not benefit from this effect until much later on. emacs could, if supported, become the vehicle for controllable text generation, or has the potential to become that, only actually surpassed when the imaginary programming environment is normal and other interfaces can be prompted into existence. Between then and now we can write prompt functions to help preserve emacs. This is my inspiration for the project. It sounds like science fiction, I know. On Sat, Jul 17, 2021 at 9:01 PM Eli Zaretskii wrote: > > From: Shane Mulligan > > Date: Sat, 17 Jul 2021 14:36:15 +1200 > > Cc: Stefan Kangas , Emacs developers < > emacs-devel@gnu.org> > > > > I think the end-goal should be to have a close collaboration with > EleutherAI, who already have an > > open-source alternative to the Copilot model. It's called GPT-j. > > =CE=B5=CE=BB=CE=B5=CF=85=CE=B8=CE=B5=CF=81=CE=AF=CE=B1 is a greek word = that means Freedom. EleutherAI are > open-sourcing language models. > > The problem is that there are very few people within EleutherAI using > emacs and few people who can help. > > I'm not sure I understand what features in Emacs this could enable. > And the references you provided don't seem to answer this question (or > maybe the answer is buried deeper than I'm prepared to dig at this > point). I understand that EleutherAI doesn't seem to support > programming at this point, only natural language (is that true?), but > that still means there could be any number of useful features where it > could help. But what are they? The stuff on the EleutherAI site is > oriented towards people who work in the machine learning domain, not > to programmers who design applications that could take advantage of > those capabilities, so it's not easy to understand what these > capabilities have in store for Emacs. > > Thus, description of relevant Emacs features, whether existing or > imaginary, with enough details for us to be able to discuss that > intelligently, will be appreciated. I don't think this discussion > will be meaningful without at least some idea of what we are trying to > accomplish. > > > If you'd please excuse my speculative musings, emacs has 40 years of > design waiting to be augmented with > > GPT3 and I believe that emacs is way ahead of the competition. It's a > gold rush really. > > Why do you think Emacs is better fitted to this than other editors? > It sounds like most of the processing is done server-side, so what > exactly is the significance of Emacs being the client? > > > Name a package and I can name an augmentation. > > Is this based on what these services (EleutherAI in particular) can > do, or are these just unrelated fantasies? We need ideas based on > capabilities that exist, not on what could exist years from now. AI > history is chock-full of ideas that didn't work out. > > > Take 'erc' and make it the first IRC client to automatically translate > all messages into any type of dialect -- > > French, Klingon or Pirate. > > How is this different from existing translation servers? > > > Company-mode + GPT =3D Copilot. > > I don't see how this is true. Copilot is not just generalized > completion, and AFAIU doesn't fit into the presentation methods used > by Company. What am I missing? > > > Org-roam + GPT =3D A multiversal prose editor ( > https://github.com/socketteer/loom) > > I couldn't understand what that does, looking at the above URL. Any > details how it works and how it helps the writer? > > > Org-brain + GPT =3D a mind map, which automatically generates and sugge= sts > nodes, then lets you talk to a > > chatbot tutor on any weird topic you can think of. > > Does this capability really exist? > > Thanks. > --0000000000001cc1bf05c74e5096 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Eli,
It's nice to talk again.

At th= is stage I am only seeking to inform you of this new technology
which wi= ll be transformative to programming and open-source and show you
that we= have some quick catching up to do to integrate into emacs so
Microsoft = does not have a monopoly on the technology.


"I understand t= hat EleutherAI doesn't seem to support
programming at this point, on= ly natural language (is that true?)"

The existing models which = are not optimised to do code, can still do
code well.

GPT-j is El= eutherAI's code model. It's designed as a direct
competitor to c= odex (the Copilot model) and trained on open-source code.

The other = part of Copilot is the automatic fine-tuning of the model to
enable it t= o learn your behaviour.

This would be trickier to distribute as a se= rvice open source and
probably isn't necessary, but GPT-j supports i= t.

"any number of useful features where it could help."Name an emacs package and I can explain how GPT will affect that package.<= br>For `dired-git-info-mode`, for instance, a model connected to GPT canexplain what files are for.

"Name a package and I can name an = augmentation."
This is not fantasy. I have many examples.

I = have blogged for this exact purpose, to explain to people what OpenAI
wi= ll be working on behind closed doors, to build a version for emacs.

= - ht= tps://mullikine.github.io/posts/explainshell-with-gpt-3/
- https:/= /mullikine.github.io/posts/nlsh-natural-language-shell/
- https://m= ullikine.github.io/posts/context-menus-based-on-gpt-3/
- https://mullikine.github.io/posts/autocompleting-anything-with-gpt-3-i= n-emacs/
- https://mullikine.github.io/posts/translati= ng-haskell-to-clojure-with-gpt-3/
- https:= //mullikine.github.io/posts/a-natural-language-database-using-a-single-gpt-= prompt/
- https://mullikine.github.io/posts/imaginary-programmi= ng-with-gpt-3/
- https://mullikine.github.io/posts/crea= ting-a-playground-for-gpt-3-in-emacs/

"How is this differen= t from existing translation servers?"
GPT can replace Google search= , Google translate, and many other
services, and GPT can repond to reque= sts with equal time for each
request. It can also be used like stackover= flow to answer questions to
many common problems.

"Org-brain= + GPT =3D a mind map, which automatically generates and
suggests nodes,= then lets you talk to a
> chatbot tutor on any weird topic you can t= hink of."

Does this capability really exist?

Yes it does= I have demonstated it.

- https://mu= llikine.github.io/posts/gpt-3-for-building-mind-maps-with-an-ai-tutor-for-a= ny-topic/

This is on my readme for my GPT project for emacs whic= h supports GPT-3
and EleutherAI.

https://github.com/semiosis/pen.el

At its heart, = emacs is an operating system based on a tty, which is a
text stream.
=
emacs supports a text-only mode. This makes it ideally suited for
tr= aining a LM such as a GPT (Generative Pre-trained Transformer).

emac= s lisp provides a skeleton on which NLP functions can built around.
Ulti= mately, emacs will become a fractal in the latent space of a future
LM (= language model). A graphical editor would not benefit from this
effect u= ntil much later on.

emacs could, if supported, become the vehicle fo= r controllable text
generation, or has the potential to become that, onl= y actually surpassed
when the imaginary programming environment is norma= l and other
interfaces can be prompted into existence.

Between th= en and now we can write prompt functions to help preserve
emacs.

= This is my inspiration for the project. It sounds like science fiction, I k= now.


On Sat, Jul 17, 2021 at 9:01 = PM Eli Zaretskii <eliz@gnu.org> w= rote:
> From:= Shane Mulligan <mullikine@gmail.com>
> Date: Sat, 17 Jul 2021 14:36:15 +1200
> Cc: Stefan Kangas <stefan@marxist.se>, Emacs developers <emacs-devel@gnu.org>
>
> I think the end-goal should be to have a close collaboration with Eleu= therAI, who already have an
> open-source alternative to the Copilot model. It's called GPT-j. > =CE=B5=CE=BB=CE=B5=CF=85=CE=B8=CE=B5=CF=81=CE=AF=CE=B1 is a greek word= that means Freedom. EleutherAI are open-sourcing language models.
> The problem is that there are very few people within EleutherAI using = emacs and few people who can help.

I'm not sure I understand what features in Emacs this could enable.
And the references you provided don't seem to answer this question (or<= br> maybe the answer is buried deeper than I'm prepared to dig at this
point).=C2=A0 I understand that EleutherAI doesn't seem to support
programming at this point, only natural language (is that true?), but
that still means there could be any number of useful features where it
could help.=C2=A0 But what are they?=C2=A0 The stuff on the EleutherAI site= is
oriented towards people who work in the machine learning domain, not
to programmers who design applications that could take advantage of
those capabilities, so it's not easy to understand what these
capabilities have in store for Emacs.

Thus, description of relevant Emacs features, whether existing or
imaginary, with enough details for us to be able to discuss that
intelligently, will be appreciated.=C2=A0 I don't think this discussion=
will be meaningful without at least some idea of what we are trying to
accomplish.

> If you'd please excuse my speculative musings, emacs has 40 years = of design waiting to be augmented with
> GPT3 and I believe that emacs is way ahead of the competition. It'= s a gold rush really.

Why do you think Emacs is better fitted to this than other editors?
It sounds like most of the processing is done server-side, so what
exactly is the significance of Emacs being the client?

> Name a package and I can name an augmentation.

Is this based on what these services (EleutherAI in particular) can
do, or are these just unrelated fantasies?=C2=A0 We need ideas based on
capabilities that exist, not on what could exist years from now.=C2=A0 AI history is chock-full of ideas that didn't work out.

> Take 'erc' and make it the first IRC client to automatically t= ranslate all messages into any type of dialect --
> French, Klingon or Pirate.

How is this different from existing translation servers?

> Company-mode + GPT =3D Copilot.

I don't see how this is true.=C2=A0 Copilot is not just generalized
completion, and AFAIU doesn't fit into the presentation methods used by Company.=C2=A0 What am I missing?

> Org-roam + GPT =3D A multiversal prose editor (https://github= .com/socketteer/loom)

I couldn't understand what that does, looking at the above URL.=C2=A0 A= ny
details how it works and how it helps the writer?

> Org-brain + GPT =3D a mind map, which automatically generates and sugg= ests nodes, then lets you talk to a
> chatbot tutor on any weird topic you can think of.

Does this capability really exist?

Thanks.
--0000000000001cc1bf05c74e5096--