From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrew Hyatt Newsgroups: gmane.emacs.devel Subject: Re: LLM Experiments, Part 1: Corrections Date: Wed, 24 Jan 2024 11:00:42 -0400 Message-ID: References: <87plxrr06f.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31535"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: contact@karthinks.com, emacs-devel@gnu.org, sskostyaev@gmail.com To: "T.V Raman" Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jan 24 16:01:19 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rSekh-00083Y-H3 for ged-emacs-devel@m.gmane-mx.org; Wed, 24 Jan 2024 16:01:19 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rSekJ-0005hw-Pi; Wed, 24 Jan 2024 10:00:55 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rSekF-0005h4-5g for emacs-devel@gnu.org; Wed, 24 Jan 2024 10:00:51 -0500 Original-Received: from mail-vs1-xe32.google.com ([2607:f8b0:4864:20::e32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rSekD-0001e3-31 for emacs-devel@gnu.org; Wed, 24 Jan 2024 10:00:50 -0500 Original-Received: by mail-vs1-xe32.google.com with SMTP id ada2fe7eead31-46afc05c192so791500137.0 for ; Wed, 24 Jan 2024 07:00:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706108447; x=1706713247; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=b5VQAz4u9LpTS0egjdrLjAZQ6Mkcn+U9ffxq2+hh/Yw=; b=jpmjUpPRYgAXtWtPLz8ibeoaKY9DGH5XrHyHRCZ/vAZl+3GCfcugctruRPcF5MaS8N yHa3tRaifiXT5xAqcmTNP26wK/P9+fgO+cGB3MwqZ36fs6EV1LMsO09R9Q8J4QPMTpe5 EACCtcZ8jY9lp9mYBNVN0M74rp4XoSIj+eNJDYEQQDBnjuS/Sn0r1dVyaIwzXABaae9s Rrw8Lh+O0Ubd66rg7vWSu1hmNtba66V4hq/1UjIXm4Q/KL1sPTxH77C/phezf9XlO0EI Tx/oKqtyTxIY4I8vcOiA01ZnI+e2EXd2t7cOI3JA+vzfj1ynNCZ3vOaxoa5mGysdKglg PdpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706108447; x=1706713247; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=b5VQAz4u9LpTS0egjdrLjAZQ6Mkcn+U9ffxq2+hh/Yw=; b=LMLl4ljsJMghEvZyCqK0KTf2WtcOj1N5KbNEPuCiOXsaunv820ti/fHaNSUFFecbX1 XBy9Q8T/Os6s4Y6cHq68REBPxh+/fSwdfV6zjKHYQOa4aebqwHcX/2ZefckT6ebvh/rB 2H6c9OpldsMuzq+zz/5Zi7vUckLs98N14NVuJHn8Ge/YqRI2zeXFMb37uGBqUzg2CuPC /B+QxFxpJhYnTrcsi4G7rKD1GS5riByXMnZDhQrTdYUEly1mOaA2zaLMmXhFVbjWmk4H MTZMpw6lMPoGICBXX54U1eb3mB9/BP6+li8IHm+kCOUlwSLA7G6SBEVxUOyJAY0AbwOl 3H6A== X-Gm-Message-State: AOJu0Yz5DLlAsPbrV8crYWJqQzqgawfpFhsnyQI51lH+gjP7ZrS3qI/c HgThEgjSPojAvY6auT1ZDbjJR7HooMeCJn+jJfvYDi5GhNBUkJdR X-Google-Smtp-Source: AGHT+IGlBhtNxhbyhvUlQrO/E8CzU6C7z7WF+sOM7+loECOysSpSMeVMKP6MHQBGQAgVzMOX5tFlnQ== X-Received: by 2002:a67:f494:0:b0:46b:4f6:ae01 with SMTP id o20-20020a67f494000000b0046b04f6ae01mr631049vsn.26.1706108446956; Wed, 24 Jan 2024 07:00:46 -0800 (PST) Original-Received: from ahyatt-home.local ([190.83.214.104]) by smtp.gmail.com with ESMTPSA id l8-20020ab038c8000000b007d28492a9d6sm1851226uaw.39.2024.01.24.07.00.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 07:00:46 -0800 (PST) In-Reply-To: (T. V. Raman's message of "Tue, 23 Jan 2024 20:17:44 -0800") Received-SPF: pass client-ip=2607:f8b0:4864:20::e32; envelope-from=ahyatt@gmail.com; helo=mail-vs1-xe32.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:315312 Archived-At: On Tue, Jan 23, 2024 at 08:17 PM "T.V Raman" wrote: > All very good points, Kartik! > > Some related thoughts below: > > 1. I think we should for now treat prose-rewriting vs code-rewriting as > separate flows -- but that said, limit our types of "flows" > to 2. More might emerge over time, but it's too early. How do you see the code and prose rewriting requiring different UI or processing? > 2. Multi-step flows with LLMs are still early -- or feel early to me; I > think that for now, we should just have human-in-the-loop at each > step, but then leverage the power of Emacs to help the user stay > efficient in the human-in-the-loop step, start with simple things > like putting point and mark in the right place, populate Emacs > completions with the right choices etc. It can't be at every step, though. Maybe you wouldn't consider this a step, but in my next demo, one step is to get JSON from the LLM, which requires parsing out the JSON (which tends to be either the entire response, or often in a markdown block, or if none of the above, we retry a certain amount of times). But agreed that in general that we do want humans to be in control, especially when things get complicated.