Re: LLM Experiments, Part 1: Corrections

[Juri Linkov <juri@linkov.net>]

> Question 3: How should we deal with context? The code that has the text
> corrector doesn't include surrounding context (the text before and after
> the text to rewrite), but it usually is helpful. How much context should we
> add? The llm package does know about model token limits, but more tokens
> add more cost in terms of actual money (per/token billing for services, or
> just the CPU energy costs for local models). Having it be customizable
> makes sense to some extent, but users are not expected to have a good sense
> of how much context to include. My guess is that we should just have
> a small amount of context that won't be a problem for most models. But
> there's other questions as well when you think about context generally: How
> would context work in different modes? What about when context may spread
> in multiple files? It's a problem that I don't have any good insight into
> yet.

I suppose you are already familiar with different methods
of obtaining the relevant context used by copilot:
https://github.blog/2023-05-17-how-github-copilot-is-getting-better-at-understanding-your-code/
https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html
etc. where tabs correspond to Emacs buffers, so in Emacs the context
could be collected from existing buffers like e.g. dabbrev-expand does.