On 2016-09-14 20:58, Michael Heerdegen wrote:
> Clément Pit--Claudel <clement.pit@gmail.com> writes:
> 
>> Ok, so we agree on this part :) Note that it may be nonsense to load
>> the file into a buffer, too; holding the full file in a list is more
>> or less as bad as holding the full file in a buffer.  If the file is
>> large, that's very inefficient.
> 
> Ok.  But how do you refer to the file's contents from elisp, then?
> As a string?

I didn't have a string in mind.  More something like running `cat` in a subprocess maybe?

>>> Do you have an example where this would be useful for streams in
>>> Emacs, different from the "last n lines" one (see my comment about
>>> that below).
>>
>> Sure: if I have a stream of numbers, I may want to calculate the
>> average of the last n numbers.
> 
>> Or if I have a stream of non-overlapping buffer spans, I may want to
>> remove just the last one.
> 
> Sure, but with what I mean, the error (inefficiency) would already be in
> the "I have..." part.

I don't think so :) These sound like reasonable streams to me ^^
The second one in particular is one that I used recently: I wanted to enumerate all spans with constant text properties, and I didn't need to keep the whole list of spans in memory.  On the other hand, I didn't want to put the buffer segmentation in each function that iterated over the spans. Using a stream for that was quite convenient.

>> To continue on the `last n lines' example, I may be interested in only
>> the last n lines of output of a running process (think dmesg | tail).
> 
> As a string, right?  Or in which way do you refer to the process output
> via streams?  Can you please explain in short how you are using streams
> in your use case?  (Sorry, I'm more a mathematician than a programmer,
> so it's not your fault if I miss the point).

I don't have a concrete use case :) This thread was born just from my looking at the original patch.  I use streams a lot more in other languages than I do in Emacs Lisp, at least for now.  And I agree that there's no hurry in implementing this feature, either (we could add it later).

> And should we add `stream'
> method implementations for building streams from files and/or processes
> to stream.el if such stuff is useful?

I think this would be great.  The comment at the top of stream.el mentions it:

;; streams could be created from any sequential input data:
;; - sequences, making operation on them lazy
;; - a set of 2 forms (first and rest), making it easy to represent infinite sequences
;; - buffers (by character)
;; - buffers (by line)
;; - buffers (by page)
;; - IO streams
;; - orgmode table cells
;; - ...

> But I guess I'm beginning to understand: if you have a string (or
> something "similar") consisting of multiple lines (or records or
> whatever), and you are interested in the last n lines, in contrast to a
> buffer, the "go to the end and then n times backwards" approach might
> not even be possible, so there is no alternative to dissect the complete
> string into entities from the start until you hit the end (and throw
> away most of the stuff without accumulation) -- i.e. to the sliding
> window approach implemented by seq-subseq with negative indexes.

Yes, essentially.  In the string case, though, you could copy the string into a buffer and apply the trick that you mentioned.  But more generally streams are good at producing their contents lazily, so an implementation that requires forcing the entire stream and then searching from the end of the resulting list is non-optimal. Of course in some cases you can just skip the production of a stream entirely; but not in all cases.

In any case, I don't think this should hold the previous patch; we can always extend the functionality of seq-subseq later.

Clément.