Re: Most used words in current buffer

[Bob Proulx <bob@proulx.com>]

Udyant Wig wrote:
> Bob Proulx wrote:
> > Hmm...  I think looking behind the abstract data type at how it might
> > or might not be implemented is a stretch to say the least.  The entire
> > reason it is an abstract data type is to hide those types of
> > implementation details.  :-)
> 
> Indeed.  The principle generalizes to abstract functionality as well,
> doesn't it?  E.g. qsort in the C library may or may not be an
> implementation of quick sort; or closer to the topic of the newsgroup,
> one ought not to care about the actual algorithm of the SORT function in
> Emacs.

Yes!  That is it exactly!

> > The naming of things usually says more about the person that named the
> > thing than the thing itself.  Associative arrays is a naming that
> > reflects the concepts involved in what it does.  This is the same as
> > when someone else names it a map table, or a dictionary.  Those are
> > all the same thing.  Just using different names because people took
> > different paths to get there.
> 
> Yes.  Just as, arguably, vectors are a special case of the general
> concept of arrays, though the terms are commonly used to name the same
> thing.

Agree completely.

> > For such things I generally prefer balanced tree structures because
> > work is amortized instead of lumped.  But the important point here is
> > that for every algorithm + data structure there is a trade-off of some
> > sort between one thing and another thing.
> 
> Hmm.  I had written a tree version of the word counter I had mentioned
> before.  I had stumbled upon the AVL tree package in Emacs and thought I
> might try using it.  This tree-based attempt turned out to be slower
> than my straightforward hashing solution.
> 
> I have no doubts this code could be written better by someone more
> experienced than I.

I don't know if the AVL package you used was implemented in elisp or
in C or otherwise.  And even though I am a long time user of emacs I
have never acquired the elisp skill to the same level as other
languages and therefore can't comment on that part.  But I know that
when people have implemented such data structures in Perl that the
result has never been as fast and efficient as in a native C version.
If so then that may easily account for performance differences.  And
also the native implementation of "hashes" in awk, perl, python, ruby
is quite optimized and very fast.  They have had years of eyes and
tweaking upon them.

> > I am in total agreement over using sed instead of head if you want to
> > do that.  Seeing 'sed 20q' should roll off the keyboard as print lines
> > until line 20 and then quit.  Very simple and to the point.  There is
> > definitely no need for a separate head command.  Other than for
> > symmetry with tail which is not as simple in sed.
> 
> I see that.  You could implement head on top of sed if you wanted to.  I
> myself have been using head for long enough for its stated purpose that
> grasping a sed equivalent was not immediately obvious.

Writing clear code that can be understood immediately by the entire
range of programmer skill is important in my not so humble opinion.
One shouldn't need to be a master experienced programmer to understand
what has been written.  Therefore I usually use 'head' specifically
for the clarity of it to everyone.  Seeing "head -n40" is not going to
confuse anyone.  Therefore I usually use it instead of "sed 40q" even
though I could remove 'head' entirely from my system if I were to
uniformly implement one in terms of the other.  Clarity is more
important.

And before someone mentions performance let me remind that we are
talking shell scripts.  In a shell script clarity is more important
than performance.  Always.  If the resulting shell script results in a
performance problem than choosing a better algorithm will almost
certainly be the better solution.  And if not than then choosing a
different language more efficient at the task is next.

I do expect some skill to be learned with 'awk' however.  It is so
very useful that seeing "awk '{print$1}' should not be that confusing
that it is printing the first field column.  Or that '{print$NF}' is
a common idiom for printing the last field.  (NF is the Number of
Fields in the line that was split by whitespace.  $NF is therefore the
last field.  If NF is 5 then $NF is saying $5 and therefore always the
last field of the line.)  A little bit of awk learning pays back a
large return on the investment.

> These things do take time to gain currency, don't they?  Under Linux,
> for example, the ip set of commands has been named the successor to
> ifconfig, and it too is taking time to diffuse into general knowledge.

Yes.  And 'ip' is an excellent example!  Even I have converted to
using ip and the iproute2 family instead of ifconfig.

One thing to note about the iproute2 family is that it is reasonably
well written.  We are not forced to use it.  Instead we are attracted
to using it in order to get access to the entire set of new networking
features available only through them.  It is a carrot not a stick.

> (And, although there have been a number of revisions of Standard C since
> 1989/1990, a lot of projects still write to that now legacy standard.
> But there may be other issues to consider here.)

Another good example. :-)

Bob