remove duplicates lines in a buffer

remove duplicates lines in a buffer

[GH]

A program give me huge logs with duplicated lines
I _need_ to simplify this huge file removing duplicated lines.

any tool to do it? maybe a func?

I think in a function that do a list with lines maybe using

  (push (NEWELT PLACE)) ...

in a while cycle

but dont push a line if it exist in the list, maybe checking it with

    (dolist ...)

then to write the list in a new buffer

Re: remove duplicates lines in a buffer

[Emanuel Berg]

GH wrote:

> A program give me huge logs with duplicated lines I _need_
> to simplify this huge file removing duplicated lines.

Linux shell,

  $ sort $log | uniq

see sort(1) and uniq(1).

Emacs:

  `cl-remove-duplicates'

-- 
underground experts united
https://dataswamp.org/~incal

Re: remove duplicates lines in a buffer

[Eli Zaretskii]

> From: GH <project@gnuhacker.org>
> Date: Thu, 22 Dec 2022 19:59:45 +0100
> 
> 
> A program give me huge logs with duplicated lines
> I _need_ to simplify this huge file removing duplicated lines.
> 
> any tool to do it? maybe a func?

There's already a command delete-duplicate-lines in Emacs.

Re: remove duplicates lines in a buffer

[Tim Landscheidt]

Eli Zaretskii <eliz@gnu.org> wrote:

>> A program give me huge logs with duplicated lines
>> I _need_ to simplify this huge file removing duplicated lines.

>> any tool to do it? maybe a func?

> There's already a command delete-duplicate-lines in Emacs.

Of course there is.  For more than eight years!  And until
today I always resorted to query-replace-regexp.

Thanks,
Tim