[ANN] Guile-DSV 0.7.0 released

[ANN] Guile-DSV 0.7.0 released

[Artyom V. Poptsov]

[-- Attachment #1.1: Type: text/plain, Size: 640 bytes --]

Hello Guilers,

I'm pleased to announce Guile-DSV 0.7.0:
  https://github.com/artyom-poptsov/guile-dsv/releases/tag/v0.7.0

This release replaces the old finite-state machine (FSM) implementation
with the new one that uses Guile State Machine Compiler (Guile-SMC) [1].
The new FSM code is generated at build-time from PlantUML description
directly.

See the full list of user-visible changes below.

As the result of re-writting FSMs of Guile-DSV with PlantUML/Guile-SMC,
Guile-DSV gained some speed improvements.  You can find an attached
LibreOffice Calc document with some benchmarking using around 5 million
lines of data (source: [2])

[-- Attachment #1.2: guile-dsv-fsm-benchmark-2023-08-14.ods --]
[-- Type: application/vnd.oasis.opendocument.spreadsheet, Size: 29577 bytes --]

[-- Attachment #1.3: Type: text/plain, Size: 2468 bytes --]


More detailed information about the data (lines/words/characters):
--8<---------------cut here---------------start------------->8---
$ $ wc datasets/FeedGrains.csv
  498929  6010097 93054624 datasets/FeedGrains.csv
--8<---------------cut here---------------end--------------->8---

I took five "dsv" runs using RFC 4180 input format with "--summary"
option on Guile-DSV 0.6.0 and Guile-DSV 0.7.0 with the same options (and
disabled logging) and in some cases the new parser implementation is 15
times faster that the old one.

As a bonus Guile-DSV FSMs are now much nicer and easier to maintain.
The downside is that now Guile-DSV depends on Guile-SMC.


* What is Guile-DSV?

Guile-DSV is a library that allows to parse Delimiter-Separated Values
format (DSV).  It supports two flavors of DSV: Unix-style[3] and RFC4180[4].

Also Guile-DSV is shipped with a program named "dsv" that allows to read
and process DSV format (including delimiter change and conversion from
one standard to another, formatting tables etc.)  See README file for
usage examples.


* List of user visible changes

Here's an excerpt from the NEWS file:

--8<---------------cut here---------------start------------->8---
* Changes in version 0.7.0 (2023-08-15)
** Guile-DSV now uses Guile State Machine Compiler (Guile-SMC)
Guile-DSV now uses Guile State Machine Compiler (Guile-SMC) for the state
machine code generation from the PlantUML description:
  https://github.com/artyom-poptsov/guile-smc

** =dsv->scm= and =dsv-string->scm= now accept additional parameters
Now =dsv->scm= and =dsv-string->scm= procedures accept additional parameters:
- =#:debug-mode?= -- Enable/disable the debug mode.
- =#:log-driver= -- Set the logging driver.
- =#:log-opt= -- Set the logging options.
** =dsv= tool now accepts =--log-driver= and =--log-opt=
** Update Texinfo documentation
--8<---------------cut here---------------end--------------->8---


Thanks and happy hacking!

- Artyom

1. https://github.com/artyom-poptsov/guile-smc
2. https://www.ers.usda.gov/data-products/feed-grains-database/feed-grains-yearbook-tables/
3. http://www.catb.org/~esr/writings/taoup/html/ch05s02.html#id2901882
4. https://tools.ietf.org/html/rfc4180


-- 
Artyom "avp" Poptsov <poptsov.artyom@gmail.com>
Home page: https://memory-heap.org/~avp/
CADR Hackerspace co-founder: https://cadrspace.ru/
GPG: D0C2 EAC1 3310 822D 98DE  B57C E9C5 A2D9 0898 A02F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: [ANN] Guile-DSV 0.7.0 released

[Aleix Conchillo Flaqué]

Great improvements! Congratulations!

On Tue, Aug 15, 2023 at 2:35 AM Artyom V. Poptsov <poptsov.artyom@gmail.com>
wrote:

> Hello Guilers,
>
> I'm pleased to announce Guile-DSV 0.7.0:
>   https://github.com/artyom-poptsov/guile-dsv/releases/tag/v0.7.0
>
> This release replaces the old finite-state machine (FSM) implementation
> with the new one that uses Guile State Machine Compiler (Guile-SMC) [1].
> The new FSM code is generated at build-time from PlantUML description
> directly.
>
> See the full list of user-visible changes below.
>
> As the result of re-writting FSMs of Guile-DSV with PlantUML/Guile-SMC,
> Guile-DSV gained some speed improvements.  You can find an attached
> LibreOffice Calc document with some benchmarking using around 5 million
> lines of data (source: [2])
>
> More detailed information about the data (lines/words/characters):
> --8<---------------cut here---------------start------------->8---
> $ $ wc datasets/FeedGrains.csv
>   498929  6010097 93054624 datasets/FeedGrains.csv
> --8<---------------cut here---------------end--------------->8---
>
> I took five "dsv" runs using RFC 4180 input format with "--summary"
> option on Guile-DSV 0.6.0 and Guile-DSV 0.7.0 with the same options (and
> disabled logging) and in some cases the new parser implementation is 15
> times faster that the old one.
>
> As a bonus Guile-DSV FSMs are now much nicer and easier to maintain.
> The downside is that now Guile-DSV depends on Guile-SMC.
>
>
> * What is Guile-DSV?
>
> Guile-DSV is a library that allows to parse Delimiter-Separated Values
> format (DSV).  It supports two flavors of DSV: Unix-style[3] and
> RFC4180[4].
>
> Also Guile-DSV is shipped with a program named "dsv" that allows to read
> and process DSV format (including delimiter change and conversion from
> one standard to another, formatting tables etc.)  See README file for
> usage examples.
>
>
> * List of user visible changes
>
> Here's an excerpt from the NEWS file:
>
> --8<---------------cut here---------------start------------->8---
> * Changes in version 0.7.0 (2023-08-15)
> ** Guile-DSV now uses Guile State Machine Compiler (Guile-SMC)
> Guile-DSV now uses Guile State Machine Compiler (Guile-SMC) for the state
> machine code generation from the PlantUML description:
>   https://github.com/artyom-poptsov/guile-smc
>
> ** =dsv->scm= and =dsv-string->scm= now accept additional parameters
> Now =dsv->scm= and =dsv-string->scm= procedures accept additional
> parameters:
> - =#:debug-mode?= -- Enable/disable the debug mode.
> - =#:log-driver= -- Set the logging driver.
> - =#:log-opt= -- Set the logging options.
> ** =dsv= tool now accepts =--log-driver= and =--log-opt=
> ** Update Texinfo documentation
> --8<---------------cut here---------------end--------------->8---
>
>
> Thanks and happy hacking!
>
> - Artyom
>
> 1. https://github.com/artyom-poptsov/guile-smc
> 2.
> https://www.ers.usda.gov/data-products/feed-grains-database/feed-grains-yearbook-tables/
> 3. http://www.catb.org/~esr/writings/taoup/html/ch05s02.html#id2901882
> 4. https://tools.ietf.org/html/rfc4180
>
>
> --
> Artyom "avp" Poptsov <poptsov.artyom@gmail.com>
> Home page: https://memory-heap.org/~avp/
> CADR Hackerspace co-founder: https://cadrspace.ru/
> GPG: D0C2 EAC1 3310 822D 98DE  B57C E9C5 A2D9 0898 A02F
>