[PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Christian Köstlin]

[-- Attachment #1.1: Type: text/plain, Size: 51 bytes --]

Aforementioned unit-test.

Kind regards,
Christian

[-- Attachment #1.2: Type: text/html, Size: 108 bytes --]

[-- Attachment #2: 0002-testing-lisp-test-ob-C.el-Add-test-for-dub-support-i.patch --]
[-- Type: application/octet-stream, Size: 1658 bytes --]

From 58204f551ff8d4eea928f127e9707117f952d47b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Christian=20K=C3=B6stlin?= <christian.koestlin@gmail.com>
Date: Sat, 1 Oct 2022 23:55:02 +0200
Subject: [PATCH 2/2] testing/lisp/test-ob-C.el: Add test for dub support in
 ob-C.el

TINYPATCH
---
 testing/examples/ob-C-test.org | 8 ++++++++
 testing/lisp/test-ob-C.el      | 6 ++++++
 2 files changed, 14 insertions(+)

diff --git a/testing/examples/ob-C-test.org b/testing/examples/ob-C-test.org
index c7a96f665..aa7945d1a 100644
--- a/testing/examples/ob-C-test.org
+++ b/testing/examples/ob-C-test.org
@@ -174,3 +174,11 @@ std::cout << "\"line 1\"\n";
 std::cout << "\"line 2\"\n";
 std::cout << "\"line 3\"\n";
 #+end_src
+
+* Dub tests
+:PROPERTIES:
+:ID:       81E17302-BE76-4E2A-93CA-0A7E121CB7C2
+:END:
+#+begin_src D :dependencies '(mir-algorithm=3.16.2) :imports "mir.combinatorics" :results pp
+    foreach (p; ['a', 'b', 'c'].permutations) { p.write(", "); }
+#+end_src
diff --git a/testing/lisp/test-ob-C.el b/testing/lisp/test-ob-C.el
index b6dbed8e3..0c0f6c8d7 100644
--- a/testing/lisp/test-ob-C.el
+++ b/testing/lisp/test-ob-C.el
@@ -192,5 +192,11 @@
                                "\"line 1\"\n\"line 2\"\n\"line 3\"\n"
                                (org-babel-execute-src-block))))))
 
+(ert-deftest ob-D/dub-basics ()
+  (if (executable-find "dub") (org-test-at-id
+       "81E17302-BE76-4E2A-93CA-0A7E121CB7C2" (org-babel-next-src-block 1)
+                      (should (equal "abc, bac, cab, acb, bca, cba, "
+                                     (org-babel-execute-src-block))))))
+
 (provide 'test-ob-C)
 ;;; test-ob-C.el ends here
-- 
2.37.3

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Max Nikulin]

On 02/10/2022 05:05, Christian Köstlin wrote:
> Aforementioned unit-test.

> +(ert-deftest ob-D/dub-basics ()
> +  (if (executable-find "dub") (org-test-at-id
> +       "81E17302-BE76-4E2A-93CA-0A7E121CB7C2" (org-babel-next-src-block 1)
> +                      (should (equal "abc, bac, cab, acb, bca, cba, "
> +                                     (org-babel-execute-src-block))))))

I am unsure concerning general policy related to org-babel test, so the 
following is rather discussion than a direct request to change the test.

Is there a way to avoid dependency on remote resources? My concern is 
test results affected by network issues. In addition, more required 3rd 
party tools means less probability that a developer after modification 
of generic babel code noticed that the changes break D backend because 
missing tool on the development machine. I have no idea how much efforts 
is required to create a mock for isolated environment.

Another point is that missed dub binary is not reflected in test report. 
If `skip-unless' feature of ert is available in Emacs-26 then it should 
be used.

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> I am unsure concerning general policy related to org-babel test, so the 
> following is rather discussion than a direct request to change the test.
>
> Is there a way to avoid dependency on remote resources? My concern is 
> test results affected by network issues. In addition, more required 3rd 
> party tools means less probability that a developer after modification 
> of generic babel code noticed that the changes break D backend because 
> missing tool on the development machine. I have no idea how much efforts 
> is required to create a mock for isolated environment.

Tests must not rely on remote resources.
However, it does not mean that we cannot test remote resources in Org
tests. One simply needs to mock the url-retrieve or whatever function is
used to retrieve remove resource.

> Another point is that missed dub binary is not reflected in test report. 
> If `skip-unless' feature of ert is available in Emacs-26 then it should 
> be used.

The current approach to optional binaries is throwing
'missing-test-dependency error when the binary is not available.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Ihor Radchenko]

<Please use Reply-All button when replying. Otherwise, the discussion is
not recorded in the mailing list and other Org contributors cannot read it>

<I am CCing Bastien and Org mailing list back in this email>

Christian Köstlin <christian.koestlin@gmail.com> writes:

>> If the programming language is using its own package manager, is it any
>> different? Or do you refer to something else?
>>
> Most of the packages that a programming language like rust/crates,
> ruby/rubygems
> or dlang/dub-packages provide will probably never make it into the package
> management system of the operating system. Also the programming language
> package management usually has finer grained dependency management.
> But it's just a theoretical question as no other ob-* integration
> integrates a package
> manager at the momen.

If we do anything about such integration, we should not enable such
testing automatically. What we certainly do not want is unsuspecting
user running make test and getting some packages installed into his/her
computer.

However, the side effects requiring installing certain packages to the
OS might be shielded behind Makefile customization that will be set to
non-nil in CI tests. Of course, such customization should be documented
in testing/README.

>>> Throwing the 'missing-test-dependency has exactly same result, but also
>> >> allows making our test suite indicate such issues simply by altering
>> >> the 'missing-test-dependency catch code. That's why prefer
>> >> 'missing-test-dependency approach.
>> >>
>> > Nice ... I just tested the aforementioned `(ert--skip-unless
>> > (executable-find "dub")`
>> > which also works nicely. I could post another patch that updates the
>> whole
>> > test-ob-C.el with one of those strategies.
>> I am confused. Why do you need to use internal ert function?
>>
> I just tried what was proposed in one of the other emails ...

I do not blame you. Do not take it for granted that every Org
contributor, maintainer, or mailing list dweller is always right.

In this particular case, I disagree with Max and I tried to put
arguments why I disagree. If you think that I am wrong, you can refute
my arguments.

>> Note that we only have skip-unless feature in
>> ob-shell/remote-with-stdin-or-cmdline, which is probably just an
>> incident.
>>
>> Most of the test files use (signal 'missing-test-dependency ...)
>> For example, at the very top of test-ob-C.el:
>>
>> (unless (featurep 'ob-C)
>>   (signal 'missing-test-dependency "Support for C code blocks"))
>>
>> We even have a dedicated function to test if executable exists:
>> `org-test-for-executable':
>>
>> (org-test-for-executable "awk")
>> (unless (featurep 'ob-awk)
>>   (signal 'missing-test-dependency "Support for Awk code blocks"))
>>
> yes .. the difference is, that with org-test-for-executable i get a failing
> test (same for `signal 'missing-test-dependency`)
> with the ert function one gets a skipped test.

I guess you tried to throw 'missing-test-dependency _inside_ the test.
AFAIK, 'missing-test-dependency is caught when test-*.el file is being
loaded, not when the ERT tests are being executed. See `org-test-load'.

I think you can create a separate file like test-ob-D.el and run
(org-test-for-executable ...) at the top-level there.

> There is only one most important policy - if we recommend anything
>> publicly, it must be Free software. The detailed rules are covered by
>> https://www.gnu.org/prep/maintain/maintain.html
>
> yes ... still i am now lawyer enough to e.g. judge if gitlab/github is
> according
> to that.

We generally avoid GitHub because of some of their practices (see
https://www.fsf.org/search?SearchableText=github).

However, FSF policy is of a secondary importance here.

WRT CI tooling, we just prefer something Org maintainers do use.
Bastien, the chief maintainer, prefers Sourcehut.
Since he maintains the Org infrastructure, we should also prefer
Sourcehut, unless there are strong reasons to use other service. (Do not
hesitate to name those reasons, if you know any)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[tbanelwebmin]

[-- Attachment #1: Type: text/html, Size: 2295 bytes --]

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Ihor Radchenko]

tbanelwebmin <tbanelwebmin@free.fr> writes:

> For unit-testing Dlang, we could mock the «dub» command (compiler+packager). For instance, we could replace it
> by the «cat» Linux command, just while running the unit-test.
>
> But the «cat» command introduces a new dependency on Linux. Is there a portable equivalent for «cat»?

Are you talking about Shell commands? That's probably too dangerous. I
meant Elisp.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Unit-test for Please add support for dlangs packagemanager to ob-C.el

[Ihor Radchenko]

Christian Köstlin <christian.koestlin@gmail.com> writes:

> Subject: [PATCH 2/2] testing/lisp/test-ob-C.el: Add test for dub support in
>  ob-C.el

Canceled, due to no response.
See https://list.orgmode.org/orgmode/CAG741+bUT_Yt3PdLuVpMNH7vo-bb3n1iiynf2Cime1LW=214NQ@mail.gmail.com/

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>