macOS and go language bindings

macOS and go language bindings

[Peter Lawrence]

[-- Attachment #1.1: Type: text/plain, Size: 386 bytes --]

Hi. I'm trying to build an email client (aerc) with support for notmuch
on macOS. aerc is written in go. I can't find an aerc package for brew 
which includes notmuch headers/library files.  Where can I get these
language bindings and how do I "install" them, or where do I place them
in the aerc source directory to get the notmuch library built and then
accessible to aerc?



[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: macOS and go language bindings

[Carl Worth]

On Sun, Nov 19 2023, Peter Lawrence wrote:
> Hi.

Hi Peter!

>     I'm trying to build an email client (aerc) with support for notmuch
> on macOS. aerc is written in go. I can't find an aerc package for brew
> which includes notmuch headers/library files.  Where can I get these
> language bindings and how do I "install" them, or where do I place them
> in the aerc source directory to get the notmuch library built and then
> accessible to aerc?

I don't have much direct experience with either macOS nor the Go
language bindings to notmuch, but I'll see how far I can get in
answering your questions.

The Go bindings for notmuch are in the contrib/go directory of the
source distribution which you can get from the latest source release


	https://notmuchmail.org/releases/notmuch-0.38.tar.xz

Or via git:

	git clone https://git.notmuchmail.org/git/notmuch

There's a brief README and a Makefile in the contrib/go directory, but
I'm not exactly clear on how things are supposed to work.

When I type "make" the first command it runs is "go install notmuch"
which fails as follows:

$ go install notmuch
go: 'go install' requires a version when current directory is not in a module
	Try 'go install notmuch@latest' to install the latest version

The advice from the error message doesn't help much:

$ go install notmuch@latest
go: notmuch@latest: malformed module path "notmuch": missing dot in first path element

I'm guessing someone who knows even a tiny bit about go modules might be
able to explain what's needed here, so we can fix the Makefile to
actually work?

-Carl

Re: macOS and go language bindings

[Peter Lawrence]

[-- Attachment #1.1: Type: text/plain, Size: 3604 bytes --]

On Tue Nov 21, 2023 at 2:19 PM EST, Carl Worth wrote:
> On Sun, Nov 19 2023, Peter Lawrence wrote:
> > Hi.
>
> Hi Peter!
>
> >     I'm trying to build an email client (aerc) with support for notmuch
> > on macOS. aerc is written in go. I can't find an aerc package for brew
> > which includes notmuch headers/library files.  Where can I get these
> > language bindings and how do I "install" them, or where do I place them
> > in the aerc source directory to get the notmuch library built and then
> > accessible to aerc?
>
> I don't have much direct experience with either macOS nor the Go
> language bindings to notmuch, but I'll see how far I can get in
> answering your questions.
>
> The Go bindings for notmuch are in the contrib/go directory of the
> source distribution which you can get from the latest source release
>
>
> 	https://notmuchmail.org/releases/notmuch-0.38.tar.xz
>
> Or via git:
>
> 	git clone https://git.notmuchmail.org/git/notmuch
>
> There's a brief README and a Makefile in the contrib/go directory, but
> I'm not exactly clear on how things are supposed to work.
>
> When I type "make" the first command it runs is "go install notmuch"
> which fails as follows:
>
> $ go install notmuch
> go: 'go install' requires a version when current directory is not in a module
> 	Try 'go install notmuch@latest' to install the latest version
>
> The advice from the error message doesn't help much:
>
> $ go install notmuch@latest
> go: notmuch@latest: malformed module path "notmuch": missing dot in first path element
>
> I'm guessing someone who knows even a tiny bit about go modules might be
> able to explain what's needed here, so we can fix the Makefile to
> actually work?
>
> -Carl


Thanks for the feedback, Carl!

I have a feeling now that this isn't just a macOS issue, but possibly a
broader issue that would effect anyone using a newer version of Go. I
followed your steps as well and got the same results. It seems that Go
changed some things around release 1.13 such that there is now a
"module" mode for Go instead of a GOPATH mode. Newer versions of Go
default to using the module mode rather than GOPATH mode. I found this 
blurb about Go environment variables: 

GO111MODULE
	Controls whether the go command runs in module-aware mode or GOPATH mode.
	May be "off", "on", or "auto".
	See https://golang.org/ref/mod#mod-commands.

So, I set my GO111MODULE to off with this command:
    go env -w GO111MODULE="off"

This helped. Then, I had to strip out some commented out lines in the
notmuch.go file under src/

Now when I compile with "make", I get some 75 lines of errors, similar
to the following:

src/notmuch/notmuch.go:89:8: could not determine kind of name for C.free
src/notmuch/notmuch.go:1047:8: could not determine kind of name for C.notmuch_bool_t
src/notmuch/notmuch.go:251:15: could not determine kind of name for C.notmuch_database_add_message
src/notmuch/notmuch.go:96:15: could not determine kind of name for C.notmuch_database_create
src/notmuch/notmuch.go:140:16: could not determine kind of name for C.notmuch_database_destroy
src/notmuch/notmuch.go:315:15: could not determine kind of name for C.notmuch_database_find_message

I am guessing that the go compiler (mine is at 1.20.8) doesn't like the
source in notmuch.go. This may have been written a while ago?  I'm
definitely over my head in the Go world at this point.

I wonder if anyone proficient with Go might know how to get the source
to build against a Go 1.20+ version?


-- 
Peter Lawrence


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: macOS and go language bindings

[Carl Worth]

[-- Attachment #1.1: Type: text/plain, Size: 1085 bytes --]

On Tue, Nov 21 2023, Peter Lawrence wrote:
> Thanks for the feedback, Carl!

No problem. I'm glad you were able to push even a little past what I was
able to do.

> I have a feeling now that this isn't just a macOS issue, but possibly a
> broader issue that would effect anyone using a newer version of Go.

Quite likely, yes!

> I am guessing that the go compiler (mine is at 1.20.8) doesn't like the
> source in notmuch.go. This may have been written a while ago?

Yes, definitely. The last non-trivial commit to contrib/go was in May
2015 (!).

> I'm definitely over my head in the Go world at this point.

That's OK. Let's look at the bright side. You also might be the most
experienced one here at the moment.

> I wonder if anyone proficient with Go might know how to get the source
> to build against a Go 1.20+ version?

It might be useful at this point to seek some porting advice from a
go-specific channel of one form or another.

If you're able to make some progress, we'll obviously gladly accept
patches that you develop to improve the Go binding within notmuch!

-Carl


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: macOS and go language bindings

[Reto]

Hi both,
I realize I am a bit late to the party, nevertheless here's some information.

> I'm trying to build an email client (aerc) with support for notmuch
> on macOS. aerc is written in go. I can't find an aerc package for brew
> which includes notmuch headers/library files.  Where can I get these
> language bindings and how do I "install" them, or where do I place them
> in the aerc source directory to get the notmuch library built and then
> accessible to aerc?

You are confusing things here.
Aerc wrote their own go notmuch bindings which uses the C library directly.
Stuff in contrib/ doesn't matter whatsoever.

So you don't install the "language bindings", it's part of the aerc source
already.

All you need to make sure is that the C compiler go invokes finds the dependencies.
Meaning you need libnotmuch installed or build from source.

> GO111MODULE
> 	Controls whether the go command runs in module-aware mode or GOPATH mode.
> 	May be "off", "on", or "auto".
> 	See https://golang.org/ref/mod#mod-commands.
> 
> So, I set my GO111MODULE to off with this command:
>     go env -w GO111MODULE="off"

Do not mess with that, aerc is a proper module aware source.
`Go build` will fetch the go dependencies it needs automatically.

> I wonder if anyone proficient with Go might know how to get the source
> to build against a Go 1.20+ version?

If by source you mean aerc, make will do (within their source tree).
However, you need to make sure that your C compiler is setup properly.

Go will execute a C compiler / linker as usual.
In your case, you basically need to setup 2 things:

	1) notmuch.h in some include dir, normally /usr/include/notmuch.h or some such,
	which is needed by the compiler.

	2) libnotmuch.so, which is normally located at /usr/lib/libnotmuch.so or some such,
	which is needed by the linker.

I have no clue where brew stores those things, shouldn't be too hard to figure out though.
Assuming that's not in some standard dir your C compiler / linker already look (else the build would work)
you can override that by setting some env vars.

CGO_CFLAGS: set that to "-I/usr/include" where /usr/include is the folder where notmuch.h resides
CGO_LDFLAGS: set that tho "-L/usr/lib" where /usr/lib is the folder where libnotmuch.so resides.

That will make the C side of things do what you expect.
The rest is handled by go.

Cheers,
Reto

Re: macOS and go language bindings

[David Bremner]

Reto <reto@labrat.space> writes:

[thanks for the explanation of how aerc uses notmuch]

>
> Go will execute a C compiler / linker as usual.
> In your case, you basically need to setup 2 things:
>
> 	1) notmuch.h in some include dir, normally /usr/include/notmuch.h or some such,
> 	which is needed by the compiler.
>
> 	2) libnotmuch.so, which is normally located at /usr/lib/libnotmuch.so or some such,
> 	which is needed by the linker.
>
> I have no clue where brew stores those things, shouldn't be too hard to figure out though.

macOS has a different naming scheme for shared libraries (and maybe they
are not ELF?). Notmuch should install libnotmuch.NN.dylib (where NN is
currently 5) and libnotmuch.dylib somewhere (given ./configure
--prefix=somewhere)

I hope this helps, even if it is second hand information.

d