But the team wants move to a compacting GC. Fine, add pin/unpin idioms. This ground has been well covered by other languages. Don't completely destroy a very productive mechanism because in some oddball cases it might not work.
I think there's another design doc being worked on that will document the situation, and possibly have proposals to make sharing memory between Go and C more convenient. There's are many projects that make heavy use of this, where copying is slow or infeasible, and malloc is inconvenient.
Where is this stated? Where is it said that it's technically incorrect?
Just to be clear, I'm not passing it and then running a thread in C that just runs with the pointer to an object that no longer exists in Go. Instead I'm calling a C function synchronously -- there is no possible way the element is going to be collected during that run. It could be compacted (though Go up until now has explicitly had a non-compacting GC, so not a current risk), of course, which is what brought up this issue, but that's the reason such interop languages have pinning (flagging that something shouldn't be moved).
It wasn't by chance -- it was by design, and it was purposefully understanding the lifetime of objects.
var ptrs = make(map[uintptr]unsafe.Pointer)
var p unsafe.Pointer = ...
id := uintptr(p)
ptrs[id] = p
C.Fn(id)
//export GoFn
func GoFn(id uintptr) {
p := ptrs[id]
// ... use p
}
All of this is requires being very careful, but the hard part is the notion of holding references to memory outside the realm of the GC. I don't believe a runtime-assisted pinning API can do any better than what you can do yourself with a map.
[1] https://godoc.org/code.google.com/p/go.mobile/cmd/gobind
This is the type of issues that nicely lead to security bugs, in the hands of the typical average enterprise developer.
Having a pin/unpin idiom might be good enough, though.
It's also the type of functionality that enables the efficient use of high performance C libraries and existing code (I use it to great effect with the Intel compiler for extremely high performance SIMD code).
The "someone might not use it right, therefore it's anathema" notion is misguided, and is orthogonal with the philosophy of Go. Even C# and Java -- the pinnacle of "enterprise" languages -- understand and respect the notion of pointer sharing, and early on built in mechanisms to support it. Go is a league above, though, given that it follows C struct layout rules.
But I see where the person proposing banning it got their notion from -- it's a classic hubris of "if it isn't important to me, it isn't important to anyone". A compacting GC has benefits, but the lazy notion of wholesale blocking broad and powerful uses because one doesn't personally benefit from them is how languages die.
With InfluxDB we want to give users the ability to define custom functions and it looks like this could be used to let people do that in Go or any language really. Whereas right now we're limited to doing something like using LuaJIT.
Of cause options are nice, I just worry that in the end I don't get to choose.
On topic: I'm really excited to see this, even though it's made clear this is just a plan and not a promise it's still great news for things like Android development.
Yes, please! We have a few potential Go projects which are blocked by the inability to separate Go code into plugins.
I have toyed with the idea of forking child processes and implementing plugins via pipe I/O, but that's a very unfriendly API interface, and I don't feel very inspired. (Go isn't that great at POSIX stuff like forking, either.)
That means the core project cannot list its plugins as dependencies. You will instead want to be able to load plugins from a configuration file.
(The problem is actually equally awkward in languages that have package systems, such as Node and Ruby. In Ruby, for example, a program can only see gems declared in the Gemfile; same problem with npm-shrinkwrap.json.)
I suppose you could set up a special build system where plugins are expected to be installed into a certain folder so that they're statically built in, with some kind of hook so that they're registered into the core. That's the Nginx approach. I'm not a big fan, especially since this method of building things need to be reinvented for each app that needs a plugin system.
You could, of course, invert the relationship and make this system a library: To set it up with your plugins, you write a small program that runs the system's main entrypoint with a list of plugins. But that just makes it harder to configure, build, use and deploy.
I suppose the namespace issue can be mitigated by a strict naming convention (something like "[application_name].[plugin_name]"), but the others are kind of a bummer.
I don't have a whole lot of positive things to say about the Python setuptools API (perhaps better called "lack-of-API internal interface"), but at least it's possible to dynamically load python plugins from arbitrary locations in the filesystem and programmatically discover their contents.
Basically the plugin API is isomorphic to dlopen/dlclose. Everything else is built on top of that.
That combined with the Plugin struct (which hopefully would have something like an iterable Map of exported symbols) resolves the issues I was concerned about. Nice.
Most plugins would I think by necessity be application specific, so I'm not sure how much gain there in in some sort of plugin discovery. But the process of loading a plugin would also allow the plugin to call into your code to register handlers & data structures in the main application through init() functions.
From a security perspective, executing arbitrary code in your address space in order to discover the contents of a plugin sounds ill-advised.
To me, it looks like they're building the low level building blocks that allow a number of different higher level APIs to be built, which is pretty neat.