Go has added Valgrind support (opens in new tab)

For the best crypto, you don’t want “within an epsilon”, you want “exactly the same number of CPU cycles”, because any difference can be measured with enough work.

Feeding random inputs to a crypto function is not guaranteed to exercise all the weird paths that an attacker providing intentionally malicious input could access. For example, a loop comparing against secret data in 32 bit chunks will take constant time 99.99999999% of the time, but is still a security hole because an attacker learns a lot from the one case where it returns faster. Crypto vulnerabilities often take the form of very specifically crafted inputs that exploit some mathematical property that's very unlikely from random data.

> But I wonder why its not trivial to throw a bunch of different inputs at your cyphering functions and measure that the execution times are all within an epsilon tolerance?

My guess is because the GC introduces pauses and therefor nondetermism in measuring the time anything takes.

Because "constant time" here means algorithms that are O(1), rather than O(n). This isn't about wall-clock execution time, it's about avoiding the number of operations performed being based on attributes of the input.

It would be another scenario to use as ammunition for "see you can't implement a language toolchain without using C", usually voiced by folks without background in compiler design, and understanding that most of the time that is a decision that spurs out of convenience and nothing else.

Assembly isn't that hard, those of us that grown around 8 bit home computers were writing Z80 and 6502 Assembly aged 10 - 12 years old, while having fun cracking games and setting the roots of Demoscene.

> it worked fine after spending an hour making a suppressions file.

So you are confirming the problem, but treating it as if ignoring it is the solution for all?

For me, a waaay outdated suppressions file for Qt + a rough understanding what syscalls and frameworks do is enough. If my app crashes in a network request and a byte sent to the X server (old example, I use Wayland now) is uninitialized, I know to ignore it.

Valgrind(-memcheck) is an extremely important tool in memory-unsafe languages.

I wrote the Lwan web server, which similarly to Go, has its own scheduler and makes use of stackful coroutines. I have spent quite a bit of time Valgrinding it after adding the necessary instrumentation to not make Valgrind freak out due to the stack pointer changing like crazy. Despite a lot of Valgrind's limitations due to the way it works, it has been instrumental to finding some subtle concurrency issues in the scheduler and vicinity.

From a quick glance, it seems that Go is now registering the stacks and emitting stack change commands on every goroutine context switch. This is most likely enough to make Valgrind happy with Go's scheduler.

IME Helgrind does an great job finding concurrency issues.

TSAN is not perfect but Go has had built-in TSAN support for a very long time (go build -race).

Also, strictly speaking all Go programs are multithreaded. The inability to spawn a single-threaded Go program is actually a huge issue in some system tools like container runtimes and requires truly awful hacks to work around. (Before you ask, GOMAXPROCS=1 doesn't work.)

TinyGo does, but it is also behind in language support.

Programs running under any Valgrind tool will be executed using a CPU emulator, making it quite a bit slower than, say, running the instrumented binaries as required by sanitizers; it's often an order of magnitude slower, but could be very well be close to two orders of magnitude slower in some cases. This also means that it just can't be attached to any running program, because, well, it's emulating a whole CPU to track everything it can.

(Valgrind using a CPU emulator allows for a lot of interesting things, such as also emulating cache behavior and whatnot; it may be slow and have other drawbacks -- it has to be updated every time the instruction set adds a new instruction for instance -- but it's able to do things that aren't usually possible otherwise precisely because it has a CPU emulator!)

I for one am still mystified how it's possible that a GC language can't expose the GC roots in a memory profile. I've lost so many hours of my life manually trying to figure out what might be keeping some objects live, information the GC figures out every single time it runs...

no, haven't heard of goref yet but will give it a shot!

usually I go with pprof, like basic stuff and it helps. I would NOT say memory leak is the biggest or most common issue I see, however as time goes and services become more complicated what I often see in the metrics is how RAM gets eaten and does not get freed as time goes, so the app eats more and more memory as time goes and only restart helps.

It's hard to call it memory leak in "original meaning of memory leak" but the memory does not get cleaned up because the choices I made and I want to understand how to make it better.

Thanks for the tool!

This was often a question asked in Java interviews as well.

In Java heap fragmentation is usually considered a separate issue but I understand go has a non-moving garbage collector so you can lose memory due to pathological allocations that overly fragment memory and require constantly allocating new pages. I could be wrong about this since I don't know a lot about go, but heap fragmentation can cause troubles for long running programs with certain types of memory allocation.

Beside that, applications can leak memory by stuffing things into a collection (map or list) and then not cleaning it up despite becoming "stale". The references are live from the perspective of the garbage collector but are dead from the application perspective. Weak references exist to solve this problem when you expose an API that stores something but won't be able to know when something goes out of scope. I wouldn't consider this to be common, but if you are building a framework or any kind of platform code you might need to reach for this at some point. Some crazy folks also intern every string they encounter "for performance reasons" and that can obviously lead to what less crazy folk would consider a memory leak. Other folk stick a cache around every client and might not tune the cache parameters leading to unnecessary memory pressure...

Golang has a feature that I love in general but that makes it very easy to keep unintended allocations around. If you have a struct with a simple int field, and you store that somewhere as an *int, the entire struct and anything it points to will be kept alive. This is super useful for short-lived pointers, and super dangerous for long-lived pointers.

Most other widely used GCed languages don’t allow the use of arbitrary interior pointers (though most GCs can actually handle them at the register level).

A GC only deallocates unreferenced memory, if you keep unused references, that's a leak the GC won't catch, as it has no way to know that you won't need it later.

It can happen when your variables have too long a lifespan, or when you have a cache where the entries are not properly evicted.

There’s ways, GC isn’t perfect.

A common one I see fairly often is opening a big file, creating a “new slice” on a subset of the file and then using the “new slice” and expecting the old large object to be dropped.

Except, the “new slice” is just a reference into the larger slice, so its never marked unused.

there are many ways:

    - you can create deadlocks
    - spawn goroutines while not making sure they have proper exit criteria
    - use slices of large objects in memory and pass them around (e.g. read files in a loop and pass only slice from whole buffer)
    - and so on

Not going to claim this is all sources, but Go makes it extremely easy to leak goroutines.

it's not hard. GC lets shit leak until it decided to clean it up...

do you think they will enable Valgrind if there's no leaks?

I sometimes dream of a GCed language with a non-escaping pointer type. However to make it really useful (i.e. let you put it inside other non-escaping structs) you need something on the scale of the Rust borrow checker, which means adding a lot of complexity.

yes, that's what I use, just wonder if there are alternatives. I am not sure how valgrind compares to it or the goref tool mentioned above, just asking around, does not hurt.

If I had to guess: because Rust engineers are like every other engineer that reads HN. They read stories, sometimes comment, and share from their own experience. Rust and Go were created roughly at the same time and are more directly comparable than e.g. Go and Ruby, so you don't see Ruby people writing about not having to use Valgrind. This means you'll see more comments from Rust devs than others in Go threads.

At least that's why I wrote that original comment.

You should see the C++ threads

There were developed around the same time so maybe that accounts for some of the comparisons, but at least in this case I think it matters that they are both relatively new languages with modern tooling.

Rust is, in my opinion, overrepresented by a vocal minority in HN. A vocal corpus that tend to be passive aggressive more often than other language communities, in my experience.

Which is sad because I like the language and find it useful. But a part of the community does a disservice with comments like your parent comment. It's often on the cusp of calling people who code in Go "stupid". But I digress.

Religious wars. C is Judaism. Go/Rust/Python/Ruby are all the different sects of Christianity. AI is Islam.

Oh, interesting. Thanks for sharing!

I guess there's also callgrind that may be useful for Gophers.

For sure, that's why I said it's possibly due to the ecosystem. We link against two C libraries other than libc: OpenSSL and librdkafka. Though they are both abstracted away with solid Rust bindings so for us, so as a consumer of these libs it hasn't been a problem (I guess it may be a problem for the people developing them).

I guess maybe the failure is not the addition of it (as it's useful for people writing the bindings), but rather how happy everyone on the thread is (which means it's more useful than it should be due to a failure with the ecosystem).

C is the language that benefits the most from tools like Valgrind. It's just so easy in C to write code with memory faults.

Memcheck (the main tool) has shortcomings (very slow, does not detect all kinds of errors). Its strongest point is that it does not need an instrumented build. That can be particularly important if you have issues in 3rd party libraries that you can't build. Its other strong point is that it checks for both addressability and initialisedness at the same time.

My favourite feature is using GDB with Valgrind+vgdb. That allows you to see what memory is addressable and/or initialised from within GDB.

If any macOS experts could help with this port it would be most welcome.

Apple have been making big changes that keep breaking things and Valgrind has not kept up. Louis Brunner has done an amazing job more or less single handedly managed to keep the basic flow working.