Calling it a 'release' is an overstatement. The docs state that it is a work in progress. It's also quite buggy (it's easy to get a segmentation fault). The version I saw in January was about 1/3 the size of this version, and also buggy. I hope that the final version of this code is less buggy and more usable.
If you want to learn the K language, don't use this version. Any of the other open source K projects are better than this (more complete, less buggy, better documented). This project is good if you want to learn more about the Arthur Whitney C coding style, because it is so small. Other projects written in this style (some open source K implementations, the J language) are huge by comparison.
One thing that puzzles me, about array languages, is that despite several open source implementations already existing, like J, its surprisingly difficult to find them packaged in Linux repositories. For example, you can't just "apt install J", or "apt install gnu-apl" on Ubuntu. In J case, it seems the default is just compiling it from source. Is there something tricky about packaging them?
The closest to a repository-friendly array language I could find was the klongpy implementation of klong[0], that is pip installable.
I also wonder if the commercial version has anti-benchmark clauses like some database vendors. I've always seen claims that K is much faster than anything else out there, but I've never seen an actual independent benchmark with numbers.
Edit: according to https://mlochbaum.github.io/BQN/implementation/kclaims.html, commercial licenses do indeed come with anti-benchmark clauses, which makes it very hard to take the one in this post at face value.
More recently, we have been working on GFQL with users at places like banks (graph dataframe query language), where we translate down to tools like pandas & cudf. A big "aha" is that columnar operations are great -- not far from what array languages focus on -- and having a static/dynamic query planner so optimizations around that helps once you hit memory limits. Eg, dask has dynamic DFS reuse of partitions as part of its work stealing. More SQL-y tools like Spark may make plans like that ahead of time. In contrast, that lands more on the user if they stick with pandas or k, eg, manual tiling.
Kdb/q is like minimalist footwear. But you can run longer and faster with it on. There's a tipping point where you just "get it". It's a fantastic language and platform.
The problem is very few people will pay 100k/month for shakti. I'm not saying people won't pay and it won't be a good business. But if you want widespread adoption you need to create and an ecosystem. Open sourcing it is a start. Creating libraries and packages comes after. The mongodb model is the right approach IMO
Quick for evaluating some idea you just had if you are a quant? Yes absolutely!
So imagine you have a massive dataset, and an idea.
For testing out your idea you want that data to be in an “online analytical processing” (OLAP) kind of database. These typically store the data by column not row and other tricks to speed up crunching through reads, trading off write performance etc.
There are several big tech choices you could make. Mainstream king is SQL.
Something that was trendy a few years ago in the nosql revolution was to write some scala at the repl.
It is these that K is competing with, and being faster than.
Yes, a very specific implementation will be faster than a generic system which includes network delays and ensures you can handle things larger than your memory in a multi-client system. But the result is meaningless - perl or awk will also be faster here.
If you need a database system, you're not going to replace it with K, if you need super fast in-memory calculations, you're not going to use a database system. Apples and oranges.
I doubt they'd give them out to a random individual or small startup, but maybe still possible for a serious potential customer.
#define _(e) ({e;})
//!< isolate expression e in its own lexical scope and clamp it with ;
//!< note the outer parens, which is a very powerful c trick: they turn _(e) into a so called
//!< r-value, which basically means we can do x=_(e) for as long as e evaluates to or returns
//!< at least anything at all, i.e. not void. this macro is fundamental to k/simple implementation.
edit:
-Wpedantic on gcc will tell me ISO C doesn't like the construct but it still compiles it happily.
Clang offers -Wgnu-statement-expression-from-macro-expansion
So it looks likely that this is the GNU statement expression extension after all and not a part of C. Shame.
The Linux kernel makes quite a bit of use of them as far as I'm aware.
the free(MIT license) version is shakti.com/k/k.zip
the supported version(supported and 10 to 100 times faster)
is $100K per month(minimum)
Good joke, though.
For example, I have an old friend from a major investment bank who used to work on an internal (proprietary) pub/sub system but who, these days, works on integrations between that system and Apache Kafka.
Wouldn't work the same way if your core customer base is elementary school teachers.
> K is a proprietary array processing programming language developed by Arthur Whitney and commercialized by Kx Systems. The language serves as the foundation for kdb+, an in-memory, column-based database, and other related financial products. The language, originally developed in 1993, is a variant of APL and contains elements of Scheme. Advocates of the language emphasize its speed, facility in handling arrays, and expressive syntax.
There was also a great thread on HN about it as well[2].
And many programming languages do, for example C# and Racket. I have a feeling it isn't very hard to implement, since that's what you typically do if you need complex numbers in Java.
It’s deliberate and powerful.
Here is a recent one: https://news.ycombinator.com/item?id=39026551
There was an epic post some years ago but couldn’t find it now from my phone.
If you'd like an antidote, have a read of Gerald Jay Sussman's books, where you'll see profound concepts from maths and physics captured in succinct and expressive (as opposed to merely terse) code, accompanied by eloquent explanations devoid of boasts or name dropping and provided free of charge online. That will change the way you think about computing too, but it will be a more pleasant experience.
The first one I found was "Structure and Interpretation of Computer Programs"
g(_M,W-=1<<f;xx=M[f];M[f]=x)Ui(M_,Ux=M[i];x?(W+=1<<i,M[i]=xx,x):30>i?_M(i,M_(i+1))+(2*n0<<i):OO())f(_r,pt?x:65535&rx?(--rx,x):e(if(!Tx)n(_r(xU))_M(mx,x-n0);x))f(r_,pt?x:65535&++rx?x:OO())
It's writing C in array-language style rather than intentional obfuscation.
I can totally believe that he didn't intentionally obfuscate it, but its incomprehensibility made it harder for other people to make a knockoff and thats why it survived and became successful.
Bonus: Go visit and do "View Source" on that website. Even HTML has fragrance of K.
I read a couple of other threads and some people try to claim less code = fewer bugs, but that's pretty clearly nonsense otherwise minifiers would magically fix bugs.
As for why people actually use this (it seems like they really do), my guess would be that it's used for write-only code, similar to regexes.
Like, using a regex you can't deny that you get a lot of power from not many keystrokes, and that's really awesome for stuff you're never going to read again, like searching code in your editor, one throwaway shell commands.
But they also tend to be completely unreadable and error prone and are best avoided in production code.
vector the journal of the British APL Association
[0] https://scholarworks.iu.edu/dspace/bitstreams/dcbd5240-8454-...
[1] https://www.bonfire.com/co-dfns-thesis-edition/
(a) come to think of it, theses are one and done
(b) thanks to Kragen for pointing out this 02019 work!
One of these days I need to (a) learn more about the Berber culture, and then (b) write an array language which exploits the ⵜⵉⴼⵉⵏⴰⵖ symbology.
https://en.wikipedia.org/wiki/Tifinagh#/media/File:Tifinagh_...
https://www.win.tue.nl/~aeb/natlang/berber/tifinagh/tifinagh...
https://www.edition-originale.com/media/h-3000-saint-exupery...
Concision is the handmaiden of clarity.
i don't think i'm going to get into the practice of doing it to that extreme, but i'll probably adopt the multiple-things-per-line part for side projects, and skip the shortening of keywords
with a wide monitor, you can have 3 files like this open at a time. you can fit a surprisingly large program on the screen at once, which is the benefit of coding like that
An ultimate expression of insider elitism and knowledge hoarding, which can be a self-interested asset to profitable, closed-source, specialized software.
For everyone else, code should present no surprises whenever possible with semantic expressiveness and reserve comments for explanation of surprises, design choices, and protocols.
There is a real tradeoff between making code friendly to the uninitiated and making code ergonomic for the expert. It's completely natural that non-initiates feel unwelcome when the code is written for domain experts. In your company's codebase, is it really best to optimize for onboarding newcomers over optimizing for the productivity of your engineers? Where along that spectrum maximizes your goals?
Are we building chairs or are we building chair-builders?
I don't know this AW guy, but to me that's a huge red flag and a sign that a programmer hasn't worked on anything substantial. Ie non-trivial stuff that's maintained by a team over time.
Being able to read the code is irrelevant, as the comments should tell you why the code is doing what it's doing.
For example, yeah I trivially can see the code is doing a retry loop trying to create a file with the same name.
That looks like a bug, if you can't create the file with that name, you change the name in the retry loop.
But the comment will tell me this is due to certain virus scanners doing dumb stuff, so we might have to try the same name a few times.
Sure, good code will have few comments as most of it should be self-documenting through good structure and names of classes and variables. But in non-trivial code there will always be places where it is not obvious why the code does what it does.
Maybe you want to check who he is?
This isn't the case. It's more likely a lack of business socialization combined with individual hyper-achievement. Reminds me of Ian Pratt in some ways.
An annoyance in tech, both startups and corporate, is technically-capable people but with outsized egos.
Perhaps a half way house is sql. The difference between ORM-style CRUD and a power user using window functions to make the data dance shows there is still art to be had in programming :)
That being said, I'm not convinced that the extremely minimal syntax is essential. I think it can be done another way ;)
If you're looking for a practical k implementation, I recommend ngn/k, and several other implementations are listed at https://k.miraheze.org/wiki/Running_K .
Oftentimes, the way something is presented and how the language is used, might be as important as the thing itself ;-)
If you want you can switch out his terse names in the .h and .c and see if that helps. I'm not so sure it does, but experience with array languages and a couple of decades with rather advanced C will. As in, experience is what matters rather than "IQ".
Is the terseness of the site mean to reproduce the terseness of the language? Is that the gimmick?
This isn‘t an advocacy piece directed at the general public. You‘re not his audience.
Fortunately, there is secondary commentary, like this thread, so we can get an idea what this is about.
E.g. it's tempting to dismiss Haskell as something invented by mathematicians more concerned with the elegance of their abstractions than actually getting things done, but Pandoc is so undeniably good and useful that you're forced to admit Haskell can be a good choice. What's the Pandoc of K?
//k(c)2024 arthur whitney(l)MITImho software size should reflect complexity of the problem domain. Not arbitrary metrics like say, the capabilities of a system executing it.
So "Hello World!" should weigh in at mere bytes. Not KBs or even MBs.
I mean faster at filtering data than a python script? Sure. Faster than a database or hand-rolled C code? Only if your benchmarks are misleading.
The new version requires ARM 64 or Intel 64 with AVX2. It requires clang-13 (clang-14 and later won't work). Gcc doesn't work.
With clang-14, I got build errors. First error: ./a.h:38:30: error: use of unknown builtin ‘__builtin_ia32_pminub256’ [-Wimplicit-function-declaration]
Seems to be related to this LLVM change which removed the above builtin: https://reviews.llvm.org/D117798
When I replaced __builtin_ia32_pminub256 with __builtin_elementwise_min and ditto for max, then it compiles and apparently works.
https://ktye.github.io/kdoc.htm
https://github.com/ktye/i/releases/download/latest/k.c
IIRC, some old UNIX versions had an APL interpeter in the userland. For me, a k interpreter could be the ultimate UNIX utility. But interoperability with pipes and other UNIX utilities is awkward to say the least, as is having to use other programming languages as duct tape.
Also when someone claims 1000x better Performance I want to know why. For example MySQL or PostgreSQL -> Clickhouse I can clearly attribute to column store, compression, vectorization, parallel execution on multiple CPU cores and machines...
EDIT: shakti.com/k/k.zip is now returning 404.
[1] https://en.wikipedia.org/wiki/Security_through_obscurity
The code does, as well. Either Mr. Whitney's brain is not wired like a regular homo sapiens sapiens, or the entire thing smells of "I am smarted than you and I don't need to lower myself to your level."
I do not buy for a single second that for Mr. Whitney debugging IOCCC-level obfuscated code is easier than plain C code. One writes "normal code" because one will have to read it later, and they don't want to spend ages doing so, unless they have to keep an air of superiority about their abilities to their peers.
I get that APL is obtuse and dense. But writing obtuse and dense C doesn't turn it into APL.
He was superb at finding errors at code review. As in looking through code someone else had written and pulling out the mistakes. Presumably everything looked completely trivial to him, regardless of how tangled the control flow had got.
Whitney may be similar.
Lagniappe: click on the circular red button underneath the comic, to the right of the orange "RANDOM" :-)