Show HN: Visual intuitive explanations of LLM concepts (LLM University)

I'm not proposing anything WRT the depth or scope of the material.

If the premise of the material is that phrases like "dot product" can be used freely or with minimal explanation then images like the "place the apple quiz" make even less sense. For that person, not much more needs to be said than "We want to represent words as vectors so we can do linear algebra with them. If the representations preserve structure we care about then here are some cool things that happen: (examples of good and bad embeddings)."

Then go deep, having given them an adequate advanced organizer.

I know this course is meant to be content marketing for your company, and I don't mean this in a derogatory sense--you're doing content marketing right by providing high quality information to an audience who could be interested in your services--but it's a bit odd that going to https://llm.university unceremoniously drops me in the middle of what looks to be your product's documentation. That's not wrong necessarily, but it all adds up to arriving at a site and having a feeling of "what am I looking at?".

Like the grandparent comment mentioned, the pitch is "visual, intuitive explanations", but I don't see that on the landing page. I'm looking for a way to get to the start of your content, but the top and left hand menus don't help and are, if anything, confusing until I realize that I'm now inside of a larger set of documentation unrelated to the course.

Below the fold we see a "Let's get started!", but the link I see, Structure of the Course" doesn't sound like getting started. It sounds like more front matter. From the nav menu I see that after that I still won't get to the content, but instead a page about the instructors. Do I really need to read blurbs of the instructors before I get to the meat of the course?

It just feels like too much wrapping paper and packaging to get to the good stuff--and it really does seem like good stuff! And I think the way that you've embedded this course into the rest of your documentation prevents you from presenting it in a structure that is more familiar and easy to navigate (e.g. an 'About' link at the top that talks about the instructors and Cohere).

It might be frustrating to put a lot of time and effort into high quality materials, only for people to not want to spend a few minutes looking around, but from the audience perspective, there's a sea of LLM-related content out there. I want to quickly determine if this is worth adding to my already-too-long list of LLM related bookmarks of things I want to read.

I understand that, of course.

Is your goal to make it feel like a typical university course or like something else?

If like a typical university course, start with a syllabus and a course description and all the logistics.

If like something else then the first 10 seconds of the experience should make people go "Oh, this is different."

What's happening in the first 1 second, 30 seconds, 1 minute, 10 minutes, etc. that are reflective of the rest of my experience? That will serve as an advanced organizer for what's to follow?

The very first graphic I see is labeled as a "quiz" and requires me to read a bunch of surrounding text to make sense of it.

That's the vibe: a promise of something visual and intuitive, first consummated by a long syllabus and a quiz.

The landing page should probably be more of a marketing page explaining why the course is worth checking out, with low information density and large visuals

start with a visual hors d'oeuvre

IDK, with only GP's comment and the image for context, and some earlier familiarity with the concept of embeddings, I'd put the apple... somewhere near the middle of the picture, nowhere near any of the lettered placement points. To me, the apple fits about equally to all the clusters, excepting bottom-left one, where I couldn't figure out the relationship in few seconds.

EDIT: in some sense, the whole idea and usefulness of embedding comes from it working like the inverse of this kind of "intelligence"/"logic" tests - tests that ask you to which, out of several groups, a new symbol belongs. Usually there are couple competing answers, but the test has you guess the one that's the Right One. Embedding is about subverting this - it's about telling the test giver, "you know what, it actually belongs to all of them", and adding enough dimensions to the problem space that you can have all the groups be far away, from each other, and the new thing close to all of them - to each along a different dimension.

Thanks, Jay!

I wasn't aware of that BERT explainability paper - will be reading it, and watching your video.

Are there any more recent Transformer Explainability papers that you would recommend - maybe ones that build on this and look at what's going on in later layers?

Sure - but it's still the interesting part!

I'm sure some of key players know at least a little, but they don't seem inclined to share. In his Lex Fridman interview Sam Altam said something along the lines of "a LOT of knowledge went into designing GPT-4", and there's a time gap between GPT-3 (2020) and GPT-4 (2022) where it seems they spent a lot of time probably trying to understand it, among other things.

It seems the way values are looked up via query/key and added must constrain representations quite a bit, and comparing internal activations for closely related types of input might be one way to start to understand what's going on.

A high level understanding of what the model has learnt may be the last thing to fall, but understanding the internal representations would go a long way towards that.

Are you saying no one really knows how these things work? I am very curious about if you can “peer into the weights”. I have seen simple examples of that with image recognition but only for early layers.

Thanks, am definitely trying to squeeze them out as I’m able!