Ask HN: How To: Internal Documentation?

Most companies don’t have a problem with too little documentation. Instead, the documentation is hard to find when needed.

I’m a documentarian and this is one of my favorite topics. My go to is creating a wiki for the team. The wiki is organized into hubs. There is a hub for each team. There is a hub for each topic such as security, new employee onboarding, etc. There might be a hub for each bigger project.

A hub contains information about people, systems, and processes. For example, there might be a section for external services. Each of those gets a hub too.

Everything is hyperlinked. Search sucks in most wiki-like tools. Hierarchies are inadequate to capture the complexities of reality. Each page or hub is often relevant in multiple places and should be linked. Links should also be included to external resources whenever they will save time. This can include linking to concrete Slack conversations, Jira tickets, and pull requests.

During onboarding each new team member is asked to improve the onboarding documentation. Anyone can change any documentation. It’s not possible to keep documentation relevant if each team member does not use it.

Here is an article that I wrote about this a while back https://koliber.com/articles/engineering-documentation-best-...

I am really passionate about this and help companies do documentation well in engineering teams.

One technique I've introduced but I don't have a long enough track record to know how effective is using a friction log when onboarding.

The core idea is to ask the person joining to keep a log of anytime something wasn't obvious, and they had to chase down an answer. IE they had to pair with someone to figure out a thing.

Then a month or so after they join, meet with them and go over the log. Doesn't mean the team fixes everything found, but it starts to build the picture of where someone joining has to spend most of their time, and can then have targeted fixes, process updates, etc. As long as the outcome isn't something vague like "improve docs" that no one will understand what the means or how to actually address it.

This also works for product UX discovery, for some types of products have every new joiner onboard as a customer would, and write down a friction log.

Docs should be as close to source code as possible, ideally in the same repo so they can be updated and reviewed in the same PR as the source changes themselves. Markdown works fine.

For docs that don’t relate to source, just have a repo for general docs. Not only is Confluence absolute crap, tech docs in it rot because it’s a separate system developers don’t want to use. Similar story for MediaWiki.

Realistically, my advice is. Document as little as possible. Build processes into your tools so it can be tested and repeated. Documentation that can get outdated, will be outdated and even worse be wrong. And, for example APIs can be organized in a way that documentation can be automated, so its never wrong or outdated, by choosing to use API contract first, like Protobuf or implementing via OpenAPI specifications. If you are implementing a specification of some kind, keep a copy of the specification version you have implemented checked in with the code.

Other things I have seen is f.ex. code that implements a business process, also can output itself as a Graphviz drawing, describing the parts of its own structure.

I strongly believe that proactively keep documentation up to date is not worth the effort. Here is what I recommend in my teams:

- There’s an onboarding guide that is usually only updated when new people onboard. If something is not right they raise it and their onboarding partner fixes it.

- When someone shares a technical document/RFC we add it to a central repository with a creation date on it. These are point in time and usually not updated but still useful for new hires.

- We sometimes do onboarding sessions to walk through the architecture, these are recorded and added to the central information repository.

Note that in the things I wrote above there’s a mix of up to date content and slightly outdated content, ultimately the code is the source of truth. It’s not worth spending time writing docs that no one is gonna read.

I've seen it done really well, really badly, and not at all!

Here's some things that I think help

- Tie documentation to code wherever possible: Automated API documentation is an obvious example, but even moves like including a 'developer guide' in the repo rather than a seperate file somewhere is a significant improvement.

- Manually written documentation will drift: This means you need capacity to maintain it. You're much better off manually documenting a few high level design choices and keeping it up to date, than trying to capture everything and it becoming inaccurate.

- Developer experience reduces the need for documentation: If you can automate stuff as much as possible, including offloading things to a type system etc to rule out mistakes, you'll reduce the amount new developers need to know to be effective, and therefore the ammount that has to be extensively documented. People are better learners when they do it slowly as they need than when they get given everything in one huge document to read.

Huawei. They have system analysts, who are basically living documentation. There's also some overlap with 'team lead' in western orgs and I'd say they have similar pay. These folks are idle by default; they're meant to be interrupted and their main focus is mentoring. They also handle a lot of the PRs, code quality, documentation.

It's also not hard to onboard someone into the same role. They took me on for a similar role and it took me about a few weeks to be familiar with nearly all the code I was responsible for.

Two places were notable, one because it was a practical sweatshop and required it... the other because of magical culture I haven't seen since.

The first place was doing support in web hosting. All the layers worked well to create feedback loops. Think phone techs, stats, and engineers all working to drop queues. Very high turn over but they had the absolute best training staff I've ever seen.

The second place wasn't in web hosting, but was similar in rigor. The culture was generally more relaxed. They relied less on trainers but still had very good ones. Their docs were more well-maintained. The work was reasonably well-distributed until they were acquired...

Most places don't have trainers (quality or not), feedback loops or docs. Opting to learn while chasing the dragon, ignoring the hoard.

You can do everything which is better than the nothing available. When I join a new company, I note *everything* during the onboarding for 2 or 3 weeks. How to get the application, how to build, who does what, what is the branching strategy...

I also note all the issues, the things that people hate, when you talk about a sensitive topic and people grin like a serial killer because it's been pissing them off for months of years.

And after 2 or 3 months, when you can work on your own, go back to your notes and put everything down in the internal wiki (even the GitLab wiki if that's all you have), and improve the onboarding guide because it exists /s No, actually, it's your job to create this doc because it never exists.

As for what can be automated by scripts, you can do that, but a wiki is always nice because it tells people where those tools are, and how to use them, even it has a short description like "to build the application, 1. install this from winget, 2. and run this script." Automation is fun, but a guide or FAQ is great for what can't be automated.

+1 for this

I agree with you, In the world where information is overflowing, and communication is async, documentation is a great way to share knowledge But the hard thing about documentation is updating it, since it keep getting outdated. What we did as a team is we discuss and agree documentation should be made as part of our OKR, add doc as part of our process, and where we should put it; in hope of making it easier to share knowledge for older codebase, new joiners, or other team.

We use inline doc for engineering, and all other usually we use gdrive for detailed source of truth and for the summary we use https://getoutline.com/

Small company, but putting READMEs in the root of each repository for specific documentation on that service and build/deploy process.

More READMEs live in a monorepo for things that aren't specific to one service (e.g. secret sharing, key rotation steps)

README.md/DEVELOPER.md in the project repo which details how to set up and get things running. That has the highest bang for buck in my experience, and is usually kept up to date as it's right next to the code and easy to update.

In my previous company, we created a ‘new hire onboarding’ checklist. A new hire got one senior team member assigned as his/her mentor. The new hire could add/change notes if something was outdated.

I work at an organization where we build mostly internally used tools to support different products/service lines. Here's what I've been advocating for and it works fairly well without requiring a lot of time to write documentation that is, often out of date. I find mileage varies as some people always just ask a senior engineer and never look at the documentation.

We have a high level wiki page about each application/micro service we support. Things like - resource locations - job schedules, purposes, inputs/outputs -FAQ's - known dependencies on other micro services, etc

We also have wiki pages for - onboarding - team norms and expectations (pr process, on-call duties, coding conventions, etc)

Inside each application's or micro service's repo the readme should have steps to - get the project running locally. - document feature flags - document conventions unique to that project - document how to trigger a release

Someone here said the code should be the source of truth and other documentation isn't needed. To me it's unclear what they mean by that exactly; I could get behind that if they are documenting via comments on functions/classes/etc. my org tends to not write much comments in the code so it's really hard to tell if the code does what it was intended to do or if there is a bug. What I love to see for "self documenting" code is tests. You write tests to either test for outcomes you expect or test outcomes that you don't expect, don't happen.

At the beginning of my career I always thought it would be amazing to have complete documentation and extensive onboarding guides. Over the years I've realized that every attempt at documentation failed and always gets out of sync too quickly. Especially as you'd have to update documentation you don't even know existed if you change some behavior that is also used / documented in other places by another team.

People sometimes don't agree and say "Chat isn't documentation", but being good at Slack search and sharing information on Slack with the right keywords (function names, column names etc.) is a super power. It always allows me to track down why a decision has been made or find the right people with context on a particular topic faster and more efficient than any documentation would ever do.

Metrics, GitHub search, Slack search, Source code comments are way more valuable than any documentation that will almost all the time be outdated in some parts. I believe that people who can work with that and know how to get information themselves vs. needing everything to be neatly summarized or collected for them in onboarding guides are the ones you want in your team.

For all the getting started and basic process documentation, you can keep it up to date by asking new hires to fix things, or update things based on their questions.

The biggest problem at larger firms is always dealing with other teams. It's inevitable that building and running your app will involve them, and their documentation may be lacking or non-existent. While you can't fix that, you can at least create a list of contacts.

"writing, reading, and maintaining documentation could get much smoother" - something we've been trying to do over at Komment. CI/CD approach to docs so they don't drift from code and a single portal where all the generated docs are available for anyone in the company. Check it out https://www.komment.ai

I think the problem with documentation is that most workers feel it's the lowest grunt task, training your replacements and not 'doing'. When in reality, you're helping onboard the next wave of people. And the best part is they don't necessarily need your time, just your docs.

I've on many occasions have been a part of detailing documents and creating them for the purpose of how-tos, I always felt a sense of accomplishment when being able to help new people into the team especially if it was something I struggled with.

A trusted colleague and I were talking recently about documentation and I was saying this much, they said it's the work that the lowest person at the company would do and I should stop viewing it as such a high level. I still enjoy documenting process but now I don't want to do it due to the fact that it is seen this way.

- readme at the root of each of application with keeping it up to date part of each relevant change. Something perhaps to consider in peer reviews.

- onboarding checklist with the new person now responsible for the checklist so when the next person needs it they can take them through it.

- mentoring with real contact between the new person and the mentor so they engage in shadowing and pairing.

- wikis are generally pretty poor, partly as they are external to the daily work, partly because nobody is made responsible. Could you delete it?

Of these I consider the README to be the most important. Imagine you are an open source application and you want to encourage others to improve your code. How would you do that? Add clear instructions on how to get it working on every platform? Include diagrams, made with simple, free tooling like Mermaid and drawio? Describe architectural decisions?

In Bookstack organized by department. Simple to update and put access controls on it.

As others have mentioned, this needs to be a living document and the person doing the task needs to update the docs where needed. No tool can replace this, but Bookstack at least makes it easy.

New hire checklist repo that the hire forks and works through.

guru is excellent too imo. docs can be set to expire which serves well if you actually want to update them regularly, or just give a marker for readers to be a bit cautious about the validity of the content.

The blank-slate of a new hire is a valuable resource which will (ideally) not remain for long.

They will essentially be testing your guides, and you should plan for existing developers to spend a little time fixing issues found by your impromptu tester.

This is a passion and pet peeve of mine. Here are my takeaways:

1) Technical documentation should be coupled with code, so devs will update it as services change (especially in a large or microservice oriented company), and that documentation is pulled into the general purpose system (read: non-tech, mgmt) This also vastly helps with versioning and finding old info (confluence versioning sucks)

2) Diagrams should be diagrams as code, for the same reasons, especially the versioning part. Can you tell me what your infra looked like exactly 3 iterations ago? This helps in problem solving when you've had long-tail undiscovered complications.

3) Make it part of product ownership, KPIs and OKRs. Product managers are the literal worst at this, and will try to push stuff to prod without it otherwise. Coming from the Ops end of things, I can't tell you how infuriating it is to get told "this product is going to prod next week" and when I look there are no ops docs!

4) Management really has to support these efforts, up to hiring someone just to manage documentation. Not just that, but the culture. One thing I've seen repeatedly, especially in high-speed/low-drag places full of PHDs and heavy engineers, is using no docs as a way to judge prod teams. I've made much progress explaining in detail why when an ops engineer is at the tailend of a shift and a sev1 happens, they don't have time or the right mental clarity to go read the code to understand your bullshit!

5) All that said, the biggest division is between the non-tech and tech teams. Do not forget to give non-techs good usability and visiblity into these systems, or management will eventually stop supporting it.

6) Some people will have a passion for documentation, despite being in teams that might not be related to it. Utilize them, and reward them!

Bonus) Include lack of documentation in your after-actions/post-mortems, to keep it relevant to uppers in a concrete way.

Internal documentation is nice, but really quite worthless for primarily three reasons:

1. At most one person is willing to create it

2. Nobody is willing to update it

3. Nobody is going to read it

I've documented how to onboard to my team and start doing dev work, and have received positive feedback from new team members as well as people who have been around a while and are starting to work in my area, saying that it was easy to follow and they went from 0 to coding in under 15 minutes.

So it can be done. The how is less about tools and process and more about simply making it a focus. I consider documentation of how to work with the codebase to be as much a core feature of the product as anything customer-facing. It is maintained, tested, and updated as deliberately as any other feature, and we log bugs on it and call out missing updates in PRs.

I use the README in our primary repo to be the starting point, and link to other places as needed. I include all the pre-reqs and instructions to get running, without assuming anything is already on their system or they know anything about our organization. Or even what OS they are running. And I include a goal-based list of FAQs - How to add a model, create a new API endpoint, change front-end routing, create new front-end components, tie everything together into a new feature, etc. How we branch, how we interact with PRs, team standards and norms, etc.

Exactly what your docs say will vary based on what type of work you are doing, but the key is to make documentation a primary feature of the product, and treat it as such when testing and reviewing code.

That's the challenge we at Komment (komment.ai) are taking on -- providing tools to maintain high-quality, up-to-date code documentation. We enable developers to spend more time developing, and less time documenting or figuring out what undocumented code does :-)

Concretely for onboarding, I create two documents. They’re both checklists. One is for the manager and the other for the new hire. It’s organized into sections.

Before the first day.

First day.

First week.

First month.

The manager is expected to keep the new hire accountable in gettin g through their checklist.