I’m used to writing on the web where the scroll is unlimited and everything is hyperlink able and potentially interactive. Journal papers are limited by length and so was our assignment. I had to cut virtually all helpful explanation needed to reproduce my results which was deeply frustrating. We were implementing an algorithm based on another paper and it was hard because key details were omitted or assumptions not stated. After that exercise I have to think some of it was intentional to get it down to size.
I find most people aren’t good at technical communication and teaching others without a LOT of practice. Even then it requires feedback and iteration to make sure the ideas are communicated well. Forcing people to be more succinct and omit details makes the final product worse to consume. I don’t know how common such limitations are these days, but I do know that the average paper is still out of reach of the average programmer (where it would likely have the most benefit).
I have always thought this was a bit silly and that it creates really weird effects that also decrease readability. An interesting point is that reviewers are not required to read the appendix of works. So everything is required to be in the front matter. This is a bit silly when we do things like research graphics or do generative works and such. You want to include images and samples but then your space is eaten up. What if you want to discuss analysis on those images and explore some? You could easily do this on a blog but you're forced to throw this into the appendix. But then a reviewer can ask a question that's explained there and your work can still get rejected because it isn't in the front matter. Another weird incentive is that people end up padding works to fit page limits. This is because if you turn in a shorter paper reviewers will frequently reject your work the same way your boss might not think you're working if they don't see you at your desk.
We live in the 21st century and we still publish like it's the 15th. Computers gave us the ability to embed images, which is why there are so many more graphs and charts now, and it's not like more pages cost more. So just remove it. Some papers should be only a few pages and there's nothing wrong with that. Some papers should be far larger and there's nothing wrong with that. It's just weird to set these up considering they were likely created under other constraints but momentum continued and we back justify the continued decisions (there is something to be said about readability, but that can just be a reason to reject).
Side note: CS groups typically publish in conferences
It's also easier to find reviewers for short papers than for long ones.
Some the issues you mention are specific to CS conferences. Because there is only time for 1-2 rounds of reviews, the reviews focus more on accepting/rejecting the paper and less on clearing any misunderstandings before judging it. Conferences are are also more likely to have one-size-fits-all page limits, while journals often have several catagories of papers with different expectations of length.
It’s not like I have a crazy illustrious career or anything, but it can feel like kind of a blur, just a rollercoaster that led inexorably towards the present, which couldn’t be further from the truth; I would love to be able to reflect on my successes (and failures!) and see the small, concrete steps I took towards each.
Even without writing it out, I know the connections I have made and the mentors / coworkers / friends who have helped me deserve much more credit than any individual strokes of brilliance on my part! Another thing that’s very easy for me to forget, day-to-day.
What a pleasant coincidence - thanks for the contribution!
Once this is ready, people will just be able to publish their "papers" there. I guess they'll be called something else then. But this sort of struggle to publish a "paper" will no longer be necessary.
Thank you even more for publishing WebGazer and for following a "systems" approach in your research, when most people produce only papers. It's systems as research artifacts that encode the exact methods as described in the papers but in sufficient detail to be executable that drive innovation. Sadly, system papers are rather hard to publish, despite taking longer (software that is released needs to be much more polished than software that you are going to keep to yourself).
Making a startup? Go and check how hard and crazy that is. Make a family? Similar convoluted process with ups and downs.
What I think is wrong is that people have a very "idealized" image of a scientist scribbling on a board and equation and getting some prize (or defeating the aliens). These images are good for kids but after high-school I think people should give it a thought and say "ok, things are not exactly how I imagined in life, lets try to understand more what I like and want". You know the same process that makes people realize there is no Santa Claus.
While the institution and national agencies measure impact in terms of number of "level 1/level 2" papers, colleagues don't care at all about this value. What's important is number of single-author papers, number of papers without their advisor, number of different small group collaborations, and most of all, having papers accepted in the top venues.
A person with 50 shit papers will not even be considered for the job.
This is an interesting note. We're talking about a student from one of the top CS schools (UIUC) and applying to another top school (UW). If you think about this a bit carefully, the paper being published did not change who he was or his capabilities, it was simply a difference in measured (distinct from measurable) signal.
It's incredible how many extremely noisy signals we use in academia but act as if we use a clear meritocracy. The review process is extremely noisy itself, with computer science in particular being generally more noisy given its preference of conferences over journals. I'm glad Jeff mentions people and opportunities, and it reminds me of the old saying about there being no self made man. But I think this is a very clear example of a instance where we need to think harder and more carefully. Counterfactually, it is almost certain that had that paper been rejected, but all else stays the same (i.e. getting into UW), his success story would also not change. Signals are definitely hard to measure and certainly schools are getting a lot of applicants, so I don't blame anyone for doing this, but I think it is incredibly important to remember these counterfactuals. To remember that metrics are guides and not causal variables themselves. Because there's a great irony in that metrics destroy meritocracies.
[1] "Almost" to account for the slight chance that he didn't actually author the paper but somehow managed to get his name put on it anyway.
I agree that you need to use metrics. But we need to be clear that metrics are not enough and very incomplete themselves. With something like admissions, I'm not sure there's anything except noisy signals and the strongest one by far is the interview.
> Since someone incapable of writing such a paper is probably unsuited for a PhD,
I very much disagree with this. The explicit purpose of schooling is to train people. Many undergrads are not going to have the opportunities to publish. It is not hard to train someone to write something publishable and this is not something I would be much concerned with myself given how much writing they're going to be doing over the next few years. The far more valuable skills are in being able to perform research which is quite ambiguous (there are at least 2 ways to read this sentence and both are correct: research type v measure). Your first 2 years of your PhD are almost exclusively training, with more class work and learning how to begin research. This isn't a job you're applying for, it is a training program.
My PhD thesis was less than 40 pages long. The introduction was 1/2 a page (basically "if you need an introduction you should not read this, here are 3,4 books to get you started").
Then I copied/pasted from my articles and then came the acknowledgments (which I actually fund valuable because I wanted to thank my advisor for his non-science-related help and a friend for her magnificent idea that turned around the thesis. And my parents, wife, dog etc.)
Then the conclusion ("brilliant work")
And then a discussion with myself about everything that I fucked up and what could be improved (my advisor fainted on that one).
The jury was 8 people. The younger/more dynamic ones were super happy (especially that they made their review a page long as well). The older ones were disgusted and said that clearly. I got my PhD.
I fought in Academia for a few years to bring some change but eventually left (also for other reasons). If I was to stay for my whole career I would have tried again and again to change the status quo.
My preliminary defense thesis had to be 50+ pages, but during the presentation, it was pretty obvious that the committee had at best looked at the table of contents. It all feels like such an unnecessary waste of effort. Even with my own thesis, over half of it is just padding with very fundamental background information because the work isn't really so complicated as to require that many pages to discuss, it's just demonstrating more advanced simulation capabilities by implementing GPU acceleration for a niche but simulation heavy field.
I clearly stated that I would not waste my time and the reviewers are free to provide comments and we will see during the defense.
I found out that a lot of these "rules" are traditions that one can challenge and suddenly they are not traditions anymore.
