First of, there is a lot more to judge in a program than just the correctness of its output. Is the code readable ? Re-usable ? Did the student understood the core concept of the language and used them correctly ? I saw some student fail to produce a correct program but had a well organized code. On the other hand, one of my student once created a program that was just made of switch instruction in switch instruction (up to 5 level of switches) and spread for more than 3000 lines, just because he failed to understand how function where working.
You can try to plug a linter in your testing stack, a quality gate, all kind of software to measure the code quality, but it will never be truly fair. It will make your evaluation a game where the student challenge is not understanding the concept of the class, but instead, understanding the rules of the various test you put in place. And those rules tend to be even more arbitrary with than the teacher himself.
Trying to automate grading is an engineer trying to apply its vision to education, and it my honest opinion, it is an awful idea. Providing tools to facilitate education is awesome, trying to automate education is a dystopia.
I think this was a good middle ground that forced you to think beyond the rigid parameters of the public tests, and discouraged things like “switch” statements.
Quality of code was also a component, as determined by a TA.
The system used to pull this all off is pretty interesting, I think. It's all open source: https://github.com/redkyn
Without automated testing, a lot of energy is spent on making sure the program runs and does what it's supposed to. Since students use all different kinds of environments (especially in beginner courses), just getting the program to run is a big challenge. Lint reports are great and I recommend adding them as part of the automated testing stack.
One huge advantage of automated testing is that students know before submitting that they understood the problem correctly, their program does what it's supposed to, and they are submitting it in the format that the instructor is expecting it in. A realtime feedback loop will always result in better submissions and grades.
Of course, once the basics are out of the way, instructors must look into the code to make an evaluation of whether the problem was solved the "right way" - whatever that means in the context of the course.
Source - I maintain a tool that automatically grades CS assignments and have collected a lot of data points over the last few years.
Perhaps it is our own fault for somehow giving administrations the impression it is possible to teach well at scales where individual attention is impossible. I don't know.
I always use automated grading to check for correctness. That way I can give more attention to other aspects of the grading rubric (a bit of style, but most of the time just understanding the thing we're supposed to be learning -- pointers, for loops, specific data structures, etc.)
For smaller programming assignments, you need to check that the right language constructs are used in the right way. E.g., you really have to go in and read to program to be certain that students are using for loops correctly. Just because the code outputs the correct answer and uses the "for" keyword, doesn't mean that the student understands how to use a for loop. I don't think it's possible to automate this check.
Similarly, it would be insane to grade a large programming assignment based on test cases alone because a) "program design" is a major part of the assignment and b) it'd be impossible to get full coverage without an insane amount of work.
I think it's a bad idea to not use automated grading, even in 20:1 classes. But of course you should also be reading the code and commenting on it.
I suspect that utilizing near peer style grading would be fairly scalable.
I agree with most of what you have to say, however you're arguing for replacing one 'game' with another. If the goal is not to understand the grading engine then by the same logic the goal is to understand what appeals to whom is marking it.
But the idea is that a teacher is not as strict as a tool and is not here to punish student, but to instead grade them fairly in order for them to see what they can improve, and provide assistance for them to improve on those point. Well at least in theory, its not that easy to do :) .
The main reason why I disliked automated grading was that, a lot of time, I could see that a student almost got the right answer, but either didn't have the time to finish, or made some small mistakes. A automated tool would have given him 0, the same grade that a student who failed to understand anything or didn't work would have had, which I see has highly unfair. He cannot have all the point, but he should have some.
Not valuing the work of students is the quickest way I found to demoralize student and ultimately have them fail the class. And this is the opposite of what a teacher should strive for.
Another fun tool is CrowdGrader (https://www.crowdgrader.org/). It's a nice way to distribute reviewing/grading throughout the class to make it more scalable. When I was teaching I really liked it, it meant I could assign more interesting creative assignments at a faster pace. It's a different type of "automation".
Someone else writes of all assignments getting a public test suite, which sounds great most of the time but is even worse in making sure that students can understand what an assignment is actually asking.
I'm about to start teaching a middle school competition robotics elective. One of the two difficult grading measures is going to be tests (both closed book and open book) about what the rulebook says and means. (The other will be documentation. All the rest will be subjective and easy and squishy and "easy A").
I don't necessarily agree with this. Programs are not designed to be readable, they are designed to solve a problem. The first goal is to get the correct program, and only after, it is important to make the code readable. A readable code that doesn't work is not worth much.
As a former student, I think automatically graded assignment are very nice because they force you to have a programmer mindset (the smallest error / typo will make your code fail).
If you want to grade also the organization of code, fine, but I think the first one is more valuable (how to organize code properly is not learned at school anyway, it is learned by months / years of practice).
This is 100% incorrect. If a developer submits a PR that works and if fast, but cannot be understood by the rest of the team, it can, should, and will be rejected.
80% of software's lifetime is spent in maintenance. That clever hack or ugly patch might work today, but you're going to hate yourself for it in a month.
There are exceptions, but they're by far the minority case and should be clearly marked as "deep magic".
As a programmer, the substance of your job is to solve a problem, which you do using various tools. Your ability to understand the problem and properly translate it within the restriction of your tool makes you a good programmer. Making no typo doesn't make you a good programmer, especially these days where you can usually really on extensive CI stack to check for this kind of small mistakes.
And, I do agree that producing a program that is correct is the first goal, but it doesn't mean that other consideration such has the quality of the code shouldn't be valued, has you said. In the same way, properly organizing code may not be the first goal, but it doesn't mean that you shouldn't reward student who clearly tried to think about their code. And, I would argue that a lot can be learned in school :) .
Overall it is all about incentive. By grading automatically, you are targeting the lowest bar possible. By grading also other aspect of the student work, you also give him intensive to work on those aspect, making him not just a machine that can pass unit test, but also a person that can reason about its own work and try to improve it.
No. Code should be readable. When a code is fresh only you and God knows what it does and how it does. 6 month later, only God knows how your code works. You need to re-read and understand it to grasp it again.
Unless you marked all magic with big comments blocks or you wrote your code explicitly, it'll take some headache-inducing hours to re-understand it.
> The first goal is to get the correct program, and only after, it is important to make the code readable. A readable code that doesn't work is not worth much.
Again no. You can leave a non-working but readable code overnight and understand the problem tomorrow morning. You can't remember random noise after a good night's sleep (unrelated: This is why regex is hard for our brains).
Related read: http://raganwald.com/2013/04/02/explicit-versus-clever.html
Programs are written to solve a problem. They are designed to be understood by others who might have to debug or modify them due to new requirements.
I assume you've not had much experience with any legacy code where: if statements span hundreds of lines, variables have cryptic names which mean nothing to anyone but the original author (who is gone), and a distinct lack of comments.
I'm in the middle of such a project, and the lack of readability is severely inhibiting my ability to move forward. In fact, I've had to write software to parse this source code, just to follow the flow of logic.
This might be an extreme example, but it highlights how important readability is.
My current favorite approach is the one taken by Udacity in their "Nanodegree" programs. Much of the coursework is tied to autograders, which offer quick (if not very nuanced) feedback on the work performed. The culmination of each unit is a project evaluated by a human being, allowing for guidance on a more personal level.
In the absence of 1:1 tutoring, a hybrid model like this would be the base for my ideal class structure.
As in person-to-person interaction when teaching, you don't have to rely in a single way of assess if the student has understand a concept correctly, a Mediocre teachers does that
It also has a number of advantages, I have lost count of the number of times I have been sent screen shots of error messages (even camera photos of a screen). I now insist that all errors a reported via issues.
This means I can fix / push code for problems and track what I have done.
It also helps with plagarism as I can usually see the progress of students work with regular commits rather than one project that appears straight away via normal submission processes.
For group work there are a number of tools to do analytics on who did what etc so it make seeing how well the group has worked.
Now we are moving to online, I'm thinking of adding some of the CI / IDE features that are being introduced to help with Lab exercises, however in our area (Animation and Games) this is not ideal as a lot of what we do is visual / interactive.
Hey, from someone who works on supporting developers every day, thank you from the bottom of my heart. I shake my head whenever an engineer, _an engineer_, feels like sending over a screenshot of an error that includes uuids that are 30+ characters is a reasonable way to report an issue.
Train those new engineers well!
I teach a computer graphics course and that seems useful. What tools do you use?
Delete repositories
Ability to delete any adminable repository
This application will be able to delete any repository to which you have admin rights.> Delete repositories
> Ability to delete any adminable repository
> This application will be able to delete any repository to which you have admin rights.
Obviously you'd probably have it saved on your computer, but this may mitigate / in some instances stop it.
I'd still say its a bit sketchy, but it makes sense in context of the everyday CS student.
And you should have a local working backup
Do technical writers within Google use Gerrit, or whatever their Perforce based review process is called, as a core part of their collaborative workflow like coders do?
I'm not entirely certain I'm understanding your concern as I don't see anything in the feature list about how they handle "pure documents". My assumption is that they expect you to use markdown for such submissions (a valid choice for programming-based learning) and you're suggesting this should be more widespread and used for teaching other subjects?
> At what age do kids embrace Markdown as the single source of truth for their documents?
I love markdown. I _want_ it to be used everywhere. But I know of adults who are barely interested in or capable of using Word/Google Docs correctly.
You want _kids_ to use _markdown_!? That is one of the more absurd things I've seen on hacker news. Up there with that guy who suggested Dropbox wasn't necessary because surely the average person can figure out how to use rsync to an external hard drive.
A sibling comment managed to take this even further by suggesting LaTeX?! You want kids, in grade school, to learn how to use LaTeX? I feel like I'm taking crazy pills! Are you all being serious? What's next, all assignments need to be submitted using JSON so teachers can program their own essay viewers?
That's obviously hyperbole, but rich text, Word or Google Docs are very clearly the best option for essay style assignments. Those options have their flaws from a technical perspective but you seriously need to look outside the hacker news echo chamber if you think this is a good idea. Many of these people are going to go into careers where if they submitted their work as a plaintext markdown document their boss is going to say "what the fuck am I supposed to do with this?"
Again, this is such a strange suggestion to me that I feel like I'm misinterpreting it. If that's the case I apologise.
It takes 30 min to learn markdown and maybe more to get hang of latex but I can assure you schools spend more than 30 mins to make students learn word.
And if teachers can't learn markdown or unwilling to, maybe they shouldn't be teachers given their job is to constantly relearn and teach it to their students?
Google docs or office might not remain tomorrow either - there maybe alternative but they are not open source. Isn't that problematic along with all the proprietary tools built on top of that?
lifetime of proprietary tools is short. Why not learn open standards?
Corporate leaking into schools need to stop. Government should fund open source alternatives to all the proprietary tools used in schools.
I think markdown would work precisely because of this. Word is too hard to use and too easy to create documents that are impossible to edit by a second person.
For context. I've just watched my wife trying for 20 minutes to add one item to a bullet list, with the proper indentation and numbering of course. Mission impossible.
That problem wouldn't exist in Markdown. There is no way to accidentally do an unmodifiable list of list in a table indented by spaces.
Of course any user adoption would require a decent editor software with live preview and export to PDF.
Markdown is excellent because it prevents the writer from formatting except where that has semantic meaning (headers, lists, etc).
I’m teaching my son (14) how to use the best tools: The first assignment I gave him was to write a short essay in markdown, using vscode, and commit and push the answer in a pull request on github. I can do a code review, pointing out changes or even add my own content in a commit of my own.
These are powerful tools that everyone should be learning for writing intellectual content. The diff tools and being able to navigate change history lead to improved quality. In the same way that git gives me confidence to delete bad code, when writing an essay, I can delete low quality content. While rewriting I can see the diff changes for reference.
Of course, once you are ready to “deploy”, you will have a suitable style transformation so that the final result is a perfectly formatted document (with emphasis on excellent content).
Markdown is incredibly straightforward. I'm quite sure it's easier to pick up than a word processor; you can explain all basic formatting on half a page instead of making kids watch a series of YouTube videos on now to use [whatever word processor is hip nowadays]. Programs like Pandoc make it easy to incrementally transition to LaTeX by inlining it in Markdown documents.
I'd say that ideally, teachers should "support" a few programs, but allow students to use any program that can conform to a reasonable style. Either that, or just have students paste their essay text into a box.
True, but I’m taking the product name “Github Classroom” seriously. Maybe it should have been named “CSLab”. I think that the Scrum-like tools and processes that developers have refined over the last decade can be used in any endeavour that generates digital artifacts from text source files.
Github/Gerrit workflows seem ideal for these types of collaborative projects but I really don’t know; thus the question marks. My conclusion from your feedback is that a Google Docs-like WebUI is a requirement and my question switches to when is it appropriate to peak under the covers and see the plumbing.
Anecdotally, never for most students. I know many of my college classes required submissions in Microsoft Word format.
We at Repl.it worked closely with them to make it really easy for students to start coding in seconds instead of hours. Announcement and HN discussion:
- https://github.blog/2020-05-26-code-in-the-browser-with-gith...
How do I quickly convince my safeguarding team of the safety of repl.it and github integration?
We’re obviously concerned about pupils having contact with strangers. A much more common problem in schools is bullying. It’s also important to not allow yet another channel for harassment. Anything where pupils can privately message each other is usually quite difficult to endorse.
I don't think what we're building is a direct competitor to IDEs -- you'll always need a repl in the same way you'll have a TV but still go to the movies, or you'll have a tablet but still have a desktop etc.
What a mixed feeling of (mostly) amazement and (some) disheartenment. I started looking into this space--particularly the autograder feature with full control of the execution environment, which looks look it was added to GH Classroom this March--back in 2016. I was helping teach life science graduate students how to code, and was surprised to find I couldn't find anything that fit my needs. I started working on CodeStories[1] during the few spare hours I had each week after grad school work. I made it decently far and built out a number of cool features. On the instructor side, course creation is tightly integrated with Jupyter notebooks, which is really handy. And the courses themselves have a cool level-based structure where main problems have associated side-tasks. I was able to use the site to teach several summers of courses.
I haven't necessarily kept a super close eye on developments in this space since 2016, so it wouldn't surprise me if there are other players with unique features at this point. But given that GH (arguably one of the biggest potential players here) basically just launched, there's clearly still work to be done. Personally, I had largely paused CodeStories after grad school until the events of early spring made it clear that remote learning is almost certainly going to make a resurgence. Since then, I've been putting in crazy after-work hours to figure out how marketing works (which has been intensely difficult given my lack of experience) and put a final layer of polish on a paid offering. Intermediate Python for Bioinformatics[2] starts in a couple weeks and looks like it's going to be a reasonable success. I'm excited for it and for whatever comes after it.
All that to say, I guess this is what it feels like getting scooped. I'm sure it's possible to argue that I'd already been scooped. But somehow it's different seeing it from GitHub.
Congrats to the team that built this! It looks like an amazing tool and I'm excited to see it put to use, particularly outside of the pure-CS world.
#2 It looks like its time to pivot. Find a niche which you can exploit (like bio it seems) and conquer it. Google Docs looked like it was going to dominate everything and no one had to do anything else in that space, yet Notion comes along and is now valued at $2B.
In a sentence, the product right now is an 8-week bioinformatics course. It happens to use the CodeStories platform, which I developed, but haven't fleshed out into its own product offering yet. Does that help?
re #2: Good point; I suppose there are lots of similar examples. Zoom is a relevant one that comes to mind.
If it works like I expect it to, you should be able to create different "classes" for different roles; SysAdmin, Programmer, Designer, etc. And you could create increasingly more difficult tests; build a web page, personalize it, add a cart, etc.
How are you guys using Classroom?
I can't really knock it, but at the same time, this has been Microsoft's strategy for a long time, with students learning Office in school but no competitors, so they take it with them in their private and professional life.
It's a difficult one though, given that both github and office are pretty standardized / ubiquitous everywhere.
> Learning through Exercism is quite different to other programming websites, with a focus on individual practice and mentor-based learning. Here's how it works. ...
What I meant with my original comment was that exercism.io had the right features for providing student feedback already, i.e. the feedback was the main point of the platform.
Additionally, they're working on the third version of the site. So a lot of people are focusing their attention on that, or dividing their attention between making new content and handling the current load.
The CI part works well (I would expect the same from github), but things like syncing up your class roster or just inviting an ad-hoc set of students.... that's very clunky.
This would be ideal and would definitely make my job a bit easier.
I have used google classroom for courses before and I really like it. It's pretty easy to use but even so, it does create a fairly big mental over-head for most students. It's easier to forget that these days many university students have no computer experience outside of smartphones. I had one class where an assignment, a paragraph of text was submitted variously as a google document, a spreadsheet, a power point presentation and a screenshot from a phone.
Git for none programmers is out of the question.
In a classroom setting, you don't have to pretend to be perfect. You have to learn. Perfection, polished history, etc is something you can save for your future career.
Overall, I think this is a well received release, since it's something I know many students have appreciated in the past rather than waiting a week for teachers to grade their code.
Accountability of submissions and recorded specific feedback for each assignment is great way to learn.
One feature I would like to see is peer reviews. Although it might cause cheating or plagiarism, it could be useful tool for peer learning. Probably can add it as post assessment step.
IDE Integration announcement: https://github.blog/2020-05-26-code-in-the-browser-with-gith...
GitHub Classroom starter guide: https://github.blog/2020-03-18-set-up-your-digital-classroom...
Autograding with GitHub Classroom: https://github.blog/2020-03-17-improve-student-success-and-i...
I'm teaching software development and was using freeCodeCamp.
But my students are designers, so the curriculum is often too much and I would like to strip it down. Maybe this is a way to streamline things a bit more in the future.
They mean “service” not “tool”.