It doesn't have to be particularly big or complicated, either. Just proving that you can accomplish the aforementioned tasks will go a long way to getting you through the door.
Another thing that helps is knowing something other than Java and Oracle, or whatever other combination of brand-name technologies is popular this week. A team I work closely with is trying to hire a Java developer. They got 50 resumes, most of them "meh". Nobody was really exciting. This is probably because after reading 48 resumes, you are not going to be in a good mood, so even if someone is really good, you are not going to notice.
Conversely, they also needed another Perl developer. They got one resume, and the guy was awesome. It was easier to spend time on him since he was the only applicant, and being the only obviously qualified person to apply is good for your odds.
The other team is thinking of changing the language requirement from Java to Scala, since people that know to put Scala on their resume are more likely to have a clue, and it will reduce the applicant pool enough to spend time talking to everyone on the phone. Plus, who wants to do a Java project when you can do a Scala project instead? ;)
Anyway -- working code + unusual skills == you getting the job.
Plus, I care. If the business I'm applying to for a job doesn't care, it'll probably reduce my interest in them.
Let me give you a strawman scenario: I walk into an interview at a startup that just landed its Series A round. I'm interviewing for a position as the chief products guy or the CTO or something. The interviewer across the table from me says "You spent a paragraph of your resume talking about the business you built from scratch into a $200k/year revenue source. That's not very interesting; let's talk about this cool IOCCC entry of yours from last year, instead."
I'm going to assume that the interviewer (likely the founder or a co-founder) is probably an idiot for dismissing a significant accomplishment of mine and focusing instead on something that is—relatively speaking—quite trivial, and that his priorities are totally off. My bozo bit's been flipped, and there's not a chance I'm going to want to work with this guy. He's going to spend the next 18 months burning KPCB's million-plus stake on architecture astronaut'ish goals that have nothing to do with solving real problems and making real money. I'm going to shake his hand, say 'no thanks,' walk out the door, and go have a pint. (bear in mind, this is a strawman).
Please note that I'm not a Perl guy, so I can't speak for you or Ingy dot Net or Larry Wall. I'm a 'business guy' who happens to have a Computer Science degree and writes his own code.
Having some working, usable sample code - whether your own project or participation in someone else's - obviously provides some gauge of what you can do but also indicates that you can make a program, maybe work with other people, maybe understand the various things involved in the process. Quite a few people come into the workforce only knowing how to write code, which while a prerequisite is nothing like actually programming. If you know the difference, you have a leg-up already.
Networking is important, and even someone like me can do some of it sort of incidentally by joining the local <Technology> User Group etc.
Being able to point to a real product being used by real people who had paid me real dollars for it has made a significant difference for me in job interviews in the past, especially for my first real job out of college at Microsoft.
Today, I'm a startup co-founder. If I'm looking to hire a developer, and I see that a candidate knows how to not only create software, but also market it and sell it, AND that they recognize the value in those other two steps, I'm significantly more likely to call them in for an in-person interview.
And besides, with any luck you'll generate enough money off your own software that you can tell all those companies you're looking to for a job to piss off ;)
Support people (users and other developers) on mailing lists and forums related to the project you're working on as well as the areas of technology that you have some experience with (note, you don't have to be an expert, just interested). Work on your page rank, so that when someone searches for you on the web, they see several pages worth of problems that you helped someone fix. For this same reason (among others), never be rude or overly sarcastic on a forum. Be helpful, and if you can't be helpful, be attentive.
In the course of doing all this, you'll sometimes come across people who work at companies looking for young hotshots (hint, that's you) or reliable developers (hint, that's also you) or an experienced programmer with an eye on customer satisfaction (yep, you again). Don't ask them for a job, but hint to them that given the right conditions, you may be open to a more permanent technical exchange.
Now if you're active enough, you can get this all done in a couple of months. When I graduated from college, my resume already had two pages of projects that I'd played a primary role in. This helped.
In that case, your answer is: network!
I've seen so many qualified candidates get passed over for someone equally or even marginally less qualified because they had an inside line on the position. It's probably too late to do much networking at school (i.e. joining clubs, etc.), but the thick part of conference season is just getting started. Get a student rate to something while you still can (JavaOne seems to have been free for students in the past...not sure what Oracle plans for this year, though). I'm guessing that your 2 years experience has come from internships, so that would be a good place to start too. They might not have been your ideal place to work, but they'll give you a paycheck while you plan your next move.
If you are going to work on an open source project, pick one that's sponsored by a big name company (i.e. not an academic project or start-up). No matter what the project leader asks you to help with, do it! If he says they need work on documentation or fleshing-out the test suite, don't complain. I know it's not sexy work, but when the project leader gets a req. to hire someone, he'll know that you're someone that can be counted on to get the job done.
Finally, learn something new. I recently saw a very qualified developer get passed over for a job not because he didn't have the skills listed in the job offer, but because he didn't show an ability to work outside of his set of core competencies. Since you're new, nobody is going to hire you based on expertise. Your secret weapon will be the fact that you can learn whatever new skill or technology it is that the hiring manager needs next. In other words, you probably want to branch out from Java (it's a skill area flush with "experts" and hiring managers who are only looking for people who can spit out code quickly).
1) Show that you are passionate and driven, which it seems you already have with your experience. Other ways could be get an iPhone or Android app out on their respective marketplaces so you can provide hard numbers (# of downloads, $ made, etc.). Contributing to OSS is always a good way to spend your time as well, but only if you really care about the project.
2) Make connections. This can be rough but a few things I've found can make it a lot easier. I've gotten a few really good leads through Twitter actually. Follow enough people from a company you'd like to work for and eventually one of them will mention something about openings. Another good avenue is going to technical talks (most universities hold a few of these a month, maybe more frequently). Go up and talk intelligently to the speaker and if they are accepting resumes definitely provide your best. The main thing is have someone inside the company turn in your resume for you. This informal referral or recommendation is huge. I read a statistic that only 1 in 20 online applications result in a face to face interview, those aren't great odds and being denied a lot is sure to affect your morale. Getting someone to turn in your resume is a sure-fire way to hear something back from the company.
Help answer people's questions on SO or irc. Learn the other JVM langauges. Learn about tools that J2EE people like, such as Maven and Hudson. Learn algorithms. Present at conferences.
Do this, and people will find you. Last time we needed to hire someone, I visited my favorite IRC channel, said "anyone want a job?", got a privmsg, and had the person interviewed in person and given an offer in just a few days. (A few because the weekend was in the middle.)
What you want is a friend who will talk you up to the people making the hiring decisions. What you don't want is to be another boring resume in a 50-resume-deep stack.
Suggestions:
Figure out where you want to look, and what the hot technologies are. In the Philly area, if you can demonstrate knowledge of Spring and Hibernate, you'll get far more offers then not anyone not having it.
Network: Join the local JUG, IEEE, etc. The more people you know, the better.
Volunteer: Something to add to your resume, and you'll get to know more people.
Read: Grab "What color is your parachute?" out of your local library. It's a great starting point for any job hunt.
Most importantly, keep at it. Actively job hunt and program. Sitting around playing games and randomly commenting on the internet are far less likely to connect you with the people you need. Do something that you can add to your resume.
Talking to our HR people, the majority of programming candidates they see are either fresh out of school with zero professional experience, or they're consulting refugees (Avanade, IBM, etc) with 10 years of experience who are just going to bolt as soon as the consulting industry recovers.
Honestly, two years of real world experience and willingness to take a junior position makes you kind of a commodity.
Are you not getting to the interview stage? If that's the case, I would echo what the others say, check craigslist in bigger metro areas.
If you're getting to the interview but not getting offers, well, that's another Ask HN in and of itself.
Where abouts are you based at the moment ?
Just say "hey, this is me, this is what I've worked on that you'd be interested in, I'd like to interview. If not, hold on to my resume"
Persistence/experience/credentials are important, but without a personal connection the job market (in any profession) is a crap-shoot.
My girlfriend had a similar situation (she's not an engineer), she finally managed to get a job by doing an internship, all the while applying to a ton of jobs. (Of course all of them were looking for 5+ years experience and a masters).
She was rejected a lot and it was demoralizing for her.
But what paid off were the personal connections that she made during the internship and interviews (she made a good name for herself), eventually one of her acquaintances gave a colleague her resume (who then offered a job).
I'm not really a people person, so I know that this can be a difficult road for us introverted folk, but I think the best thing to think about is 'where can I meet people who would be looking to hire someone like me'.
Then go to those places, meet those people, and get them on your 'team'. Eventually you are a proven commodity and are able to get something.
in the meantime, code up something cool, and post it on HN. Then in your signature or whatever, mention that you're looking for work.
Just keep at looking for open doors, and work on independent projects when you're not actively engaged in job hunting. In my personal experience, the projects don't need to be exceptionally unique and amazing -- find something that's a manageable size, and prove that you have the chops to see something through to completion with solid code.
