In the former case, the query is verifying that the object_id field cannot be used to find a foreign object--regardless of the value of object_id. This is exactly what it is asked to do.
In the latter case, the query is simply verifying that object_id is NULL, which is exactly what it's asked to do.
Part of the issue is confusing object data with metadata. The id is an implementation detail of the data store – metadata. What the user is trying to do here is “talk database” and “talk model” at the same time.
Which is a perfectly good argument against an ORM. But one should commit to a metaphor, or not: http://clipperhouse.com/2012/02/29/suspension-of-disbelief/
No, the Django ORM is not correct in unsing the correct query by default. At least not on every database backend. That's because on any database that enforeces constraints, it already created the necessary checks, and altough functionaly they are equivalent, that line can mark the difference between a 5ms or a 45 minutes runtime (that's what happens with my data, not a hypotetical case). There is more to a framework than mathematical correctness.
But then, it should certainly have an option to always use the correct query, and it is a lot of implementation work that I can understand quite well that Django developers don't want to do now. They probably have other priorities.
And by the way, as I said, this is a problem to me (but no, not important enough that makes me fix it now, maybe later), but did I stop using an ORM just because of an abstraction leakage? Of course not, I encapsulated a solution to this problem and gone on, earning lots of man-hours at the 99% of my code where Django's ORM doesn't leak. That's my main disagreement with the article. Yes, ORMs are stupid, but not using one just because of that is stupid.
And yes, if you let Django templates go, you can easily cut 9ms of your response time. That's great! I wonder how much you'll cut if you rewrite it all on assembly.
The first is retrieving the object and checking if it exists, and the second is just checking the parent's foreign key. Not sure that they are really the same query, if you have unenforced foreign keys.
This is not "dumb", this is analogous to checking if a pointer is null or that the contents of the pointer are null (which is a distinction that some people want to make).
While a lot of us who had started working with SQL in the 1980s, if not earlier, were perfectly fine with using it, many younger developers were scared away from it by these claims.
So we've had a generation of software developers who were essentially raised to hate SQL, and to embrace ORMs, even after it became clear that ORMs do come with some pretty serious trade-offs, and do not necessarily increase productivity.
Not having a solid grasp of SQL, a lot of these developers just don't realize what they're missing out on. I've seen this first-hand many times before. These developers will spent hours upon hours trying to get their ORM to perform a moderately complex query that could be easily written by hand within a few minutes, including any code necessary to perform the query and to retrieve the result. The time and effort expended on these sorts of queries will very quickly negate any time and effort the ORM may have saved for simpler queries. And these moderately-complex or complex queries always arise in real-world software.
I think that education is the only way to really solve this problem, but a lot of developers are quite set against this. Learning SQL isn't that much of an investment, but the returns it offers can be huge.
I.e., in pseudocode:
query = SQL("SELECT * FROM entities WHERE owner = ?", owner=me)
...
if some_condition:
query = query + SQL.WHERE("OR public = TRUE")
...
if other_condition:
query = query + SQL("LEFT JOIN things AS t"
" ON t.entity_id = entities.id") \
+ SQL.WHERE("things.value > 0")
...
my_nice_list_of_results = run(query + SQL("LIMIT ?", count))
That don't try to handle hidden-magic-state and lets you easily access via Raw SQL if you need to do complex queries. Many don't try to abstract anything and are simply extension methods over the underlying IDbConnection (so you never lose any flexibility), i.e. they simply exist to remove the tedium boilerplate of mapping RDBMS results back into POCOs.
Using an ORM should not exclude also using direct SQL. It should be both.
I believe this brings the best combination. Anytime there is major complexity just drop to normal SQL.
The best of both.
Now obviously, some people would complain that it doesn't make sense to do the extra join, but then people would be complaining about magical or exceptional behavior. ORM behavior is very predictable about which fields are being queried
filter(object__isnull=True)
https://docs.djangoproject.com/en/dev/ref/models/querysets/#...
...really, what advantage does Django provide at this point in this project anymore?
I've been happy with django-jinja[1] for that purpose. It replaces the context processors so it will load jinja templates if they have a .jinja extension and Django if they are .html. It also includes the django filters in jinja-land.
The ORM is more problematic. I just started a project a couple months ago and thought a lot about ditching the default ORM in favor of SqlAlchemy. I decided not to, for the reason of expedience and, as the TFA mentions, it already leaks. So, I stuck with the Django ORM and will drop to SQL directly if need be (need being defined by the ORM making the code confusing or there being performance hotspots).
Also Flask is a micro-framework and Django is a full-stack framework. Flask can be used as the backbone of a full-stack framework but figuring out a good project structure and finding out what third-party apps to use can be daunting for a person without Flask experience. If you really want to get people to switch, package up a Flask-based framework with the features of Django.
The CTO of a startup where some friends are working thought the same you did, and 2 years ago rewrote everything to Flask. Now they're going back to Django.
Django is much more that an ORM and templates.
Documentation. I use Django but without any ORM and with Jinja2, so it's basically just Flask but with more stack overflow threads and more third-party software.
Is there any actual advantage that I would be getting by using Flask instead?
I’d also like to point out that Flask-Admin has come a long way with adapters for SQLAlchemy and at least one other ORM library. Very usable and extendible.
After working with Django ORM for a large project, I started to rethink the usefulness of models. It turns out just treating data as sets lends to a functional style and is simpler down the road.
I'm curious as to why you stick with Django (other than having projects already begun relying on it)? Without the ORM and with the templates, there's is not much I get out of Django.
Also, geodjango.
I love and use Django, and have built large projects where I haven't really run into any limitations with it[1], but for the most part, nowadays my workflow is to pip install django, south, tastypie, then load in a template with Backbone and Marionette, then get to town.
Templates are either Mustache or Underscore, depending.
[1] - Yeah, it could be faster, but if you have a large, confusing database schema that you inherited, the Django ORM is great for getting things stood up, and then just tune the queries after the fact. It's still a huge timesaver vs. writing every query by hand.
FWIW, I've never bought into Spolsky's vision that re-writing code is poor strategy. Steve Jobs never thought twice about ripping something apart and starting over. If anything, code re-write can be an advantageous position -- you often have a greater understanding of the problems you're intending to solve. When well-executed, it can take the form of heavy refactoring, even when switching languages/platforms.
It's interesting that you take Jobs (and by extension Apple) as an example here, as many new projects from them which might appear to be complete rewrites from the outside are in fact heavily derivative or dependent on other projects. iOS for example is an incremental revision of OS X, removing some of the UI layer and replacing it, but leaving almost all of the underlying OS intact, using the same dev language any many of the same APIs - it is by no means a clean-room rewrite. OS X itself was heavily based on NextStep, which of course was based on Mach/BSD, so none of these 'new' platforms started from a clean slate like BeOS for example, and the same tends to happen with APIs, though sometimes these have been rewritten (Quicktime comes to mind, and arguably UIKit is a significant rewrite of AppKit, though the two still exist in parallel just now).
Sometimes rewriting is the best solution, but it does tend to take a lot longer than expected, doesn't always leave you with a satisfactory replacement, and ends in failure more often than it ends in success, particularly on very large projects or ones with fuzzy scope. I think Spolsky was talking about projects on the level of Netscape and Excel, where a rewrite would be a significant challenge very likely to fail or be delayed so long that it falls short of its initial goals. The smaller the project, the more viable a rewrite becomes, and sometimes it is the best option if the existing product is not delivering and is difficult to extend/support.
I think it's something that's true in general but false in some specific cases. Rewriting involves spending enormous time and resources to at best standstill, and at worst move backwards ( chances are your re-written product will be poorer in features, and initially buggier than your old, stable, battle-tested version). For smaller companies, it is a death knell.
> Rewriting involves spending enormous time and resources to at best standstill, and at worst move backwards ( chances are your re-written product will be poorer in features, and initially buggier than your old, stable, battle-tested version).
There is plenty of evidence that this has happened in many places and with many companies. A very real scenario that's played out before.
I'd say those scenarios were not well-executed. If the outcome of a re-write is "standstill", the re-write is pointless. There is no justification for proceeding with it.
However, if "standstill" equates only to user-facing features, chances are the re-write is to address critical issues elsewhere (I get the impression that was the situation with the OP.) In that case, "standstill" doesn't apply. It's simply a matter of deciding whether or not the effort and risk justifies the reward.
To my main point, Spolsky's hard-line essentially says re-building your application from scratch is bad strategy. I think it is short-sighted to draw that line. I prefer to exercise judgment and draw on the resources at my disposal for the given situation. I presume many others do as well.
But, if properly validated, and knowing when to NOT use the ORM, a good ORM can help you get a lot of work done very efficiently. But I've also seen improperly used ORMs turn into MASSIVE time sinks where devs spends days just configuring the stupid thing (hello Hibernate/nhibernate).
ORMs make it easy to do things like run queries inside a loop without realizing it. I worked on a site where the front page ran something like 200 queries every time it was accessed thanks to ORM magic.
ORM can abstract DB/SQL for you, but if you really ignore DB/SQL, then it can happily make some queries an order (or two ) of magnitude slower than they should be.
So, you must always think in explicit SQL-query-terms anyway; and then it's just a balance for ease of coding - does the ease of ORM syntactic sugar outweigh the effort for you to double-check if any ORM-built queries don't accidentally do something stupidly slow.
Do you really want to write SQL to retrieve data and code to populate an object for every damn thing in your system?
Annoying that it doesnt have slide numbers to refer to.
It looks like the work around he used is to cache the job queue locally and only flush it to the real job queue after the database commit, so your guaranteed whatever data may be needed for the job has been committed to the database.
Edit: 9 minutes ago, Nick posted to Twitter: "we have the recording - just need the sound cleaned up. Expect it early next week ;)"
could this performance improvement back-fire if you end up with a security issue?
I'm not saying that it definitely would. If you know what you're doing / trust your data sources or sanitize them elsewhere, you should be fine. I'd be careful turning off such a feature completely though...
http://jinja.pocoo.org/ https://www.getsentry.com/welcome/ http://graphite.wikidot.com/start https://opbeat.com/
is there a way though to use/test opbeat ?
http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/
It gives you more control and requires you to be more explicit about your queries and relationships.