Always keep your models slim. Don't stuff template related stuff in there. You need to look at those models often, so compact is a win. course_has_finished(course) is not much longer than course.has_finished(), and will allow you to expand the functionality as time goes on. Do precomputation if you need the information in a template - that keeps your templates simpler and allows you to easily expand the complexity of the precomputation.
Don't use class-based views, at least not outside very specific niches like the Django admin. Class-based views will transform a simple, composeable call stack into a ball of inheritance mud. Inheritance is not a good code reuse tool.
Don't make separate apps in the same project, unless the project actually consists of several different, completely independent projects. You can make subdirectories without making apps, and thus avoid dependency hell.
Also be wary of the formerly South, now built-in migrations stuff. It's built around a fragile model (perfect history representation), so has lots of foot guns.
And be wary of 3rd party libraries with their own models. You can spend a lot of time trying to built bridges to those models compared to just doing what makes sense for your particular project. I think 3rd party libraries are perhaps best implemented without concrete models - duck-typing in Python let us do this. This includes Django itself, by the way. User profiles didn't become good until Django allowed you to define the user model yourself.
I disagree with this. When trouble shooting or expanding code it is super convenient to import a model and have all of your methods on auto complete. Especially when you need the same functionality in a view, a cron job, a celery task, and an DRF end point.
If you want to keep it clean you can put all your methods in a mixin class and import it from another file.
Also be wary of the formerly South, now built-in migrations stuff.
Things are much better than they were in South. But yes be careful, rule of thumb is always move forward.
Don't use class-based views
Please. For the love of God, always use class based views for almost everything. Almost everything you need is a variant of one of the built in class based views, don't make me read your copy/pasted reimplementation of it.
This. 100%. Leverage as much pre-built stuff you can, especially with something as important as your HTTP layer. Whenever I run out of CRUD verbs for a model and I need to add a custom endpoint, I'll implement it in a separate APIView sublcass. Convention over configuration; write boring code.
As simple as possible, but no simpler. Django models are meant to deal not just with the data, but also with business logic. If `course.has_finished` is a property of the course, why would you want to have a separate function outside of the class?
> Do precomputation if you need the information in a template
If the precomputation is only needed in a template, you can (should, IMO) use template tags.
> Don't make separate apps in the same project (...) avoid dependency hell.
My current pattern here has been to create one "core" project where I represent all the internal models of the domain of the application, and "adapter" apps if I want to interface/integrate with anything from the external world. This makes it easier to extend or replace third-party tools.
> (Migrations) It's built around a fragile model.
I wouldn't call it fragile, quite the opposite. There are some annoying limitations for sure (I didn't find a reliable way to change the primary key of a model, except for creating a whole new model and migrating the data to it), but I think they are due to a matter of strong safety that the migration can only be done if it consistent.
My current preference is a functional core-imperative shell-style architecture where as much code lives in the functional core as possible. It's not very elegant with Django but it works fine. Cosmic Python (really accessible and fairly quick read if you have the time: https://www.cosmicpython.com/book/preface.html) has examples that are similar.
Because one should avoid passing Django models around. It leads to bad design. Have a selector or something that uses the ORM, but exposes some dataclass or pydantic model instead, and put the logic there.
People always say this but well-structured CBVs keep a generic interface that you'll be really glad that exists when you have 80 views spread across 10 apps.
Composing function-based views is a PITA and when you're building an API with a bunch of auth/serialization/cache extras being bolted on it's way easier to keep disciplined and ordered. It is _trivial_ to mess up the order of callers for these things inside function-based views.
Fat models think controllers is the suggested strategy. It works well in my opinion / experience, though I guess it could get out of hand on very large projects.
There is something that feels very unnatural and unintuitive about your course_has_finished(course) versus course.has_finished() example. This was one of the principles behind OOP, keeping your data and functions / methods together, though I know OOP isn't trendy these days. It's far more natural to have it as a method of course than some random function, stored who knows where. I worked on a system with this type of design, it was pretty bad.
One thing I would like to see for larger projects is the ability to easily split models into separate files - a bit more like Java does with one class per file. Maybe you can do this already.
Class based views mean that a lot of code is written and tested for you. I'll agree that your view does need to be somewhat "standard" in what it's doing (anything you would see in the admin, list, create, edit, detail, login, etc), so if you have something more complex multiple forms in one view, then thy don't give a great deal of advantage. In that situation I would still likely choose a basic View class over functional views, but more for consistency.
I am probably in agreement on separate apps.
Migrations are one of the best features of Django. I have just spent 4 years working on a a system without them and it was a shambles as you would expect. Everyone is scared to make database changes, so you get a ton of shitty application code to compensate for shitty database modelling. Tech debt in other words.
I can't think of any 3rd party apps where I have used the models directly, or if I did it was frictionless enough for me not to remember. So no real opinion on that one. 3rd party apps can be pretty hit and miss, especially if they get abandoned as you upgrade Django, so I would say use with caution, particularly for more obscure ones (just been revisiting django-celery-beat, that's been around for years so I doubt it's going away).
My experience has been opposite so I'd be interested in hearing your experiences if you are willing. I have been using the built in migrations since day 1 on a medium sized Django project with 350+ migrations and migration issues for our project have been exceedingly rare. edit: We have a small team of developers, so merge migrations are very rare for us, which might be a contributing factor.
Your migration code uses the model classes, but the migrations are using a rehydrated version of the model that doesn't include any methods; another footgun. Basically you need to copy in any logic that you're using into your migration file, or else that migration's logic will change as your code is refactored. You might naively think that because `model.do_the_thing()` works now, the migration is somehow pickling a snapshot of your model class. It's not.
Because of the above, you should really squash migrations frequently, but it's a big pain to do so -- particularly if you have dependency cycles between apps. ("Just Say No to Apps" is my default advice for new Django developers. If you have a real need for making a bit of library code sharable between multiple different Django projects then you know enough to break this rule. Within a single monolithic project, apps just cause pain.) Squashing migrations quickly gets to some extremely gnarly and hard-to-work-with logic in the docs.
Moving models between apps isn't supported by the migration machinery; it's an involved and dangerous process. One idea here that might save Apps is if you manually remove the App Prefix from your "owned" / internal apps; if I have Customer and Ledger apps, I don't really need to namespace the tables; `user`, `information`, `ledger_entries` are fine table names instead of `customer_user`, `customer_information`, `ledger_ledger_entries`, a normal DB admin would not namespace the table names. You neeed the app namespacing to make it safe to use someone else's apps, but I think namespacing for your own apps inside a single repo is harmful.
I find the migration framework to be worth using, but it's definitely got some sharp edges.
It's hard to get 100% right and thus our projects always have a few lazy foreign key relationships and inline imports to avoid cyclical imports... but I think our code is easier to manage because we try to model the dependency relationships by having things in separate apps.
> Also be wary of the formerly South, now built-in migrations stuff
Our experience from South to 4.x has been that the models/migrations system has matured significantly and is probably now the main selling point for Django for us.
I wrote an article on this. My goto strategy is to create a project/core/views.py,models.py,apps.py,tests.py
I disagree with the perspective on CBVs though. I've been programming Django since 0.96 and CBVs have made nearly everything easier and better for me.
_(99% kidding, but i disagree with most of this lol)_
- I think "services" is too much of a loaded term. I prefer "actions", and I always use the function-based style.
- I hate the naming of "APIs" in this document. They use the term "API" when they mean "endpoint" or "view".
- "Reuse serializers as little as possible" is the single best piece of advice when using DRF. The inline InputSerializer thing is brilliant.
- Having each each endpoint only handle a single HTTP verb is brilliant.
- URLs and views should be separate from all other business logic (models, actions etc).
- For read endpoints and associated business logic, I'd encourage https://www.django-readers.org/ (disclaimer: I'm the author).Can you expand on this? What is the InputSerializer as opposed to custom rest serializers?
Contrary to what people usually think, the shape of the serialized object is typically defined by the API endpoint, not by the object itself. Different endpoints can (and will) serve different shapes of the same object.
Even if two endpoints serve the same shape today, they can deviate tomorrow. When this happens, most people are trying to resolve it through DRF inheritance, which is wrong.
M in ActiveRecord MVC web frameworks is deeply misunderstood. M is not "data model" (it would be called DVC if that was the case). M is your domain model. It's the model of your business, model of the real world. It's the core of your application.
Another thing that I never understood, why are functions called services? Is it a subconscious desire to go back to enterprisey kingdom of nouns? (apparently it is [1])
A service is either something that lives on a network (e.g. database, payment gateway, microservice). Or a class that has a state. Your functions are neither of those, they are just functions.
You business logic should live in the "models" namespace. Whether you put it on Model classes, or onto custom Managers, or just dump them into the module is not important, as long as you keep it consistent and keep your apps fairly small and isolated from each other.
Django already gives you enough tools to support big "enterprise" applications. It is far from perfect, but you'll get much further if you embrace the framework instead of fighting it.
If you really are attached to this "services" mindset then Django API Domains [2] is your best option.
No, instead, it is that "does stuff" hast to be its own library, or god forbid, its own service that lives somewhere else, with its own communication layer, its own auth...
Why are we making this so hard on ourselves?
1. You found one case where complexity is essential.
2. That one case is not consistent with the rest of your app, and you were taught that inconsistency is bad.
3. Since you can't remove complexity from that case, for the sake of consistency you add complexity to all other cases.
Class-based views is a typical example. You found a place where CBVs are useful. Now some parts of your app use functions, some use classes, that's inconsistent. Edit your style guide to enforce CBV everywhere. Now a simple healthcheck endpoint that returns "OK" has to be a class.
As some folks used to say, you can write Java in any language.
The right approach, of course, is to say "I'd rather have inconsistency than complexity". The challenge is that perception of complexity is subjective, but inconsistency is objective. So the right approach eventually loses, and every organization turns into a bureaucratic hell.
The control part is more like Traffic Controller . Just directing traffic.
Please read this carefully: https://folk.universitetetioslo.no/trygver/1979/mvc-2/1979-1...
Business logic is supposed to be reused. Controllers in web frameworks (views in Django) expose no way to do it.
>
> Communicating with 3rd party services (sending emails, notifications, etc.)
> Offloading heavier computational tasks outside the HTTP cycle.
> Periodic tasks (using Celery beat)
Sigh. No mention of the trade-offs. There's simpler ways to do all these things. Celery is a big complex beast and it always pains me to see it as the default suggestion for simple tasks.
Celery being complicated is also entirely on the operational side, once you actually have Celery using it from within your app is simple enough.
Cron is awful for this use-case. You end up just inventing Celery but worse when you decide how your app and the cron scripts communicate. If you wanted just scheduled tasks but simpler use something like APScheduler.
Yeah and to some degree improvements in devops quality of life in the last few years has softened my view. (I used to mainly use Webfaction without access to apt-get and my own hand-rolled scripted deployment. Ugh...)
But I'd still usually prefer a pure-Python solution without additional persistent processes - assuming there is one and it's fairly well-documented. Huey is pretty good from recollection.
> Cron is awful for this use-case. You end up just inventing Celery but worse when you decide how your app and the cron scripts communicate. If you wanted just scheduled tasks but simpler use something like APScheduler.
I was advocating for something like this. There's django-cron etc which solves issues around communicating with scripts.
I do see both sides of the debate between "complexity" and "solves problems out of the box". I'm generally on the Django side when the flask vs Django discussion happens. There's always trade-offs.
I can't recommend it. We found many bugs in the more advanced features of Celery (like Canvas) we also ran into some really weird issues like tasks getting duplicated for no reason [1].
The most concerning problem is that the project was abandoned. The original creator is not working on it anymore and all issues that we raised were ignored. We had to fork the project and apply our own fixes to it. This was 4 years ago so maybe things improved since them.
Celery is also extremely complex.
I would recommend https://dramatiq.io/ instead.
> Cron is awful for this use-case. You end up just inventing Celery
Isn't it the other way around?
Crons are way more mature, well integrated (mgmt commands don't require 3rd party modules), and extremely well trodden. Crons are super predictable, have sensible defaults and plenty of tooling. Which you will have to reinvent with Celery.
There are some benefits to programmatic crons, but the downsides are huge.
https://uwsgi-docs.readthedocs.io/en/latest/Spooler.html
https://uwsgi-docs.readthedocs.io/en/latest/Cron.html
It applies to Django, Flask or any Python system using it. All of it still applies today.
It covers a few use cases on the before vs after of using Celery and touches base on why I'd consider using Celery over other solutions such as async / await. The TL;DR is Celery brings a lot to the table around tracking and retrying jobs. It's also nice to separate your web and worker workloads into different processes since they have much different requirements usually.
We’ve run into various bugs and weird performance gotchas (like the workers prefetch jobs which is terrible if they aren’t all the same size)
That's how I did a timed function for a Django project we were hosting in Azure anyway.
when I send an async job that get data from various APIs and write all in a DB, in the case of lot there is lot of data, the celery task finish properly to but my flask app becomes unresponsive. I have to restart flask to get back to a normal state.
Anyone would know where I should check?
Just wondering if I am taking the right path or if there is better alternative.
> Just wondering if I am taking the right path or if there is better alternative.
Yes, this can be a fine solution to slow queries and is used very often in many kinds of web applications.
However... 20k rows is not a very big number for a modern dB. If the db query is really the slow part then you should investigate why - ensure the relevant sql queries are written properly, and that the relevant tables are indexed properly for the queries that you are running on them.
(But it's mostly for small personal projects, so grain of salt and all.)
If complex, what does Sqlalchemy output as the SQL? Could you optimise the query? If quite complex, optimise the tables by redesigning them?
Was the move to allow Redis to cache persistently across requests? Does it do this?
Are you timing each function to look for slowness?
I'm learning node now for another mini project, is there anything similar? I know how it achieve certain tasks but to structure in a proper way I know nothing, all tutorials I've done never really go that deep
Moving models from one app to another is doable but it is a pain. It's even worse if you are relying on GFKs.
Radoslav here (one of the authors of the mentioned Django Styleguide).
First of all - I want to thank everyone to the comments The fact that someone took the time to read the styleguide & then write a comment / propose a different POV - is humbling.
I've read everything once & I'll do so at least couple more times. There are interesting ideas & comments that we can apply!
And finally, I want to add some more context:
1. That particular styleguide has served us, and it's still serving us well. It's basically a list of ideas that we found useful, thanks to the various Django projects that we've been exposed to at HackSoft.
2. One core philosophy of the style guide is the ability to cherry-pick whatever makes sense to you. Even at our company, it's very rare to have 2 Django projects following the exact same structure. The styleguide is rather a framework / direction for things that's been proven to work, from our experience.
3. And of course, the styleguide can use some more love from us. We are sitting on a lot of unshared knowledge that needs to be structured and applied back to the Django Styleguide & the corresponding Django Styleguide example project.
4. We try to keep it pragmatic, so you can actually build something. For example, DDD sounds great, but lacks pragmatism and slows you down by a lot (at least, for us).
5. And finally - this is not the "only right way" to do Django. As there is no "right way" to build software. Luckily, there are always options. We'll update the list of other suggested approaches, so people can have a choice / navigate the space better.
As an example of one of the big topics that we want to touch upon is nesting apps. Alongside the "services / selectors" layer (btw - you can call this whatever suits you best ), the ability to nest apps within apps is really powerful, when it comes to the structure and longevity of a Django project. Having 50+ flat apps is not the best experience.
Our current focus is around building the company (HackSoft). The Django Styleguide will be soon to follow. We are slowly gaining more speed & we'll eventually get there
All discussion around "How to do Django" are in fact really interesting. If we happen to meet at some future EuroPython / DjangoCon Europe - I'm always open to discuss in person. Otherwise, if you have specific comments / suggestions - you can submit it either as an issue / discussion, or just send an email to radorado@hacksoft.io
Cheers!
Cheers!
https://docs.djangoproject.com/en/dev/internals/contributing...
If you have one testclass per entity, then if you make any changes to the structure of your services/models/entities, you must restructure your tests too. This means you can't do the "dream refactor" where you don't touch your tests, and restructure your code without changing any behavior. If you rewrite your tests whenever your structure changes, how can you be sure you've not broken your tests?
Instead, I advocate for testing behaviors. In a tightly-integrated framework like Django, most of your tests are going to be integration tests (i.e. you have a database involved). You should bias those tests towards an integration-y approach that uses the public interfaces (Service Layer, if you have one) and asserts behavior. Ideally the tests should not change if the business logic has not changed. (In practice you'll often need to add some mapping/helper/setup code to make this true.)
If you have any fat-model type behavior that is explicitly scoped to a single model, then you can test that in isolation. Many Django projects call these "unit tests" even though they still involve reading and writing your model from the DB. I call them "focused integration tests". All that matters is that you have agreement on terminology inside your project. If you have extremely complex domain logic, it can be worthwhile to construct "true Unit Tests" that use dummy objects to test logic without hitting your DB. I've not found it worthwhile to mock the DB in most Django projects though.
To provide an example of where my "test behaviors not classes" advice differs from the OP's paradigm, let's say you split out a sub-object to provide a pluggable Strategy for a part of your Model's behavior -- you don't necessarily need to have detailed tests for that Strategy class if it's fully covered by the model's tests. Only the edge cases that are awkward to test at the higher level need to be tested at the granular level of the Strategy. Indeed, the first refactor that just creates a Strategy holding your existing behavior need not change any of your existing tests at all! Indeed, if you do need to change existing tests, that suggests your tests were improperly-coupled to the code under test, since a mere structural change like this should not affect the behavior of your application. Even after adding more Strategy logic, most of your old ModelTests are still good; they still test the high-level behavior, and now also test the integration between your model and the new Strategy class. Basically, test at the most-granular level that gives a clear, decoupled test for your behavior; resist testing every entity in isolation, because some entities have rich coreographies with other entites that make them hard to isolate. Sometimes you have to contort and tightly-couple in order to test things at the very-lowest-level possible.
Inspiration/further reading: https://blog.cleancoder.com/uncle-bob/2017/10/03/TestContrav.... (Grit your teeth through the "Socratic dialog" style. The principle being described is extremely valuable.)
I hate python, but love Django.
I think I have some sort of emotional attachment to Django.