In this post we’ll explore creating and running periodic tasks in django using celery.
This post assume some basic knowledge about django and celery. If you are new to celery, I’d recommend reviewing the official celery documentation here: https://docs.celeryproject.org/en/latest/index.html
But here are the basics:
Celery is a distributed task queue that requires an external message broker for sending and receiving messages. This post explores periodic tasks in a django project. The process consists of the django project, a broker (redis) a task scheduler (celery-beat), and workers (celery-worker). The intent is to create a task in django, create periodic tasks that will…
Perhaps you’ve experienced this… You have a ManyToManyField field in a model and want to update to using a through table to add additional fields. You update your models.py, make migrations, all looks good, then you hit this error when attempting to migrate:
you cannot alter to or from M2M fields, or add or remove through= on M2M fields
All seems lost, but there is a simple fix that can get you the modeling you want and preserve your data.
In this post we’ll explore this scenario and show how this can be resolved with step by step examples.
In this post we’ll look at using Redis sentinel as your broker when using celery workers and celery beat with django.
Redis Sentinel is a high availability (HA) solution for open source Redis server. In a multi-node setup, Sentinel provides monitoring of nodes and automatic fail over to keep Redis operational without human intervention.
Celery supports using Sentinel as transport broker, but getting the Django settings correct for use with Celery (workers and beat) can take some finesse.
The example we’ll be using is based on a three-node setup with Redis 6.0.5 running on a CentOS 7 systems where the…
How to setup django cache backends with Redis, Redis Sentinel, and Redis cluster
Django supports many different backends for various caching needs. One of the most popular solutions is Redis, an in-memory key-value database. While integration with a single Redis node is quite straight forward, integration with high-availability (HA) Redis solutions, can be more challenging. Luckily there are packages available to simplify this. In this post we’ll look at using Redis, Redis Sentinel, and Redis cluster as caching backends in a Django project.
The django setup for all three redis caching solutions are fairly similar, but note the differences in…
For multiple reasons, including security and historical tracking, it can be advantageous to record use login/logout activity for a service. In this post we’ll look at using signals to log such activity in a Django project.
To get started, let’s first understand logging configuration in Django. As a matter of best practice, it’s a good idea to setup multiple log files for specific activities. To do this, let’s add some logging configuration in the settings.py file. We’ll define a base log path, some formatting parameters, and set our loggers and handlers.
I like to set the log path as a…
Django signals provides a great means to launch an event in a given scenario, such as sending a message when a model instance is saved. However, there are situations that might necessitate the temporarily disabling of a signal, such as an issue with a downstream dependancy. The django-signalcontrol package was created with this in mind. With django-signalcontrol, signals with signal_control applied are automatically scanned when django starts, and can be disabled or enabled in the django admin interface, individually or in bulk.
In this post we’ll look at how django-signalcontrol works.
First let’s get django-signalcontrol installed; we can do this…
Adding default a value to a django foreign key is easier than you think.
There is sometimes debate on whether to use a ForeignKey or a CharField with choices in a model. One reason a CharField is often used is the ease of setting a default value. Adding a default value to a ForeignKey field is actually easier than you might think. In this post we’ll look at how this is done.
Let’s say we have a model to track actions. Each action will have a status and a result. …
In this tutorial we will setup, configure, and test a redis cluster using redis 5.0.6 on CentOS7.
We are working under the impression that our redis cluster will be spread across three separate regions (let’s call those us-west, us-central, and us-east),
and we want to ensure high availability and data redundancy.
For a redis cluster we need a minimum of three nodes. To provide failover and redundancy, each node needs a replica. For our cluster we’ll have a total of six individual nodes, with three primary-replica pairs, distributed across three regions. …
Automation Engineer & Django Fan