Django

Installing Rabbit MQ

Installing Erlang

Here is how to do it on 20.04

echo "deb https://packages.erlang-solutions.com/ubuntu focal contrib" | sudo tee /etc/apt/sources.list.d/rabbitmq.list
sudo apt update
sudo apt install erlang

If we get and error with the keys then, if you are sure, you can add the keys.

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXXXX

Install RabbitMQ Repository

sudo apt install apt-transport-https -y
wget -O- https://dl.bintray.com/rabbitmq/Keys/rabbitmq-release-signing-key.asc | sudo apt-key add -
wget -O- https://www.rabbitmq.com/rabbitmq-release-signing-key.asc | sudo apt-key add -
echo "deb https://dl.bintray.com/rabbitmq-erlang/debian focal erlang-22.x" | sudo tee /etc/apt/sources.list.d/rabbitmq.list
sudo apt update
sudo apt install rabbitmq-server
systemctl status  rabbitmq-server.service 
systemctl is-enabled rabbitmq-server.service

Install RabbitMQ Management

sudo rabbitmq-plugins enable rabbitmq_management
# ss -tunelp | grep 15672
tcp   LISTEN  0       128                    0.0.0.0:15672  
sudo ufw allow proto tcp from any to any port 5672,15672

Now create a user

rabbitmqctl add_user admin StrongPassword
rabbitmqctl set_user_tags admin administrator

We can see the interface at

http://server:15672

All of this was taken from the website https://computingforgeeks.com/how-to-install-latest-rabbitmq-server-on-ubuntu-linux/

Creating A Container With Composer

• Create a requirements.txt document with what is required for the container
• Create a Docker File definition
• Create a composer file

Requirements

This will differ from app to app below are examples for Django and Flask

Docker File

Again this will differ from app to app Here is a docker file to support Dango on for 8000

FROM python:3.9
ENV  PYTHONNUMBUFFERED 1
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt
COPY . /app 

CMD python manage.py runserver 0.0.0.0:8000

Composer File

Here is the composer file for Dango

FROM python:3.9
ENV  PYTHONNUMBUFFERED 1
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt
COPY . /app 

CMD python manage.py runserver 0.0.0.0:8000

Next write a yaml to install software

version: '3.8'
services: 
  django:
    container_name: djangoapp
    build: 
      context: .
      dockerfile: Dockerfile
    command: 'python manage.py runserver 0.0.0.0:8000'
    ports: 
      - 8000:8000
    volumes: 
      - .:/app
    depends_on:
      - db
    networks:
      django_demo:

  queue:
    build: 
      context: .
      dockerfile: Dockerfile
    container_name: djangoqueue
    command: 'python consumer.py'
    depends_on:
      - db
    networks:
      django_demo:

  db: 
    image: mysql:5.7.32
    container_name: djangodb
    restart: always
    environment: 
      MYSQL_DATABASE: admin
      MYSQL_USER: root
      MYSQL_PASSWORD: root
      MYSQL_ROOT_PASSWORD: root
    volumes: 
      - .dbdata:/var/lib/mysql
    ports: 
      - 33066:3306
    networks:
      django_demo:

networks:
  django_demo:
    external: true

Django

On the container you need to define the app

Setup

Requirements

Here are the specific requirements for the Django cotainer.

Django==3.1.5
djangorestframework==3.12.2
mysqlclient==2.0.3
django-mysql==3.10.0
django-cors-headers==3.6.0
pika==1.1.0

Django Docker File Specific

This command runs the django app at start up

CMD python manage.py runserver 0.0.0.0:8000

Django Docker Composer Specific

This just needs a service and a database defined. The port needs to be available. In the default above this was 8000

Create and Setup App

python manage.py startapp products

Django App Settings

We need to modify the following sections of settings.py the settings file for django

INSTALLED_APPS = [
...
    'django.contrib.staticfiles',
    'rest_framework',
    'corsheaders',
    'products'
]

MIDDLEWARE = [
...
    'django.contrib.sessions.middleware.SessionMiddleware',
    'corsheaders.middleware.CorsMiddleware',
...
]
...

# Database
# https://docs.djangoproject.com/en/3.1/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'admin',
        'USER': 'root',
        'PASSWORD': 'root',
        'HOST': 'db',
        'PORT': '3306'

    }
}
...
CORS_ALLOW_ALL_ORIGINS = True

Define the Website

Create Model Classes

Edit the model classes under the Product app

from django.db import models

# Create your models here.
class Product(models.Model):
    title = models.CharField(max_length=200)
    image = models.CharField(max_length=200)
    likes = models.PositiveIntegerField(default=0)

class User(models.Model):
    pass

Run the Migrations

This will create the tables. Log back into the container and type

python manage.py makemigrations
python manage.py migrate

Make Serializers

We make serializer for the product table with

from django.db import models
from rest_framework import serializers
from .models import Product

class ProductSerializer(serializers.ModelSerializer):
    class Meta:
        model = Product
        fields = '__all__'

Make Url (Routing)

In the main routing add the api route with an include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('api/', include('products.urls'))
]

In the app we can now add routes for the REST end points. Which are just maps to the views.

from django.urls import path

from .views import ProductViewSet

urlpatterns = [
    path('products', ProductViewSet.as_view({
        'get' : 'list',
        'post': 'create'
    })),
    path( 'products/<int:pk>', ProductViewSet.as_view({
        'get' : 'retreive',
        'put': 'update',
        'delete': 'destroy'
    }))
]

Make Views (Repository or Data Layer)

This is like the repository layer and provides the CRUD functions to support the routing.

from django.shortcuts import render

# Create your views here.
from rest_framework import viewsets, status
from rest_framework.response import Response
from rest_framework.serializers import Serializer

from .models import Product
from .serializer import ProductSerializer

class ProductViewSet(viewsets.ViewSet):

    def list(self, request): # /api/products/<str:id>
        products = Product.objects.all()
        serializer = ProductSerializer(products, many=True)
        return Response(serializer.data)

    def create(self, request): # /api/products
        serializer = ProductSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)
        serializer.save()
        return Response(serializer.data, status=status.HTTP_201_CREATED)

    def retreive(self, request, pk=None): # /api/products/<str:id>
        product = Product.objects.get(id=pk)
        serializer =ProductSerializer(product)
        return Response(serializer.data)

    def update(self, request, pk=None): # /api/products/<str:id>
        product = Product.objects.get(id=pk)
        serializer =ProductSerializer(instance=product, data=request.data)
        serializer.is_valid(raise_exception=True)
        serializer.save()
        return Response(serializer.data, status=status.HTTP_202_ACCEPTED)

    def destroy(self, request, pk=None): # /api/products/<str:id>
        product = Product.objects.get(id=pk)
        product.delete()
        return Response(status=status.HTTP_204_NO_CONTENT)

class UserAPIView(APIView):
    def get(self, _):
        users = User.objects.all()
        user = random.choice(users)
        return Response({
            'id': user.id
        })

Flask

Setup

Requirement

Flask==1.1.2
Flask-SQLAlchemy==2.4.4
SQLAlchemy==1.3.20
Flask-Migrate==2.5.3
Flask-Script==2.0.6
Flask-Cors==3.0.9
requests==2.25.0
mysqlclient==2.0.3
pika==1.1.0

Flask Docker File Specific

For Flask this is the start command in the container

CMD python main.py

Flask Docker Composer Specific

In the composer file we need to make sure the port Flask uses is unique e.g.

services: 
  backend:
    build: 
      context: .
      dockerfile: Dockerfile
    ports: 
      - 8001:5000
    volumes: 
      - .:/app
    depends_on:
      - db

Create App

Create an empty app

from flask import Flask
app = Flask(__name__)

@app.route('/')
def index():
    return 'Hello'

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

Define App

Create the Model Classes

Here we define the model

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import UniqueConstraint

from flask_cors import CORS

app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://root:root@db/main'
CORS(app)

db = SQLAlchemy(app)

class Product(db.Model):
    id = db.Column(db.Integer, primary_key=True, autoincrement=False)
    title = db.Column(db.String(200))
    image = db.Column(db.String(200))


class ProductUser(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    user_id = db.Column(db.Integer)
    product_id = db.Column(db.Integer)

    UniqueConstraint('user_id', 'product_id', name='user_product_unique')

@app.route('/')
def index():
    return 'Hello'

if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

Run the Migrations

We create the tables by creating a script e.g. manager.py

from main import app, db
from flask_migrate import Migrate, MigrateCommand
from flask_script import Manager

migrate = Migrate(app, db)

manager = Manager(app)
manager.add_command('db', MigrateCommand)

if __name__ == '__main__':
    manager.run()

And run the script using the script

python manager.py db init 
python manager.py db migrate
python manager.py db upgrade

Rabbit MQ

Example Consumer

import pika

params = pika.URLParameters('amqp://test123:PASS@heaven.com:5672/%2f')

connection = pika.BlockingConnection(params)

channel = connection.channel()

channel.queue_declare(queue='admin')

def callback(ch, method, properties, body):
    print('Received in admin')
    print(body)

channel.basic_consume(queue='admin', on_message_callback=callback, auto_ack=True)

print('Started consuming')

channel.start_consuming()

channel.close()

Example Producer

And here is a producer

import pika

params = pika.URLParameters('amqp://test:test@192.168.1.70:5672/%2f')

connection = pika.BlockingConnection(params)

channel = connection.channel() 

def publish():
    pass
    channel.basic_publish(exchange='',routing_key='admin',body='hello')

Adding Queue to Container

Simply add another service to the Django container.

  queue:
    build: 
      context: .
      dockerfile: Dockerfile
    command: 'python consumer.py'
    depends_on:
      - db

Application Events

Publishing Event

Next we can change the events to publish. First we change the publish function in the producer to add the method and the body to message using json.

def publish(method, body):
    print('Sending from django to flask')
    properties = pika.BasicProperties(method)
    channel.basic_publish(
        exchange='',
        routing_key='main',
        body=json.dumps(body), 
        properties=properties)

We now need to go to the methods to produce these events.

    def create(self, request): # /api/products
...
        publish('product_created',serializer.data)
        return Response(serializer.data, status=status.HTTP_201_CREATED)
...
    def update(self, request, pk=None): # /api/products/<str:id>
...
        publish('product_updated',serializer.data)
        return Response(serializer.data, status=status.HTTP_202_ACCEPTED)

    def destroy(self, request, pk=None): # /api/products/<str:id>
...
        publish('product_deleted', pk)
        return Response(status=status.HTTP_204_NO_CONTENT)

Consuming Events

In the flask app we now need to act on the messages being sent. There is one action for each of the CRUD operations.

import pika, json

from main import Product, db

params = pika.URLParameters('amqp://test:test@192.168.1.70:5672/%2f')

connection = pika.BlockingConnection(params)

channel = connection.channel()

channel.queue_declare(queue='main')

def callback(ch, method, properties, body):
    print('Received in admin')
    data=json.loads(body)
    print(body)

    if properties.content_type == 'product_created':
        product = Product(id=data['id'], title=data['title'], image=data['image'])
        db.session.add(product)
        db.session.commit()
        print('product created')

    elif properties.content_type == 'product_updated':
        product = Product.query.get(data['id'])
        product.title = data['title']
        product.image = data['image']
        db.session.commit()
        print('product updated')

    elif properties.content_type == 'product_deleted':
        product = Product.query.get(data)
        db.session.delete(product)
        db.session.commit()
        print('product deleted')

channel.basic_consume(queue='main', on_message_callback=callback, auto_ack=True)

print('Started consuming')

channel.start_consuming()

channel.close()

Flask App 2

Introduction

We now need to add the endpoints to Flask

Change the classes to support Json

We need to add the dataclass decorator and add the types to the class and set the end points

from flask import Flask, jsonify
from dataclasses import dataclass
...
@dataclass
class Product(db.Model):
    id: int
    title: str
    image: str

    id = db.Column(db.Integer, primary_key=True, autoincrement=False)
    title = db.Column(db.String(200))
    image = db.Column(db.String(200))
....
@app.route('/api/products')
def index():
    return jsonify(Product.query.all())

Useful Commands

Login to Container

Login to the container with

docker-compose exec backend bash

Build Container

Login to the container with

docker-compose up --build