Flask Api

Flask-api is a small API project for creating users and files (Microsoft Word and PDF). These files contain data about users registered in the project.

The project is developed in Python 3.7 and use next main libraries:

  • Flask: microframework.

  • Sqlite: SQL database engine.

  • Peewee: simple and small ORM.

  • Celery: asynchronous task queue/job.

  • RabbitMQ: message broker.

  • NGINX: web server, reverse proxy, etc.

  • UWSGI: Web Server Gateway Interface (WSGI) server implementation.

  • Flower: monitoring and administrating Celery clusters.

  • Supervisor: client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

Installation

1. Linux packages

These packages are required for the project installation:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install autoconf build-essential cmake libcap-dev libffi-dev libpcre3-dev librabbitmq-dev libreoffice-writer libtool libxml2-dev libxslt1-dev libxslt1.1 pkg-config magic nginx python3-distutils python3.7 python3.7-dev python3.7-venv rabbitmq-server uuid-dev uwsgi uwsgi-src
sudo reboot

2. RabbitMQ configuration

Required plugins for monitoring our brokers in Flower:

sudo rabbitmq-plugins enable rabbitmq_management
sudo service rabbitmq-server restart

3. Python dependencies

Install Python dependencies:

python3.7 -m venv venv
source venv/bin/activate
pip install -r requirements.txt --no-cache-dir

4. Domain configuration

Add local domain to our /etc/hosts file:

127.0.0.1 flask-api.prod

5. Environment configuration

Create a new .env file based on .env.example file.

6. uWSGI configuration

Create a new uwsgi.ini file based on uwsgi.ini.example.

username and project_path must to be filled with appropiate values.

www-data group must to be added to your user:
sudo usermod -a -G www-data username

7. Nginx configuration

Create a new flask_api file based on docs/examples/flask_api.nginx.example file.

Replace uwsgi_pass variable with the value in socket variable from uwsgi.ini file.

Move flask_api file to /etc/nginx/sites-available directory:
sudo mv docs/examples/flask_api /etc/nginx/sites-available
sudo ln -s /etc/nginx/sites-available/flask_api /etc/nginx/sites-enabled/flask_api
sudo systemctl restart nginx

8. Supervisor configuration

8.1 Main configuration

Create a new supervisord.conf file based on docs/examples/supervisor/supervisord.conf.example file in the root project.

command, directory and username variables must to be filles with appropriate values. These variables are below Mr Developer comment.

8.2 Other configurations

Create a new directory named supervisor in the root path and create next files based on docs/examples/supervisor example files:

  1. celery.conf

  2. flower.conf

  3. uwsgi.conf

username and path variables must to be replaced with appropriate values.

9. Log directories

Create next log directories:

  1. log/app

  2. log/celery

  3. log/flower

  4. log/uwsgi

10. Supervisor systemd unit file

Create a new flask_api_supervisor.service file based on docs/examples/flask_api_supervisor.service.example file.

username, usergroup and path variables must to be filled with appropiate values.

Move file to /etc/systemd/system directory and we run next commands:
sudo systemctl enable flask_api_supervisor.service
sudo systemctl daemon-reload
sudo systemctl start flask_api_supervisor.service
The systemd unit file start up the project if the system is reboot or shutdown.

For checking process status in command line:
sudo systemctl status flask_api_supervisor.service

For restart all processes in command line:

sudo systemctl restart flask_api_supervisor.service

This command reread the supervisor configuration files, stop all processes and start them again.

How to usage

The setup is finished, we only need to create the database tables and fill them with fake data. We open a terminal in the root project and run next commands:

./venv/bin/flask init-db
./venv/bin/flask migrate
./venv/bin/flask seed
You can use an API client such as Insomnia or Postman and starting to consume the API!

You can see the processes status here: http://flask-api.prod/supervisor

The credentials are user:123 by default, you can change the credentials in supervisord.conf file in inet_http_server section.

You can management the Celery tasks status here: http://flask-api.prod:5555/flower/

Optional installation

This project use logrotate for logging configuration. The config file is already defined you only need to do these steps:

  1. Create new flask_api.logrotate file based on docs/examples/flask_api.logrotate.example.

  2. path, username and usergroup variables must to be filled with appropiate values.

  3. Move flask_api_logrotate to /etc/logrotate.d:

sudo mv docs/examples/flask_api.logrotate /etc/logrotate.d

  1. Restart logrotate service:

sudo service log rotate restart

A new log file will be created every day.

Skeleton app structure

The project structure looks like this:

flask_api
├── /app
│   ├── /blueprints
│   │   └── ...
│   ├── /celery
│   │   └── ...
│   ├── /models
│   │   └── ...
│   ├── /templates
│   │   └── ...
│   ├── /utils
│   │   └── ...
│   ├── __init__.py
│   ├── extensions.py
│   └── middleware.py
├── /database
│   ├── factories
│   │   └── ...
│   ├── migrations
│   │   └── ...
│   ├── seeds
│   │   └── ...
│   └── __init__.py
├── /log
│   └── ...
├── /storage
│   └── ...
├── /tests
│   └── ...
├── config.py
├── manage.py
└── requirements.txt

app

Package for building a Flask application.

database

Package for managing the database.

tests

Package for testing the application.

config

Module loads the application's configuration.

Flask command line

Flask command line allow run scripts for managing database, start up task queues, etc.

You don’t need to start up the server for running these scripts but you must activate your virtual environment.

Changelog

All notable changes to this project will be documented in this file. See standard-version for commit guidelines.

2.0.6 (2021-12-28)

Bug Fixes

  • pip: upgrade package versions (75aff3b)

2.0.5 (2021-12-27)

Bug Fixes

  • npm: upgrade package versions (f62f335)

2.0.4 (2021-02-19)

Bug Fixes

  • users: add new fs_uniquifier column on users table (a28be38)

Build System

  • npm: update node package versions (6314da3)

  • pip: update flask-security-too version to 4.0.0 (698acb6)

  • pip: update python package versions (09755c8)

2.0.3 (2021-02-05)

Bug Fixes

  • celery: task fields are saving on database when the task is finished (bc93505)

Code Refactoring

  • add new mockups directory in storage directory (91e642e)

  • tls and ssl flask mail are getting from environment file (e548d57)

2.0.2 (2021-01-23)

Bug Fixes

  • celery: new settings name are not applies on celery 4.4.7 (f1a0bff)

  • middleware: correct middleware for allowing to let in Swagger (441b077)

  • tasks: correct task for sending an email with attachments (43d516c)

Build System

  • pip: update python dependencies (8964eed)

Code Refactoring

  • blueprints: update way to register all blueprints (b7692b2)

  • update way to test Celery tasks (eb3f72d)

  • celery: update way to run Celery (dee133d)

  • factories: update way to create a model instance from a factory (ddfcb7d)

  • swagger: update order field format (ccbfcda)

  • tasks: exclude internal_filename in Celery tasks (53dcf6d)

2.0.1 (2021-01-05)

Bug Fixes

  • tasks: correct output data on serializers and swagger (d5c050a)

Build System

  • npm: update npm dependencies for fixing a vulnerability found (15ee5de)

  • pip: update Python packages (11676b6)

Code Refactoring

  • blueprints: import dynamic way (30ee0a0)

  • blueprints: remove logging module from all blueprints (a51709e)

  • db: models available could be import from init module (2361e0e)

  • db: move user_roles model to its own module (650eb39)

  • exceptions: add error handler to marshmallow validation error class (10bb664)

  • exceptions: correct error handler to marshmallow validation error class (ba30805)

  • managers: add new logic for managing database queries through database models (758e077)

  • serializers: correct user email validation (2d733cf)

  • serializers: move app.marshmallow_schema.py to serializers package (47f75e1)

  • serializers: upgrade validation fields (24ac198)

  • services: add new logic for managing business logic to auth (3c969b5)

  • services: add new logic for managing business logic to documents (1ce7b96)

  • services: add new logic for managing business logic to roles (7fff4c1)

  • services: add new logic for managing business logic to tasks (079741d)

  • services: add new logic for managing business logic to users (572f37f)

  • swagger: move app.utils.swagger_models to app.swagger package (ef4dd87)

  • swagger: update document swagger models (7015c61)

  • swagger: update role swagger models (d618021)

  • swagger: update user swagger models (766e697)

  • tasks: add suffix to task names (998a56d)

  • utils: create new modules to constants and request query operators (cbf12c9)

2.0.0 (2020-10-27)

⚠ BREAKING CHANGES

  • order field in search requests is a list of dicts.

Features

  • factories: add prevent code for checking if a given model is registered as factory (1605a01)

  • shell: import Factory class to Flask interactive shell (8cf9c8e)

Code Refactoring

  • replace cerberus to flask-mashmallow validation (7552d5c)

  • celery: replace old Celery setting names with new ones (15e0c03)

  • celery: update way to set FLASK_CONFIG value on Flask command (a865548)

Build System

  • add .versionrc that shows build/perf/refactor/revert (0017d66)

  • add sphinx-click configuration to Sphinx and create new file for showing Click documentation (bb41b7b)

  • add sphinx-click for showing Click documentation in Sphinx (c0a8f55)

  • pip: remove cerberus package (4a4fe72)

  • pip: split python packages in two requirements local and production (f39a2a5)

1.4.1 (2020-10-07)

Bug Fixes

  • celery: correct problem when start Celery (52ad2fb), closes #3

1.4.0 (2020-10-04)

Features

  • celery: add task for exporting several files (ca8355f)

  • documentation: add sphinx integration (8c313fd)

1.3.0 (2020-09-20)

Features

  • swagger: add Swagger full integration (1eaf8d8)

1.2.0 (2020-09-18)

⚠ BREAKING CHANGES

  • install/update Node.js and Python libraries

build

  • update Node.js and Python packages (b7416cc)

1.1.0 (2020-05-31)

Features

  • security: add role-based authorization (345b57e)

  • add advanced search in documents, roles and users (8fce3e3)

  • add marshmallow package integration (a8b647e)

  • add Swagger integration (dc6ace4)

Refactor

  • replace HTTP exceptions to Werkzeug HTTP Exceptions (31e5606)

  • move Word and Excel celery tasks to them own modules (00e42e5)

Docs

  • docs: add installation project guide (b915d31)

1.0.0 (2020-05-17)

⚠ BREAKING CHANGES

  • pip: update Python dependencies

Features

  • celery: add basic installation (147dd2c)

  • db: add pewee migrations (231696c)

  • documents: add document logic (1eb7ec1)

  • emails: add send emails after creation an user (7c2cfe0)

  • log: add support for logrotate (09925e1)

  • users: add created_by column in user model (8a3d013)

  • users: add Excel and PDF users export to background processes (781e091)

  • users: add recovery password feature (e1e916e)

Build

  • pip: update requirements.txt (6193153)

0.8.0 (2020-04-29)

Features

  • roles: add role logic (d7a0535)

  • security: add jwt authentication (fb51089)

  • users: add role model integration to user model (69bc124)

  • users: add user get endpoint (018b965)

0.7.0 (2020-04-23)

Features

  • doc: add standard-version NodeJS package (c1b2cb3)

0.6.1 (2020-04-23)

⚠ BREAKING CHANGES

  • update python dependencies

Features

  • db: added script for creating database tables (c14b566)

  • logging: added logging configuration (297b9c3)

  • seeders: added user seeder (e78b4c4)

  • tests: add tests and code coverage (17317b7)

  • validation-requests: add validation requests with cerberus (a5beed6)

Bug Fixes

  • commitizen: fixed problem with the process of commitizen tags (1d3677d)

  • docs_to_pdf: fixed problem about convert a docx file to a pdf file with uWSGI (aabbc2d)

  • peewee: fixed problem about a connection already opened error (6279470)

  • peewee: problem with database connection already opened (e6c07c9)

  • users: update user endpoint cannot update data (9dfc4cc)

  • request search fields in search users, export PDF and export Excel endpoints (2ae7ab7)

Build

Note

If you find any bugs, odd behavior, or have an idea for a new feature please don’t hesitate to open an issue on GitHub.

Indices and tables