Starting a task at startup in Flask

Flask, run task at startup Flask includes a Python decorator which allows you to run a function before the first request from a user is processed. The problem I had is that the function doesn’t get run until after a user has visited a page for the first time. This article describes a way to solve that. If you haven’t heard of Flask before it’s a Python microframework for web applications. With Flask you can create small or large websites. While this article isn’t a getting started guide for Flask, it will include a complete working application.

The problem I was trying to solve

Recently I’ve been testing APIC-EM, and more specifically its Plug and Play application. PnP in the context of APIC-EM is the provisioning of new network devices. The API for APIC-EM is quite comprehensive, but the web application itself doesn’t allow for much customization. Instead of using the APIC-EM GUI, these kind of customizations can be done from an app written in Flask.

As I started to model the application in my head, I eventually ran into a problem. I would want the application to connect to the network inventory, create the device configs with Jinja2 templates, and I needed the Flask app to regularly poll the APIC-EM server and check if there any new unclaimed devices had reported in. When a new device is connected to the network it registers itself with the APIC-EM server. If no preexisting rule is defined for that device it gets classified as an unclaimed device.

In order to start the recurring job Flask has a decorator called before_first_request, or before_app_first_request if you are using a Flask Blueprint. A function decorated with before_first_request will trigger before the first request from a user is processed. Here is a simple Flask app to illustrate the concept where a decorated function is used to start a job in a new thread.

import threading
import time
from flask import Flask
app = Flask(__name__)

@app.before_first_request
def activate_job():
    def run_job():
        while True:
            print("Run recurring task")
            time.sleep(3)

    thread = threading.Thread(target=run_job)
    thread.start()

@app.route("/")
def hello():
    return "Hello World!"


if __name__ == "__main__":
    app.run()

The above code doesn’t do much, but the idea is that it should run a recurring task every three seconds. In the real application the print statement would be replaced by a function which contacts the APIC-EM server and the sleep would be longer.

The problem is that when we run the application nothing happens:

python run_flask.py
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

That is until someone visits the site for the first time after the application has loaded.

python run_flask.py
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Run recurring task
127.0.0.1 - - [27/Feb/2017 15:28:03] "GET / HTTP/1.1" 200 -
Run recurring task
Run recurring task

Here I’m faced with the problem. For this type of site it’s not certain that a user visits the site that often and it could take a long time after the application is restarted until the recurring job kicks in.

The solution

After thinking about different ways to solve it I figured that I could trigger the job to start from the application itself by starting another thread which tries to visit the site and only stops trying once the server has started. For this I used the requests module.

The new code looks like this:

import requests
import threading
import time
from flask import Flask
app = Flask(__name__)

@app.before_first_request
def activate_job():
    def run_job():
        while True:
            print("Run recurring task")
            time.sleep(3)

    thread = threading.Thread(target=run_job)
    thread.start()

@app.route("/")
def hello():
    return "Hello World!"


def start_runner():
    def start_loop():
        not_started = True
        while not_started:
            print('In start loop')
            try:
                r = requests.get('http://127.0.0.1:5000/')
                if r.status_code == 200:
                    print('Server started, quiting start_loop')
                    not_started = False
                print(r.status_code)
            except:
                print('Server not yet started')
            time.sleep(2)

    print('Started runner')
    thread = threading.Thread(target=start_loop)
    thread.start()

if __name__ == "__main__":
    start_runner()
    app.run()

Before the app starts I run the start_runner() function, within that function I start a new thread and fire up the start_loop() function. The loop will continue until it’s the Flask app is started and thereby trigger the activate_job() function decorated by before_first_request.

Running the new application, you can see that the loop fails to contact the server during the first try, then on the next run when the server is online and the recurring task is started.

python run_flask.py
Started runner
In start loop
Server not yet started
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
In start loop
Run recurring task
127.0.0.1 - - [27/Feb/2017 15:45:42] "GET / HTTP/1.1" 200 -
Server started, quiting start_loop
200
Run recurring task
Run recurring task
Run recurring task

Conclusion

This post might not help you much if you don’t use Flask. However it illustrates one of the things I really like about being able to write code. It feels so empowering to be able to just program your way around obstacles. So if you are considering to learn Python or any other programming language, don’t hesitate to start!