Http and websockets logging handlers

Posted on Sat 16 January 2016

Hello, this posts will be about 3 specific logging handlers: HTTPHandler, SocketHandler and DatagramHandler.


Let's start with HTTPHandler: reading python docs about HTTPHandler we can see that:

The HTTPHandler class, located in the logging.handlers module, supports sending logging messages to a Web server, using either GET or POST semantics.

So this will be useful to have such handler in case of many different modules in different machines that sends logs to one central server.

As an example, I will build simple flask application which prints out the logging message from the client.

To install Flask:

$ pip install Flask

Then make

from flask import Flask, request

app = Flask(__name__)

@app.route("/", methods=['POST', 'GET'])
def hello():
for key, value in request.args.items():
return 'response' # it has to return something

if __name__ == "__main__":

To send some data, create script called

import logging
import logging.handlers

logger = logging.getLogger(__name__)

server = ''
path = '/'
method = 'GET'

sh = logging.handlers.HTTPHandler(server, path, method=method)


logger.debug("Test message.")

Running and will result in this output:


Now let's move to the SocketHandler: this is what python docs say about it

The SocketHandler class, located in the logging.handlers module, sends logging output to a network socket. The base class uses a TCP socket.

Based on this we can now guess that web socket will receive logging message.Then we can process it further. It will be useful when there is a lot of logs to be sent to the server. So opening HTTP connection every time is not a good solution.

So first we need some TCP server:

class LogRecordSocketReceiver(socketserver.ThreadingTCPServer):
    allow_reuse_address = True

    def __init__(self, host='localhost',
        socketserver.ThreadingTCPServer.__init__(self, (host, port), handler)
        self.abort = 0
        self.timeout = 1
        self.logname = None

    def serve_until_stopped(self):
        import select
        abort = 0
        while not abort:
            rd, wr, ex =[self.socket.fileno()],
                                       [], [],
            if rd:
            abort = self.abort

def main():
    tcpserver = LogRecordSocketReceiver()
    print('About to start TCP server...')

if __name__ == '__main__':

What is going on here? In the main function we instantiate threading TCP server and we serve it until we don't hit Ctrl+C. In the serve_until_stopped method of LogRecordSocketReceiver we are waiting for the key combination to the stop server and if this not happening the we retrieve information about the socket by self.socket.fileno() which is a descriptor of a socket. Then we pass it to another function call: this time select(). Select is system call for examining the status of file descriptors of open input/output channels which in this case is information from the socket. If there is anything ready to be read we handle the request and process it.

To process it we need handler:

class LogRecordStreamHandler(socketserver.StreamRequestHandler):

    def handle(self):
        while True:
            chunk = self.connection.recv(4)
            if len(chunk) < 4:
            slen = struct.unpack('>L', chunk)[0]
            chunk = self.connection.recv(slen)
            while len(chunk) < slen:
                chunk = chunk + self.connection.recv(slen - len(chunk))
            obj = pickle.loads(chunk)

In method handle we read chunks of information from sent logging message. The chunk is byte type so then we need to translate it to python object by calling pickle.loads().

So running our TCP server and sending log looks like this:

Full gist with socket logger looks like this:

And with socket sender:


Lastly, there is DatagramHandler which supports sending logging messages over UDP.

The actual code is very similar to SocketHandler:

class MyUDPHandler(socketserver.BaseRequestHandler):
    def __init__(self, request, client_address, server):
        socketserver.BaseRequestHandler.__init__(self, request,

    def handle(self):
        msg, socket = self.request
        print("{} wrote:".format(self.client_address[0]))
        socket.sendto(msg.upper(), self.client_address)

Thanks to RooTer answer on stackoverflow I got this working by omitting first 4 bytes of data because they contain length of dumped object.

So the full upd_server looks as follows:

Which works like this:


  • Socket server and log sender based on python logging cookbook


  • 23.01.16 Thanks to RooTer answer I added UDP log handler

Cover image by Quinn Dombrowski under CC BY 2.0.

tags: python, logging,

Comments !