Http and websockets logging handlers
Hello, this posts will be about 3 specific logging handlers: HTTPHandler, SocketHandler and DatagramHandler.
Let's start with HTTPHandler: reading python docs about HTTPHandler we can see that:
The HTTPHandler class, located in the logging.handlers module, supports sending logging messages to a Web server, using either GET or POST semantics.
So this will be useful to have such handler in case of many different modules in different machines that sends logs to one central server.
As an example, I will build simple flask application which prints out the logging message from the client.
To install Flask:
$ pip install Flask
Then make server.py:
from flask import Flask, request
app = Flask(__name__)
@app.route("/", methods=['POST', 'GET'])
for key, value in request.args.items():
return 'response' # it has to return something
if __name__ == "__main__":
To send some data, create script called send_log.py:
logger = logging.getLogger(__name__)
server = '127.0.0.1:5000'
path = '/'
method = 'GET'
sh = logging.handlers.HTTPHandler(server, path, method=method)
Now let's move to the SocketHandler: this is what python docs say about it
The SocketHandler class, located in the logging.handlers module, sends logging output to a network socket. The base class uses a TCP socket.
Based on this we can now guess that web socket will receive logging message.Then we can process it further. It will be useful when there is a lot of logs to be sent to the server. So opening HTTP connection every time is not a good solution.
So first we need some TCP server:
allow_reuse_address = True
def __init__(self, host='localhost',
socketserver.ThreadingTCPServer.__init__(self, (host, port), handler)
self.abort = 0
self.timeout = 1
self.logname = None
abort = 0
while not abort:
rd, wr, ex = select.select([self.socket.fileno()],
abort = self.abort
tcpserver = LogRecordSocketReceiver()
print('About to start TCP server...')
if __name__ == '__main__':
What is going on here? In the
main function we instantiate threading
TCP server and we serve it until we don't hit Ctrl+C. In the
serve_until_stopped method of
LogRecordSocketReceiver we are waiting
for the key combination to the stop server and if this not happening the
we retrieve information about the socket by
is a descriptor of a socket. Then we pass it to another function call:
select(). Select is system call for examining the status of
file descriptors of open input/output channels which in this case is
information from the socket. If there is anything ready to be read we
handle the request and process it.
To process it we need handler:
chunk = self.connection.recv(4)
if len(chunk) < 4:
slen = struct.unpack('>L', chunk)
chunk = self.connection.recv(slen)
while len(chunk) < slen:
chunk = chunk + self.connection.recv(slen - len(chunk))
obj = pickle.loads(chunk)
handle we read chunks of information from sent logging
message. The chunk is byte type so then we need to translate it to
python object by calling
Lastly, there is DatagramHandler which supports sending logging messages over UDP.
The actual code is very similar to SocketHandler:
def __init__(self, request, client_address, server):
msg, socket = self.request
Thanks to RooTer answer on stackoverflow I got this working by omitting first 4 bytes of data because they contain length of dumped object.
- 23.01.16 Thanks to RooTer answer I added UDP log handler