I’m looking at developing a hobby project of a very simple multi-user game. I’ve no interest at all in computer games, if the project progresses at all I’ll explain the motivation in a later post. However in thinking about how to go about it I came up with a short demo which illustrates how to set about using Python as an application server and JQuery to talk to it from a browser.
The code is here on GitHub.
Communication methods
I wanted to be able to make three different types of call from the client to the server.
- The most straightforward is to just retrieve a static webpage. The page would then have all the client side logic coded in JQuery, which I’ll come to later.
- The second is to make a request-response call to invoke some sort of application logic on the server side. For example this could be to retrieve static data or respond to user input.
- The third is to make asynchronous calls in both directions between the server and the browser. Broadcasting data like this from the server to the browser differs from the more traditional approach of the client pulling information.
In additional to the above I wanted the server state to be shared, regardless of which of the calling methods was used. So in effect you’d have a single application server and could choose the best way to communicate with it for a given task.
Implementation
Python was the obvious choice of server-side implementation given my experience and the faster development path compared to, say, C#. It is also very easy to create web servers in Python; in fact you can create a file server without any lines of code by just running python -m SimpleHTTPServer.
The standard way of achieving the second requirement, of request-response calls, is to use ajax to perform http GET calls. These are handled in a synchronous manner on the server.
The third requirement, of asynchronous communication, suggests a websocket solution. This led me to using Autobahn on top of the Twisted framework. The demo shows how a small amount of code in this set-up can be used to provide all three communication methods.
Demo
The screenshot below shows the output of the demo; un-interesting as it looks it illustrates the three communication methods.
There are two open browsers. Both are connected to the same address from the local machine, making use of the Python server for static file content.
The browser on the left connected first, then the user clicked the “Update Client Count” button. This made an ajax call to get the number of clients connected to the server. The right-hand browser followed suit, getting a different result now that there were two clients.
The bottom section shows the asynchronous web socket communication used to achieve a publish-subscribe pattern. The browser on the left sent a message to the server which was the broadcast back to both browsers, the right-hand browser then doing the same. The browser client didn’t need to ask the server for an update, it was pushed from the server.
Although this is a trivial example it’s easy to see how these different communication methods could be useful in real applications. For instance a trading blotter could make use of the file server to load a page, the ajax communication to response to user requests and the asynchronous broadcast to stream and react to price changes.
Client code
It’s probably more illustrative to look first at some of the client code in the html file.
The client code could be written in native javascript, but I always prefer to use JQuery, it’s more compact and makes cross-browser compatibly more likely. To this end I needed to include external javascript files for jquery itself and two extenstions: jquery.json and jquery.websocket.
The ajax call to get the number of connected clients looks like this.
function updateClientCount() {
$.ajax({
url: "http://localhost:9001/get_client_count",
dataType: "jsonp",
success: function(data) {
$("#client_count_div").html("There are " + data.client_count + " clients connected as of " + data.time + ".");
}});
}
This makes a call to the given url, expecting a json response with “client_code” and “time” in the payload. The html element “client_count_div”, not shown, is updated with the results.
Note the dataType is set to jsonp rather than json. This is because this is effectively a cross-site call because the website is served on one port while the call is to a url on another. Under the jquery covers this will add a “callback” attribute to the query which we will process on the server side.
The code for the websocket communication is below.
// Create the web socket
var ws = $.websocket("ws://localhost:9002/", {
events: {
// Respond to income data of type "chat".
chat: function(e) { $('#content').append(e.data + '
') }
}
});
// Send data to web socket using a type of "chat".
function sendToServer() {
ws.send('chat', $("#textToSend").val());
}
<input type="text" id="textToSend" style="width: 200px;"/><p/>
<button onclick="sendToServer()">Send Message</button><p/>
The websocket is created to point to the relevant end point. In doing so we set up a callback to respond to messages sent from the server. Each message sent between the client and server has a “type”. Here the callback is set to respond only to messages of type “chat”. On receiving a message the content will be appended to the html element with id “content”, not shown.
To send a message we just make a call on the web socket send method, giving it a type of “chat” and some data. This will be encoded into json before being sent to the server. The sendToServer function is wired to the relevant button on the page and the contents of the message is taken from the input text box.
Server code
I’ll try to show show the simplicity of achieving this via the Autobahn and Twisted frameworks on the server side.
The static file server will just return a file when it is requested, for example on http://localhost:9000/autobahn_test.html. This can be done with just the Twisted code below.
from twisted.web.server import Site from twisted.web.static import File from twisted.internet import reactor resource = File(r'.\html') factory = Site(resource) reactor.listenTCP(9000, factory) reactor.run()
This sets up a “site” using a file resource, in this case the files under the html directory. They are then served on port 9000.
The “reactor” is the server and is common to all the communication methods. In the code below for the other communication methods we add further listenTCP commands for the same reactor.
Partial code for the ajax server is below, I’ve omitted the “client tracker” class which I’ll come to later. This will respond to ajax calls made to a url such as http://localhost:9001/get_client_count.
from twisted.web.resource import Resource
from twisted.web.server import Site
import json
import time
class ClientCountResource(Resource):
def __init__(self, client_tracker):
self.client_tracker = client_tracker
def render_GET(self, request):
# Need to return as a callback.
callback = request.args['callback'][0]
return_payload = {'time' : time.ctime(),
'client_count' : len(self.client_tracker.clients)}
return "%s(%s)" % (callback, json.dumps(return_payload))
root = Resource()
root.putChild("get_client_count", ClientCountResource(client_tracker))
factory = Site(root)
reactor.listenTCP(9001, factory)
There’s obviously a bit more here but it’s still very compact for a fully working application server. A request is dealt with by a child of the Resource class. It implements the render_GET method to handle http GET requests. In this case the only argument passed in via the url parameters is the callback method, a requirement to deal with browsers’ same-origin policy. The returned data is the current time and the number of connected clients, which it can find from the client tracker object, which is shared with the async server. The payload is returned in json format together with the callback signature.
As you can see a couple of lines are used to wire this Resource up to a url, and it is then set to be serviced over port 9001 on the same reactor as before.
The asynchronous websocket code is below. The client will set up a websocket connection on port 9002 and we will then use this for two-way communication.
from autobahn.twisted.websocket import WebSocketServerFactory, WebSocketServerProtocol
from twisted.internet import reactor
# BroadcastServerProtocol deals with the async websocket client connection.
class BroadcastServerProtocol(WebSocketServerProtocol):
def onMessage(self, payload, isBinary):
if not isBinary:
in_data = json.loads(payload.decode('utf8'))
in_message = in_data['data']
return_message = 'Message from %s : %s' % (self.peer, in_message)
return_payload = json.dumps({'type': 'chat', 'data' : return_message})
self.factory.broadcast(return_payload)
def onOpen(self):
self.factory.register(self)
def connectionLost(self, reason):
WebSocketServerProtocol.connectionLost(self, reason)
self.factory.unregister(self)
# BroadcastServerFactory is the central websocket server side component shared between connections.
class BroadcastServerFactory(WebSocketServerFactory):
def __init__(self, client_tracker, debug=False, debugCodePaths=False):
WebSocketServerFactory.__init__(self, debug=debug, debugCodePaths=debugCodePaths)
self.client_tracker = client_tracker
def register(self, client):
self.client_tracker.register(client)
def unregister(self, client):
self.client_tracker.unregister(client)
def broadcast(self, msg):
for c in self.client_tracker.clients:
c.sendMessage(msg.encode('utf8'), isBinary = False)
# Helper to keep track of connections, accessed by the sync and async methods.
class ClientTracker:
def __init__(self):
self.clients = []
def register(self, client):
if client not in self.clients:
self.clients.append(client)
def unregister(self, client):
if client in self.clients:
self.clients.remove(client)
client_tracker = ClientTracker()
factory = BroadcastServerFactory(client_tracker)
factory.protocol = BroadcastServerProtocol
reactor.listenTCP(9002, factory)
There’s a bit more code here and I won’t go through every line. In summary though, we set up a WebSocketServerProtocol sub-class, objects of which act as the interface to a single client. There are a number of override methods to respond to events, with onMessage being the one called when a new message arrives.
We then have a WebSocketServerFactory sub-class. There is only one of these, shared among clients, so this is where the shared state can sit. In the broadcast method it makes a call back to the protocol class sendMessage method. This is where data is pushed out to the clients. The messages are sent and received in json format. Note in particular that the return has a “type” set to “chat”, important as the client is set only to respond to payloads of this type.
The factory, along with the corresponding protocol, are wired into the same reactor as before.
Conclusion
This illustrates how a JQuery client and a Python web server can be combined with minimal framework code to produce a working system with request-response and publish-subscribe capabilities. Furthermore, it’s straightforward to share server-side state, creating a flexible solution in which the preferred communication protocol can be used for a particular task.
References
These very useful Twisted introductory examples formed the basis of the file and ajax server code.
This project, reached from the Autobahn websocket examples, was a help in developing the publish-subscribe set-up.