Cleaning Your Django Project With PyLint And Buildbot

There are a number of tools for checking whether your Python code meets a coding standard. These include pep8.py, PyChecker and PyLint. Of these, PyLint is the most comprehensive and is the tool which I prefer to use as part of my buildbot checks that run on every commit.

PyLint works by parsing the Python source code itself and checking things like using variables that aren’t defined, missing doc strings and a large array of other checks. A downside of PyLint’s comprehensiveness is that it runs the risk of generating false positives. As it parses the source code itself it struggles with some of Python’s more dynamic features, in particular metaclasses, which, unfortunately, are a key part of Django. In this post I’ll go through the changes I make to the standard PyLint settings to make it more compatible with Django.

disable=W0403,W0232,E1101

This line disables a few problems that are picked up entirely. W0403 stops relative imports from generating a warning, whether you want to disable these or not is really a matter of personal preference. Although I appreciate why there is a check for this, I think this is a bit too picky. W0232 stops a warning appearing when a class has no __init__ method. Django models will produce this warning, but because they’re metaclasses there is nothing wrong with them. Finally, E1101 is generated if you access a member variable that doesn’t exist. Accessing members such as id or objects on a model will trigger this, so it’s simplest just to disable the check.

Communicating With Stakeholders

Last week I had a discussion with my boss about the best way to communicate with my stakeholders about the progress of any work that they have asked my team to do. The question basically became how much information should be communicated during a project. The requirements and delivery phases obviously require close communication, but what is appropriate while the developers are hard at working, churning out code? If, like me, you are required to work a number of disparate departments then the people in the departments may want to know what work is currently on your plate before they ask you to do something. What’s the best way to keep a status board up to date?

Traditionally we have had a Bugzilla installation which was used to store a record of almost every change we made. A subversion post commit hook allows us to link every commit back to a piece of work in Bugzilla. This works well for coding in an issue-driven-development style, but does result in Bugzilla sending a lot of emails. Many of which are completely irrelevant to people outside of IT. Indeed even people inside IT, but who aren’t directly linked to that piece of work, don’t need to be informed by email of every checkin.

Recently we have begun to experiment with FogBugz. While similar to Bugzilla it has a number of subtle differences. Firstly FogBugz is designed to be used in a helpdesk environment so it provides the ability to communicate both within the team and with external stakeholders from the same interface. This gives you the ability to communicate on two different levels, with all the communication still being tracked. The second difference is that FogBugz is not known amongst the people outside of IT. Not all stakeholder know about Bugzilla, but some do, and some can even search and reply using the webinterface. With FogBugz we’ll take this ability away as, at least initially, only a limited number of IT people will have access to the cases.

Unittesting QSyntaxHighlighter

I’m using test driven development while building my pet project, DjangoDE. A key part of an IDE is the syntax highlighting of the code in the editor, so that’s one area where I’ve been trying to build up the test suite.

To test the syntax highlighter the obvious approach is to send the right events to write some code into the editor the check the colour of the text. Although the QT documentation is usually excellent, it doesn’t go into enough detail on the implementation of the syntax highlighting framework to enable you to query the colour of the text. In this post I’ll explain how the colour of text is stored, and how you can query it.

A syntax highlighting editor is normally implemented using a QPlainTextEdit widget. This object provides the user interface to the editor and manages the display of the text. The widget contains a QTextDocument instance, which stores the text. To add syntax highlighting you derive a class from QSyntaxHighlighter then instantiate it, passing the document instance as the parameter to the constructor. This is explained in detail in the syntax highlighter example.

The document stores the text as a sequence of QTextBlock objects. These store the text as well as the formatting information used to display it. You might think that you can just call QTextBlock::charFormat to get the colour of the text. Unfortunately it’s not that simple as the format returned by that call is the colour that you’ve explicitly set, not the syntax highlight colour.

Government And Website Launches

On Tuesday the UK Police service launched a new website, with much fanfair, that allows you to see the crimes that have happened near where you live. This is part of the Government’s commitment to make data much more freely available than it has been in the past.

People are nosy. There’s no way around the fact that we just love to see what our neighbours are doing and to investigate the area around us. Every time a website like the crime maps, a new census, or noise maps launches the same sequence of events happen. The are news stories on TV, on radio and online and then the website crashes. We end up with stories like this and people will, in general, never go back.

If you investigate where this new site is hosted you’ll find it’s on RackSpace, who offer a cloud hosting service so really there’s no reason for the site to go down. Although the site is interactive there is no user generated data so the site should be easily scalable.

The site cost £300,000 to develop, and although 75,000 hits a minute is a lot surely will a little bit of thought the site should have scaled fairly easily?

DjangoDE 0.1 Released

I spend most of time at work building websites in Django. My editor of choice until now has been Kate with Chromium open on another screen. Most of my co-workers use VIM inside a PuTTY session to do their editing. With multicore machines with gigabytes of RAM, surely there’s a better way?

I investigated the current state of Django IDEs and came to conclusion that none of them are that great. Most are a plugin to a giant IDE that tries accommodate many different languages, so each feels like second best, or they are designed more for traditionally programming and not webdevelopment so they don’t integrate with Django’s in built server and don’t support editing Javascript, or provide a built in webbrowser. I also don’t want to have to pay for the editor, which limits my choice even more.

Having decided that none of the existing IDEs quite fit my requirements I did what any self respecting open source programmer with an itch would do, I headed to Google Code, created a new project and then got coding.