I’ve been snatching 30 and 60 minute blocks of time in the evenings and on the weekends to get back to work on my Radar test in Python with PyGame from nearly four years ago. PyGame 1.9.1 installed easily on my Windows box and OS X laptop and integrated well with Python 3.1 (on Windows) and Stackless Python 2.6 (on OS X). The only significant changes to be made in the code between Python 2 and Python 3 had to do with exception handling.
(more…)

Here are the steps required to set up a development environment on Windows for my Python project:

1. Download the required programs and libraries
   • Komodo Edit for best-of-breed Python editing
   • Oracle Berekely DB 4.7.25 for the storage engine
   • Stackless Python 2.6 (I’ll eventually convert to 3.x)
   • setuptools-0.6c9.tar.gz to install Python eggs
   • bsddb3-4.7.6.zip
   • If you don’t have a tar.gz expander, 7-Zip
   • If you don’t have a C++ compiler, or yours is out of date, get Visual C++ Express 2009
2. Install Visual C++ Express
3. Install Komodo Edit
4. Install Stackless Python 2.6
5. Install Berkeley DB 4.7
6. Expand bsddb3-4.7.6.zip to a known directory
7. Install setuptools and PyBSDDB
   1. Expand setuptools into a temporary directory
   2. Open a command line (on Vista, Windows-R and enter cmd)
   3. Add Python to the command line environment with:
      set PATH=%PATH%;C:\Python26
   4. Change directory to the temporary directory where you expanded setuptools
   5. Install setuptools with:
      python setup.py install
   6. Install PyBSDDB with:
      
cd \Users\Chris\Temp\bsddb3-4.7.6 (where I expanded bsdb4)

mkdir db
cd db
xcopy /e /v "C:\Program Files\Oracle\Berkeley DB 4.7.25\include" include\
xcopy /e /v "C:\Program Files\Oracle\Berkeley DB 4.7.25\lib" lib\
cd ..
"C:\Program Files\Microsoft Visual Studio 9\VC\vcvarsall.bat"
python setup.py install

. . . time for bed, but I’m having a little trouble with MSVC:

snip
Modules/_bsddb.c(8358) : warning C4996: 'strncpy': This function or variable may be unsafe. Consider using strncpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
F:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(157) :
see declaration of 'strncpy'
warning: I don't know what to do with 'runtime_library_dirs': ['db/lib']
error: don't know how to set runtime library search path for MSVC++


Yeah, I’m probably nuts for trying to compile this myself. On OS X or Linux, there are still plenty of steps but no major hurdles.

I’ve been working on a personal, side-project for about a week. This morning, I finally got transactions working. I need to learn a little more about BDB tuning and I’m not planning on releasing my code until I’m happier with it and it’s been tested more thoroughly.

Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Configure
>>> cfg = Configure.Configure()
>>> class Alpha:
... def __init__(self):
... self.field = 'alpha'
...
>>> cfg.databases['objects'][1].get('a')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "RefCache.py", line 110, in get
value = self.repository.get(key)
File "Repository.py", line 104, in get
raise KeyError, key
KeyError: 'a'
>>> txn = cfg.databases['objects'][1].beginTransaction()
>>> a = Alpha()
>>> cfg.databases['objects'][1].put('a', a, txn)
>>> cfg.databases['objects'][1].commitTransaction(txn)
>>> a.field
'alpha'
>>> a = None
>>> a
>>> a = cfg.databases['objects'][1].get('a')
>>> a.field
'alpha'
>>> b = Alpha()
>>> b.field = 'beta'
>>> txn = cfg.databases['objects'][1].beginTransaction()
>>> cfg.databases['objects'][1].put('b', b, txn)
>>> cfg.databases['objects'][1].rollbackTransaction(txn)
>>> cfg.shutdown()
>>> ^D
 
$ python
Python 2.5.1 (r251:54863, Apr 15 2008, 22:57:26)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Configure
>>> cfg = Configure.Configure()
>>> class Alpha:
... def __init__(self):
... self.field = 'alpha'
...
>>> b = cfg.databases['objects'][1].get('b')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "RefCache.py", line 110, in get
value = self.repository.get(key)
File "Repository.py", line 104, in get
raise KeyError, key
KeyError: 'b'
>>> a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>> a = cfg.databases['objects'][1].get('a')
>>> a.field
'alpha'
>>> a = None
>>> a
>>> cfg.shutdown()
>>> ^D
 
$ db_stat -e -h db
Fri Sep 26 07:35:53 2008 Local time
0x120897 Magic number
0 Panic value
4.7.25 Environment version
...trimmed...


I have a working, end-to-end XMLHTTPUpdate handling framework written in Python running as a CGI application. Ewww.

Download the framework here. It’s simple and probably has some bugs in it.

You can test it as well, but don’t expect it to work under Internet Explorer. I didn’t bother writing Javascript for IE, only Firefox (which means it will also run under Safari and on an iPhone). If you try it out, give it a couple seconds to fetch the time from the server.

There are obvious points of extension, starting from the Handlers.py file. I chose to explicitly check handler names rather than use them to dynamically call into code from a string passed through the update. It just seems safer.

I figured out what I was doing wrong. Nothing except for how I chose to echo or display the XML.

Since it’s a dynamic call, I can’t view the page source to see the XML that gets returned. And I never verified that I was getting a String: I assumed that because it was inside a CDATA block that the tags would be visible, but what I actually got was the tags being evaluated, and since ^<get><time /></get> isn’t part of XHTML 1.1 . . . my XML never appeared.

So in the interest of saving some future developer a few minutes or hours or days of trouble, if you’re trying to parse POST-ed XMLHttpRequest XML DOM objects in a Python CGI web application and it looks like STDIN is empty or you’re not getting the POST contents, verify that you’re reading something by checking the length and extracting a part of the string. And convert your tags.
(more…)

Now, I’m not entirely stupid. Lacking common sense? Yes. Forgetful of dates, appointments, anniversaries, bills? Yes. Easily confused? Absolutely.

But I’ve also been programming in Python for years and wrote my first CGI scripts in 1993 or 1994 or thereabouts. I’m kind of familiar with both environments, even if this is the first time I’ve actually put them together. Everything seemed to be meshing really, really well until I tried to get XML content from the web client through a DOM document passed into the XMLHttpRequest. (And yes, I am new to XMLHttpRequest dynamic app development. I’d always had the pleasure of using someone else’s library.)


function createDocGetTime() {
    docGetTime = document.implementation.createDocument("", "", null);
    var getNode = docGetTime.createElement("get");
    var getTime = docGetTime.createElement("time");
    getNode.appendChild(getTime);
    docGetTime.appendChild(getNode);
}
 
function updateTime() {
    timerClient = new XMLHttpRequest();
    timerClient.onreadystatechange = timerHandler;
    timerClient.open("POST", "xmltimer.py");
    timerClient.setRequestHeader("Content-Type", "text/xml;charset=UTF-8");
    timerClient.send(docGetTime);
    timerId = setTimeout("updateTime()", 1500);
}


Evil code (there’s no way to stop calling updateTime() ), tutorial-simple, and pretty obvious, or at least it should be. The problem looks to be Python.
(more…)

I’ve previously identified a few elements that are needed to make a successful web based application. Since then, I’ve been prototyping, designing, coding on the bus and while making dinner, and have a pared down, minimal proof of concept for a few of those.

• XMLHttpRequest: I’ve verified XMLHttpRequest works with my wife’s iPhone. I expected it to, but had to make sure. This is the core Javascript call for dynamic web pages that retrieve content from servers.
• Timer: I still need to verify this works as expected on the iPhone, but I have a demo page to verify it ready.

In addition to the front end work (of which there will be much, much more), I needed a back-end that I could develop actively. My platform choices were limited in that it needs to work on a Dreamhost shared server, work on my desktop at home, and also on my laptop for the bus or coding around the house. Tomcat had to be discarded: I’d need to set up a separate Tomcat server and Java environment at Dreamhost. Perl was discarded as I really don’t appreciate the language. I strongly considered PHP but decided against it in the end. Ruby was also considered but I didn’t want to learn a new language while coding a new app. I toyed with using Lua. In the end, I decided on Python 2.4 so I don’t have to do any gymnastics at Dreamhost. As far as my scripts are concerned, it’s a one-line change between platforms:

#!/Python24/pythonw for Win32
#!/usr/bin/python for Linux

I have a working Python back-end that gives me a minimum of helper objects for working with HTTP requests in a CGI environment. (No WSGI here, at least not yet.) I have room to plug in different request interpreters and a complete world engine or whatever might strike my fancy as long as I keep this minimal interface in place.

Now, nobody said I had to produce anything pretty when proving this works…

Try out my first proof of concept

I’d like my next proof of concept to be an interactive “game” or other social, multi-user activity.

D’oh. Next time, I’ll just search the docs instead of thrusting blindly forward in an inefficient direction.

See 4.7.4 Unpacking Argument Lists in the Python Tutorial. It just goes to show that even when you think you know a language pretty well, there’s always something new to learn, especially if it’s a language feature you haven’t used before.

Instead of doing this:

values = self.get_field_values()
 
# build the value string
value_str = ''
 
for value in values:
    if len(value_str) > 0:
        value_str += ','
    if isinstance(value, str):
        value = replace("'", "''")
        value_str += "'" + value + "'"
    else:
        value_str += str(value)
 
result = eval("somefunc(" + value_str + ")")


you can simply do this:

somefunc(*values)

and Python will unpack the sequence into an argument list.

For the first time, I’ve had no problem finding a good binary distribution of Stackless Python for OS X. It should be everything you need to run it on either PPC or i386 Macs. It might not have all the synchronization controls I’m used to, but I’m willing to learn how to use it to write (slightly) higher performance Python apps.

A tested implementation follows. Omitted are the unit tests and exception classes.

Scalability not guaranteed.
(more…)