1. Install Apache 2
2. Install libapache2-mod-wsgi
   
# apt-get install libapache2-mod-wsgi

3. Link the project directory to /var/www
   
# cd /var/www
# ln -s /some/path/DevTools

4. Update /etc/apache2/sites-available/default
   
    <Directory /var/www/>
        Options Indexes FollowSymLinks MultiViews ExecCGI
        AllowOverride None
        Order allow,deny
        allow from all
        AddHandler cgi-script .cgi
        AddHandler wsgi-script .wsgi
    </Directory>

5. Restart Apache
   
# apache2ctl restart

6. Create the tester script, /var/www/DevTools/tester.wsgi
   
def application(environ, start_response):
    status = "200 OK"
    output = "Hello World!"
 
    response_headers = [('Content-type', 'text/plain'),
                ('Content-Length', str(len(output)))]
    start_response(status, response_headers)
 
    return [output]

7. Verify your tester by going to http://localhost/DevTools/tester.wsgi. The output should be:
   
Hello World!


If you get any 500 Server Error responses, check /var/log/apache2/error.log first. mod_python does a better job with error reporting than WSGI but is, in my experience, a more fragile framework for configuring and developing web apps.

• I lost my version of Radar that supported chat, so I had to rewrite it from scratch. This version only supports standard layout US keyboards (the shift-modifiers are hard-coded) and I haven’t tweaked the fonts, set up an input box, or limited the input string to make things look nicer.
• I likewise lost the code to add names and chat messages above the arrow sprites, so I rewrote that. This version should be more correct than the one from 2005.
• I’m using the standard Python logger (based on log4j) instead of a custom logging class.
• I’m using a Singleton logger instance for all the arrow sprites.

The arrow sprite logic has gotten complicated enough that I need to develop good unit tests for it. I’ve been developing with rapid integration tests, but I believe I’ve hit the point with the sprite where significant changes could break the correct behavior it’s exhibiting now. A good suite of unit tests against the sprite will help with later refactoring and will also provide a base for testing sprites in other projects. Unit tests should be written before I approach anything as complicated as, say, collision detection (which PyGame handles nicely now) or basic AI or pathfinding.

I’ve looked into using py2exe to package the application as a Windows executable. It looks like, with a few hours of testing and pruning, it’ll work well and still produce a relatively small executable.

I’m considering using Thrift to automate generation and interpretation of the client-server messages. I’m willing to pay a little more in bandwidth and processing time rather than spending time writing custom binary packing and unpacking code for messages. Thrift should let me abstract the process a little more, and I’m very curious to see how well it works for me in practice.

Finally, I’ll implement the server. With the minimal prediction in place on the arrow sprites, I’m very curious to see how well the client runs and how much rubberbanding will take place.

Komodo Edit is still working well for me, although I wonder if the Komodo IDE would help to speed up my develop-test iteration cycle.

Grab the Radar2.zip file here. Install Python 3.1 and Pygame 1.9.1 for 3.1, add Python to your path, and test the program with python Main.py.

1. Download the required programs and libraries
   • Komodo Edit for best-of-breed Python editing
   • Oracle Berekely DB 4.7.25 for the storage engine
   • Stackless Python 2.6 (I’ll eventually convert to 3.x)
   • setuptools-0.6c9.tar.gz to install Python eggs
   • bsddb3-4.7.6.zip
   • If you don’t have a tar.gz expander, 7-Zip
   • If you don’t have a C++ compiler, or yours is out of date, get Visual C++ Express 2009
2. Install Visual C++ Express
3. Install Komodo Edit
4. Install Stackless Python 2.6
5. Install Berkeley DB 4.7
6. Expand bsddb3-4.7.6.zip to a known directory
7. Install setuptools and PyBSDDB
   1. Expand setuptools into a temporary directory
   2. Open a command line (on Vista, Windows-R and enter cmd)
   3. Add Python to the command line environment with:
      set PATH=%PATH%;C:\Python26
   4. Change directory to the temporary directory where you expanded setuptools
   5. Install setuptools with:
      python setup.py install
   6. Install PyBSDDB with:
      
cd \Users\Chris\Temp\bsddb3-4.7.6 (where I expanded bsdb4)

mkdir db
cd db
xcopy /e /v "C:\Program Files\Oracle\Berkeley DB 4.7.25\include" include\
xcopy /e /v "C:\Program Files\Oracle\Berkeley DB 4.7.25\lib" lib\
cd ..
"C:\Program Files\Microsoft Visual Studio 9\VC\vcvarsall.bat"
python setup.py install

. . . time for bed, but I’m having a little trouble with MSVC:

snip
Modules/_bsddb.c(8358) : warning C4996: 'strncpy': This function or variable may be unsafe. Consider using strncpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.
F:\Program Files\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(157) :
see declaration of 'strncpy'
warning: I don't know what to do with 'runtime_library_dirs': ['db/lib']
error: don't know how to set runtime library search path for MSVC++


Yeah, I’m probably nuts for trying to compile this myself. On OS X or Linux, there are still plenty of steps but no major hurdles.

Download the framework here. It’s simple and probably has some bugs in it.

You can test it as well, but don’t expect it to work under Internet Explorer. I didn’t bother writing Javascript for IE, only Firefox (which means it will also run under Safari and on an iPhone). If you try it out, give it a couple seconds to fetch the time from the server.

There are obvious points of extension, starting from the Handlers.py file. I chose to explicitly check handler names rather than use them to dynamically call into code from a string passed through the update. It just seems safer.

Since it’s a dynamic call, I can’t view the page source to see the XML that gets returned. And I never verified that I was getting a String: I assumed that because it was inside a CDATA block that the tags would be visible, but what I actually got was the tags being evaluated, and since ^<get><time /></get> isn’t part of XHTML 1.1 . . . my XML never appeared.

So in the interest of saving some future developer a few minutes or hours or days of trouble, if you’re trying to parse POST-ed XMLHttpRequest XML DOM objects in a Python CGI web application and it looks like STDIN is empty or you’re not getting the POST contents, verify that you’re reading something by checking the length and extracting a part of the string. And convert your tags.


#!/Python24/python
 
import os
import sys
 
def main():
    print "Content-Type: application/xml\n\n"
 
    print "< ![CDATA[
\n";
    for key in os.environ:
        print key + "=" + os.environ[key] + "
\n"
 
    if os.environ.has_key("CONTENT_LENGTH"):
        length = int(os.environ["CONTENT_LENGTH"])
        s = str(sys.stdin.read(length))
        s = s.replace('< ', '<')
        print "

" + s + "";

 

    print "
\n]]>\n";

 

if __name__ == '__main__':

    main()


]]>
			http://www.mischiefblog.com/?feed=rss2&p=708
		0
		
		
		
		http://www.mischiefblog.com/?p=707
		http://www.mischiefblog.com/?p=707#comments
		Tue, 08 Jan 2008 00:11:59 +0000
		Chris
				

		http://www.mischiefblog.com/?p=707
		
			Now, I’m not entirely stupid.  Lacking common sense?  Yes.  Forgetful of dates, appointments, anniversaries, bills?  Yes.  Easily confused?  Absolutely.

But I’ve also been programming in Python for years and wrote my first CGI scripts in 1993 or 1994 or thereabouts.  I’m kind of familiar with both environments, even if this is the first time I’ve actually put them together.  Everything seemed to be meshing really, really well until I tried to get XML content from the web client through a DOM document passed into the XMLHttpRequest.  (And yes, I am new to XMLHttpRequest dynamic app development.  I’d always had the pleasure of using someone else’s library.)


function createDocGetTime() {

    docGetTime = document.implementation.createDocument("", "", null);

    var getNode = docGetTime.createElement("get");

    var getTime = docGetTime.createElement("time");

    getNode.appendChild(getTime);

    docGetTime.appendChild(getNode);

}

 

function updateTime() {

    timerClient = new XMLHttpRequest();

    timerClient.onreadystatechange = timerHandler;

    timerClient.open("POST", "xmltimer.py");

    timerClient.setRequestHeader("Content-Type", "text/xml;charset=UTF-8");

    timerClient.send(docGetTime);

    timerId = setTimeout("updateTime()", 1500);

}


Evil code (there’s no way to stop calling updateTime()  ), tutorial-simple, and pretty obvious, or at least it should be.  The problem looks to be Python.



From the old days of CGI, if you wanted to read the contents of a POSTed form, you’d need to split and parse the contents of the body of the HTTP request which should come in through STDIN.  (The QUERY_STRING is only useful for GET requests.)  This is old, ancient stuff, and should be possible with something like:


import os

import sys

 

print "Content-Type: text/html\n\n<b>Body Content:</b><pre><![CDATA["

if os.environ.has_key("CONTENT_LENGTH"):

    print sys.stdin.read(os.environ["CONTENT_LENGTH"])

print "]]></pre>"


Except that the body never gets read.
In these paranoid times, you can’t call a different server, port, or host than the one that served up the webpages, or you’d become vulnerable to XSS, so the browser forbids that kind of request.  I had to shut down Apache and start netcat to verify that wasn’t not a problem with how I was using XMLHttpRequest.



$ ./netcat -l -p 80

POST /demo/simple.py HTTP/1.1

Host: localhost

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Content-Type: text/xml;charset=UTF-8

Referer: http://localhost/demo/xmlupdate.html

Content-Length: 18

Pragma: no-cache

Cache-Control: no-cache

 

<get><time /></get>


So I’m stuck wondering what I screwed up with my Python CGI configuration or with how I invoke the XMLHttpRequest.
I’ve been bugged by this since Saturday and haven’t figured it out yet.  Does anyone have any idea why STDIN is empty?

Is it something to do with how Python is invoked (however unlikely)?  This error should also cause the standard cgi library to fail–but it doesn’t.
Is it something to do with the content type?  Changing to plain text/html doesn’t help, and neither does the incorrect application/x-www-form-urlencoded.

My code which dumps the POST content works fine with an HTML FORM submission:



$ ./netcat -l -p 80

POST /demo/simple.py HTTP/1.1

Host: localhost

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/2

0071127 Firefox/2.0.0.11

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai

n;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Referer: http://localhost/demo/xmlupdate.html

Content-Type: application/x-www-form-urlencoded

Content-Length: 45

 

yourname=xxx&yourmood=yyy&Submit=Submit+Query


Output



<test>

<![CDATA[

<br/>

HTTP_REFERER=http://localhost/demo/xmlupdate.html<br/>

SERVER_SOFTWARE=Apache/2.0.54 (Win32)<br/>

SCRIPT_NAME=/demo/simple.py<br/>

 

. . . skipped . . .

 

CONTENT_LENGTH=45<br/>

HTTP_ACCEPT_CHARSET=ISO-8859-1,utf-8;q=0.7,*;q=0.7<br/>

 

. . . skipped . . .

 

CONTENT_TYPE=application/x-www-form-urlencoded<br/>

HTTP_ACCEPT_ENCODING=gzip,deflate<br/>

 

<pre>yourname=xxx&yourmood=yyy&Submit=Submit+Query</pre>

<br/>

</test>

]]>


So Python with CGI doesn’t appear to be entirely at fault.
The easy solution would appear to be to shove the XML content as a String in Javascript into a form field value and submit the form.  I hate to take that approach when I should be able to do native and correct XML through the DOM object.  Another point of view may say, “If you wanted to save bandwidth, don’t use XML:  use tokens and plain text instead.”  I tend to regard that as premature optimization.
I have learned that sys.stdin.seek(0) is useless, so if you plan on using the cgi library with your own routine to read the body, pass the body to cgi.FieldStorage() after you’ve read and parsed it, or patch cgi (subclass or override) with a version that stores the unaltered body in a field.
I might find time to test if I can get the XML update request working with PHP.
]]>
			http://www.mischiefblog.com/?feed=rss2&p=707
		0
		
		
		
		http://www.mischiefblog.com/?p=706
		http://www.mischiefblog.com/?p=706#comments
		Fri, 04 Jan 2008 22:37:41 +0000
		Chris
				
		

		http://www.mischiefblog.com/?p=706
		
			I’ve previously identified a few elements that are needed to make a successful web based application.  Since then, I’ve been prototyping, designing, coding on the bus and while making dinner, and have a pared down, minimal proof of concept for a few of those.


XMLHttpRequest:  I’ve verified XMLHttpRequest works with my wife’s iPhone.  I expected it to, but had to make sure.  This is the core Javascript call for dynamic web pages that retrieve content from servers.
Timer:  I still need to verify this works as expected on the iPhone, but I have a demo page to verify it ready.

In addition to the front end work (of which there will be much, much more), I needed a back-end that I could develop actively.  My platform choices were limited in that it needs to work on a Dreamhost shared server, work on my desktop at home, and also on my laptop for the bus or coding around the house.  Tomcat had to be discarded:  I’d need to set up a separate Tomcat server and Java environment at Dreamhost.  Perl was discarded as I really don’t appreciate the language.     I strongly considered PHP but decided against it in the end.  Ruby was also considered but I didn’t want to learn a new language while coding a new app.  I toyed with using Lua.  In the end, I decided on Python 2.4 so I don’t have to do any gymnastics at Dreamhost.  As far as my scripts are concerned, it’s a one-line change between platforms:
#!/Python24/pythonw for Win32

#!/usr/bin/python for Linux
I have a working Python back-end that gives me a minimum of helper objects for working with HTTP requests in a CGI environment.  (No WSGI here, at least not yet.)  I have room to plug in different request interpreters and a complete world engine or whatever might strike my fancy as long as I keep this minimal interface in place.
Now, nobody said I had to produce anything pretty when proving this works…
Try out my first proof of concept
I’d like my next proof of concept to be an interactive “game” or other social, multi-user activity.
]]>
			http://www.mischiefblog.com/?feed=rss2&p=706
		0
		
		
		
		http://www.mischiefblog.com/?p=541
		http://www.mischiefblog.com/?p=541#comments
		Wed, 22 Nov 2006 16:57:21 +0000
		Chris
				

		http://www.mischiefbox.com/blog/?p=541
		
			D’oh.  Next time, I’ll just search the docs instead of thrusting blindly forward in an inefficient direction.

See 4.7.4 Unpacking Argument Lists in the Python Tutorial.  It just goes to show that even when you think you know a language pretty well, there’s always something new to learn, especially if it’s a language feature you haven’t used before.
Instead of doing this:



values = self.get_field_values()

 

# build the value string

value_str = ''

 

for value in values:

    if len(value_str) > 0:

        value_str += ','

    if isinstance(value, str):

        value = replace("'", "''")

        value_str += "'" + value + "'"

    else:

        value_str += str(value)

 

result = eval("somefunc(" + value_str + ")")


you can simply do this:



somefunc(*values)



and Python will unpack the sequence into an argument list.
]]>
			http://www.mischiefblog.com/?feed=rss2&p=541
		0
		
		
		
		http://www.mischiefblog.com/?p=540
		http://www.mischiefblog.com/?p=540#comments
		Wed, 22 Nov 2006 05:13:47 +0000
		Chris
				
		

		http://www.mischiefbox.com/blog/?p=540
		
			For the first time, I’ve had no problem finding a good binary distribution of Stackless Python for OS X.  It should be everything you need to run it on either PPC or i386 Macs.  It might not have all the synchronization controls I’m used to, but I’m willing to learn how to use it to write (slightly) higher performance Python apps.

]]>
			http://www.mischiefblog.com/?feed=rss2&p=540
		0