Development strategy for google-app-engine

        In developing my site for the cloud I found it a bit of a challenge. There is very little documentation in terms of infrastructure strategy and a lot of sites assume you have a clear methodology. I always strive for a clean infrastructure — having developed code for the quite awhile I have come to know that if you have a solid scalable strategy up front it makes things just work easier later on. That is why I was really turned onto google-app-engine. It gives you a clear way of building and testing your code and then pushing it to the web. It is a great place to start but it isn’t the “everything” I’m looking for.
        Because some of the limitations/restrictions with using google-app-engine (GAE) you need to understand that if you want to use it (i.e. python package) you need to include it in your application. That is where the benefits of a virtualenv come into play. virtualenv (when combined with pip) let you know exactly what packages you have because throughout the application development process if you need it you ‘pip-it’. Then pip gives you a clean manifest on what you have so you always know. But.. (you knew this was coming) GAE doesn’t like a virtual env. In fact if you try it you almost surely run into problems. For me it was a error along this line...

ImportError: No module named unittest

        So what can I do? For me the reasons to use a virtualenv/pip combination is too compelling to simply not use it. I really like knowing what packages I willbe deploying up to the cloud and since I need to package them with my application why not? Furthermore if you are used to running in a virtualenv/pip environment or you have a somewhat stock python environment this is just a good way to keep things straight. Understanding the obvious limitation that you can’t run in a virtualenv we are going to take advantage of everything else this environment has to offer.

General Development Strategy

• Use Django. While I understand that GAE now support 1.1 I want that control left to me.
• Keep the development tree streamlined for the entire site. Guido recommends (as I understand him in this video ~11:45) to use a single app-engine application per independent application. This doesn’t make much sense to me as the ability to share information between different models is really extending the multi-dimensionality of the site. So my preference is to keep it all under one hood.
• Use revision control (for me it’s perforce - I don’t have anything against the others (hg/git/svn) it’s just what I like)
• Use eclipse/pydev as my development environment

General Directory Structure

I am developing for $SITE ( i.e export SITE=“foobar.com”)

/dev
        /<$SITE>
                /bin
                /lib
                /include
                /src
                        /django
                /www
                        app.yaml
                        /appengine_django
                        manage.py
                        main.py
                        /apps
                                /app_1
                                /app_2
                                /app_3
                                /app_n
                        /settings
                                __init__.py
                        urls.py
                        django symlink to ../src/django/django

Getting started

Setup your virtualenv

 cd dev

Build up a baseline virtualenv

 virtualenv --python=python2.5 --no-site-packages ${SITE}

Install some needed apps.. (make sure you are in your dev directory..)

# pip install -E ${SITE} -e svn+http://code.djangoproject.com/svn/django/trunk#egg=Django
# pip install -E ${SITE} -e svn+http://code.djangoproject.com/svn/django/tags/releases/1.1.1/#egg=Django
 pip install -E ${SITE} -e svn+http://code.djangoproject.com/svn/django/tags/releases/1.1/#egg=Django
 pip install -E ${SITE} yolk

*Notes: 1.1.1 failed 2 tests and trunk failed more.. 1.1 passes all tests

Activate your virtualenv

 source ${SITE}/bin/activate

Create your www tree (this is where we house our app). Use the google-app-engine-django helper stuff. It’s well worth it!!

 cd ${SITE}
 svn export http://google-app-engine-django.googlecode.com/svn/trunk/ www

Link in your django tree

 cd www
 ln -s ../src/django/django

At this point you should be able to deactivate yourself and simply verify everything works.

 deactivate
 python2.5 manage.py test
  #Ran 61 tests in 8.058s
 python2.5 manage.py runserver

Now open up a web browser to http://127.0.0.1:8000/ and you should get the famous “It works!!”. At this point you should also configure the Google App Engine Launcher to point to your site directory and “www” will be your application.

Build up a quick index and push it to the web.

This is not for the faint. I am going to quickly build up a basic django index page. I want to show it works and we con complete the process.

Shuffle around my tree (personal preference)

Like I indicated above I like to have my apps all under a single apps tree (keep it clean). I also like to put settings.py under a settings tree - this allows me to mess with the __init__.py

So with that hang on..

 cd ${SITE}/www
 mkdir settings
 mv settings.py settings/__init__.py
 rm settings.pyc

 mkdir apps
 python2.5 manage.py startapp core
 mv core apps/

Build your simple app

Edit your ${SITE}/www/urls.py

 urlpatterns = patterns('',
      url(r'^', include('apps.core.urls')), 
 )

Add in your app to the ${SITE}/settings/__init__.py

 INSTALLED_APPS = (
       'appengine_django',
       'apps.core',
 )

Create your apps/core/views.py

 from django.http import HttpResponse 
 def index(request):
    return HttpResponse('Hello World -- Django rocks!!')

Get your apps/core/urls.py ready to accept the index view

 from django.conf.urls.defaults import *
 import views
 urlpatterns = patterns('',
    url(r'^$', views.index),
 )

Verify it works.. Voila - pretty simple eh??

Next up publish it..

References:

Guido's screencast - it was awesome..
        Rapid Development with Python, Django, and Google App Engine (2008 Google I/O Session Videos and Slides)