Sunday, 1 August 2010

Python web development for Google App Engine

To start your very first Python web development for GAE, you will need:
To run any application you should have GAE account ( You should also create new application on GAE administration pages.

Hello World project
Now, you can try to generate Hello World application using the Pydev's wizard. Create new project in eclipse - use "Pydev Google App Engine Project". Select Python version to 2.5 and configure interpreter to the one that you installed earlier. Next, you will have to specify path to Google App Engine SDK. On next page, set name for your application (the same as was registered on GAE account admin pages) and choose "Hello Webapp World" template. Within few seconds you should have your Hello World web app generated.

Run Google App Engine Launcher that was installed with SDK. If you are running it first time, configure paths to Python and SDK (Edit -> Preferences). Add your project to the Launcher:

Now, you should be able to run your application locally as well as deploy it to GAE. During deployment you will need to log into your GAE account.

More complex application

As usual, I created my test app as simple Book management application. It is not even CRUD as I had no time to develope update functionality. Application requires user to log in using Google Account before they enter the main page.

There, user will have links to other pages:
  • Book list - when user will be able to display and remove books
  • New book - page with form for adding new entries
  • log out
Project structure:

Projects uses Google Webapp framework that is quite simple, but on the other hand you don't need anything more sophisticated for such simple app;)

Main application code and logic is in
import os

from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from models import Book

from google.appengine.ext.webapp.util import run_wsgi_app

class MainPage(webapp.RequestHandler):
def get(self):
user = users.get_current_user()

if user:
template_values = {
'user' : user.nickname(),
'logoutUrl' : users.create_login_url("/")
path = os.path.join(os.path.dirname(__file__)+"/templates", 'index.html')
self.response.out.write(template.render(path, template_values))


class NewBookPage(webapp.RequestHandler):
def get(self):
path = os.path.join(os.path.dirname(__file__)+"/templates", 'new.html')
self.response.out.write(template.render(path, {}))

class ListBooksPage(webapp.RequestHandler):
def get(self):
books = Book.all();
template_values = {
'books' : books,

path = os.path.join(os.path.dirname(__file__)+"/templates", 'list.html')
self.response.out.write(template.render(path, template_values))

class SaveBookPage(webapp.RequestHandler):
def post(self):
book = Book();
book.title = self.request.get('title') = self.request.get('author')

class RemoveBookPage(webapp.RequestHandler):
def get(self):
book = Book.get(self.request.get('key'))

application = webapp.WSGIApplication(
[('/', MainPage),
('/list', ListBooksPage),
('/save', SaveBookPage),
('/remove', RemoveBookPage),
('/new', NewBookPage)],

def main():

if __name__ == "__main__":
In this module we have main "controller" classes that cover all functionality. Classes are mapped to particular URL. Responsibility of each class is to perform business logic, prepare model and return template that will render the page:
  • MainPage - prepares model for menu, returns index.html template
  • NewBookPage - returns new.html template with form
  • ListBooksPage - selects books from Google Datastore, puts them into model and returns list.html template
  • SaveBookPage- saves book to datastore and redirects view to book list
  • RemoveBookPage - removes book from datastore and redirects view to book list
This module contains also code that maps each controller class to particular URL.

Pages are implemented using Django templated mechanism that is included in SDK. We have a main template base.html that renders html page structure:
<link type="text/css" rel="stylesheet" href="/style/main.css" />
{% block main %}
{% endblock %}
All other templates are extending the main one.

Code of index.html:
{% extends "base.html" %}
{% block main %}
Welcome {{ user }}!
<a href="/list">Book list</a><br/>
<a href="/new">New book</a><br/>
<a href="{{ logoutUrl }}">Logout</a>
{% endblock %}
{% extends "base.html" %}
{% block main %}
{% for book in books %}
<td>{{ book.title }}</td>
<td>{{ }}</td>
<td><a href="/remove?key={{ book.key }}">remove</a></td>
{% endfor %}
<a href="/">Back</a>
{% endblock %}
{% extends "base.html" %}
{% block main %}
<form action="/save" method="POST">
<td><input type="text" name="title"/></td>
<td><input type="text" name="author"/></td>
<td><input type="submit" value="Add"></td>
<a href="/">Back</a>
{% endblock %}
I used separete main.css file to define look (don't take it seriously hi hi) of the application:
body {
font-family: Verdana, Helvetica, sans-serif;
background-color: #DDDDDD;
In app.yaml file, we define that module is responsible for handling all request to /* urls and that /style urls will be exposed as a static resource:
application: python-test-maciekm
version: 1
runtime: python
api_version: 1

- url: /style
static_dir: style
- url: /.*
Data model is defined in file:
from google.appengine.ext import db

class Book(db.Model):
title = db.StringProperty()
author = db.StringProperty()
Let's see the result
Such application should be ready to get it started using Launcher and for deployment on GAE. When it starts, you should be able to see application pages in the browser - creating new book:

And the list:

Was it hard? To be honest I really think it was not. Personally, I think it was even easier for me to create GAE application in Python than in Java. In the future, I would be happy to try to use Django instead of Webapp framework and to compare them.

Saturday, 15 May 2010

BPEL compensation handlers with Glassfish ESB example

Recently I am involved in integration project based on Glassfish ESB and I needed to get into BPEL technology. There was one thing that really impressed me - compensation handlers.

Why do we need compensation handlers?
Imagine a simple book ordering process defined in BPEL and exposed as a webservice.
Process flow would be:
  • receive needed information: bookId, customerId, creditCardNumber
  • call BookOrderWS - webservice responsible for creating order
  • call PaymentWS - webservice responsible for payment
  • return result of the process
Execution of this process looks very simple until we consider that something may go wrong. Unpredicted exception may occur in the middle of the process. For example payment component may not be working and we will get exception while calling it. Without any fault handling our BPEL process will just stop after customer invoking it via WS will receive fault information. The worst thing is that earlier part of the process was executed properly - order was created by BookOrderWS. So, the book will be posted to customer even though he was not charged for it;)

Possible solutions:
  • use catch clause to react on errors on each webservice call
  • use componsation handlers - for each webservice call define a set of actions that will undo its execution
In first solution we could catch exception while calling PaymentWS and call another webservice that would cancel the order that was created. It would work in our case, but imagine a process that orchestrates ten or more webservice calls - code would be very complicated.

That's why I prefer second solution. No matter in which process step error occurs - all previous steps will know how to undo its operation. Process definition will be much clearer.

Now I will show a draft of simple implementation that I prepared to test this concept. During development I was using Glassfish ESB 2.2 and bundled Netbeans IDE 6.7.1.

My application consists of following projects:
  • BookOrderWS
  • PaymentWS
  • BookOrderBPEL
  • BookOrderCA
BookOrderWS is responsible for creating and canceling orders. It is implemented as EJB module and consists of one class:
package com.maciekm;

import javax.jws.Oneway;
import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebService;
import javax.ejb.Stateless;

public class BookOrderWS {

@WebMethod(operationName = "orderBook")
public long orderBook(@WebParam(name = "bookId") long bookId,
@WebParam(name = "customerId") long customerId) {

long orderId = System.currentTimeMillis();
System.out.println("Order request for bookId: " + bookId + " from customerId: " + customerId + " - created orderId: " + orderId);
return orderId;

@WebMethod(operationName = "cancelOrder")
public void cancelOrder(@WebParam(name = "orderId") long orderId) {
System.out.println("Canceling orderId: "+orderId);
PaymentWS is also EJB module with very simple code:
package com.maciekm;

import javax.jws.WebMethod;
import javax.jws.WebParam;
import javax.jws.WebService;
import javax.ejb.Stateless;
import javax.jws.Oneway;

public class PaymentWS {

@WebMethod(operationName = "payForOrder")
public void payForOrder(@WebParam(name = "orderId") long orderId,
@WebParam(name = "creditCardNumber") String creditCardNumber) {

System.out.println("Payment received for orderId: " + orderId + ", creditNumber:" + creditCardNumber);
if (creditCardNumber.equals("1234")) {
System.out.println("Payment error!");
throw new RuntimeException("some unexpected errors");
System.out.println("Payment completed!");
Here our mock implementation simulates payment processing. Unpredicted exception will occur for 1234 credit card number.

Process definition and composite application
WSDL of our process is based on very simple input and output schemas:
<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd=""
<xsd:element name="order" type="tns:order"/>
<xsd:complexType name="order">
<xsd:element name="bookId" type="xsd:integer"/>
<xsd:element name="customerId" type="xsd:integer"/>
<xsd:element name="creditCardNumber" type="xsd:string"/>

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:xsd=""
<xsd:element name="orderResult" type="tns:orderResult"/>
<xsd:complexType name="orderResult">
<xsd:element name="isCompleted" type="xsd:boolean"/>
Process diagram:

<?xml version="1.0" encoding="UTF-8"?>
xmlns:sxeh="" xmlns:sxed2="" xmlns:ns0="" xmlns:ns1="">
<import namespace=""
location="BookOrderWSServiceWrapper.wsdl" importType=""/>

<import namespace="" location="BookOrderWS/wsdl/BookOrderWSService.wsdl"

<import namespace="" location="PaymentWSServiceWrapper.wsdl"

<import namespace="" location="PaymentWS/wsdl/PaymentWSService.wsdl"

<import namespace=""
location="bookOrderProcessWSDL.wsdl" importType=""/>

<partnerLink name="OrderProcessPL" xmlns:tns=""
partnerLinkType="tns:bookOrderProcessWSDL" myRole="bookOrderProcessWSDLPortTypeRole"/>

<partnerLink name="BookOrderPL" xmlns:tns=""
partnerLinkType="tns:BookOrderWSLinkType" partnerRole="BookOrderWSRole"/>

<partnerLink name="PaymentPL" xmlns:tns=""
partnerLinkType="tns:PaymentWSLinkType" partnerRole="PaymentWSRole"/>

<variable name="CancelOrderIn" xmlns:tns="" messageType="tns:cancelOrder"/>
<variable name="OrderBookOut" xmlns:tns="" messageType="tns:orderBookResponse"/>
<variable name="PayForOrderIn" xmlns:tns="" messageType="tns:payForOrder"/>
<variable name="OrderBookIn" xmlns:tns="" messageType="tns:orderBook"/>
<variable name="BookOrderProcessWSDLOperationOut"

<variable name="BookOrderProcessWSDLOperationIn"

<receive name="ReceiveOrder" createInstance="yes" partnerLink="OrderProcessPL"
portType="tns:bookOrderProcessWSDLPortType" variable="BookOrderProcessWSDLOperationIn"/>

<scope name="BookOrderScope">
<sequence name="Sequence2s">
<assign name="Assign1">
<invoke name="CancelOrder" partnerLink="BookOrderPL" operation="cancelOrder"
portType="tns:BookOrderWS" inputVariable="CancelOrderIn" xmlns:tns=""/>

<sequence name="Sequence1">
<assign name="AssignToOrder">
<invoke name="OrderBook" partnerLink="BookOrderPL" operation="orderBook"
xmlns:tns="" portType="tns:BookOrderWS" inputVariable="OrderBookIn" outputVariable="OrderBookOut"/>

<assign name="AssignToPay">
<invoke name="Pay" partnerLink="PaymentPL" operation="payForOrder" xmlns:tns=""
portType="tns:PaymentWS" inputVariable="PayForOrderIn"/>

<assign name="AssignToReply">
<reply name="Reply" partnerLink="OrderProcessPL" operation="bookOrderProcessWSDLOperation"
xmlns:tns="" portType="tns:bookOrderProcessWSDLPortType"

As you can see BookOrderScope defines its compensation handler. It calls cancelOrder method of BookOrderWS.

Composite application that connects all modules together:

Testing how it works
Composite application has a testing support in Netbeans IDE. We can use it to create test cases.
Test case one - sending valid order and payment information - processing works properly:
Order request for bookId: 1 from customerId: 2 - created orderId: 1274383922765
Retrieving schema at '', relative to 'file:/PaymentWSService.wsdl'.
Payment received for orderId: 1274383922765, creditNumber:1111
Payment completed!
Test case two - sending valid order, but credit card number that simulates error - order created by first WS is cancelled by compensation handler:
Order request for bookId: 1 from customerId: 2 - created orderId: 1274383962281
Payment received for orderId: 1274383962281, creditNumber:1234
Payment error!
The log message is null.
at com.sun.ejb.containers.BaseContainer.processSystemException(
at com.sun.ejb.containers.BaseContainer.completeNewTx(
at com.sun.ejb.containers.BaseContainer.postInvokeTx(
Canceling orderId: 1274383962281
It works indeed:)

PS. Whole application was just quick test, so forgive me messy code and totally naive business scenario;)

Thursday, 11 February 2010

In-Out to In-Only MEP transformation in ServiceMix ESB's bean component

Recently I needed to write servicemix-bean component that implements following scenario:
  • receives In-Out MEP request
  • sends response to received In-Out request
  • constructs In-Only MEP request and forwards it to another service
I had some problems to get it working and could not find help in the internet. So, when I finally solved that problem I decided to share my solution.

My component:
package org.maciekm;

import java.util.Date;

import javax.annotation.Resource;
import javax.jbi.component.ComponentContext;
import javax.jbi.messaging.DeliveryChannel;
import javax.jbi.messaging.ExchangeStatus;
import javax.jbi.messaging.InOnly;
import javax.jbi.messaging.MessageExchange;
import javax.jbi.messaging.MessageExchangeFactory;
import javax.jbi.messaging.MessagingException;
import javax.jbi.messaging.NormalizedMessage;
import javax.xml.namespace.QName;
import javax.xml.transform.Source;
import javax.xml.transform.TransformerException;

import org.apache.servicemix.MessageExchangeListener;
import org.apache.servicemix.client.ServiceMixClient;
import org.apache.servicemix.client.ServiceMixClientFacade;
import org.apache.servicemix.jbi.jaxp.SourceTransformer;

public class MyService implements MessageExchangeListener {

private DeliveryChannel channel;

private ComponentContext context;

private String targetServiceNS;
private String targetServiceName;

public String getTargetServiceNS() {
return targetServiceNS;

public void setTargetServiceNS(String targetServiceNS) {
this.targetServiceNS = targetServiceNS;

public String getTargetServiceName() {
return targetServiceName;

public void setTargetServiceName(String targetServiceName) {
this.targetServiceName = targetServiceName;

public void onMessageExchange(MessageExchange exchange) throws MessagingException {
if (exchange.getStatus() == ExchangeStatus.ACTIVE) {
try {
NormalizedMessage message = exchange.getMessage("in");
Source content = message.getContent();

// getting "in" message of In-Out request
String body = (new SourceTransformer()).toString(content);

// some your custom logic

// sending response to initial In-Out request - place some xml there
String out=".......... some xml ..............";
StreamSource respSource = new StreamSource(new StringReader(out));
exchange.setMessage(message, "out");

// some business logic to prepare message to forward based on original request

// sending In-Only message to service defined by targetServiceNS and
// targetServiceName properties
String forwardOut=".......... some xml ..............";
StreamSource forwardSource = new StreamSource(new StringReader(forwardOut));
MessageExchangeFactory mef =
channel.createExchangeFactoryForService(new QName(targetServiceNS, targetServiceName));
InOnly inOnlyExch = mef.createInOnlyExchange();
NormalizedMessage forwardMessage = inOnlyExch.createMessage();
ServiceMixClient smc = new ServiceMixClientFacade(context);
} catch (TransformerException te) {
// please do it better way in real project ;)
throw new MessagingException(te);
Declaration in xbean.xml file:
<bean id="myService" class="org.maciekm.MyService">
<property name="targetServiceNS" value="" />
<property name="targetServiceName" value="targetService" />

Injected properties are defining target service of In-Only message. Hope it helps some of you:)

Saturday, 30 January 2010

(Almost) RESTful application with Grails and Dojo

As REST techniques are getting more and more popular I wanted to give it a try. Of course I wanted to use Grails. For client side I have chosen Dojo javascript framework as I already had some experience with it.

Project setup
During my experiment I used Grails 1.2.0.
Let's start with generating simple project with Book domain object and controller - almost as in Grails tutorial:

grails create-app GrailsRestTest
cd GrailsRestTest
grails create-domain-class book

grails create-controller Book

Only one thing that surprised me was a message: "WARNING: You have not specified a package. It is good practise to place classes". I wanted to make it simple so I just ignored it by writing "y" and pushing enter;)

As in Grails tutorial, I added some sample data to /grails-app/conf/BootStrap.groovy file:
def init = { servletContext ->
new Book(author:"Stephen King",title:"The Shining").save()
new Book(author:"James Patterson",title:"Along Came a Spider").save()
Final project structure will be:

RESTful server
We need to create controller that will be performing base CRUD operations and map its methods to REST style URLs.

Edit /grails-app/conf/UrlMappings.groovy:
class UrlMappings {
static mappings = {
constraints {
// apply constraints here
"/book" (controller: "book") {
action = [GET:"list", POST: "create"]
"/book/$id" (resource: "book")
Entry with /book maps GET and POST HTTP requests to this URL to list and create controller methods. Below we map URLs like /book/$id using default grails REST mapping:
  • GET -> show
  • PUT -> update
  • POST -> save (used when you know ID of item you create - not our case;)
  • DELETE -> delete
Implementation of our controller will use GORM for data storage and will communicate with client application using JSON format. Edit /grails-app/controllers/BookController.groovy:

import grails.converters.*

class BookController {
def list = {
response.setHeader("Cache-Control", "no-store")
render Book.list(params) as JSON

def show = {
Book b=Book.get(
render b as JSON

def create = {
def json = JSON.parse(request);
def b = new Book(json)
response.status = 201
response.setHeader('Location', '/book/'
render b as JSON

def delete = {
Book b=Book.get(
render(status: 200)
I guess this code is quite easy to understand. In create operation we need to send 201 HTTP code and URL to newly created item in Location header. I did not implement save and update methods to simplify my tutorial.

Run this code with grails run-app and point your browser to http://localhost:8080/GrailsRestTest/book. You should get following response:
[{"class":"Book","id":1,"author":"Stephen King","title":"The Shining"},{"class":"Book","id":2,"author":"James Patterson","title":"Along Came a Spider"}]
To get particular item try http://localhost:8080/GrailsRestTest/book/1 - response should be:
{"class":"Book","id":1,"author":"Stephen King","title":"The Shining"}
Client interface with Dojo
Install Dojo with:
grails install-plugin dojo

Add following page to /grails-app/views/gui/books.gsp:
<script type="text/javascript" src="<g:createLinkTo file="/js/dojo/dojo-1.3.0/dojo/dojo.js" />" djConfig="parseOnLoad:true, isDebug:false">
<script type="text/javascript" src="<g:createLinkTo file="/js/dojo/dojo-1.3.0/dijit/dijit.js" />"></script>
<style type="text/css">
@import "<g:createLinkTo file="/js/dojo/dojo-1.3.0/dijit/themes/tundra/tundra.css" />";
@import "<g:createLinkTo file="/js/dojo/dojo-1.3.0/dojox/grid/resources/Grid.css" />";
@import "<g:createLinkTo file="/js/dojo/dojo-1.3.0/dojox/grid/resources/tundraGrid.css" />";
<body class="tundra">
<script type="text/javascript">

dojo.addOnLoad(function() {
var restStore = new{target:"<g:createLink controller="book" />"});

var layout = [
field: 'id',
name: 'Id',
width: '50px'
field: 'title',
name: 'Title',
width: '200px'
field: 'author',
name: 'Author',
width: '100px'

var grid = new dojox.grid.DataGrid({
store: restStore,
structure: layout,
selectionMode: 'single'



saveBook = function(book) {
// to force grid refresh

removeBook = function() {
var book = grid.selection.getSelected()[0];
if (book !=null && != null) {

showAddForm = function() {

<div id="gridContainer" style="width: 375px; height: 200px;"></div>
<button dojoType="dijit.form.Button" onclick="showAddForm()">Add book</button>
<button dojoType="dijit.form.Button" onclick="removeBook()">Remove</button>

<div dojoType="dijit.Dialog" id="dialog1" title="Add book"
<td><label for="title">Title: </label></td>
<td><input dojoType="dijit.form.TextBox" type="text" name="title" id="title"></td>
<td><label for="author">Author: </label></td>
<td><input dojoType="dijit.form.TextBox" type="text" name="author" id="author"></td>
<td colspan="2" align="center">
<button dojoType="dijit.form.Button" type="submit">Save</button></td>
Our client is based on three Dojo components:
  • JsonRestStore - responsible for transparent client-server communication
  • DataGrid - grid presenation for data
  • Dialog - simple popup for adding new book items
After starting application and pointing your browse to http://localhost:8080/GrailsRestTest/gui/books.gsp you should see:

DataGrid asked JsonRestStore for data and it made following request to server:
GET /GrailsRestTest/book/? HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv: Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Accept: application/json,application/javascript
Accept-Language: pl,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Range: items=0-24
Content-Type: application/json
X-Requested-With: XMLHttpRequest
Referer: http://localhost:8080/GrailsRestTest/gui/books.gsp
Response contains JSON data as presented earlier.

Clicking Add button will show popup that allows creating new items:

Submitting data will result in following request and response:
POST /GrailsRestTest/book/ HTTP/1.1
Host: localhost:8080
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv: Gecko/20091221 Firefox/3.5.7 (.NET CLR 3.5.30729)
Accept: application/json,application/javascript
Accept-Language: pl,en-us;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Transaction: commit
Content-Type: application/json; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://localhost:8080/GrailsRestTest/gui/books.gsp
Content-Length: 49
Pragma: no-cache
Cache-Control: no-cache
{"title":"Test author 1","author":"Test title 1"}

HTTP/1.x 201 Created
Server: Apache-Coyote/1.1
Location: /book/3
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Sat, 30 Jan 2010 13:59:06 GMT
At that is the moment when I had some problems. You probably noticed commented line with grid.sort() in gsp page. It forces grid and JsonRestStore to reload data from server after saving new item. When you remove this line, you will notice that new item is still added to store and even to the table (without reloading) - but ID of new item is missing. According to Dojo documentation JsonRestStore should pick this ID from Location http header of response. Unfortunatelly it does not. Am I doing something wrong? Or is this a bug? Hints are welcome!

To remove item select it and push Remove button. Dojo will make request similar to:
DELETE /GrailsRestTest/book/8 HTTP/1.1
Host: localhost:8080
This time item is automatically removed from store and grid and any data reloading is not needed.

The title of this article starts with "Almost" as it is not complete application. I did not implement any update operation. Also this ID issue with Dojo makes me a little worried. Application was tested only on Firefox... so to make it "production ready" you should also test (or get it working ;) on other browsers. Anyway, it was fun to try something new...