We need a little structure

We were finally able to get through to the remote server after adjusting the dates to use GMT:

>>> import time
>>> dep = SOAPpy.dateTimeType((2004, 3, 24, 12, 30, 59, 4, 86, 0))
>>> ret = SOAPpy.dateTimeType((2004, 3, 26, 12, 30, 59, 4, 86, 0))
>>> proxy.getAirFareQuote(dep, ret, 'den', 'iad')

... Long, bewildering SOAP fault snipped ...

Python's time.strptime, which has been the subject of well-deserved abuse by many developers, has no way to handle proper ISO 8601 time zone offsets. The closest format specifier, %Z, which only accepts civil time zone names such as MST, is not permitted in ISO-8601. In the end, we just used the ugly time tuples directly. A SOAP message was constructed and sent to the server. We were unable to get that far with ZSI until after the recent publication of the article (click here for an update). Unfortunately, the server was not obliging. It sent back an extremely confusing SOAP fault--a huge Java Stack trace from which the key line was: org.xml.sax.SAXException: No such operation 'getAirFareQuote. We knew something was fishy about the fault message because we could use an on-line SOAP debugger to access the getAirFareQuote method. Indeed, if we tried a little experiment:

>>> proxy.getAirFareQuote()
[]
>>>

      

We did get a result. The fault message was a confusing way of saying "I don't understand the way you are invoking this method." Googling around a bit led to the clue that we should read the WSDL more closely. The server was actually expecting a multi-ref defined in a structure called in0, comprising the four parameters we had been passing directly. So, we needed to deal with structure types. The document docs/complexTypes.txt was no help at all in finding out how to do so. We had to pore through SOAPpy code to find the probable class we needed, SOAPpy.structType(). A little experimentation led us to:

>>> in0 = SOAPpy.structType()
>>> in0._addItem('outwardDate', dep)

>>> in0._addItem('returnDate', ret)
>>> in0._addItem('originAirport', 'den')
>>> in0._addItem('destinationAirport', 'iad')

      

This time we got a different SOAP fault -- a Java traceback with initial message org.xml.sax.SAXException: No deserializer defined for array type {http://www.w3.org/1999/XMLSchema}ur-type. Time to see what WSDL was actually sending on the wire. We turned on the SOAPpy's debugging facilities. Luckily, unlike in the 0.9x versions of SOAPpy, there is a simple way of turning on and off the debugging code by setting SOAPpy.Config.debug to 0 or 1.

>>> SOAPpy.Config.debug = 1

>>> proxy.getAirFareQuote(in0)

      

This put out a huge amount of text, but we found the outbound SOAP payload buried in the output.

*** Outgoing SOAP ******************************************************
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope
  SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"
  xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/"
  xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance"
  xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
  xmlns:xsd3="http://www.w3.org/2001/XMLSchema"
  xmlns:xsd="http://www.w3.org/1999/XMLSchema">
<SOAP-ENV:Body>
<ns1:getAirFareQuote
  xmlns:ns1="urn:SBGAirFareQuotes.sbg.travel.ws.dsdata.co.uk"
  SOAP-ENC:root="1">
<v1 href="#i1"/>
</ns1:getAirFareQuote>

<xsd:v1 id="i1" SOAP-ENC:root="0">
<outwardDate href="#i2"/>
<returnDate href="#i3"/>
<originAirport href="#i4"/>
<destinationAirport href="#i5"/>
</xsd:v1>
<outwardDate SOAP-ENC:arrayType="xsd:ur-type[4]" xsi:type="SOAP-ENC:Array"
             SOAP-ENC:root="0" id="i2">
<item href="#i3"/>
<item href="#i3"/>

<item href="#i4"/>
<item href="#i5"/>
</outwardDate>
<returnDate xsi:type="xsd3:dateTime" id="i3"
            SOAP-ENC:root="0">2004-03-24T12:30:59Z</returnDate>
<originAirport xsi:type="xsd:string" id="i4"
            SOAP-ENC:root="0">den</originAirport>
<destinationAirport SOAP-ENC:arrayType="xsd:string[2]"
            xsi:type="SOAP-ENC:Array" SOAP-ENC:root="0" id="i5">
<item href="#i6"/>

<item href="#i7"/>
</destinationAirport>
<item xsi:type="xsd:string" id="i6" SOAP-ENC:root="0">phl</item>
<item xsi:type="xsd:string" id="i7" SOAP-ENC:root="0">iad</item>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
************************************************************************

      

We have added some new lines and indentation for formatting reasons. This is clearly a complete mess. The W3C XML Schema namespace and the basic data structuring is from long-obsolete versions of SOAP proposals. It is hard to imagine how to convert this into anything a current SOAP implementation would recognize without a huge amount of effort. Scouring through the SOAPpy code for flags and options that might help did not turn up anything very encouraging. At this point, we gave up trying to access the airfare quote service. Certainly, we could access the getAirlines method easily enough:

>>> SOAPpy.Config.debug = 0
>>> proxy.getAirlines()
['Alitalia', 'American Airlines', 'BMI', 'BMIBaby', 'British Airways',
 'Continental', 'EasyJet', 'EBookers', 'Expedia', 'Global Traveller',
 'Lufthansa', 'Northwest Airlines', 'Maersk Air', 'Opodo', 'Qantas',
 'Ryanair', 'Star Alliance', 'Travelocity', 'United']
>>> 

      

This, of course, is back to the trivial flavor of Web services we had been hoping to move beyond. The WSDL module is a welcome addition to SOAPpy and ZSI because it simplifies the process of working out the low-level details of the service. However, it does not help any if you need the complex data structure marshalling that Python SOAP implementations seemingly have not caught up to yet. Again, we would like to emphasize that this is an indictment of how terribly arcane SOAP RPC complex types are. This is a problem the entire SOAP community urgently needs to address. Luckily, with the emergence of the document/literal flavor of SOAP, there is some hope for sanity.

A ZSI update

After our last article was published, the core developer of ZSI, Rich Salz, sent an email indicating the following:

At some point, the return value of time.strptime() changed from a tuple to a <type 'time.struct_time'>. [...]The following fragment -- note the tuple() calls for dep and ret -- [gets] further.

A ZSI bug? Python advancing too quickly? Your call ...

We tried out the modified code he suggested:

>>> from ZSI import ServiceProxy
>>> wsdl = 'http://wavendon.dsdata.co.uk:8080\
... /axis/services/SBGGetAirFareQuote?wsdl'
>>> proxy = ServiceProxy(wsdl)
>>> import time
>>> ISO_8601_DATETIME = '%Y-%m-%dT%H:%M:%S'
>>> dep = tuple(time.strptime('2003-12-06T12:30:59', ISO_8601_DATETIME))
>>> ret = tuple(time.strptime('2003-12-12T12:30:59', ISO_8601_DATETIME))

>>> proxy.getAirFareQuote(dep, ret, 'den', 'phl')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/ServiceProxy.py", 
    line 82, in __call__
    return self.parent()._call(self.__name__, *args, **kwargs)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/ServiceProxy.py", 
    line 65, in _call
    apply(getattr(binding, callinfo.methodName), args)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 28, in __call__
    requesttypecode=TC.Any(self.name, aslist=1))
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 132, in RPC
    return self.Receive(replytype, **kw)
  File "/home/uogbuji/lib/lib/python2.3/site-packages/ZSI/client.py", line 261, in Receive
    raise FaultException(msg)
ZSI.FaultException: org.xml.sax.SAXException: No such operation 'getAirFareQuote'
>>> 

      

The use of the modified code got us to the point of getting the SOAP request transmitted to the server, but we were back to figuring out how to send out the precise structure the remote method is expecting.

Wrap-up

Even though our efforts to access the air fare Web service were frustrating, we can see that ZSI and SOAPpy have made important strides. Our intent in these articles are not to disparage these packages, but to lay out the traps we fell into so that others might avoid them. We do believe that a lot of the problem here is that SOAP is far too complex as soon as you venture outside the realm of trivial Web services. As we mentioned, the growing acceptance of document/literal message encodings reduces the problems of data architecture to the same ones that are well understood in XML. Luckily, there is a lot that you can accomplish without venturing into the snake-pit of complex structure types. In recent articles, we have had success accessing Google's and Amazon.com's Web services APIs using SOAPpy (and some testing indicates that ZSI would work as well). Even these commercial services avoid complex structure types. There is no reason the air fare quotes service could not just accept simple positional parameters. You should be able to use ZSI and SOAPpy for most SOAP tasks you come across, but it is important to be aware of their limitations.