|
|
2008 Summer School |
As more is done in a distributed fashion, it is important to learn how to interact with services using standard protocols. In this exercise, we will explore interaction with REST-type services and SOAP-type services through the Python language. The NVOSS software package contains all the infrastructure we require.
REST (Representational State Transfer) services access endpoints using defined HTTP methods (GET, POST, PUT, DELETE). These methods are then mapped by the server to specific actions. Arguments can be sent either as part of the URL in the GET and DELETE cases or as part of the body of the message in the POST and PUT cases. The response to the request will contain a status code and optionally content in the body of the message which can be parsed for useful information. The NVOSS software package contains an example REST service written in python that we will use to examine the use of python in REST type communication.
Demonstration code has been written and resides in the python branch of the NVOSS software distribution. Follow Listing 1 to import the code. The Listings are written in a *nix style and may need modification on other operating systems.
> source $NVOSS_HOME/bin/setup.csh > cd $NVOSS_HOME/python/src/webservice/ *Edit ImageAnnotationService.py. Set imgdir and commentdir to point to valid paths on your filesystem. > python ImageAnnotationService.py [19/Aug/2008:16:20:30] ENGINE Listening for SIGHUP. [19/Aug/2008:16:20:30] ENGINE Listening for SIGTERM. [19/Aug/2008:16:20:30] ENGINE Listening for SIGUSR1. [19/Aug/2008:16:20:30] ENGINE Bus STARTING [19/Aug/2008:16:20:30] ENGINE Started monitor thread '_TimeoutMonitor'. [19/Aug/2008:16:20:30] ENGINE Started monitor thread 'Autoreloader'. [19/Aug/2008:16:20:30] ENGINE Serving on 127.0.0.1:8080 [19/Aug/2008:16:20:30] ENGINE Bus STARTED > cd http_client > python HTTP_client.py 200 OK 200 OK 200 OK 200 OK 200 OK My distance is(GET method): 2832.938230 Mpc/h 200 OK My distance is(POST method): 2832.938230 Mpc/h Listing 1
For interactive access to the methods in HTTP_client.py:
> python
>>> import HTTP_client as hc
Try it out!
>>> file = r"/home/simon/nvoss2008/python/data/whirlpool.jpg"
>>> uri = "/whirlpool"
Note: The leading slash in the previous line is important
>>> hc.PUTImage(file, uri)
200 OK
>>> hc.POSTComment("This is a pretty picture of M51", uri)
200 OK
Listing 2Your browser does a GET call by default so we can use python calls or our browser to look at the results of Listing 2. If you used the same identifier as listed, you can go to http://localhost:8080/whirlpool to see the uploaded image and comment.
Python has built in methods to take care of construction of HTTP headers. POST and PUT can contain bodies in which case the Content-length keyword is automatically set in the header. Get and Delete don't have bodies. The demonstration code uses the built in httplib python package. Let's look at the code to "PUT" an image.
from httplib import *
baseurl = "localhost:8080" Location of the Service
def PUTImage (file, uri): This takes the path to the file to upload and some identifier for it
binarr = open(file, 'rb').read() Read the file into a binary array
connection = HTTPConnection(baseurl) Open a connection to the service
connection.request("PUT", uri, binarr) Use PUT method to path set by uri and set the body to contain binarr
response = connection.getresponse() Get the response so that the status can be checked
print response.status, response.reason Print out the response status (should be 200 OK)
Listing 3That's all there is to it. The request method can be set to any of the defined HTTP request methods. As shown in the GETPage method, if the service returns useful information other than the status, the read() method can be called on the response to get the content.
Johns Hopkins University (voservices.net) has made available several services that are accessible via several protocols (GET, POST, or SOAP). Let's look at the luminosity distance calculator service. Unlike most of the methods in our image annotator, the distance calculator takes several arguments and returns information in the body of the response. See http://voservices.net/Cosmology/ws_v1_0/Distance.asmx?op=Luminosity for descriptions of the messages.
Most REST services that use the GET method use the convention that
the service location is defined by the path following the base URL
with arguments following a ? separated by &. The luminosity
distance method takes four arguments: redshift, Hubble parameter, Ωm,
and ΩΛ. These arguments will be appended in name/value
pairs to the path to the services. For example, try this call to the
distance
calculator:
http://voservices.net//Cosmology/ws_v1_0/Distance.asmx/Luminosity?z=0.5&hubble=0.70&omega=0.3&lambda=0.7
We can do this programmatically in python using httplib. We will use the GET method and construct the URL by hand.
>>> args = hc.urlencode({'z':'0.5','hubble':'0.7','omega':'0.3','lambda':'0.7'}) urlencode automatically adds the & and = characters
>>> print args
hubble=0.7&z=0.5&omega=0.3&lambda=0.7
>>> ldist = hc.GETLdist(args)
200 OK
>>> print ldist
2832.93823
Listing 3Let's look inside the method that calls the calculator:
from httplib import *
from urllib import *
import elementtree.ElementTree as ET We need ElementTree to parse the returned XML
distanceurl = "voservices.net" The base URl of the service
serviceuri = "/Cosmology/ws_v1_0/Distance.asmx/Luminosity" Path to the method of interest
def GETLdist (args):
connection = HTTPConnection(distanceurl) Open a connection to the server
uri = serviceuri + "?" + args Concatenate the service path and arguments with a ?
connection.request("GET", uri) Issue the GET request
response = connection.getresponse() Get the response so that status can be checked and output parsed
print response.status, response.reason
str = response.read() Get a string representation of the response
doc = ET.fromstring(str) Parse the XML into a tree
return float(doc.text) In our case the result is in the root element, so get text. More parsing can be done for more complicated returns.
Listing 4This service also allows for communication via the POST method. This is useful if the inputs are coming from a web form. The difference is that instead of encoding the arguments in the URL, we send the arguments in the body of the POST request.
Try the example code:
>>> args = hc.urlencode({'z':'0.5','hubble':'0.7','omega':'0.3','lambda':'0.7'})
>>> print args
hubble=0.7&z=0.5&omega=0.3&lambda=0.7
>>> ldist = hc.POSTLdist(args)
200 OK
>>> print ldist
2832.93823
Listing 5Good, it worked. How does the code for calling via POST differ from that for calling via GET?
from httplib import *
from urllib import *
import elementtree.ElementTree as ET Once again we need ElementTree to parse the output
distanceurl = "voservices.net" The server url
serviceuri = "/Cosmology/ws_v1_0/Distance.asmx/Luminosity" Path to the service
def POSTLdist (args):
connection = HTTPConnection(distanceurl) Get connection to the service
headers = {"Content-type": "application/x-www-form-urlencoded"} We need to set the Content-type header keyword correctly. See message description.
connection.request("POST", serviceuri, args, headers) Issue the POST request with the path to the service, set args to the body, and add the extra header value
response = connection.getresponse() Get the response
print response.status, response.reason Check status
str = response.read() Read response body
doc = ET.fromstring(str) Parse XML return
return float(doc.text) Return value of root node
Listing 6SOAP web services use a transport protocol (usually HTTP in our case) to send verbose XML messages adhering to the SOAP specification. Both request and response are sent as SOAP XML documents. As such, SOAP is inherently platform and language independent and lends itself to being extensible. One attractive aspect of SOAP web services is that they can be described using WSDL (Web Service Description Language). Using the WSDL, many language can auto generate the code for writing and reading SOAP messages from a particular service. The NVOSS software package contains a demonstration server and client.
Follow Listing 7 to set up the server and run the test client. Once again, the examples are done in a *nix way assuming tcsh as the shell.
> source $NVOSS_HOME/bin/setup.csh > cd $NVOSS_HOME/python/src/webservice > python AbsoluteMagnitudeService.py [20/Aug/2008:11:35:34] ENGINE Listening for SIGHUP. [20/Aug/2008:11:35:34] ENGINE Listening for SIGTERM. [20/Aug/2008:11:35:34] ENGINE Listening for SIGUSR1. [20/Aug/2008:11:35:34] ENGINE Bus STARTING [20/Aug/2008:11:35:34] ENGINE Started monitor thread '_TimeoutMonitor'. [20/Aug/2008:11:35:34] ENGINE Started monitor thread 'Autoreloader'. [20/Aug/2008:11:35:34] ENGINE Serving on 127.0.0.1:8080 [20/Aug/2008:11:35:34] ENGINE Bus STARTED > cd webservice_client > wsdl2py --lazy -b "http://localhost:8080/AbsMagService?wsdl" > python AbsMagClient.py Hipp_ID Vmag Plx V_absmag 1168 4.79 10.01 -0.207829612603 71939 8.81 14.31 4.5881981688 70890 11.01 772.33 15.4490145234 Listing 7
SOAP interaction is sometimes slightly more complicated than REST type because, in general, complex data types are being used in both the request and response phase. Luckily, most code generation tools will generate the code to manipulate the complex types. In the case of this absolute magnitude calculator, both the input and output are in VOTable format. This means that we must build a VOTable to send and parse the VOTable that is returned. The methods for dealing with the VOTable are attached to the Request and Response objects.
Let's get started...
from AbsMagService_client import *
import sys
def callSVC():
loc = AbsMagServiceLocator() First get the location of the service
tracefile = open('message.soap', 'w') Optionally, set a file to contain the response and request
kw = { 'tracefile' : tracefile }
svc = loc.getAbsMagService_PortType(**kw) Get instances of the generated client code
wsreq = calcAbsMagRequest() Get an instance of the Request object
vot = wsreq.new_VOTABLE() Get instance of the VOTABLE object
.
. Read values into a VOTABLE object
.
wsreq.set_element_VOTABLE(vot) Assign the VOTABLE to the Request object
response = svc.calcAbsMag(wsreq) Call the service and get the response back
vot_ret = response.get_element_VOTABLE() Get the VOTABLE from the response and operate on it.
Listing 8That's really all there is to it. The methods exposed by SOAP services are generally wrapped to look like regular method calls. In this aspect, SOAP interaction is generally more transparent than REST calls in that it is not usually necessary to compose the message as this is done by the generated code.
The NVO Summer School is made possible through the support of the National Science Foundation.
|
|
