Web services for environmental applications Practical session

Introduction

This practical session is a tutorial intended to give a brief introduction to the use and creation of Web Services. A basic knowledge of Web Services and Java programming is assumed, although it is possible to follow the main points of this tutorial without knowledge of either (familiarity with object-oriented programming will help but is also not required). (If your preferred language is C# under a .NET environment, please note that many of the tools available under .NET are very similar to the Apache Axis tools that we shall be using here.) This practical session will be conducted in a Windows environment, but the Java code that we will create would work on any platform.

We are using Apache Tomcat as the application server and Apache Axis as the SOAP (Simple Object Access Protocol) implementation. Other applications and libraries could be used, but the Tomcat/Axis partnership is one of the most widely used, as well as being free, open-source software.

This tutorial is in four parts:

1. Creating a client program for an existing Web Service
2. Creating a simple Web Service
3. Creating a Web Service that wraps an existing piece of binary code
4. Calling a more complex Web Service, the Microsoft TerraServer

A useful reference for optional further reading is the Apache Axis User Guide (http://ws.apache.org/axis/java/user-guide.html), which will explain in greater detail what we are about to do.

Checking the local setup

Before you start, it's worth verifying that your machine has been set up correctly. To do this, open a new command prompt (Start Menu -> Programs -> Accessories -> Command Prompt) and enter the command "echo %CLASSPATH%", noting the percentage signs. You should be presented with a few lines of output containing (amongst many others) the string "axis.jar". If nothing appears, please contact a supervisor, or make sure your setup is correct.

All the code that you will create will be contained in subdirectories of one directory, referred to as your "working directory" in this document. Example solutions are available for all sections from here. Please try to complete the problems without looking at the solutions if you can, but don't hesitate to ask for help if you are stuck.

Part One: Creating a client program for a Web Service

In this first section, we'll go through the steps of creating a simple client program that is capable of calling a Web Service running on a remote machine. You will probably recall two of the major advantages of the Web Services architecture:

• Platform independence (so you don't have to know or care what sort of machine the remote Web Service is running on)
• Self-description with Web Services Description Language (WSDL)

These things mean that it's very easy to automatically generate client code for a Web Service. The WSDL document defines all the inputs and outputs of a service, the order in which they should appear, the location of the service and so forth. We shall use a tool called WSDL2Java to read WSDL files and automatically generate the majority of our Java code for us.

Haven't you always wanted a program that can tell you the temperature at any zip code in the States? Well, you'll be glad to know that there's a Web Service that does just this. It's published on the XMethods (http://www.xmethods.net) site, which contains lots of Web Services to do various things (look at the "Full List" on the site).

Step 1: Examine the WSDL

You can examine the WSDL for the service by opening a web browser and going to http://www.xmethods.net/sd/2001/TemperatureService.wsdl. If you're using a fairly modern browser you should see the WSDL nicely formatted, just like any other XML document.

Take a moment to look through the WSDL and see if you can spot the key parts. The "portType" section defines that the "getTemp" method of the service will expect a request of type "getTempRequest" and will respond with a "getTempResponse". Just above the portType, the getTempRequest and getTempResponse types are defined simply as a string and a floating-point number respectively. So the Web Service expects a single string (the zip code) and will respond with a floating-point number (the temperature).

Step 2: Create the client-side stub code

The first thing we have to do is create the "stub code" that's going to do most of the work of calling the service. (Behind the scenes, this stub code will take a String, wrap it in a SOAP message, send it to the Web Service, wait for the SOAP response, decode the response and return the decoded floating-point number. But you don't have to worry about any of that.) To create the stub code, open a command prompt and change into your working directory (e.g. "cd C:\work"). Create a new subdirectory called "part1" to hold the files from this part of the tutorial ("mkdir part1") and change into it. Then run the command (all one line):

java org.apache.axis.wsdl.WSDL2Java http://www.xmethods.net/sd/2001/TemperatureService.wsdl

If you get any error messages, check that you've entered the command correctly, then ask for help if it still doesn't work.

You should find a new directory (called "net") in the current directory. Drilling down into the subdirectories, you should find four Java files in net\xmethods\www\sd\TemperatureService_wsdl. These are the stub files, i.e. the Java representation of the client side of the Web Service.

(Note: How did the files end up in this strange directory hierarchy? Look again at the WSDL for the service. On the first line, there's a TargetNamespace. This namespace has been turned into a directory hierarchy, and hence a Java package hierarchy. You can override this default behaviour if you like; see the Axis manuals. We shan't bother here.)

We can safely ignore most of the contents of the stub files. However, one file is worth a look: TemperaturePortType.java. Open this file and examine its contents:

package net.xmethods.www.sd.TemperatureService_wsdl;

public interface TemperaturePortType extends java.rmi.Remote
{
    public float getTemp(String zipcode) throws RemoteException;
}

Remember the PortType from the WSDL document? This is its representation in Java. If you're not familiar with Java, a brief explanation is probably in order here. (If you are, please skip the rest of this paragraph.) The first line defines the package to which this code belongs. You'll notice that it corresponds exactly to the directory hierarchy in which you found this file. The second line shows that this is an interface. A Java interface is simply a definition of a set of methods, without any actual implementation of those methods (in other words, it does practically the same job as a WSDL file). What this file is saying is that the TemperaturePortType defines only one method that takes a String as an argument and returns a float. The implementation of the getTemp() function is the Web Service itself.

Step 3: Write a program to call the Web Service

Now we're ready to write the code that calls the Web Service to get the temperature at our zip code of choice (exciting, isn't it?). Change back into the "part1" subdirectory of your working directory (make sure you really are in this directory! Things won't work otherwise). Using any text editor, create a file called TemperatureServiceMain.java. If you're familiar with Java, you might like to have a go at coding this from scratch. If you do this, please note the following tips:

• You'll need to import all the stub classes that we've just generated.
• You get an instance of TemperaturePortType by first creating a ServiceLocator object, then calling getTemperaturePort() on this object.

A possible TemperatureServiceMain.java is provided with the solutions. Having created your code (or copied it from the solutions directory if you're not comfortable with Java), compile it with "javac TemperatureServiceMain.java". If you get messages about classes not being found or similar, make sure you've imported the stub classes. You may have noticed that the stub classes have automatically been compiled.

Run your code by entering "java TemperatureServiceMain" on the command line. If you're using the prepared solution, you should see a message like "The temperature at 19147 is 72.0". By the way, the units are Fahrenheit - but there was no way to tell that from the WSDL! The only way to contain this information in the WSDL would have been as a comment or between <documentation>> tags. There is no standard way (yet) to encode this semantic information in the WSDL.

If it worked, well done! It took us a few pages in this first walkthrough, but in principle what we have done is pretty straightforward, and should work for all standards-compliant Web Services:

1. Find out where the WSDL is
2. Use WSDL2Java to automatically generate Java stub code from the WSDL
3. Write a main() class that uses the stub code.

Part One and a Half (optional): another way of creating a client program

(If you're a Java novice, or if you're keen to get on, you can skip this section, but you might like to skim through to get the gist, even if you don't actually go through the exercise.)

There is an alternative way to create a client program. Instead of creating Java stub code automatically using WSDL2Java, you can write a client program directly using lower-level Axis classes. The basic parts of such a program are:

1. Create a Call object representing the operation to be performed
2. Set the endpoint address of the Call (i.e. the Web Service's location)
3. Set the name of the operation you want to perform
4. Invoke the Call object, passing an array of Objects as the parameters

As you might think, this is essentially what is done by the Java stub code that WSDL2Java creates. Note that you still (of course) need to know the location of the Web Service and the name and signature (i.e. the number and types of the input and output parameters) of the operation you want. This usually means that you need to read and understand the WSDL yourself.

Here is an example client written in this way for the same Web Service as in Part One:

import org.apache.axis.client.Call;
import javax.xml.namespace.QName;
public class TemperatureServiceMain2
{
	public static void main(String[] args) throws Exception
	{
		// Create a new Call object, setting the location (gotten from the WSDL)
		Call call = new Call("http://services.xmethods.net:80/soap/servlet/rpcrouter");
		
		// Set the name of the operation to call (from the portType in the WSDL)
		// Note that we need to specify the namespace of the operation,
		// also gleaned from the WSDL
		call.setOperationName(new QName("urn:xmethods-Temperature", "getTemp"));
		
		String zipCode = "19147"; // Zip code for Philadelphia
		// Call the Web Service, passing the parameter as an array of Objects
		// Similarly, the return value is a plain Object
		Object ret = call.invoke(new Object[]{zipCode});
		
		// Cast the return value to the correct type
		float temp = ((Float)ret).floatValue();
		System.out.println("The temperature at " + zipCode + " is " + temp);
	}
}

An advantage of this approach is that this class is all we need to call the service; no stub code is required. A disadvantage is that we have to be able to read the WSDL ourselves to extract the relevant information. Also, the code is less clean since it involves the use of casting to and from Objects; it is much harder to see the types of the input parameters and the return value. This is not a "better" or "worse" way of creating a client program; it may or may not be appropriate for you, depending on the context. It is presented here just so you know that the possibility exists. (Note that it would be a lot of work to write this kind of code from scratch to access complex services such as the TerraServer in Part Four below. In the case of services like the TerraServer, it's definitely much easier to use WSDL2Java.)

Part Two: creating a simple Web Service

(If you're not interested in how to create a Web Service, you might like to skip Parts Two and Three of this tutorial, and move on to Part Four, where you'll access a real Web Service, the Microsoft TerraServer.)

Now let's create a Web Service of our own. This is not very much more complicated than creating a client program, since the Axis tools do most of the hard work for us. We're going to create a Web Service that performs simple arithmetic operations on pairs of numbers. We shall go through the following steps:

1. Write a Java interface describing the Web Service's methods
2. Use Java2WSDL to create the WSDL for our Web Service
3. Use WSDL2Java to create server-side stub code for our Web Service
4. Implement the methods of the Web Service
5. Deploy the Web Service

Step 1: Create the Java interface

You may remember from above that a Java interface is a collection of specifications of methods, with no implementation, and that it performs much the same task as a WSDL file. The easiest way to generate our WSDL, therefore, is to write a Java interface and convert it using the Axis tool Java2WSDL.

Create a new directory in your working directory called "part2". In this directory, create a directory called "wseaworkshop" and in that directory create a directory called "exercise2". Inside the "exercise2" directory, create a new Java file called Arithmetic.java. This is the file that will contain the Java interface.

We can define as many methods as we like in this interface. Here I shall just show two methods, add() and multiply(), but you might like to add some more of your own:

package wseaworkshop.exercise2;

public interface Arithmetic
{
    public float add(float val1, float val2);
    public float multiply(float val1, float val2);
}

Please note the following important points: (1) the package declaration has to match the directory in which the interface resides; (2) the name of the interface (Arithmetic) has to match the name of the file (Arithmetic.java). These are Java's rules, not Axis's.

Now change back into the "part2" directory so that you are at the same level as the "wseaworkshop" directory. Compile the interface into a .class file with the command:

javac wseaworkshop/exercise2/Arithmetic.java

Step 2: Convert the Java interface to a WSDL file

Convert the interface into a WSDL file with the Java2WSDL command (this needs to be entered as a single line, but we shall show the different arguments on separate lines for clarity):

java org.apache.axis.wsdl.Java2WSDL
    -o Arithmetic.wsdl
    -l"http://localhost:8080/axis/services/Arithmetic"
    -n "urn:Arithmetic"
    -p"wseaworkshop.exercise2" "urn:Arithmetic"
    wseaworkshop.exercise2.Arithmetic

(Strangely enough, we have to make sure that there is no space between "-p" and "wseaworkshop...". Don't ask me why.) Let's go through those arguments in turn:

• The "-o" argument gives the name of the WSDL file that will be generated.
• The "-l" argument gives the URL at which the Web Service will reside, when it has been deployed.
• The "-n" argument gives the target namespace for the WSDL file.
• The "-p" argument specifies that everything in the wseaworkshop.exercise2 package will belong to the urn:Arithmetic target namespace (this is a package-to-namespace mapping)

Don't worry too much about trying to understand the "-n" and "-p" arguments. Open the generated WSDL file in a web browser. You should notice that the target namespace has been set to "urn:Arithmetic" and that the wsdl:soap address location has been set to http://localhost:8080/axis/services/Arithmetic.

You will probably also notice that the WSDL file does not contain the original parameter names ("val1" and "val2") for the input parameters for the methods; it has changed them to "in0" and "in1" respectively. You don't have to do this, but it's often helpful to change these parameter names back to something more meaningful (in this case it doesn't matter too much). Open the WSDL file in a text editor and change the parameter names in all of the relevant wsdl:message sections and the parameterOrder part of the wsdl:operation sections. If you don't change the strings in the parameterOrder part, it won't work! (Tip: you could just to a global search-and-replace to replace "in0" with "val1" and so on.)

Step 3: Create the server-side stub code

We can now use WSDL2Java to automatically generate the server-side stub code that will do most of the grunt work of the Web Service. This is similar to the mechanism we used in Part One to create client-side stub code, but we need to specify a few extra arguments. Run the following command (again, all on one line), from the directory containing the WSDL file (i.e. the "part2" subdirectory of your working directory):

java org.apache.axis.wsdl.WSDL2Java
    -o . -d Session -s -S true
    -Nurn:Arithmetic wseaworkshop.exercise2.impl
    Arithmetic.wsdl

Again, let's go through those arguments:

• "-o ." means "output in the current directory"
• "-d Session" is the deployment scope. Don't worry about this
• "-s" means "output server-side stub code", as opposed to client-side code.
• "-S true" means "deploy skeleton code". Don't worry about this either.
• "-N…" is a namespace-to-package mapping, specifying that everything in the "urn:Arithmetic" namespace will end up in the Java package "wseaworkshop.exercise2.impl" (in other words, this is the package to which the stub code will belong)

You should find that the wseaworkshop.exercise2.impl directory has been created, and that it contains six Java files (the stub code) and two .wsdd files (deployment and undeployment files).

At this point, the original interface file from Step 1 and the WSDL file from Step 2 play no further part. You could delete them if you like but you might like to keep them for future reference.

Step 4: Implement the methods of the Web Service

To create the actual implementation of the Web Service, you only need to edit one file; ArithmeticServiceSoapBindingImpl.java in the wseaworkshop\exercise2\impl directory. Open this file in a text editor and notice that it is very similar to the original Java interface we created in Step 1. Also notice that the names of the parameters (val1 and val2) correspond to the names of the parameters in the WSDL file, after we edited it at the end of Step 2.

Change the default method implementations to do what you want. For example, in the case of the add() and multiply() functions, just replace the default implementation with "return val1 + val2;" and "return val1 * val2;" respectively, remembering the semicolons at the end of each line. Save the file.

Now we're going to compile all of the Web Service's code. Make sure you are in the "part2" directory (i.e. at the same level as the "wseaworkshop" directory) and type:

javac wseaworkshop\exercise2\impl\*.java

No error messages should appear.

Step 5: Deploy the service

Nearly finished! In order for others to use our new Web Service, it only remains to deploy the service under the Tomcat application server. First, we will package up the compiled Java code into a Java archive (.jar file). From the "part2" directory, execute:

jar -cf Arithmetic.jar wseaworkshop\exercise2\impl\*.class

Now we copy this archive into a directory where Tomcat and Axis can find it:

copy Arithmetic.jar "%CATALINA_HOME%\webapps\axis\WEB-INF\lib"

(CATALINA_HOME is the Tomcat root directory.) Note the quotes around the second argument to the copy program; they are needed in case the CATALINA_HOME environment variable contains spaces.

Now we need to start the Tomcat application server. It should already have been installed for you, so go to the Windows Start Menu and click on Programs->Apache Tomcat 4.1->Start Tomcat. Give it a few seconds to start up and then verify that the server is running by pointing your web browser at http://localhost:8080. You should see the Tomcat welcome page (ask for help if you don't).

With the Tomcat server running, we can now deploy the Web Service. From the "part2" directory, run (all one line):

java org.apache.axis.client.AdminClient wseaworkshop\exercise2\impl\deploy.wsdd

You should get a "Done processing" message confirming that the Web Service has been deployed. Using a web browser, navigate to http://localhost:8080/axis. You should see the Axis front page. Click on "View the list of deployed Web Services". You should see your new service "ArithmeticService" alongside (probably) two other built-in services, called AdminService and Version. If you don't see your newly-deployed service here, restart the Tomcat server. If you still don't see it, check that you have followed all the above steps properly and ask for help if you need it.

Step 6: Create a client program to test your service

By this stage all the signs are good that our Web Service has been deployed correctly. But to make sure, we need to test it out. The good news is that the client-side stub code has already been generated (it is part of the server-side stub code). So all we have to do is to create a program with a main() function that calls the stub code. Try it.

Part Three: Wrap a piece of existing code as a Web Service

A very common thing to want to do is to provide access to an existing (legacy) piece of code by wrapping it in a Web Services interface. Very often the existing code can be used without modification, no matter what language it was written in. Each case is different, but the steps involved are normally:

1. Decide what methods the Web Service will have. The same executable can be used to implement many methods.
2. Create server-side stub code to create a Web Service with these methods, just as we did in Part Two above.
3. Implement the methods of the Web Service (as in Step 4 of Part Two).

The last part is usually the trickiest and there is no set formula for doing this; it depends on how the executable gets its input and how it outputs results. The executable could read input parameters from the command line, from standard input (i.e. the keyboard) or from one or more input files. The output could appear on the standard output stream (i.e. the console window), in one or more output files or the standard error stream.

We can't cover all these cases in this tutorial. We shall cover one of the simplest but most common cases, in which input data are read from an input file and output data are written to an output file. The program that we will wrap is a FORTRAN program called GULP, which you can download from here. This program calculates some thermodynamic quantities (e.g. free energy) and physical properties (e..g. elasticity) of a given mineral, using models for the forces between atoms.

GULP takes as its input a file (which is always called input.txt). This file provides GULP with information on the atomic structure of the mineral, information on the models used to describe the forces between atoms, a set of control commands (such as telling GULP to relax the structure to the lowest energy state, or to calculation the vibrations in order to be able to compute the free energy), and other control parameters (such as temperature and pressure). Here we shall not generate these input files from scratch as it would take too long to explain their structure. Instead, we have provided five example input files, one for each of the following minerals: andalusite, sillimanite, olivine, spinel, cordierite. The input to the service will simply be the name of the mineral for which we want to perform the calculation.

GULP calculates many things, but the most important of its outputs is the free energy of the resulting structure. We shall represent this as a double-precision floating-point number with units of eV (electron volts).

The Java signature of the method that we shall implement is therefore:

double calculateFreeEnergy (String mineralName);

(If you don't understand why this is the case, please ask for help before you go any further.)

Now we have decided this, we can go ahead and create the stub code of the Web Service.

Step 1: Create a skeleton Web Service

Following through the steps in Part Two (Steps 1 to 3), create the server-side stub code for our Web Service. The only method that the Web Service will implement is the calculateFreeEnergy() method. Call the Web Service "GULP".

Step 2: Implement the calculateFreeEnergy() method

Now that we have our stub code, we can edit the implementation class (probably called GULPSoapBindingImpl.java). However, to help prevent future problems it is wise to create a standalone program that implements the calculateFreeEnergy() method, so that we can test it outside of a Web Services environment.

If you are familiar with Java, you might like to have a go at coding this program on your own. This will require familiarity with manipulating files in Java and the Runtime.exec() function that executes non-Java programs. If you are not familiar with Java or these concepts, there is a sample program called GULPTest in the solutions directory that contains a working implementation of calculateFreeEnergy() plus two helper functions.

The basic steps in the calculateFreeEnergy() method will be:

1. Check for the existence of the <mineralName>.dat input file
2. Copy the <mineralName>.dat file to "input.txt"
3. Delete the previous output file (gulp_0001.txt) if it exists
4. Start the gulp_win.exe executable
5. Wait for the executable to finish
6. Open the output file (gulp_0001.txt) and search for the string "Helmholtz free-energy". This marks the line containing the free energy.
7. Get the free energy from the line in the output file and return it.

When you are satisfied that your test program is working, copy the calculateFreeEnergy() and any necessary helper functions to GULPSoapBindingImpl.java. The only potential complication with this is that the calculateFreeEnergy() method in GULPSoapBindingImpl.java can only throw Exceptions of type java.rmi.RemoteException. In the example in the solutions directory, it is arranged such that all Exceptions are caught and re-thrown as RemoteExceptions. Don't hesitate to ask for help if you need it.

N.B. This would not be suitable for a production service, for one major reason (and most probably others too). Can you see what it is? What could happen if two clients queried the service in quick succession, so that the second query arrived before the first completed? How could you solve this problem?

Step 3: Deploy the service

Following the same procedure as in Step 5 and Step 6 of Part Two, deploy the service. Just to remind you, you will be performing the following tasks:

1. compile the server-side code
2. package it as a JAR file
3. copy the JAR file to a place where Tomcat/Axis can find it
4. (re)start the Tomcat server
5. run the Axis AdminClient on the .wsdd file for the service
6. restart Tomcat (not always necessary)
7. check that the service appears in http://localhost:8080/axis/servlet/AxisServlet

Step 4: Make a simple client program

Write a client program for your service called GULPMain.java, just as you did with the Arithmetic service in Part Two. Remember that the client-side stub code has already been generated; it is part of the server-side code. An example solution is provided in the solutions directory if you require it. Verify that your service is working.

Part Four: Use Microsoft's TerraServer

If you're comfortable with creating client programs for Web Services (and with Java), you might like to have a go at interacting with Microsoft's TerraServer Web Service. This is a far more complex Web Service than we have encountered so far in this tutorial, but the principles remain the same: get the WSDL, generate the stub code and write your code to use the Service.

The WSDL for the TerraServer Web Service is at http://terraserver.homeadvisor.msn.com/TerraService2.asmx?WSDL and further documentation can be found at http://terraserver.homeadvisor.msn.com/TerraService2.asmx. Note that the documentation assumes that you are going to be using C#, Visual Basic or one of Microsoft's other languages. You don't need to do this; we can create a Java client with exactly the method that have used previously.

Create a new directory in your working directory, called "part4". From this directory, run the WSDL2Java tool on the TerraServer WSDL and your Java stub code should appear in the localhost\TerraWebSite directory. (By examining the WSDL, you should be able to figure out why the code ended up in this directory.)

The Java interface that defines the TerraServer methods is in localhost\TerraWebSite\TerraServiceSoap.java. Have a look at the contents of this file. You'll notice that things are a little more complex than in Part One. Most of the input and output data types for the methods are now Java classes (such as PlaceFacts, AreaBoundingBox etc), rather than basic types such as strings and integers. The specifications for these complex types are given in the WSDL, and so WSDL2Java has automatically generated the code for these classes for us. The classes representing complex types are Java Beans (so you can get and set individual elements of the types with get() and set() methods).

To get you going, here is an example of a main() function that calls the getPlaceFacts() method of the Web Service to get the longitude and latitude of a given place in the US:

// Import all the Java stub classes
import localhost.TerraWebSite.*;
public class GetPlaceFactsMain
{
    public static void main(String[] args) throws Exception
    {
        TerraServiceLocator locator = new TerraServiceLocator();
        TerraServiceSoap service = locator.getTerraServiceSoap();
        Place place = new Place
        place.setCity(args[0]); // The city is the first argument
        place.setState(args[1]); // The state is the 2nd argument
        place.setCountry("US");
        // Now get the facts about this place
        PlaceFacts facts = service.getPlaceFacts(place);
        // Get the longitude and latitude of this place
        LonLatPt pt = facts.getCenter();
        System.out.println("The location of " + place.getCity() + " is (" + pt.getLon() + ", " + pt.getLat() + ")");
    }
}

Once this program is compiled you can run it, passing the name of the city and state on the command line:

java GetPlaceFactsMain Philadelphia Pennsylvania

You might like to look through the TerraServer documentation and call some other methods. This has shown us several things:

1. Platform independence: the TerraServer is a .NET application, but we have called it perfectly happily from Java.
2. The Apache Axis tools do a good job in doing the hard work of creating stub code, even for complex services.

With a bit of practice, we can generate code for calling pretty much any Web Service in a few minutes.