Home‎ > ‎

GALEON WCS "Tests"

Very Rudimentary Exercises on
GALEON WCS Servers
Using the BADC WCS Client Python Library

Ben Domenico <ben@unidata.ucar.edu>: July 28, 2008

Updated July 31, 2008 with discussion among:
Aaron Braeckel
<braeckel@ucar.edu>
Dominic Lowe <d.lowe@rl.ac.uk>
Roy Mendelssohn <Roy.Mendelssohn@noaa.gov>
Maartin Plieger <plieger@knmi.nl>
Paolo Mazzetti <mazzetti@imaa.cnr.it>

Updated August 20, 2008 by Dominig Lowe:
Added code examples.

 

Basically the code just attempted to get a coverage returned from the server. If multiple coverages were available, the first was requsted. Likewise, if multiple times were available, the initial time was requested. Where problems arose, the notes below show either the error message returned in the output file or the error message from the python interpreter. Apologies to those who don't use python, but perhaps the message will be useful anyway.

Update: Code Now Attached

 
Attached are two sample files which show how these tests were done.
  •  wcsThreddsExample.py
  •  serverTests.py
wcsThreddsExample.py is an illustration of how to contact the Unidata Thredds WCS using the python OWSLib client. This can be adapted to test other clients.
serverTests.py is the test script used in these exercises to test multiple servers. This code is published with the caveat that it was very much thrown together quickly to see how many servers a coverage could be retrieved from.


No problems from the following servers:

This simply means the server APPEARS to have returned a valid coverage. But no check was made to see if the coverage indeed made any sense.

  • 'http://oceanwatch.pfeg.noaa.gov:8081/thredds/wcs/satellite/AA/ssta/1day', version='1.0.0'

  • 'http://motherlode.ucar.edu:8080/thredds/wcs/fmrc/NCEP/NAM/CONUS_40km/conduit/NCEP-NAM-CONUS_40km-conduit_best.ncd', version='1.0.0'

  • 'http://coast-enviro.er.usgs.gov/thredds/wcs/bathy/adria15', version='1.0.0'

  • 'http://nomads.ncdc.noaa.gov:8085/thredds/wcs/narrmonthly/197901/19790101/narrmon-a_221_19790101_0000_000.grb'

As noted in the list below, some servers can be made to work with alterations to the default parameter list, e.g.:

  • 'http://oceanwatch.pfeg.noaa.gov:8081/thredds/wcs/satellite/AI/icov/mday', version='1.0.0'

  • 'http://geoservices.knmi.nl/cgi-bin/mapserver-5.0.2.fcgi', version='1.0.0'

PFEG discussion:

Error message in output file from:

'http://oceanwatch.pfeg.noaa.gov:8081/thredds/wcs/satellite/AI/icov/mday', version='1.0.0'

when the WSG84 bounding box supplied by the server is used.

<ServiceExceptionReport version='1.2.0'>
<ServiceException code='Server Error'>
Request Bounding box does not intersect Grid
</ServiceException>
</ServiceExceptionReport>

Roy Mendelssohn Comments:

Not certain why the one PFEL failed when most of the others worked.  Will check into that when I get a chance. Also, is the code you used available, so that we can replicate it in house?

Dominic Lowe Comments:

Have discussed this with Roy and it is a suspected problem with the Bounding Box extents on global datasets.

Roy, in answer to your questions, the development trunk WCS client code can be checked out from subversion here:
$svn co http://svn.gispython.org/svn/gispy/OWSLib/trunk OWSLib

A versioned release is also available for download here, but this will not contain any recent fixes:
http://pypi.python.org/pypi/OWSLib

The tests directory contains some doctests which serve as examples and should help you get started with the client. I have also asked Ben to post some sample code on the GALEON wiki.

Further Discussion:

On Tuesday 29 July 2008 16:20:09 Roy Mendelssohn wrote:
> I only have internet off and on.  If you get a chance, some of our
> datasets are clearly global  (labeled so) and some clearly are not.
> Could you do a quick test to see if it the global ones that are
> failing?  Perhaps it is that it goes form 0,360 rather than 0,359 is
> what is fouling things up.
>

Yes I think that may be the case.

e.g. This one was also 'broken', but worked ok with 0, 359:
http://oceanwatch.pfeg.noaa.gov:8081/thredds/wcs/satellite/MH/chla/mday'

Regards,

Dominic

ARGOSS discussion:

At one time, I thought I got this server to work, but I can't find a combination of fixes that will do so.

Error message in output file from

'http://services.argoss.nl/motiive/cgi/tide', version='1.0.0'

when no CRS is provided

<?xml version='1.0' encoding="ISO-8859-1" ?>
<ServiceExceptionReport version="1.2.0"
xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
si:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.1.1/OGC-exception.xsd">
<ServiceException code="MissingParameterValue" locator="crs">msWCSGetCoverage(): WCS server error. Required parameter CRS was not supplied.
</ServiceException>
</ServiceExceptionReport>

Error message in output file from

'http://services.argoss.nl/motiive/cgi/tide', version='1.0.0'

when no resx, resy is provided

<?xml version='1.0' encoding="ISO-8859-1" ?>
<ServiceExceptionReport version="1.2.0"
xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.1.1/OGC-exception.xsd">
<ServiceException code="MissingParameterValue" locator="width/height/resx/resy">msWCSGetCoverage(): WCS server error. A non-zero RESX/RESY or WIDTH/HEIGHT is required but neither was provided.
</ServiceException>
</ServiceExceptionReport>

Maarten Plieger Comments:

Apparently there are problems with some required parameters not being
filled in.
The CRS is a required parameter (coordinate reference system). And
either width/height or resx/resy are required parameters.
(ref
http://cite.opengeospatial.org/OGCTestData/wcs/1.0.0/specs/03-065r6.html#9.2.2_Key-value_pair_encoding)

I am happy with the python client code from Dominic Lowe. This helps us
improving the service. I have tested the wcs client and I get the same
errors as you do.
The problem is that I am not sure whether we should support default CRS
and Width/Height parameters...

Dominic Lowe Comments:

This server requires Width and Height parameters.

Adding "width=600, height=400" to the parameters returns a different error
possibly suggesting some type of file access problem on the server side ?

<?xml version='1.0' encoding="ISO-8859-1" ?>
<ServiceExceptionReport version="1.2.0"
xmlns="http://www.opengis.net/ogc"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.opengis.net/ogc
http://schemas.opengis.net/wms/1.1.1/OGC-exception.xsd">
 <ServiceException>

msDrawRaster(): Unable to access file. (netCDF:
%tidethamdata%:u)
 </ServiceException>
</ServiceExceptionReport>


Maarten, In terms of CRS, resX, resY, width, height I think you should be able to just pass these args through the client (or indeed any other args). The "cvg.supportedCRS" attribute should list the available CRSs for a coverage (cvg). Let me know if you have problems with this.

BADC discussion:

Error message in output file from

wcs=WebCoverageService('http://ndgbeta.badc.rl.ac.uk/wcs/badc.nerc.ac.uk__NDG-A0__AWQX8gTc', version='1.0.0'

resxFixFlag = 'YES'

<open file '/var/www/ndg/data/tmp/csml_wxs_DC8uUR.nc', mode 'r' at 0xb551d968>

Got the following message from the python interpreter one time, but can't seem to repeat it:

Making getCoverage request for PvqedYEf in cf-netcdf format.
get coverage for no initial time given
Traceback (most recent call last):
File "C:/Python25/Scripts/wcsExamplesBens.py", line 276, in <module>
output=wcs.getCoverage(identifier=CovID, bbox=bb, format=formatType)
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs100.py", line 143, in getCoverage
ServiceException: axis length 36 does not match corresponding dimension 12
>>>

Dominic Lowe Comments:

There is a bug with the BADC server which I've fixed in development but not
deployed yet.

KNMI discussion:

Nota bene that the knmi server works fine if both crs and resx/resy parameters are supplied.

Error message in the output file from

'http://geoservices.knmi.nl/cgi-bin/mapserver-5.0.2.fcgi' , version='1.0.0'

with no crs parameter

<?xml version='1.0' encoding="ISO-8859-1" ?>
<ServiceExceptionReport version="1.2.0"
xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.1.1/OGC-exception.xsd">
<ServiceException code="MissingParameterValue" locator="crs">msWCSGetCoverage(): WCS server error. Required parameter CRS was not supplied.
</ServiceException>
</ServiceExceptionReport>

with no resx parameter

<?xml version='1.0' encoding="ISO-8859-1" ?>
<ServiceExceptionReport version="1.2.0"
xmlns="http://www.opengis.net/ogc" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ogc http://schemas.opengis.net/wms/1.1.1/OGC-exception.xsd">
<ServiceException code="MissingParameterValue" locator="crs">msWCSGetCoverage(): WCS server error. Required parameter CRS was not supplied.
</ServiceException>
</ServiceExceptionReport>

NSIDC discussion:

Error message from ouput file from:

http://nsidc.org/cgi-bin/atlas_south

POST body is short
Content-type: text/html
<h1>Software error:</h1>
<pre>Bad file descriptor at /WEB/INTERNET/cgi-bin/atlas_south line 104
</pre>
<p>
For help, please send mail to the webmaster (<a href="mailto:www@nsidc.org">www@nsidc.org</a>), giving this error message and the time and date of the error.</p>

John Maurer of NSIDC suggested:

Thanks for your reply. I see in your e-mail that no crs (coordinate reference system) is specified. This is a required parameter for a GetCoverage request (at least in WCS 1.0--I didn't check later versions). This would explain why you are not getting any results. Another requirement is that you must specify either WIDTH/HEIGHT or RESX/RESY parameters. For more details on the WCS spec, see the OGC site at:

http://www.opengeospatial.org/standards/wcs

or my summary at:

http://nsidc.org/data/atlas/ogc_services.html#WCS

Here are two GetCoverage requests that would successfully download the GeoTIFF you listed; the first is in a polar projection (EPSG:3031) and the second is in the latlong projection (EPSG:4326) you were apparently testing for (since your bounding box was in degrees); note that the layer you picked is not that interesting in this case (not much snow in the Southern Hemisphere) but that's beside the point:

http://nsidc.org/cgi-bin/atlas_south?service=WCS&request=GetCoverage&version=1.0.0&crs=EPSG:3031&format=GeoTIFFInt16&resx=25067.525&resy=25067.525&bbox=-12500000,-12500000,12500000,12500000&coverage=snow_water_equivalent_05_excluding_antarctica
http://nsidc.org/cgi-bin/atlas_south?service=WCS&request=GetCoverage&version=1.0.0&crs=EPSG:4326&format=GeoTIFFInt16&width=800&height=200&bbox=-180,-90,180,0&coverage=snow_water_equivalent_05_excluding_antarctica

Hope this helps! Let me know if I can be of further assistance.
Cheers,
John

Dominic Lowe added:


It turns out this problem was actually to do with the way I was encoding the
request URL in python. I've changed this now in the code.

I hadn't encountered this problem before - it's only through exposure to
various servers that these issues come to light, so thanks for the feedback.

Ben Domenico comments:

I got the latest owslib via subversion and, sure enough, the error message disappeared and the client successfully acquired the geoTIFF from the NSIDC server when I included the crs, height, and width parameters.  I have to think through what the implications of those parameters are for our THREDDS Data Server WCS interface.

GMU (WCS 1.1.0) discussion:

Error message from the python interpreter for

http://data.laits.gmu.edu:8080/cgi-bin/wcs110

Initializing the WCS object
Traceback (most recent call last):
File "C:/Python25/Scripts/wcsExamplesBens.py", line 64, in <module>
wcs=WebCoverageService('http://data.laits.gmu.edu:8080/cgi-bin/wcs110', version='1.1.0') # George Mason server
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs.py", line 22, in WebCoverageService
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcsBase.py", line 29, in __new__
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs110.py", line 80, in __init__
TypeError: __init__() takes exactly 4 arguments (3 given)
>>>

Dominic Lowe Comments:

Have fixed bug in client wcs code on subversion, but now the server has XML
content errors.

e.g. this document seems to have invalid characters in it (and won't render in
Firefox).

http://data.laits.gmu.edu:8080/cgi-bin/wcs110?service=WCS&request=DescribeCoverage&version=1.1.0&identifiers=NETCDF%3A%22%2FData%2FG12IR04D20011013014200.nc%22%3ABand4_TEMP&format=text%2Fxml

U of Florence CNR (WCS .1.10) discussion:

Error message from the python interpreter for

'http://kronos.pin.unifi.it:8080/wcs-1.1.0-service/ogc-wcs', version='1.1.0'

Initializing the WCS object
Traceback (most recent call last):
File "C:/Python25/Scripts/wcsExamplesBens.py", line 65, in <module>
wcs=WebCoverageService('http://kronos.pin.unifi.it:8080/wcs-1.1.0-service/ogc-wcs', version='1.1.0') # U of Florence CNR server
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs.py", line 22, in WebCoverageService
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcsBase.py", line 29, in __new__
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs110.py", line 53, in __init__
File "owslib\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\coverage\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\wcs110.py", line 214, in __init__
AttributeError: 'NoneType' object has no attribute 'text'
>>>

Dominic Lowe Comments:

I have fixed a couple of bugs in the client code and can now access the
server.
However I get a 'Content not allowed in prolog' error when trying to unpack
the Multipart Mime response from a getCoverage request.

I think that there may be some headers missing from the multipart mime
response.

i.e. I suspect there should be something like this:
Content-Type: multipart/mixed; boundary="===============

1992083144=="
MIME-Version: 1.0

Before the XML and NetCDF parts.

Here you can compare the response from the BADC server:

wget "http://ndgbeta.badc.rl.ac.uk/wcs/badc.nerc.ac.uk__NDG-A0__AWQX8gTc?Service=WCS&request=GetCoverage&version=1.1.0&TimeSequence=2024-01-15T00:00:00.0&Format=cf-netcdf&store=false&BoundingBox=-80,10,80,100&Identifier=EhWv0qW5"

With the University of Florence server:

wget "http://kronos.pin.unifi.it:8080/wcs-1.1.0-service/ogc-wcs?service=WCS&format=image%2FnetCDF&request=GetCoverage&version=1.1.0&identifier=23lug07H&store=false"

But, I am by no means very familiar with MIME so I may easily have
misinterpreted something.  If anybody knows which (if either) is the correct
way to deliver multipart MIMEs that would be most helpful.

Paolo Mazzetti comments:

Concerning your test with our WCS 1.1.0 server we (Mattia Santoro and I) analyzed our server responses.
Actually they look correct, but the response "content not allowed in prolog" during XML parsing usually comes from some encoding problem. Thus we are examining our code to verify if an encoding mismatch occurs somewhere between xml prolog, http header declaration and actual data encoding.

By the way comparing our and your servers responses we found the following differences and problems:

1) Your response includes the "MIME-Version" header field in each body part while ours does not. We do not think that this should be a problem, but if your client expects it an error could be raised. Please note that the specifications (RFC 2045) state that "the MIME-Version header field is required at the top level  of a message.  It is not required for each body part of a multipart entity.  It is required for the embedded headers of a body of type "message/rfc822" or "message/partial" if and only if the embeddedmessage is itself claimed to be MIME-conformant."

2) We found a small error in our response: a double ";" before the "charset" attribute in the HTTP header. This is sintactically correct but depending on the http client library behaviour, it could be one of the reasons for the possible character encoding error previously described.

3) We also found a problem in your server responses (e.g. http://ndgbeta.badc.rl.ac.uk/wcs/badc.nerc.ac.uk__NDG-A0__AWQX8gTc?Service=WCS&request=GetCoverage&version=1.1.0&TimeSequence=2024-01-15T00:00:00.0&Format=cf-netcdf&store=false&BoundingBox=-80,10,80,100&Identifier=EhWv0qW5). It does not include the XML prologue and the first tag (<coverages>).

Generally speaking we are currently revising the multipart response. In particular we are considering the following modifications:

        a) The message should be multipart/related instead of multipart/mixed since the semantics of multipart/related better fits our case.
        b) The Content-ID should be world-unique.

We still have not implemented this modifications in our server, but I am going to better detail them in a paragraph of the document "WCS extensions for CF-NetCDF 3" that Stefano is preparing.

Weather.aero (WCS 1.1.1) discussion:

Error message from the python interpreter for

http://weather.aero/wcs/soap', version='1.1.1'

Initializing the WCS object
Find out general information about the WCS
-----------------------------------------------
Traceback (most recent call last):
File "C:/Python25/Scripts/wcsExamplesBens.py", line 76, in <module>
%wcs.identification.title
AttributeError: 'NoneType' object has no attribute 'identification'
>>>

Aaron Braeckel Comments:


the weather.aero (Experimental ADDS/NNEW) servers are implemented on top of SOAP.  Based on the error I see listed, it seems as if either we are not properly implementing KVP-based getCapabilities or SOAP requests and responses are the issue.

Dominic Lowe Comments:


If I use http://weather.aero/wcs/kvp (as opposed to /soap) then I can
communicate with your server.

However the client didn't quite work as it is looking for some elements with
the "owcs" namespace, which (I have just discovered) was removed in the 1.1.1
corrigendem.

So for now I think it's fair to say the client doesn't support 1.1.1 but I'll
try and fix it up so it does at some point.


ċ
serverTests.py
(9k)
Unknown user,
Aug 20, 2008, 7:03 AM
ċ
wcsThreddsExample.py
(4k)
Unknown user,
Aug 20, 2008, 7:04 AM
Comments