GIS functionalities in GISolve2

Currently, we are exploring online solutions for digital mapping and spatial analysis visualization. Particularly, we are looking at geoserver+openlayer for creating a result visualization overlay on digital map for the DMS analysis application.

Problem description

How to use GIS tools to draw points as image directly?

DMS input and output datasets contains spatial points, represented as “longitude latitude Z-value”. Input datasets usually contains 1000-100,000 points. Output datasets contain all points of a grid. Depending on the resolution of grid, dataset size can be very large: 250,000 (500*500) to 4,000,000 (2000*2000) points, typically. The task of visualization is to quickly visualize DMS input and output datasets on online digital mapping system. Current solution use overlay image on google map. Efficient overlay image creation method is being sought for GISolve2.

Originally, we draw a box for each point by calculating box height and width according to bounding box and the resolution of output results. However, due to inevitable floating point precision error, we could see obvious lines b/w rows and columns.

Later, we developed direct draw method using Java 2D package. Each point is drawn as a pixel in image buffer directly. The size of buffer is higher than the resolution of output results. It works well.

Now, we'd like to find GIS tools to do the work for us. We could convert output result data to a shape file, then find a GIS function to create image.

Input data Input data plot Output result Output visualization with Java 2D Source code for direct draw

Geoserver+OpenLayer+PostGIS solution

Input dataset

Potential new approach: Store the input dataset from GISolve into a postgis table. The following perl source code will create a series of SQL statements needed to store the postgres database. The resulting spatial data was published through geoserver.

Source code Code output Result Map(Input data overlayed)

Output dataset

For output datasets there is potential of using postgis and geoserver.

Approach 1: In the first approach we created one polygon per data point. So for point x,y we store a rectangle(x,y,x+delta_x,y+delta_y) and associate the z value for the point with this rectangle. The following perl source code create a series of SQL insert statements needed.

Source code Code output Result Map(Output data overlayed)

As you might be able to notice dynamically creating an image with quarter million polygon can be a bit slow (takes about ~10 secs to load the page). This would be a drawback associated with this method.

Approach 2:In the second approach instead of creating one polygon per point, we strip the image. So we take one line (latitude value) at a time and create the necessary polygons. For every line one polygon is created for each z-value range (color). The source code to do this can be found here.

Source code Code output Result Map(Output data overlayed)

As you can see this map loads much faster because the number of polygons is much less (~5000) as compared to 0.25M. So this approach is like a slightly lossy compression technique, since we no longer store the values of each of the point. It has its drawback too since we may see striations at some zoom levels. The cause of these striations could also be the same precision errors discussed earlier. More investigation is needed in this front.

We may also be able come up with better algorithms to create the image with some effort.

Styles

The style files associated with these maps were created using GeoExt Styler. An installation of this styler can be found here.

Challanges

Currently we have been able to automate the mechanism of getting data into the postgis DB from DMS input and output data. But the last step of adding a new feature type into the Geoserver still has to done through the user interface. We will have to explore Geoserver further to see if it provides a API for data management functionality.

Discussion 1

02/06/2009 by Anand, Wenwu, and Yan

How it works in geoserver
  • A template is created to generate postgis table (data table + geometry_column meta table) in postgre db and import DMS result dataset into data table
  • In geoserver, “new” feature and coverage by linking postgis db to geoserver and specify various configuration items
  • openlayer will retrieve overlay from geoserver with the feature and coverage specified for DMS table
Issues
  • overlay creation is dynamic per user request. since drawing an overlay with 2000*2000 is slow even on geoserver side, Anand developed two approaches to try to speed it up. for fast visualization, is it possible for us to generate geo-jpeg or geo-tiff and let geoserver display the image directly as overlay? this way geoserver doesn't need to dynamically render an overlay from data source again and again per user request (note: geoserver has limited cache functionality to cache previous request results). it will be on our todo list.
  • currently, a user job has a table in postgis. creating an overlay based on a user data table means manually creating feature and coverage configurations in geoserver. we need to automate that. an alternative is to create a visualization buffer that contains an array of temporary user data tables (empty, initially). in geoserver, feature and coverage configurations will be created for each table in visualization buffer. upon a user visualization request, that user's designated job data table is copies to one of temporary table in visualization buffer and geoserver is used to create an overlay with the content in that temporary table. of course, we will have buffer overflow problem to deal with. since automating the creation of feature and coverage configuration in geoserver is our ultimate solution, it is on our todo list. the visualization buffer idea kicks in if automation does not work
  • current overlay shape is rectangle. can we create an overlay with bounding box as a polygon, e.g., US boundary polygon? postgis has such capability as a database function. this function can be exposed as a service for generic use, and it is on our todo list. as an alternative, we could eliminate data points in data table at the time of importing result datasets. however, this method does not allow dynamic polygon filter (e.g., we have data for US, but user wants an overlay plot of IL. IL boundary polygon will filter out data points fallen into other states. then how about another user requests a IA overlay plot?)
Agreed
  • about the role of database, we need to store data in spatial database as spatial data source for user query, dynamic geoprocessing, and analysis input data generation. at the same time, we need to look at capability of geoserver to generate overlay from files, e.g., shape files.

Todo list

create an overlay with bounding box as a polygon
automate the creation of feature and coverage configuration in geoserver
generate geo-jpeg or geo-tiff and let geoserver display the image directly as overlay
  1. display a geotiff as overlay in geoserver and openlayer. Sample GeoTiff file How to display tiff with geoinfo in geoserver
  2. create geotiff from DMS output datasets. Currently, there are two methods to create geotiff from a regular tiff file: gdal and libgeotiff tool. Both need two input files, a regular tiff and a ESRI World file for georeferencing. However, we don't know how to convert DMS output metadata information (bounding box and resolution) into World file configuration parameters. About size of geotiff files, we should use the default LZW compression. Sample tiff Sample World file Sample geotiff