cartoninjas.net

It's been a while since the last post appeared here so this one will be rather longish ;-)

Recently in one of our projects we had a requirement to allow users to upload their images to the database so they can later be viewed by other users of the application.

For the clientside we used the ExtJs framework while the serverside was ASP.NET and c#.

Basically the process can be divided into three parts: image upload on the clientside, image processing on the server side and also image retrieval from the database so it can be displayed again in a browser.

More...

Recently I have played a bit with manifold to find out how to prepare a cost surface. There are no out of the box tools in manifold to magically create such a surface but a bit of sql does the job.

Lets assume we have two components - a surface and a drawing. Surface will be a container for our output data and the drawing will provide the input - in this case a bunch of simple lines imitating a road or stream network.

Lets start with preparing an output surface - simply copy the drawing and paste it as a surface and when done run an update query to set the cell values to 0:

UPDATE [Surface] SET [Height (I)] = 0

When our surface is ready we can start playing with generating a cost surface (in this case this will be a distance from the nearest vector object):

UPDATE
(SELECT
	[Surface].[Height (I)],
	Distance(
		AssignCoordsys(
			NewPoint([Center X (I)],[Center Y (I)]),
			CoordSys("Surface" AS COMPONENT)
		),
		Line.[Geom]
	) AS [Distance]
FROM
	[Surface],
	(SELECT AllBranches([Drawing].[Geom (I)]) As [Geom]
	FROM [Drawing]) AS Line)
SET [Height (I)] = [Distance];

 After running this query our surface should look like this:

So far s good but let's have a look at raster reclassification to make the output a bit more useful for further usage. We can write another query that will update our already generated surface, though for this example I have decided to combine it with the query above:

UPDATE
(SELECT
	[Surface].[Height (I)],
	Distance(
		AssignCoordsys(
			NewPoint([Center X (I)],[Center Y (I)]),
			CoordSys("Surface" AS COMPONENT)
		),
		Line.[Geom]
	) AS [Distance]
FROM
	[Surface],
	(SELECT AllBranches([Drawing].[Geom (I)]) As [Geom]
	FROM [Drawing]) AS Line)
SET [Height (I)] = 
	CASE
		WHEN  [Distance] < 5 THEN 0
		WHEN  [Distance] < 10 THEN 1
		WHEN  [Distance] < 15 THEN 2
		WHEN  [Distance] < 20 THEN 3
		WHEN  [Distance] < 25 THEN 4
		WHEN  [Distance] < 30 THEN 5
		WHEN  [Distance] < 35 THEN 6
		WHEN  [Distance] < 40 THEN 7
		WHEN  [Distance] < 45 THEN 8
		ELSE 9
	END

 So now our surface looks like this:

What's next? Having prepared more surfaces for analysis one can now combine them using surface tools. If surface tools are not avaialble it is easy enough to combine surfaces using the UPDATE technique shown above.

If you want to play a bit with the examples shown here just download this file: example.map (1.00 mb)

We have recently been asked to write an article about what we do for a new CAD & GIS magazine that is to be introduced to the Polish professional market soon. We have decided to introduce Manifold to the readers as it is still rather not present in the Polish market.

Below is a translation of the article Andre, Greg and myself managed to put together. Perhaps it is not as readable as its Polish version due to the translation process, still it may be fairly interesting for some of you thinking whether using Manifold may make your GIS work easier and more fun.

Manifold System – a powerful GIS tool still new to the Polish professional market.

We first met Manifold System in 2004. We were positively surprised by its functionality and the ‘functionality to price’ ratio. We gave Manifold a try and it won over all other software packages available in the market at the time.

Because of the specific approach to working with the spatial information, Manifold required us to change our habits and the way we work with the GIS tools. Our decision to make a use of Manifold System quickly appeared to be a good one and it soon became our main tool for interacting with the spatial data.

Since then we have used Manifold in many projects focused on automated cartographic productions systems (single maps, series of maps and atlases) that not only had to provide an outstanding cartographic quality but also serve as data management tools or name index generators.

Manifold also appeared to be a very powerful tool for batch data processing – loading, transformation, analysis and visualization of the data sets that in fact positioned it as a valuable ETL software (Extract, Transform and Load). Manifold IMS engine (Internet Map Server) available already at the Professional license level has been very often used in our projects as a core of the Web-GIS systems we have built, taking care of making them complaint with the OGC standards – WMS, WFS, WFS-T. Our own implementation of the WMS on top of Manifold IMS has been successfully used by recognized European orto imagery vendors as a high performance service targeted at delivering the data to their customers.

Flexibility of Manifold and the availability of the API at the lowest license level make Manifold even more powerful in hands of advanced users, while at the same time Manifold is a user friendly application for the beginners trying to make their way into the world of GIS. It is also a good choice for the customized systems vendors looking for a core GIS component for their products.

In spite of its functionality Manifold System is rather not popular in the Polish GIS market. The question then is – what can Manifold offer to its potential users? Manifold is an advanced GIS software that can be used as a desktop tool and is easily scalable to the enterprise level. At the same time it is a good choice for the beginners due to the design of its easy to understand user interface.

Manifold System is a rather young product. Its first versions were developed in 1997 as a result of Manifold.net team engagement in the project managed by the U.S. Department of Defense and Intel focused on improving the math operation libraries used by super computers. In its first form Manifold was a product offering visual tools for working with network data sets but soon, thanks to suggestions made by its users, Manifold become a fully featured GIS toolkit.

An interesting fact about Manifold Limited is that despite the company being perceived an U.S. based enterprise, not even a single line of the code of Manifold System was written in America. Manifold was developed in the company European and Asian research centers and the company itself is based in Hong Kong.

Since 1998 when the first version of Manifold System hit the GIS market, the software has been upgraded to version 8 and the next version is eagerly expected by the user community. During this time Manifold was constantly evolving introducing improvements and new functionality with every minor and major release. The current version offers an unmatched comfort of working with a very responsive interface and robust data analysis tools designed to work seamlessly with every modern database (Oracle, MS Sql Server, Posgres, DB2, MySql) without a need for any middleware. The system is empowered by the x64 architecture enabling users to use their hardware resources more efficiently but also by extensive use of parallelism and leveraging the power offered by the modern GPU technologies such as NVIDIA CUDA – some of the raster algebra operations benefit from processing times shortened by 100 up to 300 times.

Manifold is a fully featured GIS tool designed to operate on vector, raster, elevation and tabular data. Being able to handle over 100 GIS and CAD data formats along with the ability to work with the native geometry types used by database engines and its own implementation of SQL and Spatial SQL makes Manifold a powerful ETL tool.

The first contact with Manifold might be surprising. Terminology used in the software is a bit different than what most GIS users may be accustomed to. To the rescue comes the help file that quickly puts a user back on tracks.

In Manifold language a Project is made of components that provide functionality for the different data types the software is able to work with. The main component is called ‘Drawing’ and is treated as a container for the vector data. Unlike the alternative GIS tools Manifold can work with different geometry types within one component – it can handle points, lines and polygons in one place. This initially may feel unusual but after a while becomes a very convenient feature popular also in many CAD tools.

Rasters are in Manifold called ‘Image’ and ‘Surface’. The former one handles rasters visualized by colorizing pixels using one of the color modes – one channel color, palette, RGB or RGBA). The latter works with digital elevation models but is not limited only to the actual height data and can work with other data sets expressed as grids such as acoustic measurements, pollution data or statistical data.

Descriptive data associated with the vector and raster datasets are handled by the ‘Table’ component. It operates as a nested component with vector drawings and as a virtual component with rasters. It can also be a standalone component, either stored within the project file or linked from a data source. Table is an equivalent of the ‘tabular’ mode known from the alternative packages but provides an instant access to the intrinsic values specific for the owner components – for example geometry objects expose data such as area, length or centroid coordinates and this data is not stored within the table but rather calculated on demand, whenever a user decides to make use of it.

All of the mentioned components can be independent components stored in the project file but also, what is a common situation, they can operate as components linked to a data source – be it a table in a database, a SQL query in the project or another component. In fact linked components can be treated as data views known in the database world. Linked drawings and tables offer live data editing of local data sources and when linked from a database, drawings also offer multi-user editing with the conflict resolution.

A ‘Map’ component is a container used for cartographic visualization of the data. It can display many layers at once providing an on-the-fly coordinate conversion for the components with different coordinate systems. Map also provides the access to the editing tools for the displayed components and can be used as a link between components for spatial analysis. In one project a user can create as many maps as needed – each will remain independent allowing custom layer styling or visibility, yet the source components displayed on a map are shared between many maps.

Other components available in Manifold are as follows:

• ‘Chart’ – offers charting functionality
• ‘Comments’ – lets user save textual information within a project
• ‘Elevation’ – provides elevation charts based on ‘Surface’ and ‘Profile’ components
• ‘Form’ – form building functionality offering GUI for the scripts
• ‘Labels’ – used for labeling maps
• ‘Layout’ – used to prepare printouts
• ‘Palette’ – handles color palettes of the indexed color images
• ‘Profile’ – used to specify a line for the ‘Elevation’ component
• ‘Query’ – used to write and execute SQL scripts
• ‘Script’ – used to write and execute scripts
• ‘Terrain’ – 3D visualization of the ‘Surface’ components
• ‘Theme’ – virtual component used to style vector data
• ‘Folder’ – used to organize all the components within the project

The main analysis tool in Manifold is a ‘Transform Toolbar’ – a toolbar offering a quick access to the analytic functionality of the system. Available operators are changing according to the component the transform toolbar is interacting with – there are almost 60 functions for images, around 40 for the surfaces, around 90 for the vector drawings and over 50 for the tables. With tables and drawings a user can also use a simple select tool enabling him to perform the analysis on subsets of the data or use selections present in other components as the scope for the performed analysis.

Other analysis functions of Manifold are:

• Spatial Overlay – transfers data between vector components but also from rasters to vectors
• Topology Overlay – overlays vector datasets and create an output by intersecting, merging or identifying objects
• Topology Factory – advanced topology correction tools
• Visible areas – generates visible areas of a surface from locations specified by a user
• Watersheds – generates watersheds
• Routing – calculates optimal routes, drive-time zones
• Contours – generates contour lines from a surface component
• Districts – generates districts
• Send Email – used for mass mailing based on the input tabular data

For advanced users who need more flexibility Manifold offers scripting tools. One can write and execute SQL and Spatial SQL scripts by using a ‘Query’ component but also can use a ‘Script’ component for writing own code in one of the popular languages such as JScript, VBScript, VB.NET, C#.NET, or Python. Power users can easily use Manifold in external application by utilizing its API or can extend Manifold functionality directly by developing an Add-In that will be available in GUI.

Most of the mentioned functionality is available at the ‘Personal’ license level starting at $245. More demanding users can opt for a higher license or extend the ‘Personal’ license when needed. The most powerful license of Manifold does not exceed $1000 while the upgrades are available at $50 to $300.

Although our affair with Manifold started quite a while ago, we are still fascinated by this software and its functionality and flexibility that enables us to offer our customers services precisely tailored to their GIS needs.
Because Manifold makes our work so much fun we are always happy to introduce it to our customers. So far all of them are happy Manifold users too.

Cartomatic.pl & Cartoninjas.net - Grzegorz Marchut, Dominik Mikiewicz, Andrzej Siedy

More information available at:
http://www.manifold.net
http://georeference.org


Images

1. A topographic map designed completely in Manifold

2. A 3D view based on a Surface component with overlaid contour lines

3. An example of a spatial SQL query that selects all the objects in one drawing touching objects in another drawing

4. A modern Web-GIS solution utilising Manifold IMS as a WMS and WFS data source

5. Classic AJAX enabled Manifold IMS application

In this post Andre explained how to set up a dev environment on win7 so Geoserver can coexist with IIS 7.5.

When it comes to deploying Geoserver to a machine that is exposed to the Internet things may go a bit more difficult. It actually took me a while to figure out how to make all the pieces work together so if you are in a similar situation - trying to run Geoserver on Windows Server - keep on reading.

We have decided to create a subdomain that will take all the traffic targeted at our Geoserver. Apart from looking unusual it certainly makes life easier when setting up rewrite rules for IIS. But let me explain it step by step.

1. Installing Apache Tomcat

I had some problems with Tomcat 7.0.6 and Geoserver 2.1RC1 (betas failed too) but luckily Tomcat 6.0.3 was ok. When installing the server it is worth to choose the service startup option so the Tomcat service starts automatically with WIndows. If you are running win x64 make sure you choose x64 JRE as well.

2. Making Tomcat pick a proper host name

By default Tomcat binds itself to the port 8080 an I have let it do so in this case as well. The problem though is that geoserver will pick the localhost:8080 for the capabilities documents and also for the example pages generated by the layer preview links. This is not a problem when working with geoserver locally but when accesssing it from another machine, the urls have to be resolved properly.

To make Tomcat know how of the host its pages are requested from I had to edit the tomcat/conf/server.xml file. By default the Connector tag does not have the proxyPort and proxyName properties so I needed to add them. proxyPort is the actual port the resource is requested through (the default is 80 and IIS listens at this port of course) and proxyName specifies the host name that will be used. Adding these two properties to the Connector section makes the geoserver report proxyName:proxyPort as the host name instead of localhost:8080 (in my case geoserver now uses http://geoserver.cartoninjas.net:80).

3. Deploying Geoserver

Since I have fixed the server.xml file I can start the Tomcat service and navigate to localhost:8080. After logging in I deployed Geoserver by using the war file available at the download page:

4. Setting a rewrite rule in IIS

Having installed Geoserver I was ready to route all the trafic to geoserver.cartoninjas.net to my Tomcat. I had the subdomain already created so there had to be a rewrite rule set appropriately. In my case I needed to have all the incoming traffic routed to localhost:8080:

5. Testing the geoserver from a remote computer

The final step was to test if everything worked ok. I have navigated to geoserver.cartoninjas.net/geoserver and tested the tiger ny layer group. It looked like everything was ok now ;-)

Why bothering with all the steps above? Initially I had the rewriting set in IIS and I could connect to geoserver through my subdomain. The actual problem though was with the capabilities documents but also with some of the automatically generated preview pages. Of course when knowing the actual resource location it was already possible to connect to the services exposed by geoserver. But clients trying to connect to the geoserver services automatically without knowing there was a problem with the host resolution would obviously fail. A dirty solution was to make the client application replace localhost:8080 with the actual host name and we had it working for quite a while. Luckily there was a bit more elegant way of fixng things and now we have our geoserver instance work as expected. I found that little proxy thing here.

Printing a map created in OpenLayers or other commercial APIs is still not that easy. There are some problems with transparency of the gif and png images but also with the transparency of the vector canvas used to display vectors. It is possible though to print a road map or layers that do not use transparency without problems. What if we use transparent overlays or vector layers? If our project is utilising geoserver then the problem is solved - there is a mapfish printing module available for geoserver and it does a great job. What if the project does not utilise geoserver but some other custom data sources? Well, we'll need to do some work serverside.

In order to do a serverside tile stitching we will need to collect some data on the clientside so we can then grab all the necessary tiles and assemble them together into one piece:

var tiles = [];
for (var l = 0; l < map.layers.length; l++) {

	//grab the layer
	var layer = map.layers[l];
		
	//skip vector layers	
	if (layer.isVector) continue;

	//now check if it is visible and in range (wms)	
	if (!layer.getVisibility()) continue;
	if (!layer.calculateInRange()) continue;

	// iterate through their grid's tiles, collecting each tile's extent and pixel location at this moment
	for (var r = 0; r < layer.grid.length; r++) { //tile rows (grid is an array of rows)
		for (var c = 0; c < layer.grid[r].length; c++) {//columns

			//grab the tile
			var tile = layer.grid[r][c];

			//when using round there would be some gaps between tiles from time to time so ceil is used instead
			var tilePosX = Math.ceil((tile.bounds.left - mapBounds.left) / resolution);
			var tilePosY = Math.ceil((mapBounds.top - tile.bounds.top) / resolution);                 

			//get the layer opacity
			var opacity = layer.opacity ? parseInt(100 * layer.opacity) : 100;

			//collect data for a tile
			tiles[tiles.length] = {
				url: layer.getURL(tile.bounds),
				x: tilePosX,
				y: tilePosY,
				tileSizeW: layer.tileSize.w,
				tileSizeH: layer.tileSize.h,
				opacity: opacity
			};
		}
	}
}

//data to be sent to the serverside
var printData = {
	mapPixWidth: map.getSize().w,
	mapPixHeight: map.getSize().h,
	tileData: tiles
}

If you searched for some OpenLayers printing examples you may have found examples that use:

tile.position.x,
tile.position.y

instead of:

tilePosX = Math.ceil((tile.bounds.left - mapBounds.left) / resolution);
tilePosY = Math.ceil((mapBounds.top - tile.bounds.top) / resolution);

This is quite weird as I expected both yield the same results but when using OpenLayers within ExtJs layouts I encountered some strange results and found out there were some tile origin positioning shifts. Using the 'manual' tile origin calculation seems to fix the problem so I decided to stay with the adjusted code of course ;)

Since we have the tile data collected already it is the time to move to the serverside. The job to be done here is to download all the necessary tiles (the ones that overlap with the map's viewport), stitch them together and save the final image to jpeg, png, pdf, etc.

I had to do printing to jpeg so the example will use jpeg output.

In order to stitch the images together I had to download them first:

//downloads a remote image from given url
private System.Drawing.Bitmap grabImageFromWeb(string requestUrl, int tileWidth, int tileHeight)
{
    //output image
    System.Drawing.Bitmap outputImage = new System.Drawing.Bitmap(tileWidth, tileHeight);

    //test if the request string was passed and if so request data from the destination server
    if (requestUrl != null)
    {
        //create a new HttpWebRequest
        System.Net.HttpWebRequest webRequest;
        webRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(requestUrl);
        webRequest.Method = "GET";

        System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)webRequest.GetResponse();

        //check if the data was successfully retrieved
        if (response.StatusCode.ToString().ToLower() == "ok")
        {
            System.IO.Stream stream = response.GetResponseStream();
            outputImage = (System.Drawing.Bitmap)System.Drawing.Image.FromStream(stream);
        }
    }

    return outputImage;
}

 Having created a method to grab the images off the web I could now do the actual tile collection and stitching (the output of the code below is an image that maps 1:1 to the map extent visible at the user's display):

//output bitmap
System.Drawing.Bitmap mapBitmap = new System.Drawing.Bitmap(printData.mapPixWidth, printData.mapPixHeight);

//compose the map image
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(mapBitmap))
{
    //stitch all the tiles together 
    for (int t = 0; t < printData.tileData.Length; t++)
    {
        //test if a tile overlaps with the output image and grab it only if so
        //tile origin + tile size must be > 0
        //and tile origin < bitmap size
        if (printData.tileData[t].x + printData.tileData[t].tileSizeW > 0 && printData.tileData[t].x < mapBitmap.Width && printData.tileData[t].y + printData.tileData[t].tileSizeH > 0 && printData.tileData[t].y < printData.mapPixHeight)
        {
            g.DrawImage(
                grabImageFromWeb(printData.tileData[t].url, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH), //source image
                new System.Drawing.Rectangle(printData.tileData[t].x, printData.tileData[t].y, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH),//destination rect
                new System.Drawing.Rectangle(0, 0, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH),//source rect
                System.Drawing.GraphicsUnit.Pixel //drawing unit
            );
        }
    }
}

//output file name
string fileName = "Printout_" + DateTime.Now.Ticks.ToString() + ".jpg";

//save bitmap
pageBitmap.Save(Server.MapPath (System.Configuration.ConfigurationSettings.AppSettings["printedFiles"] + "\\" + fileName), System.Drawing.Imaging.ImageFormat.Jpeg);

There are a few things worth remembering here:

• google layers will not print as there is no direct access to the google tiles through OpenLayers. OL 3.0 though should have direct access to the Bing Maps tiles, so it should be possible to create a printout off the Bing tiles
• when collecting the tile data I was testing for a few conditions specific to my set up, you may require some more tests (for example if you have gmaps layers the js example shown here will fail as gmaps layer does not have a grid property
• there may be some other issues with the code shown but the generic idea should be easy to follow
• with a bit more work one could collect vector data as well and draw the features on the top of the stitched tiles (I actually did it for my app but it wouldn't make sense to show it here as the code was simply too customised)