Creating a cost surface in Manifold

Recently I have played a bit with manifold to find out how to prepare a cost surface. There are no out of the box tools in manifold to magically create such a surface but a bit of sql does the job.

Lets assume we have two components - a surface and a drawing. Surface will be a container for our output data and the drawing will provide the input - in this case a bunch of simple lines imitating a road or stream network.

Lets start with preparing an output surface - simply copy the drawing and paste it as a surface and when done run an update query to set the cell values to 0:

UPDATE [Surface] SET [Height (I)] = 0

When our surface is ready we can start playing with generating a cost surface (in this case this will be a distance from the nearest vector object):

UPDATE
(SELECT
	[Surface].[Height (I)],
	Distance(
		AssignCoordsys(
			NewPoint([Center X (I)],[Center Y (I)]),
			CoordSys("Surface" AS COMPONENT)
		),
		Line.[Geom]
	) AS [Distance]
FROM
	[Surface],
	(SELECT AllBranches([Drawing].[Geom (I)]) As [Geom]
	FROM [Drawing]) AS Line)
SET [Height (I)] = [Distance];

 After running this query our surface should look like this:

So far s good but let's have a look at raster reclassification to make the output a bit more useful for further usage. We can write another query that will update our already generated surface, though for this example I have decided to combine it with the query above:

UPDATE
(SELECT
	[Surface].[Height (I)],
	Distance(
		AssignCoordsys(
			NewPoint([Center X (I)],[Center Y (I)]),
			CoordSys("Surface" AS COMPONENT)
		),
		Line.[Geom]
	) AS [Distance]
FROM
	[Surface],
	(SELECT AllBranches([Drawing].[Geom (I)]) As [Geom]
	FROM [Drawing]) AS Line)
SET [Height (I)] = 
	CASE
		WHEN  [Distance] < 5 THEN 0
		WHEN  [Distance] < 10 THEN 1
		WHEN  [Distance] < 15 THEN 2
		WHEN  [Distance] < 20 THEN 3
		WHEN  [Distance] < 25 THEN 4
		WHEN  [Distance] < 30 THEN 5
		WHEN  [Distance] < 35 THEN 6
		WHEN  [Distance] < 40 THEN 7
		WHEN  [Distance] < 45 THEN 8
		ELSE 9
	END

 So now our surface looks like this:

What's next? Having prepared more surfaces for analysis one can now combine them using surface tools. If surface tools are not avaialble it is easy enough to combine surfaces using the UPDATE technique shown above.

If you want to play a bit with the examples shown here just download this file: example.map (1.00 mb)

Printing a OpenLayers map in ASP.NET

Printing a map created in OpenLayers or other commercial APIs is still not that easy. There are some problems with transparency of the gif and png images but also with the transparency of the vector canvas used to display vectors. It is possible though to print a road map or layers that do not use transparency without problems. What if we use transparent overlays or vector layers? If our project is utilising geoserver then the problem is solved - there is a mapfish printing module available for geoserver and it does a great job. What if the project does not utilise geoserver but some other custom data sources? Well, we'll need to do some work serverside.

In order to do a serverside tile stitching we will need to collect some data on the clientside so we can then grab all the necessary tiles and assemble them together into one piece:

var tiles = [];
for (var l = 0; l < map.layers.length; l++) {

	//grab the layer
	var layer = map.layers[l];
		
	//skip vector layers	
	if (layer.isVector) continue;

	//now check if it is visible and in range (wms)	
	if (!layer.getVisibility()) continue;
	if (!layer.calculateInRange()) continue;

	// iterate through their grid's tiles, collecting each tile's extent and pixel location at this moment
	for (var r = 0; r < layer.grid.length; r++) { //tile rows (grid is an array of rows)
		for (var c = 0; c < layer.grid[r].length; c++) {//columns

			//grab the tile
			var tile = layer.grid[r][c];

			//when using round there would be some gaps between tiles from time to time so ceil is used instead
			var tilePosX = Math.ceil((tile.bounds.left - mapBounds.left) / resolution);
			var tilePosY = Math.ceil((mapBounds.top - tile.bounds.top) / resolution);                 

			//get the layer opacity
			var opacity = layer.opacity ? parseInt(100 * layer.opacity) : 100;

			//collect data for a tile
			tiles[tiles.length] = {
				url: layer.getURL(tile.bounds),
				x: tilePosX,
				y: tilePosY,
				tileSizeW: layer.tileSize.w,
				tileSizeH: layer.tileSize.h,
				opacity: opacity
			};
		}
	}
}

//data to be sent to the serverside
var printData = {
	mapPixWidth: map.getSize().w,
	mapPixHeight: map.getSize().h,
	tileData: tiles
}

If you searched for some OpenLayers printing examples you may have found examples that use:

tile.position.x,
tile.position.y

instead of:

tilePosX = Math.ceil((tile.bounds.left - mapBounds.left) / resolution);
tilePosY = Math.ceil((mapBounds.top - tile.bounds.top) / resolution);

This is quite weird as I expected both yield the same results but when using OpenLayers within ExtJs layouts I encountered some strange results and found out there were some tile origin positioning shifts. Using the 'manual' tile origin calculation seems to fix the problem so I decided to stay with the adjusted code of course ;)

Since we have the tile data collected already it is the time to move to the serverside. The job to be done here is to download all the necessary tiles (the ones that overlap with the map's viewport), stitch them together and save the final image to jpeg, png, pdf, etc.

I had to do printing to jpeg so the example will use jpeg output.

In order to stitch the images together I had to download them first:

//downloads a remote image from given url
private System.Drawing.Bitmap grabImageFromWeb(string requestUrl, int tileWidth, int tileHeight)
{
    //output image
    System.Drawing.Bitmap outputImage = new System.Drawing.Bitmap(tileWidth, tileHeight);

    //test if the request string was passed and if so request data from the destination server
    if (requestUrl != null)
    {
        //create a new HttpWebRequest
        System.Net.HttpWebRequest webRequest;
        webRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(requestUrl);
        webRequest.Method = "GET";

        System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)webRequest.GetResponse();

        //check if the data was successfully retrieved
        if (response.StatusCode.ToString().ToLower() == "ok")
        {
            System.IO.Stream stream = response.GetResponseStream();
            outputImage = (System.Drawing.Bitmap)System.Drawing.Image.FromStream(stream);
        }
    }

    return outputImage;
}

 Having created a method to grab the images off the web I could now do the actual tile collection and stitching (the output of the code below is an image that maps 1:1 to the map extent visible at the user's display):

//output bitmap
System.Drawing.Bitmap mapBitmap = new System.Drawing.Bitmap(printData.mapPixWidth, printData.mapPixHeight);

//compose the map image
using (System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(mapBitmap))
{
    //stitch all the tiles together 
    for (int t = 0; t < printData.tileData.Length; t++)
    {
        //test if a tile overlaps with the output image and grab it only if so
        //tile origin + tile size must be > 0
        //and tile origin < bitmap size
        if (printData.tileData[t].x + printData.tileData[t].tileSizeW > 0 && printData.tileData[t].x < mapBitmap.Width && printData.tileData[t].y + printData.tileData[t].tileSizeH > 0 && printData.tileData[t].y < printData.mapPixHeight)
        {
            g.DrawImage(
                grabImageFromWeb(printData.tileData[t].url, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH), //source image
                new System.Drawing.Rectangle(printData.tileData[t].x, printData.tileData[t].y, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH),//destination rect
                new System.Drawing.Rectangle(0, 0, printData.tileData[t].tileSizeW, printData.tileData[t].tileSizeH),//source rect
                System.Drawing.GraphicsUnit.Pixel //drawing unit
            );
        }
    }
}

//output file name
string fileName = "Printout_" + DateTime.Now.Ticks.ToString() + ".jpg";

//save bitmap
pageBitmap.Save(Server.MapPath (System.Configuration.ConfigurationSettings.AppSettings["printedFiles"] + "\\" + fileName), System.Drawing.Imaging.ImageFormat.Jpeg);

There are a few things worth remembering here:

  • google layers will not print as there is no direct access to the google tiles through OpenLayers. OL 3.0 though should have direct access to the Bing Maps tiles, so it should be possible to create a printout off the Bing tiles
  • when collecting the tile data I was testing for a few conditions specific to my set up, you may require some more tests (for example if you have gmaps layers the js example shown here will fail as gmaps layer does not have a grid property
  • there may be some other issues with the code shown but the generic idea should be easy to follow
  • with a bit more work one could collect vector data as well and draw the features on the top of the stitched tiles (I actually did it for my app but it wouldn't make sense to show it here as the code was simply too customised)

ASP.NET xDomainProxy for OpenLayers getInfo requests

After a long time of just talking about using geoserver we have eventually installed it on our server. Making it work behind IIS is a subject for another article and I am hoping to post it soon.

Anyway, we decided to make our geoserver available at geoserver.cartoninjas.net, while the applications we write are likely to be hosted under different subdomains or even under different domains. Since x-domain requests are not allowed in JavaScript due to some security restrictions I needed to create a simple server side proxy that would be exposed to application as a local resource and would take care of pulling the info when a getInfo requests are issued by OpenLayers.

I started with setting up a proxyHost in OpenLayers:

//proxy host
OpenLayers.ProxyHost = 'xDomainProxy.ashx?url=';

Then a server side script for a generic handler:

<%@ WebHandler Language="C#" Class="xDomainProxy" %>

using System;
using System.Web;

public class xDomainProxy : IHttpHandler {
    
    public void ProcessRequest (HttpContext context) {

        //OpenLayers.ProxyHost = 'xdomainProxy.aspx?url=' so the requested url is passed in a url param
        string requestUrl = HttpUtility.UrlDecode(context.Request.QueryString["url"]);

        //test if the request string was passed and of so request data from the destination server
        if (requestUrl != null)
        {
            //create a new HttpWebRequest
            System.Net.HttpWebRequest webRequest;
            webRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(requestUrl);
            webRequest.Method = "GET";

            System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)webRequest.GetResponse();

            //check if the data was successfully retrieved
            if (response.StatusCode.ToString().ToLower() == "ok")
            {
                //set the appropriate response content type
                context.Response.ContentType = response.ContentType;

                //Read the stream associated with the response.
                System.IO.StreamReader reader = new System.IO.StreamReader(response.GetResponseStream());

                //and write it to response
                context.Response.Write(reader.ReadToEnd());
            }
        }
    }
 
    public bool IsReusable {
        get {
            return false;
        }
    }
}

And voila, it's ready to be used ;)

A side note: although in many cases this proxy will work properly this is a mini version of the code and should not be implemented in the production environment - it does not catch any errors and will fail if the connection is not available. Also it was written to work with OpenLayers specifically (OpenLayers.ProxyHost) and needs some extra work before it can act as a bit more flexible proxy allowing one to pull data without having to escape the passed url.

A generic error occurred in GDI+

Recently I have been working on a tile serving utility that would generate tiles on the fly but also cache them at the same time for future usage. After releasing our map tiling tool for manifold this was the next step.

The tile rendering functions were working nicely and the process was fairly quick so enabling data caching functionality could only speed things up ;-) So far so good... It was supposed to be just a matter of saving the output bitmap to a file... So I did it the way I usually do and tried to save my tile this way:

mapImageBitmap.Save(path, _outputTileFormat);

 Apparently this was throwing an error. A very descriptive one: A generic error occurred in GDI+... Not very helpful, is it?

After googling for a while it looked like this was supposed to be a permissions problem but allowing my IUSR to write to the specified folder did not help at all. What's worse I have found some info on the msdn that one should avoid using the System.Drawing namespace in ASP.NET: Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.

Nice huh?

Another solution I found on the web was to clone the bitmap in question and then save it, though that gave the same error. No avail in my case.

Luckily after messing with the problem a bit more I have discovered that writing a bitmap to a memory stream and then saving the data using Sytem.IO.File.WriteAllBytes did the trick:

System.IO.MemoryStream outStream = new System.IO.MemoryStream();
mapImageBitmap.Save(outStream, _outputTileFormat);
System.IO.File.WriteAllBytes(_requestedTilePath, outStream.ToArray());

Cutting image from a bigger raster source

This script has already been posted on the georeference.org (http://forum.manifold.net/forum/t99935.7) so nothing new will be presented...

There was one interesting thing though - how to get from a bbox of a geom used by the input drawing to the actual size of the output image in pixels. The script needed to cut images and preserve their actual resolution - as one would crop an image in photoshop.

First I had to prepare a coordinate converter and to grab some data off the image coordinate system needed later for calculating the actual size of a cut image in pixels:

//prepare coordinate converter in order to properly calculate image extent in pixels later
Manifold.Interop.CoordinateConverter coordConverter = manApp.NewCoordinateConverter();
coordConverter.Prepare((Manifold.Interop.Base)map.CoordinateSystem, (Manifold.Interop.Base)inputImage.CoordinateSystem);

//also grab the input image local scales
double imageLocalScaleX = inputImage.CoordinateSystem.ParameterSet["localScaleX"].Value;
double imageLocalScaleY = inputImage.CoordinateSystem.ParameterSet["localScaleY"].Value;

The next step was to grab the bounding box of a source geometry used to cut a new image from the source raster:

//grab the bounding box of an object
Manifold.Interop.Rect geomBbox = geomSet.get_Item(n).Box;

After that I used the bottom left and top right corners of the bbox and coverted them to the source image coordsys in order to calculate the size of the new image in pixels:

//get the corner points of the geom's bbox
Manifold.Interop.Point bottomLeft = manApp.NewPoint(geomBbox.XMin, geomBbox.YMin);
Manifold.Interop.Point topRight = manApp.NewPoint(geomBbox.XMax, geomBbox.YMax);

//convert them to the image coordsys
coordConverter.Convert((Manifold.Interop.Base)bottomLeft, null);
coordConverter.Convert((Manifold.Interop.Base)topRight, null);

//image size in pixels
int imageSizeX = (int)((topRight.X - bottomLeft.X) / imageLocalScaleX);
int imageSizeY = (int)((topRight.Y - bottomLeft.Y) / imageLocalScaleY);

//and then cut tile
map.RenderAreaTo(fileName, imageSizeX, imageSizeY, geomBbox, true);

Fairly straight forward isn't it?

Anyway, if you would like to use this script it is attached below. There are some input params and they need to be set prior to running the script. The reason behind using a RenderAreaTo method of a map object instead of the image object is explained in the script.

Although perhaps it would be easier to use gdal for the task than writing a script this exercise seemed to be interesting enough to give it a go. Having a GUI environment to choose areas of interest by simply drawing a rectangle over the image is a good reason isn't it ;-) Make sure though you switch off the input drawing before rendering the new images...

Also bear in mind that if you work with a high resolution ecw for example, the image you want to cut may be quite large since and the script calculates its size based on the source image resolution - to make it simpler: trying to cut a too big image out of a high res source image may make your pc unresponsive for a longer time ;-)

EDIT: I would almost forget - the input drawing is expected to have a column with names for the new images.

tileCutter.cs (7.53 kb)