Why Manifold?

We have recently been asked to write an article about what we do for a new CAD & GIS magazine that is to be introduced to the Polish professional market soon. We have decided to introduce Manifold to the readers as it is still rather not present in the Polish market.

Below is a translation of the article Andre, Greg and myself managed to put together. Perhaps it is not as readable as its Polish version due to the translation process, still it may be fairly interesting for some of you thinking whether using Manifold may make your GIS work easier and more fun.

Manifold System – a powerful GIS tool still new to the Polish professional market.

We first met Manifold System in 2004. We were positively surprised by its functionality and the ‘functionality to price’ ratio. We gave Manifold a try and it won over all other software packages available in the market at the time.

Because of the specific approach to working with the spatial information, Manifold required us to change our habits and the way we work with the GIS tools. Our decision to make a use of Manifold System quickly appeared to be a good one and it soon became our main tool for interacting with the spatial data.

Since then we have used Manifold in many projects focused on automated cartographic productions systems (single maps, series of maps and atlases) that not only had to provide an outstanding cartographic quality but also serve as data management tools or name index generators.

Manifold also appeared to be a very powerful tool for batch data processing – loading, transformation, analysis and visualization of the data sets that in fact positioned it as a valuable ETL software (Extract, Transform and Load). Manifold IMS engine (Internet Map Server) available already at the Professional license level has been very often used in our projects as a core of the Web-GIS systems we have built, taking care of making them complaint with the OGC standards – WMS, WFS, WFS-T. Our own implementation of the WMS on top of Manifold IMS has been successfully used by recognized European orto imagery vendors as a high performance service targeted at delivering the data to their customers.

Flexibility of Manifold and the availability of the API at the lowest license level make Manifold even more powerful in hands of advanced users, while at the same time Manifold is a user friendly application for the beginners trying to make their way into the world of GIS. It is also a good choice for the customized systems vendors looking for a core GIS component for their products.

In spite of its functionality Manifold System is rather not popular in the Polish GIS market. The question then is – what can Manifold offer to its potential users? Manifold is an advanced GIS software that can be used as a desktop tool and is easily scalable to the enterprise level. At the same time it is a good choice for the beginners due to the design of its easy to understand user interface.

Manifold System is a rather young product. Its first versions were developed in 1997 as a result of Manifold.net team engagement in the project managed by the U.S. Department of Defense and Intel focused on improving the math operation libraries used by super computers. In its first form Manifold was a product offering visual tools for working with network data sets but soon, thanks to suggestions made by its users, Manifold become a fully featured GIS toolkit.

An interesting fact about Manifold Limited is that despite the company being perceived an U.S. based enterprise, not even a single line of the code of Manifold System was written in America. Manifold was developed in the company European and Asian research centers and the company itself is based in Hong Kong.

Since 1998 when the first version of Manifold System hit the GIS market, the software has been upgraded to version 8 and the next version is eagerly expected by the user community. During this time Manifold was constantly evolving introducing improvements and new functionality with every minor and major release. The current version offers an unmatched comfort of working with a very responsive interface and robust data analysis tools designed to work seamlessly with every modern database (Oracle, MS Sql Server, Posgres, DB2, MySql) without a need for any middleware. The system is empowered by the x64 architecture enabling users to use their hardware resources more efficiently but also by extensive use of parallelism and leveraging the power offered by the modern GPU technologies such as NVIDIA CUDA – some of the raster algebra operations benefit from processing times shortened by 100 up to 300 times.

Manifold is a fully featured GIS tool designed to operate on vector, raster, elevation and tabular data. Being able to handle over 100 GIS and CAD data formats along with the ability to work with the native geometry types used by database engines and its own implementation of SQL and Spatial SQL makes Manifold a powerful ETL tool.

The first contact with Manifold might be surprising. Terminology used in the software is a bit different than what most GIS users may be accustomed to. To the rescue comes the help file that quickly puts a user back on tracks.

In Manifold language a Project is made of components that provide functionality for the different data types the software is able to work with. The main component is called ‘Drawing’ and is treated as a container for the vector data. Unlike the alternative GIS tools Manifold can work with different geometry types within one component – it can handle points, lines and polygons in one place. This initially may feel unusual but after a while becomes a very convenient feature popular also in many CAD tools.

Rasters are in Manifold called ‘Image’ and ‘Surface’. The former one handles rasters visualized by colorizing pixels using one of the color modes – one channel color, palette, RGB or RGBA). The latter works with digital elevation models but is not limited only to the actual height data and can work with other data sets expressed as grids such as acoustic measurements, pollution data or statistical data.

Descriptive data associated with the vector and raster datasets are handled by the ‘Table’ component. It operates as a nested component with vector drawings and as a virtual component with rasters. It can also be a standalone component, either stored within the project file or linked from a data source. Table is an equivalent of the ‘tabular’ mode known from the alternative packages but provides an instant access to the intrinsic values specific for the owner components – for example geometry objects expose data such as area, length or centroid coordinates and this data is not stored within the table but rather calculated on demand, whenever a user decides to make use of it.

All of the mentioned components can be independent components stored in the project file but also, what is a common situation, they can operate as components linked to a data source – be it a table in a database, a SQL query in the project or another component. In fact linked components can be treated as data views known in the database world. Linked drawings and tables offer live data editing of local data sources and when linked from a database, drawings also offer multi-user editing with the conflict resolution.

A ‘Map’ component is a container used for cartographic visualization of the data. It can display many layers at once providing an on-the-fly coordinate conversion for the components with different coordinate systems. Map also provides the access to the editing tools for the displayed components and can be used as a link between components for spatial analysis. In one project a user can create as many maps as needed – each will remain independent allowing custom layer styling or visibility, yet the source components displayed on a map are shared between many maps.

Other components available in Manifold are as follows:

  • ‘Chart’ – offers charting functionality
  • ‘Comments’ – lets user save textual information within a project
  • ‘Elevation’ – provides elevation charts based on ‘Surface’ and ‘Profile’ components
  • ‘Form’ – form building functionality offering GUI for the scripts
  • ‘Labels’ – used for labeling maps
  • ‘Layout’ – used to prepare printouts
  • ‘Palette’ – handles color palettes of the indexed color images
  • ‘Profile’ – used to specify a line for the ‘Elevation’ component
  • ‘Query’ – used to write and execute SQL scripts
  • ‘Script’ – used to write and execute scripts
  • ‘Terrain’ – 3D visualization of the ‘Surface’ components
  • ‘Theme’ – virtual component used to style vector data
  • ‘Folder’ – used to organize all the components within the project


The main analysis tool in Manifold is a ‘Transform Toolbar’ – a toolbar offering a quick access to the analytic functionality of the system. Available operators are changing according to the component the transform toolbar is interacting with – there are almost 60 functions for images, around 40 for the surfaces, around 90 for the vector drawings and over 50 for the tables. With tables and drawings a user can also use a simple select tool enabling him to perform the analysis on subsets of the data or use selections present in other components as the scope for the performed analysis.

Other analysis functions of Manifold are:

  • Spatial Overlay – transfers data between vector components but also from rasters to vectors
  • Topology Overlay – overlays vector datasets and create an output by intersecting, merging or identifying objects
  • Topology Factory – advanced topology correction tools
  • Visible areas – generates visible areas of a surface from locations specified by a user
  • Watersheds – generates watersheds
  • Routing – calculates optimal routes, drive-time zones
  • Contours – generates contour lines from a surface component
  • Districts – generates districts
  • Send Email – used for mass mailing based on the input tabular data

For advanced users who need more flexibility Manifold offers scripting tools. One can write and execute SQL and Spatial SQL scripts by using a ‘Query’ component but also can use a ‘Script’ component for writing own code in one of the popular languages such as JScript, VBScript, VB.NET, C#.NET, or Python. Power users can easily use Manifold in external application by utilizing its API or can extend Manifold functionality directly by developing an Add-In that will be available in GUI.

Most of the mentioned functionality is available at the ‘Personal’ license level starting at $245. More demanding users can opt for a higher license or extend the ‘Personal’ license when needed. The most powerful license of Manifold does not exceed $1000 while the upgrades are available at $50 to $300.

Although our affair with Manifold started quite a while ago, we are still fascinated by this software and its functionality and flexibility that enables us to offer our customers services precisely tailored to their GIS needs.
Because Manifold makes our work so much fun we are always happy to introduce it to our customers. So far all of them are happy Manifold users too.


Cartomatic.pl & Cartoninjas.net - Grzegorz Marchut, Dominik Mikiewicz, Andrzej Siedy

More information available at:
http://www.manifold.net
http://georeference.org


Images

1. A topographic map designed completely in Manifold

2. A 3D view based on a Surface component with overlaid contour lines

3. An example of a spatial SQL query that selects all the objects in one drawing touching objects in another drawing

4. A modern Web-GIS solution utilising Manifold IMS as a WMS and WFS data source

5. Classic AJAX enabled Manifold IMS application

Defitions of Polish projections for manifold

With version 8.0.19 manifold team added some of the Polish projections to manifold projection presets. Basically they added five zones of the National Coordinate System 1965 (PUWG 1965, zones I, II, III, IV and V). There seems to be a mistake though with the way they named the projections: National Coordinate System 1965 was named 1942 while the only one that has 5 zones for Poland is 1965.

For those interested in using Polish coordinate systems with manifold attached is a custom projection xml (it will display in a folder Poland_Custom in the assign / change projection dialog). This is the best we came up with so far so please bear in mind that it may not be 100% perfect. For some more details please see a discussion at georeference.org.

Styling Navteq data using Manifold

Recently we were asked to prepare some map layers for a web application. The exercise was about styling Navteq 2010 data in manifold so created maps can be then used for rendering a tiled base map but also used by a map server.

Although manifold lacks some of the carto tools available in other GIS packages or graphic software it does offer enough to complete such task. It actually does offer some other functions that let one create some nice map features - for example the road shields labels were created from a drawing linked to a SQL query that created road label points spaced evenly along the road lines. This backed with the automatic resolution of label conflicts allowed us to nicely place the road shields on the map.

Cutting image from a bigger raster source

This script has already been posted on the georeference.org (http://forum.manifold.net/forum/t99935.7) so nothing new will be presented...

There was one interesting thing though - how to get from a bbox of a geom used by the input drawing to the actual size of the output image in pixels. The script needed to cut images and preserve their actual resolution - as one would crop an image in photoshop.

First I had to prepare a coordinate converter and to grab some data off the image coordinate system needed later for calculating the actual size of a cut image in pixels:

//prepare coordinate converter in order to properly calculate image extent in pixels later
Manifold.Interop.CoordinateConverter coordConverter = manApp.NewCoordinateConverter();
coordConverter.Prepare((Manifold.Interop.Base)map.CoordinateSystem, (Manifold.Interop.Base)inputImage.CoordinateSystem);

//also grab the input image local scales
double imageLocalScaleX = inputImage.CoordinateSystem.ParameterSet["localScaleX"].Value;
double imageLocalScaleY = inputImage.CoordinateSystem.ParameterSet["localScaleY"].Value;

The next step was to grab the bounding box of a source geometry used to cut a new image from the source raster:

//grab the bounding box of an object
Manifold.Interop.Rect geomBbox = geomSet.get_Item(n).Box;

After that I used the bottom left and top right corners of the bbox and coverted them to the source image coordsys in order to calculate the size of the new image in pixels:

//get the corner points of the geom's bbox
Manifold.Interop.Point bottomLeft = manApp.NewPoint(geomBbox.XMin, geomBbox.YMin);
Manifold.Interop.Point topRight = manApp.NewPoint(geomBbox.XMax, geomBbox.YMax);

//convert them to the image coordsys
coordConverter.Convert((Manifold.Interop.Base)bottomLeft, null);
coordConverter.Convert((Manifold.Interop.Base)topRight, null);

//image size in pixels
int imageSizeX = (int)((topRight.X - bottomLeft.X) / imageLocalScaleX);
int imageSizeY = (int)((topRight.Y - bottomLeft.Y) / imageLocalScaleY);

//and then cut tile
map.RenderAreaTo(fileName, imageSizeX, imageSizeY, geomBbox, true);

Fairly straight forward isn't it?

Anyway, if you would like to use this script it is attached below. There are some input params and they need to be set prior to running the script. The reason behind using a RenderAreaTo method of a map object instead of the image object is explained in the script.

Although perhaps it would be easier to use gdal for the task than writing a script this exercise seemed to be interesting enough to give it a go. Having a GUI environment to choose areas of interest by simply drawing a rectangle over the image is a good reason isn't it ;-) Make sure though you switch off the input drawing before rendering the new images...

Also bear in mind that if you work with a high resolution ecw for example, the image you want to cut may be quite large since and the script calculates its size based on the source image resolution - to make it simpler: trying to cut a too big image out of a high res source image may make your pc unresponsive for a longer time ;-)

EDIT: I would almost forget - the input drawing is expected to have a column with names for the new images.

tileCutter.cs (7.53 kb)

Preparing the data description for the metadata docu

Today I had to prepare a description of all the vector drawings I had in a project in a semi tabular form listing a folder, contained drawings and their column, column types and the length of the data in each column. In other words I needed to create a dataset description for the metadata document needed in my current project.

Obviously collecting such data for a drawing or two is fairly quick but manually fetching data for multiple components would a pain in the neck…

A script seemed to be the best option here so I decided to write one ;-) It searches for the drawings contained in folders (one level of nesting) and then lists the needed stuff. It can easily be adapted to list drawings in a root folder as well and to dig deeper in nested folders. It can also be adjusted to look for other types of components or to grab some info about the coordsystems used by them - I just needed the drawings though hence a rather simplistic version of the script was enough.

Anyways, feel free to use it if you like (simply add a c# script and replace its content with the attached). Bear in mind script lists info only for drawings contained in a folder.

Output example:

Topo200k
========================================

Drogi
----------------------------------------
ColumnName	DataType	FieldLength
ID	ColumnTypeInt32U	4
LENGTH	ColumnTypeFloat64	8
KLASA	ColumnTypeInt16	2
ID 2	ColumnTypeInt32	4
NUMER	ColumnTypeAText	16
----------------------------------------

Duze_rzeki_i_jeziora
----------------------------------------
ColumnName	DataType	FieldLength
ID	ColumnTypeInt32U	4
NAME	ColumnTypeAText	100
SHAPE_LENG	ColumnTypeFloat64	8
SHAPE_AREA	ColumnTypeFloat64	8
TOPO_CLASS	ColumnTypeInt32	4
----------------------------------------

 And the script:

using Manifold.Interop.Scripts;
using System;

class Script {
	static void Main() {
		pullDataDescription();
	}

    static void pullDataDescription()
    {
        //reference the app object first
        Manifold.Interop.Application manApp = new Manifold.Interop.Application();

        //grab doc object
        Manifold.Interop.Document manDoc = (Manifold.Interop.Document) manApp.ActiveDocument;

        //create output comments component
        Manifold.Interop.Comments cmt = manDoc.NewComments("Drawings&Data", false);
        cmt.Folder = null; //so it's always in the root folder

        //record the time this summary was created
        DateTime date = DateTime.Now;
        cmt.AddText("Report generated " + date.ToLongDateString() + " " + date.ToLongTimeString() + Environment.NewLine);
        cmt.AddText(writeBreakLine("*", 40) + Environment.NewLine);

        //iterate through components
        foreach (Manifold.Interop.Component cmp in manDoc.ComponentSet)
        {
            //check if this is a folder
            if (cmp.Type == Manifold.Interop.ComponentType.ComponentFolder)
            {
                Manifold.Interop.Folder fld = (Manifold.Interop.Folder)cmp;
                
                //write the folder name
                cmt.AddText(fld.Name + Environment.NewLine);
                cmt.AddText(writeBreakLine("=", 40) + Environment.NewLine);

                //iterate through drawings
                foreach (Manifold.Interop.Component fldCmp in fld.Children)
                {
                    //check if this is a drawing
                    if (fldCmp.Type == Manifold.Interop.ComponentType.ComponentDrawing)
                    {

                        Manifold.Interop.Drawing drw = (Manifold.Interop.Drawing)fldCmp; 

                        //write drawing name
                        cmt.AddText(drw.Name + Environment.NewLine);
                        cmt.AddText(writeBreakLine("-", 40));

                        //write headers for the tables
                        cmt.AddText("ColumnName" + "\t" + "DataType" + "\t" + "FieldLength" + Environment.NewLine);
                        //iterate through colums
                        foreach (Manifold.Interop.Column col in ((Manifold.Interop.Table)drw.OwnedTable).ColumnSet)
                        {
                            if (col.Category == (int)Manifold.Interop.ColumnCategory.ColumnCategoryNative)
                            {
                                cmt.AddText(col.Name + "\t" + (Manifold.Interop.ColumnType)col.get_Type()  + "\t" + col.Size + Environment.NewLine);
                            }
                        }
                        cmt.AddText(writeBreakLine("-", 40) +    Environment.NewLine);
                    }
                }
                //write the folder name
                cmt.AddText(writeBreakLine("=", 40));
                cmt.AddText("eof " + fld.Name + Environment.NewLine + Environment.NewLine + Environment.NewLine);
            }
        }
        //open the comments component
        cmt.Open();
    }

    //writes a 'break line'
    static string writeBreakLine(string character, int length)
    {
        string output = string.Empty;
        for (int n = 0; n < length; n++)
        {
            output += character;
        }
        output += Environment.NewLine;
        return output;
    }  
}