This project is read-only.

Working w/ Raster Layers

Dec 6, 2012 at 9:51 PM

I am hoping to use DotSpatial for working with raster layers in a very simple way. From reading the discussions here, it appears you need to use the GDAL library in some way to work with the DotSpatial data provider. Unfortunately, I can not find any detailed help on how to do this when building a non-GUI application. There is one tutorial for doing this with the windows forms based plugin architecture. Can anyone show tell me the steps necessary to set up my system to simply open a raster (any format really) and simply get a raster cell value at a particular coordinate?

Dec 6, 2012 at 11:37 PM

I don't have a simple example to point you at, but I do have a non-GUI application that reads and writes rasters through DotSpatial/GDAL so I can tell you it can be done. One hint I would give is that you can reference the DotSpatial.Data.Rasters.GdalExtension library directly in your project rather than have it loaded as a plugin. This application has all its own binaries along with all the relevant DotSpatial and gdal libraries in one folder with sub-folders "gdal-data" and "gdalplugins" containing additional files needed by gdal.

Dec 7, 2012 at 2:22 PM

Thanks for your help. I actually figured it out using GDAL w/ C# bindings for working with rasters. It is really tricky and took a very late night of trial and error programming. It is very old school and procedural, but massively fast. I did a "point on raster" routine and it was incredibly fast. I will experiment with getting this setup in DotSpatial next. I'll try to post it once I have gotten it working. Now that I understand the underlying GDAL library it should be easier to get it going in DotSpatial.

Dec 7, 2012 at 4:38 PM

This is interesting also for me. If you could post some examples it would be nice

Oscar

Dec 7, 2012 at 9:15 PM

Sure, here are the steps...

I hope this helps. If anyone else can help wire this up with DotSpatial, that would be great.

1. Download the GDAL binaries from here... http://vbkto.dyndns.org/sdk/. Make sure to get 32 or 64 bit to fit your hardware/OS.

2. In your program make sure to set the Built type to x86 or 64 bit, depending on your hardware/OS. You get weird errors it the architecture is not correct. 

3. Add Reference to the 3 GDAL/csharp bindings and add using statements for these references... 

/bin/gdal/csharp/gdal_csharp.dll

/bin/gdal/csharp/ogr_csharp.dll

/bin/gdal/csharp/osr_csharp.dll

using OSGeo.GDAL;
using OSGeo.OGR;
using OSGeo.OSR;

4. Use this method to setup all the path and environment variables for your application. You can add these to your computers path as well if you like.

 

public static bool SetupGDAL()
    {
        bool result = false;

        //We're adding the libs\GDAL-1.6.3 folder to our path

        string path = Environment.GetEnvironmentVariable("PATH");
        string gdalPath = @"C:\Temp\TestGdal";

        FileInfo info = new FileInfo(gdalPath);
        if ((info.Attributes & FileAttributes.Directory) > 0)
        {
            String oldPath = Environment.GetEnvironmentVariable("PATH");
            String PATH = gdalPath + @"\bin;" + gdalPath + @"\bin\gdal\python\osgeo;" + gdalPath + @"\bin\proj\apps;" + gdalPath + @"\bin\gdal\plugins;" + gdalPath + @"\bin\gdal\plugins-optional;" + gdalPath + @"\bin\gdal\apps;" + gdalPath + @"\bin\ms\apps;" + gdalPath + @"\bin\gdal\csharp;" + gdalPath + @"\bin\ms\csharp;" + gdalPath + @"\bin\curl";

            if (!path.Contains(PATH))
            {
                path = PATH + ";" + path;
                Environment.SetEnvironmentVariable("PATH", path);
            }


            String GDAL_DATA = gdalPath + @"bin\gdal-data";
            Environment.SetEnvironmentVariable("GDAL_DATA", GDAL_DATA);
            OSGeo.GDAL.Gdal.SetConfigOption("GDAL_DATA", GDAL_DATA);

            String GDAL_DRIVER_PATH = gdalPath + @"\bin\gdal\plugins";
            Environment.SetEnvironmentVariable("GDAL_DRIVER_PATH", GDAL_DRIVER_PATH);
            OSGeo.GDAL.Gdal.SetConfigOption("GDAL_DRIVER_PATH", GDAL_DRIVER_PATH);

            String PYTHONPATH = gdalPath + @"\bin\gdal\python\osgeo";
            Environment.SetEnvironmentVariable("PYTHONPATH", PYTHONPATH);
            OSGeo.GDAL.Gdal.SetConfigOption("PYTHONPATH", PYTHONPATH);

            String PROJ_LIB = gdalPath + @"\bin\proj\SHARE";
            Environment.SetEnvironmentVariable("PROJ_LIB", PROJ_LIB);
            OSGeo.GDAL.Gdal.SetConfigOption("PROJ_LIB", PROJ_LIB);

            OSGeo.GDAL.Gdal.SetConfigOption("GDAL_CACHEMAX", "100000");
            OSGeo.GDAL.Gdal.SetConfigOption("CPL_TMPDIR", @"C:\");

            //Test to see if EPSG spatial reference was registered
            new OSGeo.OSR.SpatialReference("EPSG:4326");

            //This line throws an exception if the the wrong version of GDal was found in the path somewhere OR the path didn't point to GDal correctly.
            OSGeo.OGR.Ogr.RegisterAll();
            OSGeo.GDAL.Gdal.AllRegister();

            result = true;
        }
        return result;
    }

 

5. Now do something like I did to return a raster cell value from a lat/long point (intersection)...

 

		if(SetupGDAL())
		{
			try
			{
				double x = -117.5;
				double y = 33.75;

				//Open Raster
				Dataset dataset = Gdal.Open(@"C:\arcgisserver\data\pmap_results2\000aSLTemp.img", Access.GA_ReadOnly);
				
				//Get the only band
				Band band = dataset.GetRasterBand(1);
				
				//Set "No Data value"
				double dblVal = -1;
				int intVal = -1;
				band.GetNoDataValue(out dblVal, out intVal);
				if (intVal == 0)
				{
						band.SetNoDataValue(-9999);
				}

				//Get row and column count of raster
				int cols = band.XSize;
				int rows = band.YSize;
			
				double[] geotransform = new double[6];
				double  dfGeoX, dfGeoY;
				dataset.GetGeoTransform(geotransform);
			
				//Get cell size dimensions
				double cellSizeX = geotransform[1];
				double cellSizeY = -1 * geotransform[5];
			
				//Getting min and max geographic values for entire raster
				double minx = geotransform[0];
				double maxy = geotransform[3];
				double maxx = minx + (cols * cellSizeX);
				double miny = maxy - (rows * cellSizeY);
			
				//Validation of input values
				if ((x < minx) || (x > maxx) || (y < miny) || (y > maxy))
				{
			    		return;
				}
			
				//# calc point location in pixels
				int xLoc = Convert.ToInt32((x - minx) / cellSizeX);
				int yLoc = Convert.ToInt32((maxy - y) / cellSizeY);
			
				if ((xLoc < 0.5) || (xLoc > cols - 0.5))
			    		return;
			 
				if ((yLoc < 0.5) || (yLoc > rows - 0.5))
			    		return;
			
				double[] r = new double[cols * rows];
				band.ReadRaster(0, 0, cols, rows, r, cols, rows, 0, 0);
			
				int locVaue = xLoc + yLoc * cols;
				double value = r[locVaue];
				Console.Write(value);
			}
			catch (Exception ex)
			{
				Console.WriteLine(ex.Message);
			}
		}

		Console.ReadLine();

 

Dec 8, 2012 at 2:17 PM

Thank you Atv

as soon as I will have some spare time I will try

Thank you again

Oscar

Dec 8, 2012 at 7:46 PM

@atv

you might want to try this nuget package http://nuget.org/packages/GDAL.

It has the setup code and works with AnyCPU setting.

Hth FObermaier

Jan 12, 2013 at 9:12 AM

Dear FObermaier

 

I finally bought Visual Studio Pro (I am working with vb.net).

I could load Nuget Gdal and tested it with a customized version of point 5 code kindly described by atv.

How can I link dotspatial data structures with gdal data structures? better, how can I visualize raster images loaded with gdal in a dotspatial map (say adding as a new layer to the map)? 

Is there any example code that shows how to do that?

 

Thank you

Oscar

Jan 15, 2013 at 1:54 PM
Edited Jan 15, 2013 at 3:16 PM

I'll look into that ASAP

EDIT: please evaluate the provided patchfile from this issue.

FObermaier

Jan 17, 2013 at 10:45 AM

Thank You FObermaier

Look forward for the Patch issue to be handled by Dotspatial team.

One more question: does it happen only to me that when I load into my application (but the same happens in demomap) a raster map (say 100 Mb) the memory usage increases by more than three times the file weight?

Also, when I remove the image the memory usage still the same, no memory is freed. I tried to call GC.collect and the result is that the memory is freed by almost the size of the original file (100 Mb) while 300 Mb usage is still there. Is there any issue with layer's remove method?

Thank you

Oscar

Jan 22, 2013 at 3:09 PM

Some things you should check:

  • If your raster has a dimension of more than 8000 pixels, the GdalImageProvider will give you a PyramidImage. Maybe that is the cause to your issue.
    (see Issue)
  • Another thing is that GdalImage holds references to r-, g-, b- byte arrays if you have an RGB image. This is double effort since the values are stored in the vals -byte array, too. Remove instance field r, g, b and use local variables.
  • There are several places throughout the GdalExtensions project where GDAL objects are not disposed properly, mainly OSGeo.GDAL.Band objects.

I fixed some of them for me, but am still not satisfied. I have the impression that on removing the layer all memory is freed, but while zooming in and out the application keeps taking more and more memory.

 

Hth FObermaier

Jan 22, 2013 at 3:41 PM

Thank you FObermaier,

 

the thing is that it does the same even if I do not load the Gdal Extension, therefore using the bgd reader inside dotspatial.

The rasters I tried to load are smaller than 8000x8000, say in average 1000x1000.

I think that it has something to do with the rasterlayer structure. It sounds like some hidden buffer/image/copy/band of the rasterlayer is allocated and not necessary, and when the layer is disposed they are still there.

This is an important issue to dotspatial, because when you have to deal with several different maps in the same working session you get to a situation in which you saturate your memory although removing layers when not needed anymore.

This is also why I tried to work in 64bit, where the memory limit is not so important, and found problems with loading gdalextension.

By the way, I wanted to try to apply your patch myself, but my experience in dealing with this stuff is still too poor to get through it. I will wait for people to test it and maybe insert it in a future dotspatial release. 

Thank you

Oscar

Jan 22, 2013 at 4:41 PM

You should put some effort in applying the patch. I have a 19000x16000 ECW raster that pops up just like that. It may very well be that the memory issues I'm seeing are related to GDAL's own caching.

Hth FObermaier

Jan 22, 2013 at 5:27 PM

I made it!

I loaded the dotspatial project.

Went to Gdalextension subproject and I modified all the files as told in your patch file, but then I had a problem in resolving Osgeo.Gdal Osgeo.OGR and Osgeo.Osr.

Installed them from nuget.gdal (in the whole dotspatial project otherwise it does not work...at least to me...don't know why).

rebuilt dotspatial project in release mode, copied the built GDAL directories in my project release\plugins folder and it works.

Thank you very much!!!!!

However, regarding the memory problem I am not sure it has to do with the GDAL, since it happens also when I do not load the GDAL plugin.

If you try to run demomap after removing the GDAL directories from the plugin folder (so GDAL is not loaded) and load a .bgd raster map you'll see that the memory usage increases far more than needed, and as you remove the layer the memory usage will remain the same, and if you call GC.collect() you get some free memory but not all of that used to load the bgd.

I think that it is very important to solve this issue, since it makes dotspatial not very appealing, and this is a pitty

 

Thank you again

Oscar 

Jan 23, 2013 at 3:42 PM

AHG

I changed something in my bin directory to tidy up unused things and now when I run my code I get an exception:

can't load requested DLL ...\release\gdal\x64\plugins\ogr_OCI.dll

126: The specified module could not be found

Resulting in: An exception occurred while trying to create an instance of type 'Dotspatial.Data.OgrExtension.OgrVectorProvider'.

resulting in: Cannot activate part 'Dotspatial.Data.OgrExtension.OgrVectorProvider'.

Element: DotSpatial.Data.OgrExtension.OgrVectorProvider -->  DotSpatial.Data.OgrExtension.OgrVectorProvider -->  DirectoryCatalog (Path="D:\LAVORO\RigottiSUAP\ConcessioniConsequenziale\readDBF\readDBF\bin\x64\Release\Plugins\GDAL")
Resulting in: Cannot get export 'DotSpatial.Data.IDataProvider (ContractName="DotSpatial.Data.IDataProvider")' from part 'DotSpatial.Data.OgrExtension.OgrVectorProvider'.Element: DotSpatial.Data.IDataProvider (ContractName="DotSpatial.Data.IDataProvider") -->  DotSpatial.Data.OgrExtension.OgrVectorProvider -->  DirectoryCatalog (Path="D:\LAVORO\RigottiSUAP\ConcessioniConsequenziale\readDBF\readDBF\bin\x64\Release\Plugins\GDAL")
Resulting in: Cannot set import 'DotSpatial.Data.DataManager.DataProviders (ContractName="DotSpatial.Data.IDataProvider")' on part 'DotSpatial.Data.DataManager'.Element: DotSpatial.Data.DataManager.DataProviders (ContractName="DotSpatial.Data.IDataProvider") -->  DotSpatial.Data.DataManager
   at System.ComponentModel.Composition.CompositionResult.ThrowOnErrors(AtomicComposition atomicComposition)   at System.ComponentModel.Composition.Hosting.ComposablePartExportProvider.Compose(CompositionBatch batch)   at DotSpatial.Controls.AppManager.LoadExtensions() in c:\testdir\dotspatial-70810\DotSpatial.Controls\Extensions\AppManager.cs:line 386   at readDBF.mapDSform.mapDSform_Load(Object sender, EventArgs e) in D:\LAVORO\RigottiSUAP\ConcessioniConsequenziale\readDBF\readDBF\mapDSform.vb:line 95   at System.EventHandler.Invoke(Object sender, EventArgs e)   at System.Windows.Forms.Form.OnLoad(EventArgs e)   at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible)   at System.Windows.Forms.Control.CreateControl()   at System.Windows.Forms.Control.WmShowWindow(Message& m)   at System.Windows.Forms.Control.WndProc(Message& m)   at System.Windows.Forms.Form.WndProc(Message& m)   at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

 

 

do you know what is the reason?

 

 

Please help me understand

Thank you

Oscar

Jan 23, 2013 at 5:21 PM
Edited Jan 23, 2013 at 5:43 PM

Removing

  • ..\release\gdal\x86\plugins\ogr_OCI.dll
  • ..\release\gdal\x64\plugins\ogr_OCI.dll

should resolve your issue. These files are needed if you want to access Oracle database with GDAL/OGR. In this case you'd have to install oracle client, too.

Hth FObermaier

Jan 23, 2013 at 6:21 PM

thank you

 

now it works perfectly

Oscar

Jan 24, 2013 at 9:47 AM

Sorry Felix if I keep disturbing you (I fear next time you will send me an invoice to pay your help),

regarding memory usage, I was looking into the dotspatial source code and more specifically in the controls.MapRasterLayer.cs file

There, for what I understand of C# (almost nothing) I see that for each rasterlayer added to the map a BMP image (inram) is created.

So it seems to me that when one loads a raster layer, in ram you find the raster dataset plus the BMP image used to render it. This could partly justify what I see in memory usage where for 100 Mb raster I get three times memory usage.

Is it correct?

Then, when I decide to remove this raster file is it possible that the BMP image will remain loaded into memory? even if the raster layer is disposed?

If what I wrote is correct, is there a way to dispose the BMP image?

Hope I have been clear enough.

Thank you

Oscar

Jan 24, 2013 at 9:05 PM
oscarafone77 wrote:

Then, when I decide to remove this raster file is it possible that the BMP image will remain loaded into memory? even if the raster layer is disposed?

If what I wrote is correct, is there a way to dispose the BMP image?

Override the Dispose Method in the MapRasterLayer and dispose both _stencil and _backBuffer if they are not equal and not null. Don't forget to call base.Dispose.

Raise an issue about it!

Hth FObermaier

Jan 27, 2013 at 9:43 AM

Hello,

I want to load Ascii Grids or other Raster layers in my GUI.

So I followed the tutorial (http://mudnug.wordpress.com/2011/11/23/how-to-support-loading-extensions-in-dotspatial/) to support loading extensions step by step. But it doesn't work for me.

Vatavian wrote that 'you can reference the DotSpatial.Data.Rasters.GdalExtension library directly in your project rather than have it loaded as a plugin.' So i added a referens con DotSpatial.Data.Rasters.GdalExtension.dll. But this didn't work either. Loading raster data apart from bgd-Files is not possible.

Can somebody give me a hint?

Thanks in advance,

Claudia

Jan 28, 2013 at 8:35 AM

Dear Claudia,

I think the first check you have to make is that when you run your project you should not get any exception. It happened to me that having exceptions avoided plugins from being loaded. Remove from your building directory all the unnecessary dlls, you might solve your problem.

The second way I advice you to follow is to read exactly the instructions written by FObermaier. To do that you need to have visual studio professional or above, otherwise you wouldn't be able to download the nuget Gdal package.

Basically you need to apply the patch he points to in his 15th January post (follow the issue link). This means downloading the dotspatial source, open it and modify by hand all the lines reported in the patch. Recompile it and copy the GDAL output in your project bin directory.

Hope you get through it 

Oscar

Jan 28, 2013 at 5:09 PM

Hi Felix,

I tried to do what you suggest, but had no luck (maybe my poor (absent) knowledge of c# or the wrong changes).

I know I am a bit  annoying you, but can you please give me detailed instructions as in your GDAL patch?

If this is too much don't worry, I'll understand.

Thanks

Oscar

Feb 22, 2013 at 9:30 AM
i rised an issue on the memory problem. Hope it will be considered soon.

Oscar