6
Vote

Add Support for loading large rasters

description

Allow loading several layers of large rasters piecemeal.

file attachments

comments

jartieda wrote May 16, 2012 at 5:00 AM

hi,
I have modified GdalImage.cs so that big images are not loaded in memory for representation.
most of the modifications are on the GetBitmap method.
The work is istill unfinished as it only works now for not rotated rgb images.
However I would like to know your opinion about my approach.
Regards
Jorge

mudnug wrote May 16, 2012 at 5:15 PM

I didn't understand why you use if (_dataset != null) in ReadBlock(...) instead of if (_dataset == null)

We were glad to see this contribution though there are a number of issues (visually and performance-wise), we're excited to see where you can take this!

jartieda wrote May 18, 2012 at 10:38 AM

Hi,
I have some advances. I have added the overviews capabilities and solved some things.
I attach the files so you can see the progress and provideme some feedback
regards
jorge

mudnug wrote May 18, 2012 at 9:46 PM

I notice you added an unsafe block. Is that the preferred way to call GDAL?

BTW, _red.GetOverview(_overview) is null, causing an exception in GetBitmap(...) with your new changes.

I add a constructor to GdalImageProvider so I can open a jpg to get this result.
    public GdalImageProvider()
    {
        GdalHelper.Configure();

        // Add ourself in for these extensions, unless another provider is registered for them.
        string[] extensions = { ".jpg" };
        foreach (string extension in extensions)
        {
            if (!DataManager.DefaultDataManager.PreferredProviders.ContainsKey(extension))
            {
                DataManager.DefaultDataManager.PreferredProviders.Add(extension, this);
            }
        }
    }

jartieda wrote May 21, 2012 at 12:30 PM

Ok, a better version...
removed the unsafe block. I was a try to solve a problem with .net not freeing the big memory blocks. This way to call gdal avoids one memory copy (I think) but doesn't solve the memory segmentation problem. However the overviews solved it.
I had to remove the static identifier in some methods to access the _overview variable. I am not sure about the implications of this.
I have update the ARGB, grey and palleted readers but haven't tested them.
I am not sure why it tries to build pyramids when I load a gray scale .img file. The pyr. building method fails with big images.
The white stripes artifacts are only solved for images which can be read in stripe blocks.
Tiled images must be tested.
please let me know any comment you have or what else must be done to put this on the main branch
Thanks for your previous suggestions.
Regards.
Jorge

mudnug wrote May 21, 2012 at 8:20 PM

I think you have made great advances, here. I'm checking in your contribution with only spelling changes, but I would like to see GetBitmap broken into several smaller methods, and additional features added.
We'll get this in the extension manager quickly so that users can start testing it.
Also, in GetColorPalette(), please dispose the ColorEntry if that is appropriate and dispose red in WriteArgb(...) to prevent memory leaks.

wrote May 21, 2012 at 8:25 PM

Associated with changeset 67230.