Symbology:Value Rendering

Developer
Sep 21, 2010 at 3:52 PM

Hey All,

We use a special case of attribute-driven symbology.  I think it is commonly called "value rendering".  We set up a symbology table that maps an integer 'value' to a symbology; This table can contain thousands of entries.  We have something like 40 attributes on a shapefile that control symbology.  User settings determine at any given time which  single attribute is to be used for symbology.  The value of the attribute on the shape determines the symbology to be used when drawing the shape.  So, the capabilities we need are:

  • Ability to set the rendering value to symbology map on a layer (this is a one-time setup)
  • Ability to tell the layer from which attribute to get the rendering value (this changes when the user changes a setting).

I tried to implement with the current symbology system in Dot Spatial using a line layer example.  I'd like to know if there is a better approach.  Here is what I did in English:

  • Created a LineScheme for each of the 40 attributes. 
  • Each LineScheme contained 1000 categories with the appropriate filter expressions.
    • LineScheme A (which is for attribute A) had categories with filter expressions like "[A]=16897", "[A]=16898" and so on.
    • LineScheme B (which is for attribute B) had categories with filter expressions like "[B]=16897", "[B]=16898" and so on.
    • Repeat LineSchemes until all 40 have been generated.
  • Based on user input, set the appropriate LineScheme on the map layer.

Here are some code snippets:

 First, we build the LineSchemes for each of the 40 attributes.

        private void btnGCSymbology_Click(object sender, EventArgs e)
        {
            // Build dictionary to lookup LineScheme from Attribute name.
            if (_lineDictSchemes.Count == 0)
            {
                LineScheme lineSchemTemplate = new LineScheme();
                lineSchemTemplate.EditorSettings.ClassificationType = ClassificationTypes.UniqueValues;
                lineSchemTemplate.EditorSettings.UseColorRange = false;
                lineSchemTemplate.EditorSettings.UseGradient = false;
                lineSchemTemplate.EditorSettings.StartSize = 1;
                lineSchemTemplate.EditorSettings.EndSize = 5;
                lineSchemTemplate.EditorSettings.UseSizeRange = true;
                // Create the categories for each rendering value.
                for (int j = 0; j < 1000; j++)
                {
                    ILineCategory lc = (ILineCategory)lineSchemTemplate.CreateRandomCategory();
                    lc.Symbolizer.SetWidth(1.0);
                    lineSchemTemplate.Categories.Add(lc);
                }

                // cmbSymbField holds the 40 some odd attribute names
                foreach (string strField in cmbSymbField.Items)
                {
                    LineScheme ls = new LineScheme();
                    int index = 16000;
                    foreach (ILineCategory cat in lineSchemTemplate.Categories)
                    {
                        LineCategory lc = new LineCategory(cat.GetColor(), cat.Symbolizer.GetWidth());
                        // Build FilterExpression using the attribute name and rendering value.
                        lc.FilterExpression = String.Format("[{0}]={1:D}", strField, index++);
                        ls.Categories.Add(lc);
                    }
                    _lineDictSchemes.Add(strField, ls);
                }
            }
        }

 Next, when the user selects which attribute should be used, we set the scheme on the layer and redraw the map:

private void cmbSymbField_SelectedIndexChanged(object sender, EventArgs e)
        {
            if (_lineDictSchemes.Count > 0)
            {
                LineScheme ls = _lineDictSchemes[(string)cmbSymbField.SelectedItem];
                IMapLineLayer[] lineLayers = map1.GetLineLayers();
                foreach (IMapLineLayer ill in lineLayers)
                {
                    ill.Symbology = ls;
                    MessageBox.Show(this, "Loaded Symbology");
                    //ill.ApplyScheme(ls);
                }
            }
            map1.Refresh();
        }

Observations:

  • Building up the categories on the scheme was slow.  I need to figure out how to make this faster.  We might be able to do use serialization since these settings do not change often.  I'm not particularly worried about this one.
  • Setting the LineScheme on the layer was slow (took 8 seconds for my example, I will look further into this to see where the time is being spent... may be data dependent... I was using a shapefile with 100K shapes).
  • Drawing was slow (5X slower than the case with one category).
    • I looked at the drawing code and it is looping over the categories, doing a query to find the shapes pertaining to that category, then drawing the shapes.

Discussion:

  1. Is there a better way to do this with current DS technology?
  2. Might other users have a similar use case?
  3. If answer to 1 is No, and answer to 2 is Yes, might we consider having a "ValueRendering" mode that uses a Dictionary to lookup symbologies based on the attribute value?

Thanks,

Kyle

 

 

Developer
Sep 21, 2010 at 4:46 PM

If you open a shapefile, the first optimization is that it doesn't even use categories unless it has to.  This speeds up the default case where everything on the layer has one symbology.  The second optimization is that if you are working with shapes in "IndexMode" which you should be if you opened the shapefile from disk, it caches the selection and symbology information in an array of "FastDrawnState" classes, which just match up the appropriate category to a given feature, as opposed to testing the actual attributes from the dbf file every time.  The code then cycles through this array in order to try to make as few actual calls to the drawing API as possible.  Each call adds a tiny amount of overhead.  If you call DrawPolygon() 100,000 times, it is quite a bit slower than if you call DrawGraphicsPath 40 times, with each graphicspath having something like 2500 shapes.  I don't think this was true at all for points, however, and so I think that code used a simpler technique.  For lines, the case is arguable.  For numbers of lines below about 50,000 or so and with very few categories, the graphics path works best.  As you have larger numbers of categories (and so make more and more cycles through the array), this approach is probably actually slower, especially as the number of members in the array also increases.  This particular aspect of working with GDI+ for our rendering has always been particularly vexing.  In unmanaged C++ you just cycle through each polygon and draw it according to its category and it still works fast.  In .Net, you really have to work for every once of speed that you see, and even then your so-called optimizations may end up working against you in cases that weren't being used much for your original performance evaluations.  We probably need another conditional test for the case of a large number of categories that switches back to making one drawing pass and drawing the lines one at a time.  Some performance evaluation could tell us where best to make that cutoff.

The good news is that the array of FastDrawnState classes (the dictionary you mentioned) is already there.  To fix the code, all we would have to do is update the DrawFeatures method, adding a test based on the number of categories and the number of features to be drawn and if it is larger than whatever, we would just draw each line separately, rather than creating graphics paths.  We might need to make sure that the code that constructs the line itself is accessible independently from where it is currently being used to build the graphics path.

The 8 seconds is probably the construction of the FastDrawnState array from the data table, which, with 100,000 shapes should be loading in pages.  This test currently involves reading all the values from the file, and might take a bit for 100,000 shapes, but one thing we could look at exposing in the paging data access is the option for reading only the values from the one column that you are working with instead of all of the values on each row.  It might make it faster in cases where you know the filter expressions are all related to a single row.  This would involve a tweak at the data access level.

Anyway, so yes, any input we can get on what might help get our rendering faster will likely be helpful.  This is a very young framework and has had precious few eyes go over it that have any real programming experience, and I thanks a bunch for your involvement.  Even letting me know this is an issue puts things moving in the right direction.

Watch out for the major refactoring going on right now though.  It will cause you a large headache with rewriting the using expressions in your code, but once it is done, it will be a lot easier to find what you are looking for in the libraries, and the ways the libraries can be used will increase.

Thanks,

Ted

 

 

 

Developer
Sep 21, 2010 at 5:34 PM

Yes, when setting the scheme on the layer, the bulk of time is spent in AttributTable.ReadTableRowFromChars. 

I suppose for GDI+, since h/w acceleration is not used, then whatever switching threshold we determine (num categories vs. num features) will work consistently on all machines.  I know I have seen posts on different drawing technologies, but it appears that the Map*Layer implementations are drawing technology-dependent and would need to be rewritten/optimized for each drawing technology?

When I get the opportunity, I will take a look at:

  1. Targeting a single column when reading the attribute table.
  2. Getting more familiar with the FastDrawnStates.

Thanks,

Kyle