OmegaMan's Musings

A user wanted to parse basic text. Here is a regex pattern which  breaks out the user text.

string @pattern = @"
(?:OS\s)                     # Match but don't capture (MDC) OS, used an an anchor
(?<Version>\d\.\d+)          # Version of OS
(?:;)                        # MDC ;
(?<Phone>[^;]+)              # Get phone name up to ;
(?:;)                        # MDC ;
(?<Type>[^;]+)               # Get phone type up to ;
(?:;)                        # MDC ;
(?<Major>\d\.\d+)            # Major version
(?:;)
(?<Minor>\d+)                # Minor Version
";

string data = 
@"Windows Phone Search (Windows Phone OS 7.10;Acer;Allegro;7.10;8860)
Windows Phone Search (Windows Phone OS 7.10;HTC;7 Mozart T8698;7.10;7713)
Windows Phone Search (Windows Phone OS 7.10;HTC;Radar C110e;7.10;7720)";

 // Ignore pattern white space allows us to comment the pattern, it is not a regex processing command
var phones = Regex.Matches(data, pattern, RegexOptions.IgnorePatternWhitespace)
                  .OfType<Match>()
                  .Select (mt => new 
                  {
                    Name = mt.Groups["Phone"].Value.ToString(),
                    Type = mt.Groups["Type"].Value.ToString(),
                    Version = string.Format( "{0}.{1}", mt.Groups["Major"].Value.ToString(),
                                                        mt.Groups["Minor"].Value.ToString())
                  }
                  );
                  
Console.WriteLine ("Phones Supported are:");            

phones.Select(ph => string.Format("{0} of type {1} version ({2})", ph.Name, ph.Type, ph.Version))
      .ToList()
      .ForEach(Console.WriteLine);
      
/* Output
Phones Supported are:
Acer of type Allegro version (7.10.8860)
HTC of type 7 Mozart T8698 version (7.10.7713)
HTC of type Radar C110e version (7.10.7720)
*/

Most of the time one needs the power of the xml parser whether it is the XmlDocument or Linq to Xml to manipulate and extract data. But what if I told you that in some circumstances regular expressions might be faster?

Most conventional development thinking has branded regex processing as slow and the thought of using regex on xml might seem counter intuitive. In a continuation of articles I again want to dispel those thoughts and provide a real world example where Regular Expression parsing is not only on par with other tools in the .Net world but sometimes faster. The results of my speed test may surprise you;  and hopefully show that regular expressions are not as slow as believed, if not faster!

See: Are C# .Net Regular Expressions Fast Enough for You?

Real World Scenario

There was a developer on the MSDN forums who needed the ability to count URLs in multiple xml files. (See the actual post count the urls in xml file on Msdn) The poster received three distinct replies, one to use XMLDocument, another provided a Linq to XML solution and I chimed in with the regular expression method. The poster took the XMLDocument method and marked as the answer, but could he have done better?

I thought so…

So I took the three replies and distilled them down into their core processing and wrapped them in a similar IO extraction layer and proceeded to time them. I created 48 xml files with over one hundred thousand urls to find for a total of 13 meg on disk. I then proceeded to run the test all in release mode to get the results.  (See below section Setup to get a gist repository of the code).

Real World Result

Five tests, each test name is the technology and the user as found on the original msdn post. In red is the slowest and fastest time. Remember XmlDoc is the one the user choose as the answer.

Test 1
Regex           found 116736 urls in 00:00:00.1843576
XmlLinq_Link_FR found 116736 urls in 00:00:00.2662190
XmlDoc_Hasim()  found 116736 urls in 00:00:00.3534628

Test 2
Regex           found 116736 urls in 00:00:00.2317883
XmlLinq_Link_FR found 116736 urls in 00:00:00.2792730
XmlDoc_Hasim()  found 116736 urls in 00:00:00.2694969

Test 3
Regex           found 116736 urls in 00:00:00.1646719
XmlLinq_Link_FR found 116736 urls in 00:00:00.2333891
XmlDoc_Hasim()  found 116736 urls in 00:00:00.2625176

Test 4
Regex           found 116736 urls in 00:00:00.1677931
XmlLinq_Link_FR found 116736 urls in 00:00:00.2258825
XmlDoc_Hasim()  found 116736 urls in 00:00:00.2590841

Test 5
Regex           found 116736 urls in 00:00:00.1668231
XmlLinq_Link_FR found 116736 urls in 00:00:00.2278445
XmlDoc_Hasim()  found 116736 urls in 00:00:00.2649262

Wow! Regex consistently performed better, even when there was no caching of the files as found for the first run! Note that the time is Hours : Minutes : Seconds and regex’s is the fastest at 164 millseconds to parse 48 files! Regex worst time of 184 milleseconds is still better than the other two’s best times.

How was this all done? Let me show you.

Setup

Ok what magic or trickery have I played? All tests are run in a C# .Net 4 Console application in release mode. I have created a public Gist (Regex vs Xml) repository of the code and data which is actually valid Git repository for anyone how may want to add their tests, but let me detail what I did here on the blog as well.

The top level operation found in the Main looks like this where I run the tests 5 times

Enumerable.Range( 1, 5 )
            .ToList()
            .ForEach( tstNumber =>
            {
                Console.WriteLine( "Test " + tstNumber );
                Time( "Regex", RegexFindXml );
                Time( "XmlLinq_Link_FR", XmlLinq_Link_FR );
                Time( "XmlDoc_Hasim()", XmlDoc_Hasim );
                Console.WriteLine( Environment.NewLine );
            }

while the Time generic method looks like this and dutifully runs the target work and reports the results in “Test X found Y Urls in X [time]”:

public static void Time<T>( string what, Func<T> work )
{
    var sw = Stopwatch.StartNew();
    var result = work();
    sw.Stop();
    Console.WriteLine( "\t{0,-15} found {1} urls in {2}", what, result, sw.Elapsed );
}

Now in the msdn post the different methods had differing ways of finding each xml file and opening it, I made them all adhere to the way I open and sum the ULR counts. Here is its snippet:

return Directory.EnumerateFiles( @"D:\temp", "*.xml" )
            .ToList()
            .Sum( fl =>
            {

            } );

Contender  –  XML Document

This is one which the poster marked as the chosen one he used and I dutifully copied it to the best of my ability.

public static int XmlDoc_Hasim()
{
    return Directory.EnumerateFiles( @"D:\temp", "*.xml" )
                .ToList()
                .Sum( fl =>
                {
                    XmlDocument doc = new XmlDocument();
                    doc.LoadXml( System.IO.File.ReadAllText( fl ) );

                    if (doc.ChildNodes.Count > 0)
                        if (doc.ChildNodes[1].HasChildNodes)
                            return doc.ChildNodes[1].ChildNodes.Count;

                    return 0;

                } );

}

I used the sum extension method which is a little different from the original sum operation used, but it brings the tests closer in line by using the Extension.

Contender – Linq to Xml

Of the other two attempts, this one I felt was the more robust of the two, because it actually handled the xml namespace. Sadly it appeared to be ignored by the original poster. Here is his code

public static int XmlLinq_Link_FR()
{
    XNamespace xn = "http://www.sitemaps.org/schemas/sitemap/0.9";

    return Directory.EnumerateFiles( @"D:\temp", "*.xml" )
                    .Sum( fl => XElement.Load( fl ).Descendants( xn + "loc" ).Count() );

}

Contender – Regular Expression

Finally here is the speed test winner. I came up with the pattern design Upon by looking at the xml and it appeared one didn’t need to match the actual url, but just the two preceding  tags and any possible space between. That is the key to regex, using good patterns can achieve fast results.

public static int RegexFindXml()
{
    string pattern = @"(<url>\s*<loc>)";

    return Directory.EnumerateFiles( @"D:\temp", "*.xml" )
                    .Sum( fl => Regex.Matches( File.ReadAllText( fl ), pattern ).OfType<Match>().Count() );

}

XML1 (Shortened)

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>http://www.linkedin.com/directory/companies/internet-web2.0-startups-social-networking/barcelona.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/internet-web2.0-startups-social-networking/basel.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/internet-web2.0-startups-social-networking/bath.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/computer-networking/sheffield.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/computer-networking/singapore.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/computer-networking/slough.html</loc><changefreq>weekly</changefreq></url>
<url><loc>http://www.linkedin.com/directory/companies/computer-networking/slovak-republic.html</loc><changefreq>weekly</changefreq></url>
</urlset>

Xml2 Shortened

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>http://www.linkedin.com/groups/gid-2431604</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2430868</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/Wireless-Carrier-Reps-Past-Present-2430807</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2430694</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2430575</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2431452</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432377</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2428508</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432379</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432380</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432381</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432383</loc><changefreq>monthly</changefreq></url>
<url><loc>http://www.linkedin.com/groups/gid-2432384</loc><changefreq>monthly</changefreq></url>
</urlset>

Summary

It really comes down to the right tool for the right situation and this one regex really did well. But Regex is not good at most xml parsing needs, but for certain scenarios it really shines. If the xml has malformed or the namespace was wrong, then the parser has its own unique problems which would lead to a bad count. All the technologies had to do some upfront loading and that is key to how they performed. Regex is optimized to handle large data efficiently and as long as the pattern is spot on, it can really be quick.

My thought is don’t dismiss regular expression parsing out of hand, while the learning of it can pay off in some unique text parsing situations.

Finding Identical Strings using GroupBy

List<string> theList = new List<string>() { "Alpha", "Alpha", "Beta", "Gamma", "Delta" };

theList.GroupBy(txt => txt)
        .Where(grouping => grouping.Count() > 1)
        .ToList()
        .ForEach(groupItem => Console.WriteLine("{0} duplicated {1} times with these values {2}",
                                                 groupItem.Key, 
                                                 groupItem.Count(),
                                                 string.Join(" ", groupItem.ToArray())));
// Writes this out to the console:
//
// Alpha duplicated 2 times with these values Alpha Alpha

Line By Line Explanation

GroupBy Usage with Only Two Defined Keys Regex Example

List<string> theList = new List<string>() { "Alpha", "Alpha", "Beta", "Gamma", "Delta" };

theList.GroupBy( txt => Regex.IsMatch( txt, "ta$" ))
       .ToList()
       .ForEach(groupItem => Console.WriteLine("{0} found {1} times with these values: {2}",
                                                 groupItem.Key,
                                                 groupItem.Count(),
                                                 string.Join(" ", groupItem.ToArray())));
// Output
// False found 3 times with these values: Alpha Alpha Gamma
// True found 2 times with these values: Beta Delta

GroupBy Usage with one Key or a Non Related Key

List<string> theList = new List<string>() { "Alpha", "Alpha", "Beta", "Gamma", "Delta" };

theList.GroupBy(txt => 1 )
        .ToList()
        .ForEach(groupItem => Console.WriteLine("The Key ({0}) found a total of {1} times with a total letter count of {2} characters",
                                                 groupItem.Key,
                                                 groupItem.Count(),
                                                 groupItem.Sum(it => it.Count())));

// Output:
// The Key (1) found a total of 5 times with a total letter count of 24 characters

This snippet is from my archives and reminds me how to setup column names for a WPF/Silverlight datagrid and bind them to the data by not using AutoGenerateColumn feature. (See below for the visual end result.)

Ω Check out the related post Xaml: Adding Visibility Behaviors Using Blend to A DataGrid for WPF or Silverlight

DataGrid Column Names Setup in Xaml

In this example we will have two columns where the data will be filename strings. We will specify the columns in the Xaml to use the DataGridTextColumn and not to dynamically generate columns on our Datagrid:

<DataGrid x:Name="dgOperation" AutoGenerateColumns="False">
    <DataGrid.Columns>
        <DataGridTextColumn Header="File Name Before" Binding="{Binding Path=Original}"/>
        <DataGridTextColumn Header="File Name After"  Binding="{Binding Path=New}"/>
    </DataGrid.Columns>
</DataGrid>

The result is that the header row will have two columns with the name “File Name Before” and “File Name After” will subsequently bind them to a data object with the property names of “Original” and “New”.

Actual Binding During the Loading of the Grid

We will read in a directory for the WPF example from the hard drive and fill the original file names to the first column and a generated name for the second column. Since we have specified in the Xaml that we are binding to “Original” and “New” properties the dynamic linq object created will have those properties.

dgOperation.ItemsSource = Directory.GetFiles( @"C:\" )
                                   .Select( ( nm, index ) => new
                                       {
                                          Original = System.IO.Path.GetFileName( nm ),
                                          New = string.Format( "{0}_{1}{2}", System.IO.Path.GetFileNameWithoutExtension( nm ),
                                                                             index,
                                                                             System.IO.Path.GetExtension( nm ) )
                                        } );

The result is as below:

Note: Out of the box editing the rows will throw an exception. To fix that make each of the rows read only.

(Update 5/16/2011: Fixed mispelling)
I ran across a situation where the code I was working on had an enum with values and I needed to display a user friendly text which was related to the enum but not the enum’s actual text value.

The standard way of mapping values across cultures is to create a localized resource file(s) (.resx) and to put in string key and a string value, where the value will be shown. The code can then do an if check to map between the two; but that gets laborious real quick. It would be better to have a dictionary where the enum is the key and the value is the value of the resources file.

The following snippet of code takes in an enum and its corresponding resource resx data and creates that dictionary. See the section Steps to Test for an example of its usage.

/// <summary>
/// Take an enum and a created corresponding resx resource file and join the two together 
/// into a dictionary. The dictionaries key is the actual enum and the value is the user readable text
/// found in the value of the resource file.
/// </summary>
/// <typeparam name="T">The Enum type which contains the target enums</typeparam>
/// <param name="rm">The .resx resource's manager which contains the mapped text</param>
/// <returns>A dictionary whose key is the enum and the result is the mapped text in the resource file.</returns>
public static Dictionary<T, string> LoadResourceEnumMappings<T>( ResourceManager rm )  
{
    T[] eValues = (T[])System.Enum.GetValues( typeof( T ) ); // Gets the actual values of the enum (T) type

    // Puts those into a local dictionary/hash for use in later.
    var dictEnum = eValues.ToDictionary( it => it.ToString(), it => it ); 

    // Work through the key value pairs (KVP) of the resource set and marry them
    // to a new dictionary where the key is the enum and the value output is the user string
    // as found in the resource's value of its kvp.
    return rm.GetResourceSet( Thread.CurrentThread.CurrentCulture, true, false )
             .OfType<DictionaryEntry>()
             .ToDictionary( kvp => dictEnum[kvp.Key.ToString()], kvp => kvp.Value.ToString() );

}

NOTES

• One can have more resource keys than enums and the above will work. But if there are more enums than resource keys, the above code will throw a KeyNotFound exception.

Enumerate the Embedded Resource

The above code uses the ability to enumerate or iterate the resource file by calling GetResourceSet. That calls returns a Dictionary entry which has the key value pair of the resource file and could be used with a foreach.

Steps To Test

1. Create console application
2. Create enum named MappedValues with these enums : Alpha, Beta, Gamma.

   public enum MappedValues
   {
       Alpha,
       Beta,
       Gamma
   }

3. Create Resource File named UserText.Resx with these values: 

Test as such with this code by calling EnumMapper.Usage():

public static class EnumMapper
{
    /// <summary>
    /// This is just for show and not meant for production.
    /// </summary>
    public static void Usage()
    {
        try
        {
            Console.WriteLine( "Load resource and show: " + UserText.Alpha );

            Dictionary<MappedValues, string> mapped = LoadResourceEnumMappings<MappedValues>( UserText.ResourceManager );

            Console.WriteLine( mapped[MappedValues.Alpha] );

            Console.WriteLine( mapped[MappedValues.Beta] );

            Console.WriteLine( mapped[MappedValues.Gamma] );
        }
        catch ( KeyNotFoundException )
        {
            Console.WriteLine("The Resource File has a key which is not found in the enum.");
        }

/* outputs (Note "first value" was shown to initialize the ResourceManager otherwise it would be null from GetResourceSet)
Load resource and show: First Value
First Value
Second Item
Third Wave
*/
    }

    /// <summary>
    /// Take an enum and a created corresponding resx resource file and join the two together 
    /// into a dictionary. The dictionaries key is the actual enum and the value is the user readable text
    /// found in the value of the resource file.
    /// </summary>
    /// <typeparam name="T">The Enum type which contains the target enums</typeparam>
    /// <param name="rm">The .resx resource's manager which contains the mapped text</param>
    /// <returns>A dictionary whose key is the enum and the result is the mapped text in the resource file.</returns>
    public static Dictionary<T, string> LoadResourceEnumMappings<T>( ResourceManager rm )  
    {
        T[] eValues = (T[])System.Enum.GetValues( typeof( T ) ); // Gets the actual values of the enum (T) type

        // Puts those into a local dictionary/hash for use in later.
        var dictEnum = eValues.ToDictionary( it => it.ToString(), it => it ); 

        // Work through the key value pairs (KVP) of the resource set and marry them
        // to a new dictionary where the key is the enum and the value output is the user string
        // as found in the resource's value of its kvp.
        return rm.GetResourceSet( Thread.CurrentThread.CurrentCulture, true, false )
                    .OfType<DictionaryEntry>()
                    .ToDictionary( kvp => dictEnum[kvp.Key.ToString()], kvp => kvp.Value.ToString() );

    }

}

I write this because if I ran into this, someone else will. Now first off it wasn’t Linq that failed, but it looked like it and here is my story of failure I found in a WPF background worker threading code which can happen in other areas as well.

The perennial advice to all people in the MSDN forums as well as others is never, never, never  write to a GUI control in a thread or the BackgroundWorker’s do work event. All GUI work must be done on a the GUI. I very well knew that…but the reverse (don’t read from a GUI in a different thread)  is also true and that bit me. Let me explain.

Anecdotal Evidence

Imagine my surprise when I created a WPF project at a new job and started getting this exception in the DoWork code of my BackgroundWorker when the first use of a Linq query’s delayed execution point was accessed :

The calling thread cannot access this object because a different thread owns it.

The code threw an exception, on line 16 below, where I was loading data from after the Linq call.  I began to think, does this have something to do with the Linq DataContext?

void bcLoad_DoWork(object sender, DoWorkEventArgs e)
{
    try
    {
        List<string> Results = new List<string>();

        DatabaseDataContext dbc = new DatabaseDataContext();

        var data = dbc.SystemData
                      .Where(ac => ac.Account_id == tbAccount.Text)
                      .Where(ac => ac.TimeStamp == 0)
                      .Where(ac => ac.Category_id == tbCategory.Text)
                      .Where(ac => (int)ac.Category_seq_nbr == int.Parse(tbSequenceNumber.Text))
                      .Select(ac => ac.Unit_Code);

        Results.Add("Unit Code: " + data.First());

        e.Result = Results;
    }
    catch (Exception ex)
    {
        lbxData.Dispatcher.BeginInvoke( new Action(() => lbxData.Items.Add( "Exception Caught: " + ex.Message )));
    } 
}

No the problem was within the setup of the Lambdas for the Linq query. All the highlighted lines above is where the actual problem originates and not on the final highlighted line.

The problem was that I was accessing Gui controls data, and not changing; that was the nuance. For in my mind that was ok, it was a passive read action and not a direct writing one. Obviously not.

Note: If you have come to this blog experiencing this problem but for the writing of items to a control, one method to solve it is to use the Dispatcher off the control on any thread not just BackgroundWorker. That code is shown in my exception catch blog above. That line is perfectly fine to do and is not another issue. The lbxData is a Listbox on the main Xaml and because of the immediacy of the exception, I write

Resolution

Since I was already using the plumbing of the DoWorkEventArgs, it seemed a natural choice to pass in the data using that object. I changed the call to pass in a Dictionary of values to extract the data as such:

bcLoad.RunWorkerAsync(new Dictionary<string, string>() 
                        { 
                            { "AccountID",      tbAccount.Text }, 
                            { "CategoryID",     tbCategory.Text },
                            { "SequenceNumber", tbSequenceNumber.Text }
                        });

Then to consume the Dictionary as such:

void bcLoad_DoWork(object sender, DoWorkEventArgs e)
{
    try
    {
        List<string> Results = new List<string>();

        Dictionary<string, string> UserInputs = e.Argument as Dictionary<string, string>;

        if (UserInputs != null)
        {

        DatabaseContext dbc = new DatabaseContext();

        var data = dbc.SystemData
                      .Where(ac => ac.Account_ID == UserInputs["AccountID"])
                      .Where(ac => ac.TimeStamp == 0)
                      .Where(ac => ac.Category_id == UserInputs["CategoryID"])
                      .Where(ac => (int)ac.Category_seq_nbr == int.Parse(UserInputs["SequenceNumber"]))
                      .Select(ac => acc.UnitCode);

        Results.Add("Unit Code: " + data.First());

        e.Result = Results; // Pass the results to the completed events to process them accordingly.
        }
    }
    catch (Exception ex)
    {
        lbxData.Dispatcher.BeginInvoke( new Action(() => lbxData.Items.Add( "Exception Caught: " + ex.Message )));
    }

}

I simply convert the object property of Argument to a Dictionary, as highlighted, and go do the work. One doesn’t have to use a Dictionary. One can pass in any object off of the Argument property. Hope This Helps

This was an issue I answered in the forums which a user presented and felt that the response was intricate enough to share with the world as a whole. The user wanted to have a string converted to title case but also wanted to have the first letter of any article or preposition to not be be upper case along with the rest of the sentence. This article discusses how to do that in C#.

For example the user was interested in changing

“ALL QUIET ON THE WESTERN FRONT”

   to

“All Quiet on the Western Front.”

.Net Framework Almost Does It

Thanks to the TextInfo class and a helping hint from a current CultureInfo object we can use the method ToTitleCase to work with our current language. The problem is that when ToTitleCase is called with the original sentence we get this:

“All Quiet On The Western Front”

Give it some Help

The .Net code is not robust enough to ignore the articles and prepositions so we will augment it. The following code using Linq-to-Object and Regex and processes majority of the target articles and prepositions . I have placed it into an extension method below:

/*
using System.Globalization;
using System.Threading;
using System.Text.RegularExpressions;
*/

/// <summary>
/// An Extension Method to allow us t odo "The Title Of It".asTitleCase()
/// which would return a TitleCased string.
/// </summary>
/// <param name="title">Title to work with.</param>
/// <returns>Output title as TitleCase</returns>
public static string asTitleCase ( this string title)
{
    string WorkingTitle = title;

    if ( string.IsNullOrEmpty( WorkingTitle ) == false )
    {
        char[] space = new char[] { ' ' };

        List<string> artsAndPreps = new List<string>()
            { "a", "an", "and", "any", "at", "from", "into", "of", "on", "or", "some", "the", "to", };

        //Get the culture property of the thread.
        CultureInfo cultureInfo = Thread.CurrentThread.CurrentCulture;
        //Create TextInfo object.
        TextInfo textInfo = cultureInfo.TextInfo;

        //Convert to title case.
        WorkingTitle = textInfo.ToTitleCase( title.ToLower() );

        List<string> tokens = WorkingTitle.Split( space, StringSplitOptions.RemoveEmptyEntries ).ToList();

        WorkingTitle = tokens[0];

        tokens.RemoveAt(0);

        WorkingTitle += tokens.Aggregate<String, String>( String.Empty, ( String prev, String input )
                                => prev +
                                    ( artsAndPreps.Contains( input.ToLower() ) // If True
                                        ? " " + input.ToLower()              // Return the prep/art lowercase
                                        : " " + input ) );                   // Otherwise return the valid word.

        // Handle an "Out Of" but not in the start of the sentance
        WorkingTitle = Regex.Replace( WorkingTitle, @"(?!^Out)(Out\s+Of)", "out of" );
    }

    return WorkingTitle;

}

Explanation

• Line 21: Here is our English list of words not to capitalize. We would have to change this for other languages.
• Line 25: We get the current culture from the running thread so that ToTitleCase can do its job.
• Line 30: ToTitleCase does the first run and upper cases all the first letters and drops any following upper case letters if they exist.
• Line 32: We split the line on space between the words into word tokens and put them in a list.
• Line 34: We save off the first word because regardless of what it is, it is correct.
• Line 36: We remove the first word so not to process it.
• Line 40: Using the Aggregate extension to accumulate each word token we will add a space. We are using the aggregate method in-lieu of string.Join to add spaces to our words (the accumulation), but also to check each word as it goes by which string.Join can’t help us with.
• Line 42: As the tokens (words) are handed to us, check to see if they are in the list we setup in line 21. If it exists, add a space in front and make the whole word lower case (Line 43) other wise ad a space and just return the word.
• Line 46: Handle any two word Out Of issues, but ignore if it is the first word as found in “Out of Africa”.

Tests and Results

Console.WriteLine( "ALL QUIET ON THE WESTERN FRONT".asTitleCase() );
Console.WriteLine( "Bonfire OF THE Vanities".asTitleCase() );
Console.WriteLine( "The Out-of-Sync Child: Recognizing and Coping with Sensory Processing Disorder".asTitleCase() );
Console.WriteLine( "Out OF AFRICA".asTitleCase() );

/* Results
All Quiet on the Western Front
Bonfire of the Vanities
The Out-Of-Sync Child: Recognizing and Coping With Sensory Processing Disorder
Out of Africa
*/

What if you are stuck having to deal with older technology such as INI files while using the latest and greatest C# and .Net there is available? This article discusses an alternate way to read INI files and extract the data from those dusty tomes while  easily accessing the resulting data from dictionaries. Once the data resides in the dictionaries we can easily extract the data using the power of the indexer on section name followed by key name within the section. Such as IniFile[“TargetSection”][“TargetKey”] which will return a string of the value of that key in the ini file for that section.

Note all the code is one easy code section at the bottom of the article so don’t feel you have to copy each sections code.

Overview

If you are reading this, chances are you know what INI files are and don’t need a refresher. You may have looked into using the Win32 Kern32.dll method GetPrivateProfileSection to achieve your goals. Ack!  “Set the Wayback machine Sherman!” Thanks but no thanks.

Here is how to do this operation using Regular Expressions (Kinda a way back machine but very useful) and Linq to Object to get the values into a dictionary format so we can write this line of code to access the data within the INI file:

string myValue = IniFile[“SectionName”][“KeyName”];

The Pattern

Let me explain the Regex Pattern. If you are not so inclined to understand the semantics of it skip to the next section.

string pattern = @"
^                           # Beginning of the line
((?:\[)                     # Section Start
 (?<Section>[^\]]*)         # Actual Section text into Section Group
 (?:\])                     # Section End then EOL/EOB
 (?:[\r\n]{0,}|\Z))         # Match but don't capture the CRLF or EOB
 (                          # Begin capture groups (Key Value Pairs)
   (?!\[)                    # Stop capture groups if a [ is found; new section
   (?<Key>[^=]*?)            # Any text before the =, matched few as possible
   (?:=)                     # Get the = now
   (?<Value>[^\r\n]*)        # Get everything that is not an Line Changes
   (?:[\r\n]{0,4})           # MBDC \r\n
  )+                        # End Capture groups";

Our goal is to use Named Match groups. Each match will have its section name in the named group called  “Section”  and all of the data, which is the key and value pairs will be named “Key” and “Value” respectively.  The trick to the above pattern is found in line eight. That stops the match when a new section is hit using the Match Invalidator (?!). Otherwise our key/values would bleed into the next section if not stopped.

The Data

Here is the data for your perusal.

string data = @"[WindowSettings]
Window X Pos=0
Window Y Pos=0
Window Maximized=false
Window Name=Jabberwocky

[Logging]
Directory=C:\Rosetta Stone\Logs
";

We are interested in “Window Name” and “Directory”.

The Linq

Ok, if you thought the regex pattern was complicated, the Linq to Objects has some tricks up its sleeve as well. Primarily since our pattern matches create a single match per section with the accompany key and value data in two separate named match capture collections, that presents a problem. We need to join the the capture collections together, but there is no direct way to do that for the join in Linq because that link is only an indirect by the collections index number.

How do we get the two collections to be joined?

Here is the code:

Dictionary<string, Dictionary<string, string>> InIFile
= ( from Match m in Regex.Matches( data, pattern, RegexOptions.IgnorePatternWhitespace | RegexOptions.Multiline )
 select new
 {
  Section = m.Groups["Section"].Value,

  kvps = ( from cpKey in m.Groups["Key"].Captures.Cast<Capture>().Select( ( a, i ) => new { a.Value, i } )
     join cpValue in m.Groups["Value"].Captures.Cast<Capture>().Select( ( b, i ) => new { b.Value, i } ) on cpKey.i equals cpValue.i
     select new KeyValuePair<string, string>( cpKey.Value, cpValue.Value ) ).ToDictionary( kvp => kvp.Key, kvp => kvp.Value )

  } ).ToDictionary( itm => itm.Section, itm => itm.kvps );

Explanation:

• Line 1: Our end goal object is a Dictionary where the key is the Section name and the value is a sub-dictionary with all the keys and values found in that section.
• Line 2: The regex needs IPW because we have commented the pattern. It needs multiline because we are spanning multiple lines and need ^ to match each individual line and not just the beginning.
• Line 5: This is the easiest item, simply access the named capture group “Section” for the section name.
• Line 7 (.Captures) : Each one of the keys and values are in the specialized capture collection property off of the match.
• Line 7 (.Cast<Capture>) : Since capture is specialized list and not a true generic list, such as List<string> we are going to Cast it(Cast<(Of <(TResult>) it (to IEnumerable<(Of <(T>)>),so we can access the standard query operators, i.e. the extension methods which are available to IEnumerable<T>. Short answer, so we can call .Select.
• Line 7 (.Select): Because each list does not have a direct way to associate the data, we are going to create a new object that has a property which will have that index number, along with the target data value. That will allow us join it to the other list.
• Line 7 (Lambda) : The lambda has two parameters, the first is our actual regex Capture object represented by a. The i is the index value which we need for the join. We then call new and create a new entity with two properties, the first is actual value of the Key found of the Capture class property “Value” and the second is i the index value.
• Line 8 (Join) : We are going to join the data together using the direct properties of our new entity, but first we need to recreate the magic found in Line 7 for our Values capture collection. It is the same logic as the previous line so I will not delve into its explanation in detail.
• Line 8 (on cpKey.i equals cpValue.i) : This is our association for the join on the new entities and yay, where index value i equals the other index value i allows us to do that. This is the keystone of all we are doing.
• Line 9 (new KeyValuePair) : Ok we are now creating each individual linq projection item of the data as a KeyValuePair object. This could be removed for a new entity, but I choose to use the KeyValuePair class.
• Line 9 (ToDictionary) : We want to easily access these key value pairs in the future, so we are going to place the Key into a Key of a dictionary and the dictionary key’s value from the actual Value.
• Line 11 (ToDictionary) : Here is where we take the projection of the previous lines of code and create the end goal dictionary where the key name is the section and the value is the sub dictionary created in Line 9.

Whew…what is the result?

Console.WriteLine( InIFile["WindowSettings"]["Window Name"] ); // Jabberwocky
Console.WriteLine( InIFile["Logging"]["Directory"] );          // C:\Rosetta Stone\Logs

Summary

Thanks to the power of regular expressions and Linq we don’t have to use the old methods to extract and process the data. We can easily access the information using the newer structures. Hope this helps and that you may have learned something new from something old.

Code All in One Place

Here is all the code so you don’t have to copy it from each section above. Don’t forget to include the using System.Text.RegularExpressions to do it all.

string data = @"[WindowSettings]
Window X Pos=0
Window Y Pos=0
Window Maximized=false
Window Name=Jabberwocky

[Logging]
Directory=C:\Rosetta Stone\Logs
";
string pattern = @"
^                           # Beginning of the line
((?:\[)                     # Section Start
     (?<Section>[^\]]*)     # Actual Section text into Section Group
 (?:\])                     # Section End then EOL/EOB
 (?:[\r\n]{0,}|\Z))         # Match but don't capture the CRLF or EOB
 (                          # Begin capture groups (Key Value Pairs)
  (?!\[)                    # Stop capture groups if a [ is found; new section
  (?<Key>[^=]*?)            # Any text before the =, matched few as possible
  (?:=)                     # Get the = now
  (?<Value>[^\r\n]*)        # Get everything that is not an Line Changes
  (?:[\r\n]{0,4})           # MBDC \r\n
  )+                        # End Capture groups";

Dictionary<string, Dictionary<string, string>> InIFile
= ( from Match m in Regex.Matches( data, pattern, RegexOptions.IgnorePatternWhitespace | RegexOptions.Multiline )
    select new
    {
        Section = m.Groups["Section"].Value,

        kvps = ( from cpKey in m.Groups["Key"].Captures.Cast<Capture>().Select( ( a, i ) => new { a.Value, i } )
                 join cpValue in m.Groups["Value"].Captures.Cast<Capture>().Select( ( b, i ) => new { b.Value, i } ) on cpKey.i equals cpValue.i
                 select new KeyValuePair<string, string>( cpKey.Value, cpValue.Value ) ).ToDictionary( kvp => kvp.Key, kvp => kvp.Value )

    } ).ToDictionary( itm => itm.Section, itm => itm.kvps );

Console.WriteLine( InIFile["WindowSettings"]["Window Name"] ); // Jabberwocky
Console.WriteLine( InIFile["Logging"]["Directory"] );          // C:\Rosetta Stone\Logs