Posts Tagged Linq

XPath meets Linq To XML in C#

There are two technologies which I feel that every coder should learn regardless of the language one programs in…those are Regular Expressions and XPath.

XPath is a quirkier language than Regular Expressions, but it is the the linqua-franca for working with xml documents. By specifying an XPath query which mirrors the target nodes one can get nodes from an xml document to be processed in the code.

The forum poster had xml data with a missing node, he had a car example. He wanted all the parent car nodes which had three tires (child nodes) and did not have four. Three wheel cars verse four wheel cars.

The following C# shows how to manipulate the Xpath and retrieve the nodes needed. Note one uses an xml editor with Xpath editing to figure out the stranger derivations.

using System.Xml.Linq;
using System.Xml.XPath;

...

string xml= @"<?xml version='1.0' encoding='UTF-8'?>
<Main>
<cars>
    <car name='Twingo'>
        <wheel1>abc</wheel1>
        <wheel2>def</wheel2>
        <wheel3>ghi</wheel3>
    </car>
    <car name='quattro'>
        <wheel1>some info</wheel1>
        <wheel2>more info</wheel2>
        <wheel3>blur</wheel3>
        <wheel4>We have four tires</wheel4>
    </car>
    <car name='Triumph'>
        <wheel1>some info</wheel1>
        <wheel2>more info</wheel2>
        <wheel3>blur</wheel3>
    </car>
</cars>
</Main>";

XDocument loaded = XDocument.Parse( xml );

IEnumerable<XElement> list1
   = loaded.XPathSelectElements( "//cars/car[not(wheel4)]" );

foreach ( XElement el in list1 )
    Console.WriteLine( el );

/* Outputs
<car name="Twingo">
  <wheel1>abc</wheel1>
  <wheel2>def</wheel2>
  <wheel3>ghi</wheel3>
</car>
<car name="Triumph">
  <wheel1>some info</wheel1>
  <wheel2>more info</wheel2>
  <wheel3>blur</wheel3>
</car>*/
Share

Tags: , , ,

Linq Orderby a Better IComparer in C#

Sometimes IComparer falls short when on has a need to sort on different, for lack of a better term, data columns. Before writing an IComparer interface for sort, try using Linq’s Orderby.

In the forums the user had data, in string lines, which looked like this

3 months ending 9/30/2007
9 months ending 9/30/2007
3 months ending 9/30/2008
9 months ending 9/30/2008

The user needed the white items sorted first in ascending fashion and the red year items sorted descending. Because the data was all in a string and needed differing sorts, je was having problems with sort with a custom IComparer class.

I recommend that he use regex to parse out the items then use linq to sort. Here is the result.  Note I merged all data into one string where each line is a true line.

string input =
@"3 months ending 9/30/2007
9 months ending 9/30/2007
3 months ending 9/30/2008
9 months ending 9/30/2008";

string pattern = @"(?<Total>\d\d?)(?:[^\d]+)(?<Date>[\d/]+)";

var items =
    from Match m in Regex.Matches( input, pattern )
    select new
    {
        Total = m.Groups["Total"].Value,
        Date = DateTime.Parse( m.Groups["Date"].Value ),
        Full = m.Groups[0].Value
    };

var values = from p in items
             orderby p.Total, p.Date.Year descending
             select p;

foreach ( var itm in values )
    Console.WriteLine( itm.Full );

/* Outputs
3 months ending 9/30/2008
3 months ending 9/30/2007
9 months ending 9/30/2008
9 months ending 9/30/2007
             */
Share

Tags: , , , ,

Regex To Linq to Dictionary in C#

This article demonstrates these concepts:

  1. Regex extraction of Key Value pairs and placing them into named capture groups.
  2. Linq extraction of the Key Value pairs extracted from the matches of Regex.
  3. Dictionary creation from Linq using the ToDictionary method.

I answered this on the MSDN forums, the user had this data in key value pairs delimited by the pipe:

abc:1|bbbb:2|xyz:45|p:120

Keys values separators

The need was to get the keys and values into a dictionary. The following code uses named regex group matches which are used in Linq to extract the keys and their values. Once that is done within the linq the extended method ToDictionary is used to create the dictionary on the fly. Here is the code:

string input = "abc:1|bbbb:2|xyz:45|p:120";
string pattern = @"(?<Key>[^:]+)(?:\:)(?<Value>[^|]+)(?:\|?)";

Dictionary<string, string> KVPs
    = ( from Match m in Regex.Matches( input, pattern )
      select new
      {
          key = m.Groups["Key"].Value,
          value = m.Groups["Value"].Value
       }
       ).ToDictionary( p => p.key, p => p.value );

foreach ( KeyValuePair<string, string> kvp in KVPs )
    Console.WriteLine( "{0,6} : {1,3}", kvp.Key, kvp.Value );

/* Outputs:
 abc :   1
bbbb :   2
 xyz :  45
  p  : 120
 */
Share

Tags: , , ,

C# Dictionary<T> Tricks

>(Post Updated 4/2/2011 : Added extra example)

This article demonstrates the Tribal Knowledge of using a Dictionary to do the things that the Swiss Army Knife Hashtable used to do.

For .Net 3 through .Net 4.

I love the generic Dictionary collection so much, I think that Microsoft might charge me usage on it. Here I present some of the things that I have learned which I call the tribal knowledge of the class.

The first use of the Dictionary was found in .Net 1 HashTable class. That class is a dictionary for storing lists with keys and has the ubiquitously Key Value Pair. With .Net 2 the generic class Dictionary was used, less upfront functionality, but same concepts; data in key value pairs.

While the old hash table could be sorted the dictionary cannot. So how does one sort? The Dictionary is just two lists under the covers. One is for keys and one is for values. Operations don’t go to the hash table but to the lists. Once you remember that and understand it, the obtuseness of the Dictionary goes away…

Sorting

Ok, so we can sort…but one complaint or question found in the MSDN forums that I have run into is that, the Hashtable could be sorted because it exposed IComparer and its cousin the Dictionary does not! How does one get around that?

As mentioned one must think of what a dictionary really is…just two lists. There are two properties exposed on the Dictionary and those are the Keys and Values. Those are shown to the world as ValueCollection and that my friends is the entry pass into the club. One of the interfaces that is exposed is the ValueCollection and it adheres to the ICollection. Bingo! With that we can do sorting! Here is the code snippet:

Dictionary<int, string> myDict = new Dictionary<int, string>()
            {
                { 2, "This" },
                { 1, "is" },
                { 5, "radio" },
                { 12, "clash" },
            };

List<string> song = new List<string>( myDict.Values );

song.Sort();

// This writes out: "clash is radio This"
Console.WriteLine( string.Join( " ", song.ToArray() ) );

We simply use the sorting which is found on the List<T>.

Enumerating over a Dictionary

Sometimes the need is there to run through a dictionary and show the keys and values. When enumerating over a Dictionary it returns a Key Value Pair object. That object holds the key and the value.

Dictionary<string, string> ColumnValuesHash = new Dictionary<string, string>()
// ... load with values...
foreach (KeyValuePair entry in ColumnValuesHash)
    Console.WriteLine("{0}: {1}", entry.Key, entry.Value);

Dictionary of Dictionaries

I have used this gem many times. Create a top level dictionary that will hold other dictionaries…its not as bad as it sounds.

Dictionary<int, string> myStringHash = new Dictionary<int, string>();

myStringHash.Add(41,      "Jabberwocky");
myStringHash.Add(8675309, "Jenny");

Dictionary< string, Dictionary<int, string>> myDicts = new Dictionary<string, Dictionary<int, string>>();

myDicts.Add("Test", myStringHash);

Console.WriteLine( myDicts["Test"][8675309] ); // Prints Jenny

Notice how easy it is to drill down via indexes off of each dictionary.

Dictionary Meets Linq and Gets Sorted

Here is how you can access the dictionary, sort it and enumerate it all in Linq. We will take our above example and do it

Dictionary<int, string> myDict = new Dictionary<int, string>()
                     {
                         { 2, "This" },
                         { 1, "is" },
                         { 5, "radio" },
                         { 12, "clash" },
                     };

var sorted = from item in myDict
             orderby item.Value ascending
             select item.Value;

// This writes out: "clash is radio This"
Console.WriteLine( string.Join( " ", sorted.ToArray() ) );

Extra Example (Loading on the fly and Linq extension Select Many)

Dictionary<string, Dictionary<int, string>> TopLevel = new Dictionary<string, Dictionary<int, string>>()
{
    {
        "Dictionary 1",
        new Dictionary<int, string>()
                                    { // Sub "dictionary 1" contains two objects
                                        {41, "The Meaning of Life" },
                                        {90125, "Owner of a Lonely Heart"}
                                    }
    },

    {

        "Dictionary 2",
        new Dictionary<int, string>()
                                    { // Sub "Dictionary 2" contains one object
                                        {8675309, "Jenny!" }
                                    }

    }

};

Console.WriteLine(TopLevel["Dictionary 1"][41]);      // Prints out "The Meaning of Life!"

//     Console.WriteLine(TopLevel["Dictionary 1"][8675309]); // Fails and would throw an exception! That is in Dictionary 2!

// Use selectmany to grab all sub objects
// (which are a Key value pair of KVP<string, Dictionary<int, string>)
// and return the value (the sub dictionaries) and combine
// all the those dictionaries from both sub dictionary into one IEnumerable KVP.
TopLevel.SelectMany( dict => dict.Value ) // Gets all the DIctionary<int, string> pairs between both dictionaries
        .ToList() // Convert from IEnumerable to list so we can do the for each below
        .ForEach( kvp => Console.WriteLine( "{0} is {1}", kvp.Key, kvp.Value ) ); // Outputs the lowest dictionaries.

/* above outputs
41 is The Meaning of Life
90125 is Owner of a Lonely Heart
8675309 is Jenny!
*/

Hope this helps!

Share

Tags: , ,

Asp.Net ListView: Linq Allows Creation of Dynamic Hyperlinks Example

(Update 03.28.2012 Cleaned up article and verified it  works with Asp.net versions 3.5 and 4.0)

The following ASP.Net article details how to create a ListView, bind it to a Linq query and create a dynamic list of hyperlinks to show on an Asp.net web page in C#. To recreate this, use Visual Studio (VS2008/VS2010/VS 11) and create a default Asp.Net Web Application project all working in the default page setup.

On the screen there will be a button which will cause a refresh to the page when pressed. On that refresh the ListView will be cleared of all data and the button disabled to show that the control has not been rebound to actual data.

Here is what it will look like on its first run with a button and dynamically created html links in a list.

Example showing button and dynamic links

HTML: which has the button and the ListView. We will hand create the LayoutTemplate and the ItemTemplate. Our goal is to have a list of links and dynamically create them. The ItemTemplate is expecting to be bound to Url field and UrlText which Linq will nicely create for us in the code behind.

<asp:Button ID="btnClear" runat="server" Text="Clear" /> 

<br /> 
<asp:ListView ID="lvPanelGroup1" runat="server">     
    <LayoutTemplate>         
        <div>             
            <ul>                 
                <asp:PlaceHolder runat="server" ID="itemPlaceholder" />             
            </ul>         
        </div>     
    </LayoutTemplate>     
    <ItemTemplate>         
        <li>             
            <asp:HyperLink ID="HyperLink1"                            
                           NavigateUrl='<%# Eval("Url") %>'                            
                           runat="server">
                <%# Eval("UrlText") %>
            </asp:HyperLink>         
        </li>     
    </ItemTemplate> 
</asp:ListView>

C#:The following is the code in the PageLoad. Its job is to either bind the data in the ListView using Linq or clear the ListView of all items and disable the control. Notice the Linq code where we create our object to have a Url field and UrlTextField which the ListView will use.

protected void Page_Load(object sender, EventArgs e) 
{ 
    if (IsPostBack == false) 
    {     
        List<string> Titles = new List<string>()        
            { "Title 1", "Title 2", "Title 3", "Fini"};     

        int index = 0;     

        var links = from item in Titles                 
                    select new                 
                            {                     
                                Url = "~Default.aspx?articleid=" + (++index).ToString(),                     
                                UrlText = item                 
                            };     
        lvPanelGroup1.DataSource = links;     
        lvPanelGroup1.DataBind();     
        btnClear.Enabled = true; 
    } 
    else // Clear the ListView 
    {     
        lvPanelGroup1.Items.Clear();     
        lvPanelGroup1.DataSource = null;     
        lvPanelGroup1.DataBind(); 

        // The rebinding will clear the ListView.     
        btnClear.Enabled = false; 
    } 
}

Here is what happens after a button press:

What happens after Clear is pressed.

Hope this helps those learning about the ListView and Linq.

Share

Tags: , , ,

Linq: Flatten Tercerary Table Data Problem

Update: Problem has been solved! See answer section at the bottom!

TablesI have run into a problem which I am unable solve using Linq to Sql. I want flattened data from a database which holds knowledge base type information to get as flattened data. My need is to pass it on to a databound list control to display to the user. Shown to the right are the tables which are needed basically an article and associated tags.

The Article table holds the textual articles a summary and each article has an ID which is its primary key. The textual tag(s) which may describe the articles are saved on a one by one basis in the KB_Tag table and like the article table, it has a primary key ID named Tag_ID. Linking the articles to the actual tag(s) is done by the KB_ArticleTag table which have the foreign key relationships back to the Article table and the KB_Tag table.

Download SQL 2005 database backup here.

Linq Tried

My goal is to extract each article summary and its corresponding tags in data form such as -Summary, “tag1 tag2…”-

“Data by Numbers”| “Data”
“Linq overview”    | “Linq Data C#”

But what I get with my Linq query against my databaselooks like this (shown here using the output of LinqPad):


Linq result showing two columns Summary and tags where the tags are IEnumerable of string and not flattened

Here is the query used:

from ar in dc.Articles
   join atgs in dc.KB_ArticleTags on ar.Article_ID equals atgs.Article_ID into iTags
   select new
   {
       ar.Summary,

       tags = from cpt in iTags
              join tgs in dc.KB_Tags on cpt.Tag_ID equals tgs.Tag_ID
               select tgs.Text

   };

Other things Tried

  • Posted to MSDN forums (Flatten FK Tercerary Table)
  • Response from Lawrence Parker suggested using the Aggregate extension:
    from ar in dc.Articles
    join atgs in dc.KB_ArticleTags on ar.Article_ID equals atgs.Article_ID into iTags
    select new
    {
     ar.Summary,
    
     tags = (from cpt in iTags
      join tgs in dc.KB_Tags on cpt.Tag_ID equals tgs.Tag_ID
      select tgs.Text).Aggregate<String, String>
       ( String.Empty,
       ( String prev, String input ) => prev + (prev == String.Empty ? input : ", " + input) )
     }

But unfortunately the Aggragate generates a runtime error “The query operator ‘Aggregate’ is not supported.” by Linq parser.

Answer

Now Lawrence had the right idea with the Aggregate, but the problem lies in the fact that Linq-To-Sql cannot handle that operation on the backend due to the structure being in an IQueryable. The solution is to bring the data to the front end by using the IEnumerable data source. By bring it away from the backend, the data contained can be operated on locally on the client.

from ar in dc.Articles.AsEnumerable()
join atgs in dc.KB_ArticleTags on ar.Article_ID equals atgs.Article_ID into iTags
select new
{
 ar.Summary,

 tags = (from cpt in iTags
   join tgs in dc.KB_Tags on cpt.Tag_ID equals tgs.Tag_ID
   select tgs.Text)
    .Aggregate<String, String>
     ( String.Empty, ( String prev, String input )
      => prev + (prev == String.Empty ? input : ", " + input) )
}
Share

Tags: , ,

C# Regex MatchCollection Meets Linq

Here is a code snippet which accomplishes these following goals:

  • It marries a C# Regular Expression MatchCollection to a property list using Linq.
  • It uses a Regex Pattern which creates named capture groups which Linq can easily exploit in the join of two data lists.

Let me show you the code. Don’t get hung up on the pattern or what it is doing. What needs to be known is that the pattern places the data matched into Named Capture Groups of Key and Value. The actual Key value corresponds to a property on a real class. Using reflection we will find that property on the class and link its property name to the value stored. That will allow us to change that properties value on the class from the Value we get from the regex match.

The goal of the Linq code is to join into another list, which is the list of properties from the class and the commonality is the PropertyInfo.Name found in that list. Once that data is joined a new object will be created which will have the actual property object and the value of Value. That new list will allow the following operations to set target properties value to Value of the match in Match collection.

public static T ASCIISerializeOut<T>( string targetSerialized )
     where T : new()
{

     T targetInstance = new T();

     string pattern = string.Format( @"(?<Key>[^{0}]*)(?:{0})(?<Value>[^{1}]*)(?:{1}?)",
           Seperators.cnKVPSeperator,   // "±"
           Seperators.cnSeperator );    // "¶"

     MatchCollection mcKVPs = Regex.Matches( targetSerialized,
                                             pattern,
                                             RegexOptions.Compiled );

     var kvps = from Match m in mcKVPs
                where mcKVPs != null
                where mcKVPs.Count > 0
                join prp in GetPublicProperties<T>() on m.Groups["Key"].Value equals prp.Name
                select new
                {
                    prop  = prp,
                    Value = m.Groups["Value"].Value ?? string.Empty
                };

     foreach (var item in kvps)
         item.prop.SetValue( targetInstance, item.Value, null );

     return targetInstance;

 }

 /// <summary>
 /// Return all public properties which are of string type from T class.
 /// </summary>
 public static IEnumerable<PropertyInfo> GetPublicProperties<T>()
 {
     return from p in typeof( T ).GetProperties()
            where p.PropertyType == typeof( string )
            select p;
 }
  • Line 01: The function takes in text such as "AProp±AValue¶BProp±BValue" which needs to be serialized into a newly created class of type T. The first item in the pattern is the property name AProp followed by a seperator ± then the value of the property AValue and finally a key value seperator: ¶. our regex will create individual matches for each of the key value pairs.
  • Line 07: This pattern when used will get key and value pair combinations and place them in named groups of Key and Value of the match.
  • Line 11: Get all the key/value pair combinations into the match collection.
  • Line 15: Linq starts here: We define a Var object kvps (key value pairs) which will use /loop each match from the match collection.
  • Line 16: Make sure the collection is not null.
  • Line 17: Make sure there are one or more matches.
  • Line 18: Get all the public properties of class T and make a join to our collection data. Key should match the property Name.
  • Line 19: Each match found within the property where the names are the same will create this new object below with two properties.
  • Line 21: Save the actual property object, we need that later to load data.
  • Line 22: Get the value out of the Value group and save that as well. Note, if it is null, just use string.Empty. Thanks Null Coalescing operation (??).
  • Line 25: Now for each var object created enumerate through it and load the target values into our newly minted class object of T.
  • Line 26: Set the target item’s property to the value found from the regex matches.
  • Line 28: Return the new object with the original text data serialized in.
  • Line 35: Return an enumeration of all generic string properties of the type T.

For completeness see my post entitled A C# ASCII Serializer Generic Method for Class Objects which has the actual downloaded project and working test example. (Post coming soon!)

Share

Tags: , ,

Linq in C# Learn to Return IEnumerable and not List<>

(Updated 3/27/2011 and works with .Net 3-4)

When I wrote this article I was fixated on returning IEnumerable instead of of List<>. Being a wiser developer my advice now would be to return IList<> instead of IEnumerable in most situations after using the extension ToList() to not have any differed execution due to IQueryable. One should return IList<> because List<> is malleable and can be changed by the consumer in some situations which may have undesired effects elsewhere. My intent now with this article is just to understand how IQueryable uses IEnumerable, such as does List<> and really when to use IQueryable and how it may effect the code.. To that end I have reworked this article.

IQueryable and IEnumerable and List<>

In .Net 2.0 with the release of generic collections one would return List<> and was not too concerned with IEnumerable which List<> derives from. This changed with .Net 3 and the iQueryable returns.

With Entity Framework and Linq to Object/SQL the ability to build queries (differed execution) was presented to the user. A querable item means that the operation in question is held off until the data values are actually needed. The IEnumerable which List<> and IQueryable inherits from simply provides the common base for operations for items to be enumerated (listed).

Stating again, when one uses or returns IQueryable, the operation of getting the list is not done until the data is actually needed; it is differed.  (One can see that in the Visual Studio debugger when viewing the querable list in question, one has to click on it to enumerate it in the debugger before values are shown! When data is not IQuerable the debugger can show it immediately.)

Whereas wiht List and other IEnumerable derived lists,  such as IList<>, the process is done immediately (unless the parent object is IQueryable then one needs to covert it to a hard List<>). Differed operations are good when building a larger set of data outside the method returing IQueryable. But it is not always needed.

Since the execution is differed changes in the list will be reflected at the time of acquisition. Meaning that if you know the list may change and the user should not see those changes, send back a list and not a Queryable. That is done by calling ToList with any Linq operation.

Original Article

When you find a pattern in programming that is a sign that you need to place it into a function for reuse. With .Net 3.5 there is value in returning IEnumerable of items where before one would return a list.

// Take an object and print out the property name and its value
// for all public properties. public static
void DisplayPublicPropertiesAndValues<T>( T target )
{
   foreach (PropertyInfo prp in GetPublic<T>())
      Console.WriteLine( prp.Name + " " + prp.GetValue( target, null ) );
} 

// I have to put things into a list to enumerate over it in the calling method.
public static List<PropertyInfo> GetPublic<T>()
{
   List<PropertyInfo> retList = new List<PropertyInfo>();     

   foreach (PropertyInfo prp in typeof( T ).GetProperties())
      if (prp.PropertyType == typeof( string ))
         retList.Add( prp );     

   return retList;
}

For example in 2.0 I would have to return a List<> of objects to accommodate my foreach needs such as found in GetPublic when I was reflecting off of a objects instance.

Now we return IEnumerable<> thanks to Linq and let it do the work for you. The above GetPublic in .Net 3.5 looks like this:

public static IEnumerable<PropertyInfo> GetPublicV2<T>()
{
    return (from p in typeof( T ).GetProperties()
           where p.PropertyType == typeof( string )
           select p) // Returns IQueryable
           .ToList();  // Otherwise we would be returning IQueryable so change it to a List to get the data.
}

// Same as above but using the extension methods
public static IEnumerable<PropertyInfo> GetPublicV3<T>()
{
    return typeof( T ).GetProperties()
                      .Where( p => p.PropertyType == typeof( string ))
                      .ToList(); // Otherwise we would be returning IQueryable

}

// As I would actually do it now in 2011
public static IList<PropertyInfo> GetPublicV4<T>()
{
    return typeof( T ).GetProperties()
                      .Where( p => p.PropertyType == typeof( string ) )
                      .ToList(); // Otherwise we would be returning IQueryable

}

A lot nicer. We achieved the same functionality but we don’t have to do the dirty work of creating a List! So advice…get used to working with IEnumerable<>.

Share

Tags: