Dynamic and ExpandoObject

Intro

Ahh, dynamic types. For those who love anarchy, disorder, and a total lack of type-safe compile-time code wrangling, dynamic types are quite a boon. There are many languages out there that facilitate such things, JavaScript being one of the largest among them. And while I consider myself to be firmly in the camp of "type-safety is good!", I do recognize that there are a few valid uses for dynamic types in my cozy little type-safe bubble that is c#.

As you can guess from the title of this post and the preceding paragraph, this post is going to be about dynamic and ExpandoObject. I won't get into the nitty gritty details of either, as I prefer the 30,000 ft view here. We'll cover how to use them and what you might do with them, but without opening up the hood too much. If you wish to dig further you can read the links towards the bottom of the blog.

When to Use

Have you ever had to interact with type-unsafe services and languages? For example, have you ever integrated a c# app with Facebook, Twitter, or any other such 3rd party service that returns a JSON object? Or, have you perhaps written a JSON Web API yourself? Being the client of such a service will almost invariably involve either you creating proxy classes that represent the data returned so that you can parse the JSON into your friendly object structure, or hey, we will soon see how to do such things with dynamic objects!

I'm sure there are other uses too, but this is the main one I've run into and I've run into it multiple times so I assume it must be fairly common. After all, a sample size of 1 among the millions of programmers in the wold has to be statistically significant.

How to Use

Our first sample will be a simple one; we're just going to create a dynamic object and add some properties and a method to it, without declaring the object type ahead of time. Let's jump right in:

        
        public string DoStuff()
        {
            dynamic myDynamicVariable = new ExpandoObject();
            myDynamicVariable.Name = "Fred";
            myDynamicVariable.Age = 34;
            myDynamicVariable.ToString = new Func<string>(() => { return myDynamicVariable.Name + "::" + myDynamicVariable.Age; });
            return myDynamicVariable.ToString();
        }

In the above code, we first declare a variable named myDynamicVariable of type dynamic, and initialize it to a new instance of ExpandoObject. The combination of dynamic and ExpandoObject is the magic sauce that makes the code below it function properly. Notice how we never declared or used some special class that has a Name or Age property, or a ToString method? Yet, through the awesomeness of dynamics the value we want is returned by the method DoStuff.The return statement returns the result of a call to the ToString method of myDynamicVariable, and in this case returns the string "Fred::34". When I first saw code similar to this, I was amazed it even compiled much less ran! All we had to do was declare our type and start putting new properties and values into our class, and we can even declare methods inline. Groovy! What would happen if you tried the same thing with an object of type "Object"? If you guessed compiler error, treat yourself to a hearty pat on the back, because you're right! Dynamic types give you the power to modify object properties with a clean syntax, without declaring the type ahead of time.

For our second sample we'll be using the JSON.Net NuGet package to import a JSON string into a dynamic object. In my opinion, this is the best type of usage for dynamic objects. Let's look at the sample code before I get ahead of myself here:

        public KeyValuePair<string, int> DoJsonStuff()
        {
            string json = "{ 'SomeStringProp': 'Hey, a string!', 'SomeIntProp': 42  }";
            dynamic deserializedObject = JsonConvert.DeserializeObject<dynamic>(json);
            return new KeyValuePair<string, int>((string)deserializedObject.SomeStringProp, (int)deserializedObject.SomeIntProp);
        }

The first line sets the JSON value we are going to parse. The 2nd line uses JSON.Net to deserialize the JSON string into a dynamic object using the call JSONConvert.DeserializeObject. The 3rd line returns our KeyValuePair, which as you have guessed contains the values "Hey, a string!" and 42.

Now imagine you have just called the Twitter API to retrieve a user's feed. Twitter responds with a JSON list of object that is fairly large in many cases, and contains lots and lots of properties and sub-object. Do you want to go map what comes back to statically typed classes? Well I do actually, but hey it's not everyone's cup of queso and as you can see, dynamics aren't hard to work with in .Net.

What's Next?

• Read up more on the dynamic keyword
• Read up more on ExpandoObject
•  Learn how to create anonymous methods using the Func type (and others!)
• Go hog-wild and read up on JSON.Net

Extension Methods

Intro

What is the airspeed velocity of an unladen swallow? African AND European.

Oh wait, wrong question. What I meant was, what are extension methods? Extension methods allow you, the humble developer, to add methods to existing types. You can do this without inheritance and without modifying the source class.

Example

I think the best way to illustrate the concept is with an example. We will extend the int class for the sake of dice rolling. Specifically, I want to be able to roll 2 x-sided die at the same time. I know this seems like a waste of effort to use an extension method, but too bad; it's my blog! So, we'll extend Integer to have a DoubleDieRoll() method which returns a Tuple<int, int> that is 2 rolls of a die with x sides, where x is the source integer. So for example, if I were to call 6.DoubleDieRoll(); I would expect to receive a 2-part result tuple that contains 2 random numbers between 1 and 6. Believe it or not, this is quite easy to do in .net. Fire up a .net project of your favorite type (winforms, xaml, webforms, mvc, whatever) and add a class called GameExtensions. In the sample code below I've created an mvc project ccalled BlogExtensionMethods. Create yourself a new class named "GameExtensions" (note that this name is purposely meaningless, to illustrate that the name of the class doesn't matter). Here's the code for my class:

using System;

namespace BlogExtensionMethods
{
    public static class GameExtensions
    {
        public static Tuple<int, int> DoubleDieRoll(this int val)
        {
            Random rnd = new Random();
            var roll1 = rnd.Next(1, val);
            var roll2 = rnd.Next(1, val);
            return new Tuple<int, int>(roll1, roll2);
        }
    }
}

As you can see, we have a single static class name GameExtensions. Within this class we have a single static method named DoubleDieRoll. The key part of the method is the parameter. the parameter "this int val" means that our method DoubleDieRoll will work on the int class. The body of the method is nothing special, but it does illustrate returning a tuple of int, int. So, key points:

1. static class
2. static method
3. single parameter with the keyword this, the data type we're extending, and a name for the parameter (name doesn't matter).

Now let's see how to call the spiffy new method:

        public string Roll2Die()
        {
            var dieRolls = 20.DoubleDieRoll();
            return String.Format("die roll1: {0}, die roll2: {1}", dieRolls.Item1, dieRolls.Item2);
        }

Cool huh? You can call this on any old int value. In the above code we're calling 20.DoubleDieRoll(). It's pretty fun, and you even get intellisense on your call within the IDE! You'll get back 2 random rolls of a 20-sided die with the above code.

Pitfalls

This stuff is pretty fun, but guess what? There are a lot of people that recommend you use extension methods very judiciously if at all. The reasoning is that it can get kind of confusing if in some projects you have methods on pre-canned classes that don't exist in other projects, such as the odd DoubleDieRoll we defined above for the int class. I must confess that I've never used extension methods, partially for this reason. However like any other tool, I'm sure it has it's uses and if you try hard enough you just might find one yourself.

Resources

Extension Methods, from the c# Programming Guide on MSDN

LINQ - LINQ to SQL

Intro w/Recap

In part 1, part 2, and part 3 we discussed LINQ basics, some more advanced techniques for dealing with objects in LINQ, and LINQ to XML. Here in our final post of the series we'll discuss LINQ to SQL, and briefly discuss Entity Framework in order to give some context to LINQ to SQL. What is LINQ to SQL? As you can guess, it is the usage of LINQ with data stored in a sql-compatible database such as MS SQL Server. For the purposes of our discussion we will be using SQL Express 2008 R2, though these samples will work fine with newer versions as well.

Entity Framework Basics

At this point hopefully you're wondering why I plan on discussing Entity Framework (EF). The reason is that LINQ to SQL works directly with Entity Framework objects. Entity Framework is MS's Object Relational Mapper (ORM) that helps save you time by linking your Plain-Old-CLR-Objects (POCO's) to your database. If you've ever had to link objects with database tables/fields you know it can be very tedious and time-consuming work, and ORM's help automate much of this tedium.

As part of today's LINQ lesson I'll walk you through how to get a database all setup and ready to map with EF, then I'll show you how to have EF automatically create your tables/fields based on object(s) that you've created in code.

Let's start with setting up the database. If you don't already have it, go download sql server express edition. You can find it here. Install that little sucker (get the one called "SQL Server Express With Tools") and then return. If you need help with installation, please post a reply here in the blog so that everyone can benefit from the shared knowledge. After you have completed installation, you're done! Yeah EF is so cool that it creates your database, tables, and columns for you. Can't get much easier than that. But hey we haven't done that yet so keep reading.

Code-First DB Setup w/EF

That's it for the database for now. We're going to use something called code-first setup of our EF, meaning we'll write our storage classes first and we'll use some spiffery to automatically create the tables for us.

I've created a couple classes myself for storage; Animal and Show. Here is the code for both:

using System.ComponentModel.DataAnnotations;
namespace BlogLinq
{
    public class Animal
    {
        [Key]
        public string name { get; set; }
        public string animalType { get; set; }
    }
}

using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;

namespace BlogLinq
{
    public class Show
    {
        [Key]
        public string name { get; set; }
        public virtual List<Animal> animals { get; set; }
    }
}

As you can see, we have an Animal with 2 properties (name and animalType), and a Show with 2 properties (name and animals, which is a list of type Animal). We use the Key attribute to denote which field is the primary key for our class/table, so that EF can avoid creating a heap table. Trust me (or ask Bobby, my resident DBA, they're bad). The only other oddity here is that I made the animals property of the Show class virtual. This is necessary for EF to load such a list at runtime from a SQL data source.

Next you need to install the EntityFramework package from NuGet. If you don't, your code won't compile as that namespace and the Key attribute I used above are from EF. I'm assuming you know how to work with NuGet package manager, but if that's not the case feel free to ask in the comments below. Now you need to create a "Context" class, which is just a fancy way of saying that you need something that ties LINQ to your database table(s) and classes. Create a class called ZooContext. Here's the code of my ZooContext:

using System.Data.Entity;

namespace BlogLinq
{
    public class ZooContext : DbContext
    {
        public DbSet<Show> Shows { get; set; }
        public DbSet<Animal> Animals { get; set; }
    }
}

Because this blog post isn't about EntityFramework (let me know if you want to see such a beast!) I won't go into any further detail on this ZooContext class, so we'll just say for now that this class as defined above will let us use LINQ to query the database rather than having to roll our own SQL queries.

Inserting Data With EF

What fun is querying data when there's no data? none at all! We're going to write a quick method here that will insert data into our database and tables for us. "But Pete", you say, "We don't have a database, or tables, or columns!". Yeah I know that, and so does EF. Trust me, it will create this crap for you. Write a function like this:

        protected void btnEFInsert_Click(object sender, EventArgs e)
        {
            var zooContext = new ZooContext();
            var show = new Show() { name = "Early", animals = new List<Animal>() };
            show.animals.Add(new Animal() { animalType = "Bird", name = "George" });
            var show2 = new Show() { name = "Late", animals = new List<Animal>() };
            show2.animals.Add(new Animal() { animalType = "Ferret", name = "Fred" });
            show2.animals.Add(new Animal() { animalType = "Bear", name = "Bear" });
            zooContext.Shows.Add(show);
            zooContext.Shows.Add(show2);
            zooContext.SaveChanges();
        }

In the first line of our function we create an instance of our ZooContext class that links EF to our POCO's. We then proceed to create some shows, add animals to them, and add our shows (which contain the animals) to the zooContext. Then we call zooContext.SaveChanges(), and voila! EF created a database for us, a few tables, setup our columns, and even created primary and foreign keys for us. That's so darn cool I could pinch myself. Here's a screenshot full of awesomesauce:

Querying Data With LINQ to SQL

Back to LINQ...now that we have a little bit of data in our database our foray into pure EF is over, so let's query that data using LINQ. This should look pretty familiar by now:

        protected void btnLinqSqlSelect_Click(object sender, EventArgs e)
        {
            var zooContext = new ZooContext();
            var query = from show in zooContext.Shows
                        where show.name.Equals("Late", StringComparison.OrdinalIgnoreCase)
                        select show;
            foreach (var show in query)
            {
                Response.Write("show: " + show.name);
                foreach (var animal in show.animals)
                    Response.Write(String.Format("<br / >    {0} the {1} is in the {2} show!", animal.name, animal.animalType, show.name));
            }
        }

We start by creating a ZooContext object in order to pull junk from the database, then it's all just standard-looking LINQ from there. In the above LINQ query we retrieve all the shows (there's just 1) named "Late", and EF/LINQ goes and pulls all the shows and their child animals from the database for us. Simple and powerful stuff here!

Here's the output in case you don't trust me about the code working:
show: Late
    Bear the Bear is in the Late show!
    Fred the Ferret is in the Late show!

Drawbacks

With great power comes great headaches. The cool time-saving and readability afforded by LINQ comes with a price. Ask your local DBA what they think about ORM's and you'll get a good first-person ear-blasting about the evils of auto-generated queries. Fine-tuning a SQL database really is a profession unto itself, which is part of why the DBA was invented. EF and LINQ generate some decent SQL, but in many cases it's not optimized. This isn't much of a concern with small apps, but if you expect to scale, dig further into LINQ/EF and see what all options you can find for optimization.

What's Next?

• You could spend some time cozying up to Entity Framework. Learn things like changing the name of the database your data gets plopped in, view the sql queries it generates (it's still using sql behind the scenes), how to add new fields to your classes and have them added to the table(s), etc.

Resources

My Code (Created Using VS Express 2013)!
Entity Framework
SQL Express

LINQ - LINQ to XML

Intro

This week we'll build on the foundation we learned in part 1 and part 2, adding LINQ to XML to our repertoire. As you have probably surmised, LINQ to XML technology lets you use the LINQ query syntax, with which you've now familiarized yourself, to query data from an in-memory representation of XML data. To an extent, this frees you from learning even more technology for XML parsing such as XPath, so you can have a common framework of knowledge (LINQ) to query many different types of things (objects, XML, and next week SQL data). Perhaps I should have stated and emphasized this better up-front in part 1 to drum up more interest in LINQ, but learning LINQ is almost like a 3-for-1 deal. You learn LINQ, and you get to query 3 types of data. It saves code and saves time, and what else can you ask for in the world of coding? No it won't write your code for you. I know you were gonna ask :)

Loading XML

Step 1 in our journey today is loading an XML string from memory. There are other ways to get XML into a LINQ to XML object such as loading from disk, creating the xml object tree ourselves, etc, but this is the simplest. Note that you'll need to add System.Xml.Linq to your using clause. Here is how to load XML from a string:

private XElement ZooXml
        {
            get 
            {
                return XElement.Parse("<zoo><show name='Early'><animal name='Simba' animalType='lion'></animal><animal name='Bill' animalType='baboon'></animal></show><show name='Lunch'><animal name='Bjork' animalType='baboon'>Bjork says bye!</animal></show><show name='Evening'><animal name='Pete' animalType='panda'></animal></show></zoo>");
            }
        }

The only thing noteworthy here is XElement.Parse. XElement is the class you'll work with most frequently in LINQ to XML as it represents an XML element. The parse method parses the string that you pass in and returns a new object of type XElement. Pretty simple stuff so far.

Querying Basic Data

I know you all paid the price of admission to see the real show, so let's start querying the data. What I want is to pull a list of all the shows and display the names of all the shows. Here's our first example:

        protected void btnLinqXmlMultipleElements_Click(object sender, EventArgs e)
        {
            XElement zooElement = ZooXml;
            var shows = from show in zooElement.Elements("show")
                        select show;
            foreach (var show in shows)
                Response.Write("< br />" + show.Attribute("name").Value);
        }

and here's the output:
Early
Lunch
Evening

Pretty simple code huh? The .Elements() method will return all matching elements of the specified element/node (in this case zooElement, which is the root element). We specify that we only want elements named show in this case, so we only get back show elements. Then we loop through the result set and display the name of each show.

Elements Containing Specific Sub-Elements

What if we wanted only shows that have a baboon in them?

        protected void btnLinqXmlShowsWithBaboons_Click(object sender, EventArgs e)
        {
            XElement zooElement = ZooXml;
            var shows = from show in zooElement.Elements("show")
                        where show.Elements("animal").Any(animal => animal.Attribute("animalType").Value.Equals("baboon", StringComparison.OrdinalIgnoreCase))
                        select show;
            foreach (var show in shows)
                Response.Write("< br />" + show.Attribute("name").Value);
        }

And our output (hey, it's even correct!) is as follows:
Early
Lunch

As you can see above, the where clause is what's different. We tell the compiler that we want all show elements who contain an element named "animal" which have an attribute "animalType" with the value "baboon". There's nothing complicated here, with a little practice you can get the hang of it quite easily.

Getting Element Value(s)

Scroll back up near the top for a moment where we defined the property ZooXml. Notice how Bjork the baboon has a value within her xml element, with the value "Bjork says bye!"? Putting values into xml tags is common so let's see filter on and use them:

        protected void btnLinqXmlElementWithValue_Click(object sender, EventArgs e)
        {
            XElement zooElement = ZooXml;
            var elements = from element in zooElement.Descendants()
                            where !String.IsNullOrWhiteSpace(element.Value) && element.Name.ToString().Equals("animal", StringComparison.OrdinalIgnoreCase)
                            select element;
            foreach (var element in elements)
                Response.Write("< br />" + element.Name + "::" + element.Attribute("animalType").Value + "::" + element.Attribute("name").Value + "::" + element.Value);
        }

Output:
animal::baboon::Bjork::Bjork says bye! 

The first thing you'll notice different is that we used Descendants() this time instead of Elements(). What's the diff yo? Elements are just the direct children of where we're looking. For our prior queries that was good enough. For this query, we know that an animal is what has a value, and an animal is not a direct child of the root zoo element. Rather, animals are children of the show in which they perform, so we need to use the Descendants method().

The next difference is our use of element.Value. This isn't voodoo or anything, it's exactly what you think; the element value. We use it in our where clause to pull only those elements that have a value.

What's Next?

There are lots more things that you can do with LINQ to XML. We don't have time to cover all of them, but take a look on MSDN; there's lots to read.

• Other methods such as Ancestors(), ElementsAfterSelf(), etc.
• XNode: look it up! It's powerful. Finer level of detail than XElement, though you won't often need it's functionality.
• Manipulate and save XML.
• Create an XML tree entirely in code.
• Peek ahead to LINQ to SQL which will be next week's post.
• Experiment with last week's topics while playing with LINQ to XML. Joins, ordering, grouping, projections. This is by far the best thing you could do, as you'll put the pieces of the puzzle together and learn them as a whole.

Resources

LINQ to XML on MSDN