Wednesday, May 28, 2014

LINQ - Ordering, Grouping, Joining, and Projections

Intro


In part 1 of our series on LINQ we discussed the very basics of LINQ, pulling a few choice bits of data out of an array based on a simple filtering clause. This week we'll dive a little deeper into LINQ, exploring how to sort the results, group results, join 2 sources together to get a single output, and how to manipulate the structure of the output using projections. Hold onto yer hats (or other similar headgear such as beanies, earmuffs, etc)!

Ordering (orderby)

Everybody needs to sort data at some point in their coding career. To help illustrate my point with many of these LINQ operations, I'm going to compare using T-SQL against a fictional database in a zoo, versus using LINQ queries to pull the same data from C# objects. For starters, let's see a sample T-SQL query for pulling a list of baboons, sorted by aggressiveness:

select * 
from tblAnimals
where AnimalType = 'baboon'
order by Aggressiveness


Now let's see the same thing in LINQ using objects already loaded in memory:

        private dynamic CreateAnimal(string animalType, string name, int aggressiveness)
        {
            dynamic animal = new ExpandoObject();
            animal.AnimalType = animalType;
            animal.Name = name;
            animal.Aggressiveness = aggressiveness;
            return animal;
        }

        private List<dynamic> PopulateAnimalsList()
        {
            var animals = new List<dynamic>();
            animals.Add(CreateAnimal("lion", "Simba", 10));
            animals.Add(CreateAnimal("baboon", "Bobby", 4));
            animals.Add(CreateAnimal("baboon", "Bill", 1));
            animals.Add(CreateAnimal("baboon", "Bjork", 7));
            animals.Add(CreateAnimal("panda", "Pete", 2));
            return animals;
        }

        protected void btnLinqOrdering_Click(object sender, EventArgs e)
        {
            var animals = PopulateAnimalsList();
            var query = from animal in animals
                        where animal.AnimalType.Equals("baboon")
                        orderby animal.Aggressiveness
                        select animal;
            foreach (var val in query)
                Response.Write("<br />" + val.Name + " the " + val.AnimalType + " has aggro val of " + val.Aggressiveness);
        }


The first 2 methods are just setup, putting a list of animals into memory for us to manipulate. The LINQ-ified portion of our code is the 3rd and final method btnLinqOrdering_Click which creates the following output:
Bill the baboon has aggro val of 1
Bobby the baboon has aggro val of 4
Bjork the baboon has aggro val of 7

The orderby clause, just under the filtering where clause, is the sauce which produces this little bit of magic. By default, just like T-SQL, the sort order is ascending as you can see in the above output. If you wanted to order descending, you would change the orderby line to "orderby animal.Aggressiveness descending".
(Hey, wondering about dynamic and ExpandoObject? Keep paying attention to my blog, it'll probably make an appearance in a month or so).

Grouping (group)

Let's say you wanted to return a list of the types of animals and the count of each type. This will involve grouping. As with the last section, we'll start things with a sample bit of T-SQL:

select AnimalType, Count(1) as [NumAnimals]
from tblAnimals
group by AnimalType
order by Aggressiveness


Here were are selecting a list of types of animals and the number of each type of animal. How would we accomplish the same thing in LINQ? Here's that bit o' sample code:

        protected void btnLinqGrouping1_Click(object sender, EventArgs e)
        {
            var animals = PopulateAnimalsList();
            var query = from animal in animals
                        group animal by animal.AnimalType into animalGroup
                        select animalGroup;
            foreach (var val in query)
                Response.Write("<br /> there are " + val.Count() + " " + val.Key + "ses");
        }


This isn't the most grammatically correct output ever, but it's correct at least:
there are 1 lionses
there are 3 baboonses
there are 1 pandases

Notice how in our foreach loop where we display the output, we are accessing val.Count() and val.Key? This is because when you use the group clause in LINQ as denoted by group...by...into..., you actually create a list of lists. The outer list is a list of your groups which has a Key property that  in this case AnimalType property (because it's the "by" in the group clause).

While we do get a nice count by group from the above LINQ, we can't tell which animals belong to each group. What could we do in our code to show the contents of the groups/inner lists? the query is already selecting them, so in this case it's just a matter of changing our output. Here's the updated code:

            foreach (var val in query)
            {
                Response.Write("<br /> there are " + val.Count() + " " + val.Key + "ses");
                foreach (var animal in val)
                    Response.Write("<br />    " + animal.Name + " the " + animal.AnimalType + " has aggro val of " + animal.Aggressiveness);
            }


The output is pretty much what you'd expect:
there are 1 lionses
    Simba the lion has aggro val of 10
there are 3 baboonses
    Bobby the baboon has aggro val of 4
    Bill the baboon has aggro val of 1
    Bjork the baboon has aggro val of 7
there are 1 pandases
    Pete the panda has aggro val of 2

As you can see, I didn't lie to you earlier when I said the result of a group'd LINQ statement is a list of lists. The above code loops through the outer and inner lists, producing a combined output of summary and detail data. Spifferiferous!

Joining (join), as well as Projections

Joining lets you combine 2 source data sets into a single output data set. Let's pretend for this example that we want to see a list of which animals are in which shows. The shows are stored in a different table for T-SQL and in a different list for LINQ. Here's what the query might look like in T-SQL:

select tblShows.Name as [ShowName], tblAnimals.Name as [AnimalName]
from tblShows
join tblAnimals
on tblShows.AnimalName = tblAnimals.Name


This is pretty simplified as it assumes that the primary key of the Animal (unique property) is it's name, and it assumes only a single animal can be in any show, but hey you gotta lose realism for simplicity sometimes.

Now let's see what that might look like in LINQ:

        private dynamic CreateShow(string showName, string animalName)
        {
            dynamic show = new ExpandoObject();
            show.Name = showName;
            show.AnimalName = animalName;
            return show;
        }

        private ListPopulateShows()
        {
            var shows = new List<dynamic>();
            shows.Add(CreateShow("Early", "Simba"));
            shows.Add(CreateShow("Lunch", "Bjork"));
            shows.Add(CreateShow("Evening", "Pete"));
            return shows;
        }

        protected void btnLinqJoin_Click(object sender, EventArgs e)
        {
            var animals = PopulateAnimalsList();
            var shows = PopulateShows();
            var query = from animal in animals
                        join show in shows on animal.Name equals show.AnimalName
                        select new { ShowName = show.Name, AnimalName = animal.Name };
            foreach (var val in query)
                Response.Write("<br />the " + val.ShowName + " show has " + val.AnimalName);
        }



As in our first example, the first 2 methods are just setup. They create our lovely new list of shows. The 3rd and awesomerest method is our little bit o special LINQ. You can see we've joined the list "animals" to the list "shows", via the join clause. Notice the weird new "select new..." syntax? This is what's known as a projection. You can see the resource link at the bottom of this post for a full definition, but basically in this query our output is a list of a newly defined type of object that has 2 properties, ShowName and AnimalName. It's a pretty cool concept that you can just us a dynamic object like this for your LINQ output, it makes things very flexible. Other than that, we have what you'll recognize by now as some pretty standard output:
the Early show has Simba
the Lunch show has Bjork
the Evening show has Pete


What's Next?

I hope you enjoyed Part 2 in our series on LINQ. I learned a bit along the way myself. For the next post I'll show you guys how to use a different kind of data source, either SQL server or XML files. I haven't decided which yet. If you've read this far and have a preference, let me know in the comments and I'll take your thoughts into consideration when I make my final decision.

Resources

Basic LINQ Query Operations

Friday, May 16, 2014

LINQ - An Introduction

Intro

LINQ stands for Language Integrated Query. It was introduced with Visual Studio 2008, but it is probably still a new concept for many so I figured it just might be worth a blog on the topic. To me, LINQ is basically an extension of .Net that lets you use sql-like query syntax to retrieve some types of data. Let's say for example you have a list of customers and you want to retrieve all customers who have blue hair. If you were to do this in sql, it might look something like "select * from tblCustomers where HairColor = 'blue'". In C# you'd have to create a temp list, then write a loop where you find all blue-hairs and insert them into your temp list. It's just more work. LINQ however lets you treat some aspects of .Net as though you were using something akin to SQL.

On top of that, LINQ lets you pull data from many different types of sources. You can use LINQ to SQL to use LINQ for querying databases, you can use LINQ to XML to use link for querying XML files, and of course there's just plain old LINQ for querying lists of objects in .Net. By learning LINQ, you learn how to, in some fashion, query many different types of data so it saves you a little bit of code and a little bit of extra brainpower for learning other new things. Neat huh?

Constructing a LINQ Query

A LINQ query is comprised of 3 clauses: from, where, and select. If you're familiar with SQL syntax you'll notice that this isn't quite the same. In SQL, it's select, then from, then where. But hey, if this were the exact same order then life wouldn't be fun at all, and hippos would rule the seas. Or something. The from clause is your data source, the where clause is your filtering, and the select clause is what you're returning. Let's see a quick sample:

        protected void btnLinq1_Click(object sender, EventArgs e)
        {
            var blurbs = new string[] { "fred", "george", "fred rocks", "george doesn't" };
            var query = from blurb in blurbs
                        where blurb.StartsWith("fred")
                        select blurb;
            foreach (var val in query)
                Response.Write(" " + val);
        }


The above sample is a button click event in a webforms project. We first define our data source, which in this case is a simple array of strings named blurbs. We then construct our query object. Our query has a from clause that shows we are pulling data from blurbs, the where clause states that we only want to retrieve those blurbs who start with the word "fred", and the select clause states we are selecting individual blurbs from the list of blurbs. The last 2 lines loop through our query results (tip: the query isn't actually run until its results are used! see Introduction to LINQ Queries for more info) and write them out to the web page. As you can guess, we only retrieve and thus only display the values "fred" and "fred rocks" from the list of blurbs.

What's Next?

Either wait until next week's blog post when I delve a little deeper, or be a suck-up and read some of the resources below for more info.

Resources

LINQ
Introduction to LINQ
Introduction to LINQ Queries
Basic LINQ Query Operations

Thursday, May 8, 2014

A Chess Project, Part 13 (The Final Chapter)

Intro

I can see the light at the end of the tunnel! It's an alternating black and white checkered light, but still, a light nonetheless. How many of you thought it would take 13 blog posts to get here? Well I sure didn't, but I'm glad we're almost done. Here in week 13 we'll be modifying our Web API that we created months ago. We'll make it call into our spiffy chess AI dll in order to determine the best move from a given board position. After all, that's what we were trying to do from the very start!

 

 Code Changes

First, here is the absolute latest code as it stands right now, before today's modifications. It's not the most optimized, and I dare say the AI isn't really all that great, but hey the point of this blog was learning new technologies not to make the perfect chess AI.

Let's get to work. Open up BlogChessController from the BlogChessApi project. That method named PostBestMove is the one we want to do our modifications in. Here's what it looks like currently:

        public HttpResponseMessage PostBestMove(ChessGame game)
        {
            try
            {
                //validate the game
                var validator = new ChessGameValidator(game);
                if (!validator.Validate())
                    return Request.CreateResponse(HttpStatusCode.BadRequest, validator.ValidationIssues);

                //calculate the best move
                IChessValidMoveCalculator moveCalculator = new ChessValidMoveCalculator(game);
                var validMoves = moveCalculator.CalculateValidMoves();

                //return the best move wrapped in an http "ok" result
                return Request.CreateResponse(HttpStatusCode.OK, validMoves);
            }
            catch (Exception ex)
            {
                return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
            }
        }


You'll notice we don't yet have a call into the AI dll. At the time we originally made this method we had no such AI dll, so I can forgive us. Before we can do the code though, we need to reference BlogChess.AI.Basic1 from the Web API project. Go ahead and do that now (I'm assuming you've seen that enough that no screenshots are necessary). Add a using statement up at the top of the unit while you're at it.

All we really need to do is add in a call to our Negamax evaluation, then add in a little bit of logic to look for win/draw conditions. All of this is just calling methods we've already written. Your newly modified should look like the following:

        public HttpResponseMessage PostBestMove(ChessGame game)
        {
            try
            {
                //validate the game
                var validator = new ChessGameValidator(game);
                if (!validator.Validate())
                    return Request.CreateResponse(HttpStatusCode.BadRequest, validator.ValidationIssues);

                //calculate the best move
                var bestMove = Negamax.Evaluate(game, 2, game.GameStatus == GameStatus.WhitesTurn);
                var responseGame = bestMove.Item2;
                responseGame.CurrentEvaluation = bestMove.Item1;

                //is a win or draw, set game move accordingly
                IChessValidMoveCalculator moveCalculator = new ChessValidMoveCalculator(game);
                var availableMoves = moveCalculator.CalculateValidMoves();
                if (availableMoves.Count == 0)
                {
                    var isKingInCheck = moveCalculator.IsKingInCheck(game.GameStatus == GameStatus.WhitesTurn, ((IList<ChessBoard>)game.Positions)[0]);
                    //look for checkmate
                    if (availableMoves.Count == 0 && isKingInCheck)
                        responseGame.GameStatus = game.GameStatus == GameStatus.WhitesTurn ? GameStatus.WhiteWin : GameStatus.BlackWin;
                    //look for draw
                    else if (availableMoves.Count == 0 && !isKingInCheck)
                        responseGame.GameStatus = GameStatus.Draw;
                }
                else //just the next turn now
                {
                    responseGame.GameStatus = game.GameStatus == GameStatus.BlacksTurn ? GameStatus.WhitesTurn : GameStatus.BlacksTurn;
                }

                //return the best move wrapped in an http "ok" result
                return Request.CreateResponse(HttpStatusCode.OK, responseGame);
            }
            catch (Exception ex)
            {
                return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
            }
        }



As advertised, just a couple minor additions to call our evaluator and check for end of game.


Testing It

Remember way back when, that web forms project BlogChessApiFlexer? It's right there in the solution, and it's just itching to call the modified web api method. We had already coded a call into the web api (in default.aspx.cs), but prior to now we were just passing in a dummy request and expecting a dummy response. Well now it's time for an appropriate request and response!

        public ChessGame Game
        {
            get
            {
                return Session["Game"] as ChessGame;
            }
            set
            {
                Session["Game"] = value;
            }
        }

        protected void btnServerTest_Click(object sender, EventArgs e)
        {
            //1. Create and setup client object
            using (var client = new HttpClient() { BaseAddress = new Uri("http://localhost:11482/"), Timeout = TimeSpan.FromSeconds(300) })
            {
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                //2. Create a blank chess game to send up in the post request
                if (Game == null)
                {
                    Game = new ChessGame() { GameStatus = GameStatus.WhitesTurn };
                    var positions = new List<ChessBoard>();
                    var sampleBoard = new ChessBoard(true);
                    sampleBoard.Board = new short[8, 8] { { -4, -2, -3, -5, -6, -3, -2, -4 }, { -1, -1, -1, -1, -1, -1, -1, -1 }, { 0, 0, 0, 0, 0, 0, 0, 0 }, { 0, 0, 0, 0, 0, 0, 0, 0 }, { 0, 0, 0, 0, 0, 0, 0, 0 }, { 0, 0, 0, 0, 0, 0, 0, 0 }, { 1, 1, 1, 1, 1, 1, 1, 1 }, { 4, 2, 3, 5, 6, 3, 2, 4 } };
                    positions.Add(sampleBoard);
                    Game.Positions = positions;
                }
                int numPositions = ((IList<ChessBoard>)Game.Positions).Count;

                //3. Send the request
                var response = client.PostAsJsonAsync("api/BlogChess/BestMove", Game).Result;
                if (response.IsSuccessStatusCode)
                {
                    //4. Read the result from the response, display the "best move"
                    var result = response.Content.ReadAsStringAsync().Result;
                    var resultObject = JsonConvert.DeserializeObject<ChessGame>(result);
                    Game.GameStatus = resultObject.GameStatus;
                    ((IList<ChessBoard>)Game.Positions).Clear();
                    for (int positionIndex = 0; positionIndex < numPositions + 1; positionIndex++)
                    {
                        if (((IList<ChessBoard>)resultObject.Positions).Count > positionIndex)
                            ((IList<ChessBoard>)Game.Positions).Add(((IList<ChessBoard>)resultObject.Positions)[positionIndex]);
                    }
                    lblResult.Text = "Success! :" + result;
                }
                else //5. Request failed; tell the user what happened
                    lblResult.Text = "Failzor'd!: " + response.StatusCode.ToString() + "::" + response.ReasonPhrase;
            }
        }



Let's start with the easiest part, the propert Game of type ChessGame. If you've done webforms before, this shouldn't require much of an explanation. This is just a handy, type-safe way for me to be able to reference the current game object from the session so I can persist information about a single game in memory. Moving on to the button click event...

And here's the meat. Some of this was already here. As I said above, we were already calling the web api. What's new is we're now creating and serializing a chess game object (the one from our session), and we're passing this game object to the web api. Then we're taking the result of this call and updating the Game property. This means that, if you keep clicking that button, the AI will play itself! How cool is that? Well it's moderately cool, as it's incredibly slow. But hey, baby steps.

Wrapping it Up

Thanks for sticking with this series of articles everybody. As can happen with coding endeavors, it got away from me a bit. I thought this might be a 4 or 5 article series, not 13. I hope you learned a few new things, and maybe even gained a little bit of interest in chess in the process of reading these.

Oh and here's the absolutely final code, all fanciful and whatnot.

What's Next?

There are a few things you could do really. You could load AI dlls dynamically, configurable via database entry or config file. This would make it easier for other people to create AI dlls that plug into your web api, and you could have clients configured to choose a specific AI. You could also optimize the existing AI dll further (or just make your own) to make it much much quicker (hint: bitboard!). Or just go have a beer, or martini, or diet dr pepper. Whatever.


Resources

online chess board editor at Apronus.com
Chess.com, a great site for everything chess related