Entity Framework and some Architectural Decisions

I recently read an article on getting around how Entity Framework works in regards to trying to simulate it into the MVC Storefront app that Phil Haack did a video series on recently. Muhammad Mosa does an excellent job writing on the same experiences I faced when trying to mock up a architecturally sound design utilizing Entity Framework (swapping out Linq to Sql).

In this article, Muhammad mentions 2 ways to get around the short comings of what it can project client side vs server side and how to map that into a Repository that flows with fluent filters as extension methods like the MVC Storefront application series.

The first way he mentions is the Virtual Proxy pattern class that wraps the entity object, and allows you to lazy load its references and the object itself when you reference its properties.

The second way is to explicitly define a complex projection (the select new from a Linq statement into another typed object - such as a data transfer object (DTO)) as a client side query with the AsEnumerable method, to pull all of the data from the database into memory to do some complex shaping, but leave its child references as IQueryable to lazy load.

Please refer to the above link to read Muhammad's article to get a clearer picture of what this means.

I decided to discuss these approaches in the comments, and realized how long the comment got, so I wanted to post it here for any readers to join in the discussion, so that hopefully, we can come up with a cleaner solution.

Entity Framework Architectural Discussion

The bad thing about the second way (i.e. the Client side evaluation), besides the mass amounts of data you need to pull into memory, is that now you are segregating your methods into some, like GetCategory7() with a AsEnumerable() and the GetProducts() with no AsEnumberable(). So you are segregating how you handle each pull from the Data Store based on what is the Main object, and its child references, which isnt all that bad if you have a repository per main object call.

However, supposebly you would have a Product Repository which has to duplicate what you are pulling from your Category Repository but with an AsEnumerable to correctly shape it.

This will make for duplicate logic in pulling data depending on if you need that entity as a child reference or base object.

Based on that alone, I would stick with the first option, which is basically a Virtual Proxy Pattern on top of each entity object. The problem here is the mass amount of mock up for each Property, especially when you already went through the trouble of creating your Model in the EDM. Now you have do it through the Entity Framework Model, and your wrapper proxy class, every time something changes. Again, excessive duplicate work.

Unfortunately, I hate to say it, but the third way it seems is just using Entity Framework as the Repository object that calls into each entity is probably the less work way, but couples you completely to EF. You can then marry your Business rules and logic to the partial of each entity. This also allows you to utilize ADO.NET Data Services in a seamless way to expose your domain layer to multiple applications (which you can do with the first and second way but you will need to implement IUpdateable in your Repository -> which can be a pain).

There is no "utimate" way to do this at this time. But to me, saving work up front, especially when it could change for v2 is probably a simpler way. But layered wise, the proxy pattern approach is definately more architecturally sound, the problem is you are back to building out your DTO objects that match your entity model, basically regurgitating out each new property in your proxy DTO class.

I dont know, but for situations where I want to utilize ADO.NET Dataservices, approach 3 seems the better route now, I just hope when POCO objects make there way, I dont get burnt in having to re-write much.

Essentially, the typical data layer model has you making a Repository Layer, a Service Layer (your business layer), and the application Layer (your MVP/MVC layer).

By utilizing EF at the Repository Layer, you completely couple yourself at that layer to EF, but gain the flexibility to easily change your model independently from your data schema (which is typically what the Repository does anyway).

I am testing out using the ADO.NET DataService as the service layer, since you can inject before query calls, and on updates/creates/etc. Unfortunately, to fully utilize my rules logic, it is simpler to push that into the partial for each entity, and throw an exception which I can trap in the DataService layer and respond out.

From there my Applications Model Layer in a MVC application will handle making a Application Business Layer that is specific for that application that simply consumes the ADO.NET Data Service to get the data and utilize the entity in the application. So rather than trying to make my Domain Layer with EF/ADO.NET Data Services be the business rules for everything, it just handles making sure my domain rules are correct for my model, and the application itself handles business rules that are correct for that specific application (since this model is for multiple applications to consume this -- it cant be everything to everyone, but it can make sure it is valid at a domain layer).

Hope this helps, this is just some of the things I have played around with. Maybe we can poke holes in both our ideas enough to come up with a cleaner solution.

Extension methods as a new Fluent Decorator Pattern?

So recently I have been on this kick to learn Design Patterns in Ruby, and to figure out all kinds of neat ways to do the same design patterns in a dynamic language that is as versatile and elegant as Ruby. While playing around with the decorator pattern in Ruby, I started trying to think of slick ways to accomplish the pattern in C# 3.0.

In Ruby, there is a really slick (and somewhat dangerous) way to handle the means to accomplish the decorator pattern in Ruby (hold on guys, I am getting to the C# part in a bit, I am trying to explain how I got there):

Imagine a string class that holds a string with a method called “write_line”. Suppose for some reason you wanted to apply a decorator pattern to this, and decorate the string at runtime to make the string perform in various output ways. In Ruby, you could tackle it the GoF (Gang of Four) ways, or you could do the following:

Alias method Wrapper:

Decorate with Modules:


Ok, so since Ruby is so dynamic, it can on the fly edit the class instance and inject objects in the inheritance chain after the class itself. So you can utilize this to decorate your base functionality at runtime.

However, C#, since it is statically typed, really won’t allow you to tackle it this way. I started thinking about some of the new features in C# 3.0, and some of the things I have read about, and some of the things I have played around with using Piping, Filtering, and a Fluent Interface (I first started learning about this on the MVC Storefront series by Rob Connery [I highly recommend watching this series btw]). It occurred to me, that although we can’t accomplish the decorator pattern the Ruby way, we can accomplish it a nice readable way using Piping and a Fluent Interface way.

Take for example the standard Decorator Pattern typically seen in C# (using the above example in the classic GoF way):

var outputString = new List<string>{“hello”, “world”, “this”, “is a”, “test”};
var myDecoratedString = new NumberingString(new TimeStampString(new SimpleString()));

myDecoratedString.Write(outputString);


The above would use the chain of sequences called by Write to build out the string that is Timestamped, Numbered, and outputted.

This is fine and dandy, but not very pretty to look at in the least. And since the Decorator Pattern, is basically hiding the base object (that does the real work) in itself, it is simply chaining it out till it gets to the base object (“chaining” being the operative word here – similar to Piping).

So what if we do this in a fluent type of way?

var outputString = new List<string>{“hello”, “world”, “this”, “is a”, “test”};
var myDecoratedString = new SimpleString().WithNumbering().WithTimeStamp();
myDecoratedString.Write (outputString);

This is much more readable if you ask me, and it is very clear and concise into what is happening. It is saying that you want a SimpleString object that has Timestamp and Numbering to decorate it.

The Decorator objects have been transformed into extension methods that act on the SimpleString object and decorate it from the extension method.

A simple example of this would be the following:

Output would be the following:

Success!

Now I am fully aware this is only decorating one function on the concrete component. But there is no reason you couldn’t make the Class Decorator (in this example the SimpleClassDecorator) have multiple Action<T> (or Action<T, S> …, or even Func<T, TResult> if you need return values).

In most cases however, there is mostly one entry point into making a class start doing something. Keep in mind, while the SimpleClassDecorator class may look somewhat ugly (because you are encapsulating the block in a delegate), it pays off when you call your decorators. And you only have to write one SimpleClassDecorator, but you can write tons of Decorating Extension Methods to decorate your concrete component object, that actually look quite easy to make going forward.

Anyways, this certainly isn’t the only way to do a Fluent Decorating Pattern, but it was my first attempt at 2am in the morning (I hate it when I think of something in the middle of the night and need to figure it out).

Also, it’s worth noting that this would also be trivial to do the same thing in Ruby, just using blocks.

Utilizing Ninject with ASP.NET MVC Framework

Lately, I have been using ASP.NET MVC exclusively on a lot of projects and have been looking for a clean way to enforce the DIP (Dependency Inversion Principle) utilizing the Strategy Pattern throughout my applications. If all of these terms seem alien to you, then you should really look into reading about them. If you want to write highly maintainable code (that is subject to spec changes every month), work very little to achieve this, and in general have clean code, these principles will go a long way to helping you out. Also, once you start doing things this way, you will never want to go back to the old clunky way of over-inheritance, and coupled-ridden code. Before you go farther, if you don’t know what these things are, do your-self a favor and Google “Dependency Inversion Principle” and “Strategy Pattern” and do a quick read on the subject. Once you get the hang of these ideas, a Dependency Injection Framework (also referred to as Inversion of Control Containers), essentially helps you very quickly enforce and setup your DIP throughout your code seamlessly and effectively. It’s a tool to help you achieve this principle and pattern throughout your code.

So after playing around with several Dependency Injection (DI) Frameworks (Unity, Spring, etc) I finally settled on one that I instantly fell in love with because of its simplicity, speed, and complete flexibility: Ninject (www.ninject.org). It is by far one of the simplistic dependency injection frameworks I have found, and fits within how I like to do things. A lot of other DI Frameworks want to be setup through a configuration file (most likely XML) to state how to handle casting and creating of your abstractions to concrete classes. However, Ninject is all about coding modules to setup your dependencies (which you can refer to configuration files there if necessary). It is also completely open so that you can modify and build on it to suit your needs.

Now don’t get me wrong, there are several great DI Frameworks out there, so before you flame me, here me out. DI Frameworks are like opinions, people have several of them. Each person thinks theirs is the best, and you cant tell them any differently. The beauty here is that you just use what works for you. The ideas I will post here should be usable through-out any DI Framework with some massaging.

So, back on course, I wanted to write a blog post about using the Dependency Injection Framework Ninject with the ASP.NET MVC Framework. I utilized it in a standard ASP.NET Web Forms project and it work as advertised. Let us take a real quick look into how it looks (if you really want to see what it is all about take a look at these URLs (http://dojo.ninject.org/wiki/display/NINJECT/Home , Screencast by Justin Etheredge: #1 Introduction to the Ninject IoC Container #2 Diving Deeper into Ninject -- Contextual Binding )

Lets start with the common MVP Pattern in a Classic ASP.NET Web Form application and then when we understand those concepts, apply it to a ASP.NET MVC Web Application.

Typically you will need to setup a Module to define how Ninject should handle your Abstractions to Concrete classes:

In this example I am telling Ninject how to inject my abstractions IRepository and IService to what Concrete class I want it to be (the AppHelper is a static class that handles grabbing which configuration details are needed for my application – similar to how Rails does this). You can see the syntax is very readable. You simply tell it that you want to bind this abstraction to this concrete type: Bind<AbstractionType>().To<ConcreteType>(). The only weird one up there is the Bind<PagePresenter>().ToSelf(). This one is simply telling Ninject that if I ask you go give me this type, just instantiate it (you could also specify default constructor parameters for Ninject to use, or when you want it to make it). You could have just omitted this, as it does this by default, but I like to be explicit in my rules.

There is no voodoo going on here. To further clarify what is going on and how it handles creating these abstractions to your concrete classes down the chain, let us look at a brief implementation of the presenter, service, and repository classes (because when we create instances of it we will be creating a Presenter, that will in turn need a Service, that will in turn need a Repository implementation instance):

Presenter:

Service:

Repository:

As you can see, using MVP (Model View Presenter), in the view we will need the following:

The only piece necessary now, is to set your view (or you can simply use NinjectHttpApplication for your Global.asax.cs and PageBase for your View located in Ninject.Framework.Web) to load up your Presenter using Ninject:


The injecting of the this object on the view will see the Property with the attribute “Inject” and start the chain of creating all of your concrete class instances from looking at the Abstract type it requires (using the Module we defined earlier). We didn’t have to write a single new line in our code, Ninject handled all of this for us.

Ok, so it works really nice in ASP.NET Web Forms, but what about ASP.NET MVC. Well, this is where it required me to extend Ninject, as it doesn’t add helper support to ASP.NET MVC that fits nicely in the ASP.NET MVC style of doing things.

Typically as you saw above, we get the view to inject all the way down the tree of instantiations. What you didn’t see is you can use Ninject.Framework.Web to help in your Global.asax class file (by changing it from HttpApplication to NinjectHttpApplication) to handle a lot of this setup for us, and make your Page inherit from PageBase to auto-inject the view and start it down the chain.

So how would we do this in MVC? Well you could do the same thing with the Global.asax file and make a base Controller class to inject the controller instance object, or you can do it the MVC way, and make it nice and clean.

So what is one way to do that?
I am glad you asked. First I had to think about how a classic ASP.NET page works using the MVP pattern and how it works using the MVC pattern. The classic ASP.NET page request first comes to a web page, so the view is the one to get the initial response, so it makes sense to start the chain of injections from there on down. But in MVC, the request first comes to the Controller, so logically we need to start the chain of injects starting there. The most effective way to do this (without tying the Controller to a base class of say NinjectControllerBase – in case we want to utilize some swanky open source ideas out there that need this) is to use a controller factory to handle all of this for us.

Here is such an implementation of one (keep in mind that KernelContainer is located in Ninject.Framework.Web, and the bottom code would be put into a dll that you can add to all of your MVC applications) :

So what does all of this allow you to do? Well assuming you package up the above into its own dll (say Ninject.Framework.MVC), you can just add it to your project and do the following to add Ninject support to your application.

1) Use the simple default Ninject controller factory to add in support in your Global.asax file in the RouteTable function.

2) Make your own Controller Factory and inherit from NinjectAbstractControllerFactory and just override CreateKernel (this is where you would make a new Ninject Kernel with the Module we created early on), then in the Global.asax file add it when building the routes for you MVC application.


As you can see here you can call it 2 ways. If you don’t care about utilizing a Controller Factory, then you can just use the first method, that builds the controller factory for you and sets up your injections for all controllers for you. You can alternatively make your own that inherits from the above NinjectAbstractControllerFactory. You can chose how you want it to load.

This is a great way to have IoC/DI in your MVC application without getting in the way of your other code. You set it and your rules and forget about it. From there you can just start marking up “[Inject]” to those constructors, properties, etc you want Ninject to take care of.

This example will load DataService and the Repository (in the IService implementation) that we specified in our module. Similar to how it worked in the MVP pattern mentioned above, except our Controller Factory did all the setup for us behind the scenes and tucked away.

Feedback, questions, comments. Keep in mind this is merely one approach, there are several others that can be taken, but I like this one. I have used it on 3 projects so far, and it really is efficient and simple to setup.

If anyone knows of a good place to store code, I can zip up all of this code and make a simple example for you to play with and tweak to how you would like it to work.

For now, as per suggestion, you can copy the above code (the Factory Code) from google docs here.

“The Next Web aka Web 3.0” and RIA’s importance to it.

I have recently been having some debates on Silverlight / Flash / (and even AIR) importance to the web. Some of these people in the discussion often say that AJAX enabled web sites are enough for what you need on the web. I whole heartedly disagree with this assessment, so I thought I would make a blog posting about what why I feel the importance of RIA development on the web is an important next step for our view of the web in the coming years.

Recently, I would say the past 5 or so years, we have been hearing a trendy word passed around called “Web 2.0”. To me, this merely means socializing on the Web in new and exciting ways. Sites such as Digg, Twitter, etc. have crafted a new age of our society and new ways for us to communicate. This is a great step forward for our generation. However, many have started to wonder what the next version of the Web will look like (“Web 3.0”).

I have a hypothesis of what will be more important to our next version of the web, and it will be somewhat seamless to the typical user that surfs the web. It will be a better organization of the domain layer, meaning, it will be spreading out your business logic and domain layer across multiple web services and Restful services that can allow for multiple consumption of the users data. This will allow you to access the data through Desktop application, Web applications, or consumption from other companies trying to tie into your data. Now, from this definition, there are a lot of people who are already there: Amazon, Ebay, Microsoft, etc. This is true, but once this becomes the norm way of creating content on the web, I think it will be an expected feature across all applications. To me the idea of “the next web version” will be the multiple ways of getting to your data and consuming that data.

All that being said, how does Silverlight, Flash, and AIR (web like desktop apps that consume web resources) play into this. There is an excellent article I read from nikhilk.net that talks about this: http://www.nikhilk.net/Entry.aspx?id=190. In it he describes the Reach vs Capabilities approach, where Classic Simple HTML is the broader reach, then AJAX apps, then AJAX + RIA (Silverlight and AIR), RIA apps, then Desktop apps that finally goes to greater capability. I think this is an accurate assessment of the technology at hand.

To me, Silverlight (etc) will be the new way to present web controls and feature components that give some depth and control to the user, in a presentation that has both speed and power. Silverlight gives the same flexibility as a windows application where the separation of concerns (domain objects) rests on your web services. It helps you really build out an enterprise system that will force you to create the idea of the “next generation of the web”. It will do this by really making you think how to structure your domain objects in interesting ways. This will enable you to create your domain layer mechanisms so that they can be consumed in multiple ways, such as an AJAX Website, Desktop application, and the RIA application. During this process, it will allow you to space out your business logic and domain logic across multiple servers, but still provide a very quick interface for the client, as well as maintain state storage locally and give the user a better environment to manipulate their data.

One thing I keep hearing is Silverlight is the next Active-X. It is really nothing like Active-X, except for the fact that it runs on the client side. Active-X was a way for the client to consume a dll created by the site and run that business logic locally to that client. Silverlight is a complete framework that runs on the client’s side. The domain layer “should be” created and utilized from the Server side (web services / Rest ful APIs). The point here is that Active-X can be any user created DLL that is registered to your browser and interacts with your OS. Silverlight is the partial .NET Framework running on your system, it doesn’t interfere with the OS or register any components to it. It is completely self contained, unlike Active-X. An active-x object could wreak havoc on your system, where-as a Silverlight application has restrictions on what it can do, because it is self contained in the Framework for it. It is nothing more or less than Flash, which is not considered Active-X in nature. What it is, is a fraction of the .NET 3.5 Framework running locally on the client, with a WPF/E (Windows Presentation Foundation Everywhere) front-end that provides vector style graphics and a rich GUI and mechanisms to consume data in multiple facets.

All that being said, I do not think have a majority of your website in Silverlight is a good thing; far from it. Silverlight, to me, is great for doing complex User controls and Components that require speed, complex GUI, depth, and simplicity of use. I can foresee a lot of Admin Sections being in Silverlight, as well as key complex components, such as Events Calendar, Attendance components, Form Builder, etc. Really anything that requires a complex GUI that requires simplicity and depth.

We all know that JavaScript is great for simple things, but the more complex a page gets and the more complex the GUI needs to perform, the more of a headache JavaScript can become for the typical developer. Not to mention, JavaScript will result in a much slower interface. Looking back at all of the issues I have ran into lately with several projects, over 35-40% of the issues was trying to get Javascript and .NET to play nicely together. Now, if I had been using ASP.NET MVC, a lot of those problems could have been mitigated, however, the more complex your JavaScript code is, the more cumbersome the code will get, and the more difficult to maintain. I am not saying JavaScript is bad and convoluted, but it is something to me that should be mitigated to making simple features seem more alive.

I am working on a proof of concept application that is build on the ASP.NET MVC framework, that makes and includes a User Control written in Silverlight (which is developed to use the same MVC pattern), to complement and see how all the pieces fit together. I read an article at http://marlongrech.wordpress.com/2008/03/20/more-than-just-mvc-for-wpf/ by Marlon, that I am very fascinated to try out in Silverlight. The article talks about using the MVC + Mediator pattern to communicate between all of the pieces in WPF. However, since Silverlight is a subset of WPF, and I cannot find a way to do the EventManger and register events of a XAML view from another class at the moment, I am having to do a mix between MVC + M and a little more work on the view’s side to create events that the controller can consume. I am trying to do as little code as possible on the Views side and keep it all confined in the Controller side. It is coming along great, and I am really thing this is the way to bring MVC in Silverlight with ASP.NET MVC Framework to keep the separation consistent on both sides.

I am currently trying to write a lot of domain base classes and interfaces to help speed up the connection of these two technologies, to help mitigate the time spent including them in the same project. I am also working on a "drag in and work" Security ASP Membership feature for MVC ASP.NET by extending a starter kit found at http://www.squaredroot.com/post/2008/04/MVC-Membership-Starter-Kit.aspx by Troy to fit the needs that I typically use for the Memberships.

Before I leave off today, I would like to leave some links to some good Silverlight Examples:

Quick Silverlight Examples:

MSPaint Type Example: http://silverlight.net/Samples/2b1/ImageSnipper/testpage.html

Awesome Weather Tool: http://silverlight.r2musings.com/weatherwidget/default.aspx

Google with a stylus or mouse: http://www.tabletpcpost.com/search/

Upload Tool: http://fluxtools.net/emailphotos/

Slide show Image App: http://www.vertigo.com/SlideShow.aspx

some more neat ones at silverlight.net

Tips and Tricks: Forcing LINQ to SQL to load Child Objects without deferred execution and why

Today I would like to discuss how to force LINQ to SQL to load child objects, as well as shape its load, without deferred execution. Before we begin, let’s first discuss why that is even necessary.

What does Deferred execution give us:

Composable queries and deferred execution work together to make LINQ a unusual rich query language. If you properly understand these features of LINQ you will be able to write less code that executes faster, in order to accomplish more.

What problems can it create:

var query = from customer in db.Customers
where customer.City == "Paris"
select customer;

foreach (var Customer in query) <<>

{

Console.WriteLine(Customer.CompanyName);
foreach (var order in Customer.Orders) <<>

{

Console.WriteLine(order.OrderID);

}

}

As you can see above, you are sending SQL across the wire each time you loop through the Customer object. This is not an ideal way to handle this load if you already know you will need to pre-load all of the child objects ahead of time.


Forcing it to load up the child objects if you know you are going to need them:

//Create a new Data Context to load data objects

NorthwindDataContext nwind = new NorthwindDataContext();

//Create a DataLoad Options object to tell the Datacontext

// how to load this object, if it

// loads the specified object (through the generic part)

DataLoadOptions options = new DataLoadOptions();

options.LoadWith<Product>(p => p.Category);

options.LoadWith<Product>(p => p.Order_Details);

options.LoadWith<Order_Detail>(od => od.Order);

//Set the load options for this dataContext

nwind.LoadOptions = options;

//This products object has preloaded the Parent object Category

// and Child object OrderDetails,

// as well as each OrderDetails parent Order object

IEnumerable<Product> products = nwind.Products.ToList<Product>();

Changing what data actually gets loaded for the child objects for the Customer’s Order:

//Create a DataLoad Options object to tell the Datacontext

// how to load this object, if it

// loads the specified object (through the generic part)

DataLoadOptions newOptions = new DataLoadOptions();

//This will tell the Datacontext to load the Order Object

// for the Customer (when it loads through deferred or not)

// to load only those Orders with OrderID < style="">

// in descending order by OrderDate

newOptions.AssociateWith<Customer>(c => from o in c.Orders

where o.OrderID < 10700

orderby o.OrderDate descending

select o);

As I hope you can see, LINQ to SQL gives you a lot of flexibility on how to load and when to load. I really do enjoy working with LINQ and all its pieces. I still have a lot to learn with it, but the more I use it the more I appreciate it and become reliant on it.

Detailed Look: Key components in LINQ to SQL and their Key Roles

Detailed Look: Key components in LINQ to SQL and their Key Roles

As part of this blog, I plan to have an on-going set of articles that takes a detailed look into some part of the .NET Framework. I plan to bring as much knowledge as I can find on the topic, but drill down into a subset of its components, to make aware and have discussions. This will bring about more understanding for me and the .NET Community. So if you have suggestions, comments, or a different idea on the subject, please share it.

Before I begin on discussing its key components, I would like to point you to some very good articles written about LINQ to SQL, which should be read to have a deeper understanding of this new technology, and prepare you to be able to use it comfortably. This is a brief listing of articles written to help you better understand how to use LINQ to SQL:

Scott Gu’s Multi-part Tutorials:

· Part 1: Introduction to LINQ to SQL

· Part 2: Defining our Data Model Classes

· Part 3: Querying our Database

· Part 4: Updating our Database

· Part 5: Binding UI using the ASP:LinqDataSource Control

· Part 6: Retrieving Data Using Stored Procedures

· Part 7: Updating our Database using Stored Procedures

· Part 8: Executing Custom SQL Expressions

Rick Strahl has many articles written on practical usages of LINQ:
http://www.west-wind.com/WebLog/ShowPosts.aspx?Category=LINQ

Pro LINQ by Joseph C. Rattz, Jr. (a book I have read cover to cover multiple times and actually reference some of his explanations).

Today I would like to discuss some of LINQ to SQL’s key components, specifically the DataContext and Entity objects. I have read a lot of articles dealing on how to effectively use LINQ to SQL, and how to structure it. However, I would like to drill down more into what two very specific parts are and what they are responsible for. Understanding these objects more fully and being aware of it, will help bring about understanding and correct usage of these components.

Before we drill down into what the DataContext and Entity objects are and what they are responsible for, let’s take a brief look at the key difference between LINQ to SQL and LINQ to Objects / XML.

What are the key differences between LINQ to SQL and LINQ to Objects / XML?

· LINQ to SQL needs a Data Context object (generic or a Custom Inherited version).

· LINQ to SQL returns IQueryable
LINQ to Objects / XML returns IEnumerable

· Normal LINQ queries are performed on arrays/collections that implement IEnumerable.

LINQ to SQL queries are performed on classes that implement the TQueryable interface, such as Table.

· LINQ to SQL is translated to SQL unlike LINQ to Objects / XML which are translated to Intermediate Language (IL).

LINQ to SQL is translated to SQL by way of Expression Trees, which allow them to be evaluated as a single unit and translated to appropriate and optimal SQL Statements.

· Normal LINQ is executed in local machine memory, LINQ to SQL is translated to SQL calls and executed on the specified Database.

What are the similarities shared between LINQ to SQL and LINQ to Objects / XML?

The similarities shared between all aspects of LINQ are the concept of Deferred Loading and Execution. I will not go into detail on this topic here, as it has been discussed countless times on various other blog articles. Suffice it to say, that it allows you define a LINQ query that maintains what you want to query, but doesn’t actually query that item until you use it.

Quick Example:
var query = from customer in db.Customers
where customer.City == "Paris"
select customer;

foreach (var Customer in query) <<>{

Console.WriteLine(Customer.CompanyName);
foreach (var order in Customer.Orders) <<>

{

Console.WriteLine(order.OrderID);

}

}

This allows you to define what you query, through various code paths, until you actually need to use the query for data.

Example:
NorthwindDataContext northwind = new NorthwindDataContext();
var products = from p in northwind.Products
select p;

if (somecondition)
{ products = from p in products
where p.Discontinued
select p;
}

foreach (Product p in products)
{ // do something }

Responsibilities of the Entity Classes and the DataContext Class

Let us continue on to our discussion about what the DataContext and Entity objects responsibilities are and what they actually do for us. We will discuss the core concepts of what the DataContext and the Entity classes do and a description of how it does it. For the Entity class part, we will actually get down and dirty with how it actually accomplishes its responsibilities, since we can visibly see how it does it via the code generated for us.

Please note, that all of this is automatically handled if you use the SQLMetal or OR/M designer in Visual Studio 2008. This knowledge is to understand what it automatically provided to us, to better understand these objects and also if we decided to implement these features ourselves.

DataContext class is responsible for identity tracking, change tracking, and change processing. All of this is automatically handled by the base DataContext Class.

Identity Tracking:
When a record is queried from the database for the first time since the instantiation of the DataContext object, that record is recorded in an identity table using its primary key, and an entity object is created and stored in cache. Subsequent queries that determine that the same record should be returned will check the identity table, and if the record exists in the identity table, the already existing entity object will be returned from the cache. This is an important concept to grasp, so I will reiterate it in a slightly different way. When a query is executed, if a record in the database matches the search criteria, and its entity object is already cached, the already cached entity object is returned. This means that the actual data returned by the query may not be the same as the record in the database. The query determines which entities will be returned based on the data in the database, but the DataContext’s identity tracking service determines WHAT data is returned. Lucky for us, we can refresh the DataContext’s cache and have it retain our changes. I will show an example of this later.

Change Tracking:
Once the identity tracking service creates an entity object in its cache, change tracking begins for that object. Change tracking works by storing the original values of an entity object. Change tracking for an entity object continues until you call the SubmitChanges method. Calling the SubmitChanges method causes the entity objects’ changes to be saved to the database, the original values to be forgotten and the changed values to become the original values. This allows the change tracking to start over. This works fine as long as the entity objects are retrieved from the database. However, merely creating a new entity object by instantiating it will not provide any identity or change tracking until the DataContext is aware of its existence. To make the DataContext aware, simply insert the entity object into one of the Table properties (that represents the collection of the Table). To accomplish this just call InsertOnSubmit or Attach method on the DataContext’s Table property passing this new entity instance as a parameter. When this is done, the DataContext will begin identity and change tracking on that entity object.

Change Processor:
When you call SubmitChanges() method, the DataContext object’s change processor manages the update to the database. First, the change processor will insert any newly inserted entity objects to its list of tracked entity objects. Next, it will order all changed entity objects based on dependency. Then, if no transaction is in scope, it will create a transaction so that all SQL commands carried out during this invocation of SubmitChanges will have transactional integrity. It uses SQL Server’s default isolation level of ReadCommited, which means that data read will not be physically corrupted and only committed data will be read, but since the lock is shared, nothing prevents the data from changing before the end of the transaction. Lastly, it will enumerate through the ordered list of changed entity objects, creating necessary SQL and executing them.

Entity classes are responsible for change notification, graph consistency, and implementing good practices.

Change Notification:
The DataContext must be able to monitor this Entity class and know what changed and when it is changed. The Entity class must notify the DataContext that something has changed, in some form or fashion.

Graph Consistency:
Updating the relationship between two entity objects, such as Products and Orders. The reference on each side of the relationship must be properly updated so that each entity object refers to each other (or no longer refers if removed).

Implementing Good Practices:
Entity classes should implement INotifyPropertyChanging and INotifyPropertyChanged. If you decide to make an entity class by hand, and do not implement this and change notification, the DataContext will need to create 2 copies of each entity object: one with original copy, to compare and determine changes (highly inefficient).

Add OnCreated to the constructor of the entity class so that the DataContext knows it is created and create partial methods of [PropertyName]Changing / [PropertyName]Changed for each property and add it before and after each set of that property.

Example of Property for an Entity Class and how it accomplishes change notification, and graph consistency (as taken from the book Pro Linq):

[Column(Storage="_ShipCountry", DbType="NVarChar(15)")]

public string ShipCountry

{

get

{

return this._ShipCountry;

}

set

{

if ((this._ShipCountry != value))

{

this.OnShipCountryChanging(value);

this.SendPropertyChanging();

this._ShipCountry = value;

this.SendPropertyChanged("ShipCountry");

this.OnShipCountryChanged();

}

}

}

As you can see in this example, the get part of this property is relatively simple. When we take a look at the set part of this property, we can see it is calling the INotifyPropertyChanging and INotifyPropertyChanged versions of SendPropertyChanging and SendPropertyChanged. This is to let the DataContext know that this property is about to change and has actually changed.

The other two odd looking methods to notice is the OnShipCountryChanging and OnShipCountryChanged methods. This is the partial methods that the user can attach to (if they decide to partial out this entity class) to add validation or additional logic if they so choose. Partial methods are like lightweight Event Handlers, in that you can stub them out in one partial class definition, and actually define them in another partial class definition. But if they are never defined in another partial implementation, the compiler removes all traces that the partial method ever existed. So it is a very efficient way to add functionality, and if you never use it, it isn’t added to the compiled code – hence like a lightweight event.

Examples of a Property for an Entity Class and how it accomplishes change notification and graph consistency (as taken from the book Pro LINQ):

public Order()

{

this._Order_Details = new EntitySet<Order_Detail>(

new Action<Order_Detail>(this.attach_Order_Details),

new Action<Order_Detail>(this.detach_Order_Details));

this._Customer = default(EntityRef<Customer>);

OnCreated();
}

private void attach_Order_Details(Order_Detail entity)

{

this.SendPropertyChanging();

entity.Order = this;

}

private void detach_Order_Details(Order_Detail entity)

{

this.SendPropertyChanging();

entity.Order = null;

}

This part of the example sets up the relationship to the Order Entity object. It sets up the child objects Order_Details, and its parent object Customer. The interesting thing here to note is the fact that the parent object is of type EntityRef and the child object is of type EntitySet. EntityRef and EntitySet are generic types that allow the deferred loading to occur. Meaning, they simply state what the relationships are, but do not actually load the references, until it is absolutely necessary. The main difference is EntityRef is a single object that represents the fact that it is a parent and EntitySet is a collection of objects that represents a set of children.

You can see above in this example that EntitySet gets instantiated with two Action delegates. This is to specify how to attach and how to detach. It will use these mechanisms to add the children and remove the children to this relationship. Adding will take place when you manually add a new or existing Order_Detail object to this Order object, and remove will occur when you Remove it. This setup is there because to remove or add a relationship, requires it be done on two sides. If you are adding a child object to a parent object (such as adding a Order_Detail object instance to an Order object), you must tell both the Order_Detail object that it is now a child of the Order object (setting its parent), and tell the Order object that it now contains a new child object of that Order_Detail instance. The above example deals with this new child object and how to attach to this parent object (as you can see in the attach_Order_Details method above).

Simply put, LINQ to SQL will use these two action delegates to assign a Order to an Order_Detail, or remove an assignment of Order from Order_Detail. This and the previous example is how Change Notification works and functions in LINQ to SQL, and how the DataContext knows what to update, remove, and insert.

[Association(Name="Order_Order_Detail", Storage="_Order_Details", OtherKey="OrderID")]

public EntitySet<Order_Detail> Order_Details

{

get

{

return this._Order_Details;

}

set

{

this._Order_Details.Assign(value);

}

}

In the Order_Detail (children reference) example above, it simply returns out the EntitySet collection of the children objects and maintains the deferred execution until you absolutely need it. When the child objects are needed, it executes the relevant SQL and grabs the child objects. It then calls the set part of this property which re-assigns the EntitySet with the loaded child objects.

[Association(Name="Customer_Order", Storage="_Customer", ThisKey="CustomerID",
IsForeignKey=true)]

public Customer Customer

{

get

{

return this._Customer.Entity;

}

set

{

Customer previousValue = this._Customer.Entity;

if (((previousValue != value)

|| (this._Customer.HasLoadedOrAssignedValue == false)))

{

this.SendPropertyChanging();

if ((previousValue != null))

{

this._Customer.Entity = null;

previousValue.Orders.Remove(this);

}

this._Customer.Entity = value;

if ((value != null))

{

value.Orders.Add(this);

this._CustomerID = value.CustomerID;

}

else

{

this._CustomerID = default(string);

}

this.SendPropertyChanged("Customer");

}

}

}

In the Customer (parent reference) example, we will skip the get part of the Property as it is fairly obvious what it is doing and focus on the set part.

Customer previousValue = this._Customer.Entity;

You can see that the first line of the set method code, they store off a copy of the original Customer assigned. Don’t let the fact that it is calling Entity on the _Customer member confuse you. _Customer is of type EntityRef, so in order to get the actual customer, we actually have to call directly to it.

if (((previousValue != value)

|| (this._Customer.HasLoadedOrAssignedValue == false)))

The above statement is checking to see if the Customer is currently being assigned to an existing customer. If it is the same customer that is already assigned, there is nothing more to do.

this.SendPropertyChanging();

As part of the change tracking system, this is notifying the DataContext that we are about to change the Customer Property as part of the Change Tracking / Notification piece.

if ((previousValue != null))

{

this._Customer.Entity = null;

previousValue.Orders.Remove(this);

}

This next part of the code determines if the previous Customer object is null. If it isn’t null, then clear out the relationship between the previous parent and this child object. The inner two lines simple removes the parent object reference and the child reference to this object (in the previous parent). Calling the Remove method above will cause the Customer class’s detach_Orders (similar to the top example) to get called and the passed Order object to be removed. In the detach_Orders method, the passed Order object’s Customer property is set to null and looks like the following:

private void detach_Orders(Order entity)

{

this.SendPropertyChanging();

entity.Customer = null;

}

As you can see, when the Customer property is set to null, this will cause the Order object’s Customer property’s set method to be called, which is the method that invoked the code that called the detach_Orders method. So the very method that started this process of removal is getting called recursively.

set

{

Customer previousValue = this._Customer.Entity;

if (((previousValue != value)

|| (this._Customer.HasLoadedOrAssignedValue == false)))

Remember how we set the Customer parent object to null before we called the detach_Order method? Well, because of this, the previousValue is set to null, and since we are passing in null, stops the execution here without doing anything else.

So, once the recursion call to the set method returns, we no longer have a reference between the prior parent and that parent to this child object. We are now back to the next line of our code.

this._Customer.Entity = value;

if ((value != null))

{

value.Orders.Add(this);

this._CustomerID = value.CustomerID;

}
else

{

this._CustomerID = default(string);

}

The above first line sets the new parent object to the _Customer member to maintain the parent relationship. The next line will check the value to make absolutely sure it is not null. If it was null, it would just assign the default value. If it isn’t null, we will set the new parents child reference to point to this object. The current Order object will be passed to the Customers collection of child Order objects. If it was null, it would just assign the default value.

The result of this will cause the attach_Order method to be called. This will assign the current Order object’s Customer object to the passed Customer, resulting in the Order object’s Customer property’s set part being called again (the second part of the recursion).

if (((previousValue != value)

|| (this._Customer.HasLoadedOrAssignedValue == false)))

Just like previous, this line will break the recursion. Remember “this._Customer.Entity = value;”, before our recursion out, we set the Oreder object’s Customer property to the new Customer, who was passed this set part again from the attach_Orders method. Since they are the same, this exits out the recursion the same way the detach did.

The last thing of relevance is setting the Customer ID from the new Customer parent, and the letting the DataContext know that this property was changed via the this.SendPropertyChanged("Customer") part.

This whole system is required to maintain a one-to-many relationship, between the Entity objects and maintain the graph consistency between them. If you decide to write an Entity class by hand, you must remember to implement a feature similar to this, as the Entity class is responsible for its own graph consistency. A typical approach (if you really want to make your own entity class), is to allow the ORM create these classes for you, and copy the code and paste it into your own Entity class. This will cut down on the amount of work you need to do to maintain change tracking and graph consistency.

It may take a couple of re-reads to fully get what is accomplished here, but it will be worth it to fully understand how all of the change tracking and graph consistency “magic” occurs. If you truly want a deeper understanding of these concepts, I highly recommend reading the book “Pro LINQ” I mentioned earlier. It opened my eyes to a lot of the inner workings of LINQ in general.

Some of the key components not mentioned in this article is the concept of Expression Trees. This is essential to how LINQ to SQL works. In future blog articles I will try to get more in depth with this concept, and how they relate to the .NET Framework 3.5 as a whole. For now I highly recommend reading the article posted here. Another thing I would like to touch on in the future is the inner workings of the DataContext, and how it accomplishes the responsibilities.

Look for more Detailed Look blog articles in the future, as well as Tips and Tricks (quick excerpts on useful features in .NET), Common Design Patterns (review of common design patterns and how they relate to you as a .NET developer), and Hair Pullers (those things that are frustrating gotchas and simple ways around them).