Mehran Nikoo's Notes

I find myself answering questions related to the presentation patterns repeatedly. Since I am a lazy person, I am writing this post to illustrate my understanding and interpretation of the Model-View-* patterns and their variations so I can refer back to this post later.

[Update] I have discussed these patterns and their contexts with my colleagues Josh and Rupert and have updated the post the reflect the result of our conversation. Thanks guys!

I understand the concerns the Model-View-* purists might have around the correctness and/or pureness of these diagrams so I do apologize for potential inaccuracies in advance and I welcome any suggestions / comments!

I am not going to explain every single Model-View-* pattern as you can find better explanations in other places. Instead, I will focus on the relationship between participant components in each pattern.

Let’s start with a list of patterns we are going to cover:

• MVC: Model-View-Controller
• MVP (Supervising Presenter): Model-View-Presenter with Supervising Presenter
• MVP (Passive View): Model-View-Presenter with Passive View
• MVVM: Model-View-ViewModel
• MVPoo: Model-View-ViewModel with some unwanted stuff (in the ViewModel)

This is a typical implementation of the good old MVC pattern. The Controller displays the View, make changes to its state and manage the navigation. The Controller can also update the Model (or ask the Model to update itself) and these updates are subsequently reflected in the View.

* What Happened to the MVP Pattern? *

As you may have noticed, the plain MVP (Model-View-Presenter) pattern is not in the above list. If you want to know why, read Martin Fowler’s note on retirement of the MVP pattern. Basically, he has split the MVP pattern into two: Supervising Presenter and Passive View. By looking at the next two diagrams, you will notice why they need to be separated.

In the Supervising Presenter variation of the MVP pattern, Controller becomes responsible for implementing the complex view logic where the data binding between the View and the Model is not able to support that complex logic. Data binding is still the primary mechanism for the communication between the View and the Model and the Controller is used only in those scenarios where data binding support falls short.

Passive View is the other variation of the MVP pattern and it brings the level of supervision of the Controller to the next level. The View no longer updates itself based on the Model and Controller becomes responsible for updating the View based on the data in the Model.

In the MVVM pattern, the View Model inherits most of the functionality that would otherwise be in the Controller. The user gestures are captured by the View and the View uses the commanding feature of the underlying framework to invoke the operations defined in the View Model. Note that there is a dotted line showing the data binding between the View and the Model. This highlights the ability for the view to be bound to the Model via the View Model. Although the View Model can have its own properties the View binds to, but it is very likely for the View Model to publish the underlying Models as properties so the View is effectively bound to the Model (via the View Model).

MVVM makes an assumption that the framework the application is built on provides strong support for two-way data binding between the View and the Model. This diagram shows how simple the MVVM pattern is but this pattern would be really hard (if not impossible) to implement before WPF because of the lack of strong support for commanding and data binding. Even with WPF, implementing a non-trivial application that follows the design shown above can be tricky and will usually require a lot of additional helpers and constructs if we want to rely on data binding only.

The MVPoo (or M-V-poo to be more precise as defined by its creator, Dr. WPF) pattern acknowledges the fact that there is a difference between the ideal world and the real world so that nice and clean implementation of the MVVM pattern is not always achievable. As a result, there will be scenarios where:

• View Model needs to update the View with mechanisms other than data binding.
• The user gesture captured by the View or the events raised by UI components needs to be handled by the View Model with mechanisms other than commanding
• …

So it is very likely that we will end up with a stinky version of the MVVM pattern. The MVPoo pattern acknowledges this fact and says this is okay but we need to try and make the components (with specific attention to the View Model!) less stinky as much as possible (by keeping the number of non-ideal elements/features in the View Model to a minimum).

Windows Server AppFabric simplifies the process of creating, scaling and managing web and composite applications that run on IIS and includes the following components:

• AppFabric Caching Services (code-named “Velocity”)
  • Caching services
  • Cache client
  • Cache administration
• AppFabric Hosting Services (code-named “Dublin”)
  • Hosting services
  • Hosting administration

You can get the RTW bits using the Web Platform Installer or downloading it directly from here.

The supported operating systems are both 32-bit and 64-bit editions of:

• Windows 7
• Windows Server 2008 R2
• Windows Vista SP2
• Windows Server 2008 SP2

You will also need a version of .NET Framework, depending on the features you want to use (more info):

• Hosting administration: .NET 4
• Hosting services: .NET 4
• Cache administration: .NET 4
• Caching service: .NET 4 and optionally .NET 3.5 (.NET 3.5 adds new capabilities)
• Cache client: .NET 4 and .NET 3.5 (depending on client application)

There are a number of additional resources linked from the landing page, including the Windows Server AppFabric wiki.

In Application Lifecycle Management (ALM) consulting engagements with customers, we usually cover the automation of various activities in the development process. Most of the development teams I work with have some sort of automated build system in place. The automated build usually goes as far as compiling the solution(s) and then running some unit tests to verify the build. Build automation adds a lot of value but the true value of the development process automation can only be realised when we take that final step and automate the deployment and testing process too. This is because the automated build (without deployment) cannot tell if all of the system components are going to work together to meet the requirements. It is also very hard (if not impossible) to run any integration tests in such environment because there is no deployment so the tests we can run will be limited to those that are related to individual units or those that rely on mocking frameworks.

When I ask development teams why they don’t take that final step, this is what I usually hear back:

• Our environment and deployment process is very complex so we can’t automate it.
• We have a shared database server we use for development and it is constantly changing so we don’t have a reliable baseline to test against.
• We don’t store our configuration files in the source control because it contains sensitive information.
• We have different environments we use for development and testing and every one of those has its own settings so which one should we keep in the source control?
• We need some data to be available in the database for testing and since the database is changing, the base data set is changing too.
• If we want to include the deployment and testing in the process, it is going to take n (n>5) hours.
• …

There are various solutions to any of these issues and to be honest, none of these are a good excuse for not automating the deployment and testing. Points similar to the ones listed above highlight fundamental problems somewhere else in the development process so even if we don’t want to automate the deployment and testing, we are still going to be affected by those inefficiencies and problems.

For example, if our argument is that the database schema is constantly being changed (by a different team and probably on a different release cycle), this means when we decide to have a release, we have no idea if the latest version of the code is in sync with the database schema and scripts at that point. We may be lucky and manage to release in few hours but next time we may end up spending a day or two sorting out inconsistencies. The typical solution to this is to make sure the database artifacts are always in sync with the source code and are stored together with the code in the source control. Historically, maintaining the database scripts (especially the DDL change scripts) in source control has been a huge problem but today, we have great support from development tools in the market for managing the database artifacts (and their changes). Many of the teams I talk to are simply unaware of the developments we have made in these areas.

The point about having a dependency on the base test data is interesting too. If we don’t have a reliable set of test data for the code we are working with, we will end up copying a database from a previous release and then start making changes to that database to bring it in line with the latest version of the database schema. And guess what? Every developer needs to go through this error-prone process. As the testing team and other developers start testing, they will discover new sets of data that allows them to test new scenarios but since all of the developers have spent a considerable time on creating their “precious” test data, they will hang on to what they have. I am sure this is one of those experiences we have all had and we can spend a lot of time on maintaining our test data instead of implementing features for our product. The easiest solution to this challenge is to maintain a set of base test data and store it together with the source code in the source control. Yes somebody needs to maintain this but we are in a better position as less people need to be worried with maintenance of that data and other developers will get the latest test data when they get latest from the source control. This base test data can also be used by the automated processes for testing purposes.

Implementing an automated deployment and testing process is not trivial but is certainly achievable and there are things you can do to make it simpler. If you ask me to come up with #1 guideline in this area, I will say put all of the artifacts required for deployment in the source control but even more importantly, make sure they are in the right location! Nobody questions why we need to put the source code under source control but people usually miss other artifacts such as database artifacts and build and deployment scripts. The interesting point is that many teams actually put these artifacts somewhere under source control but they put it in the wrong location, which makes them less effective and those artifacts will be unusable when we want to automate deployment and testing. So what is the right location for the artifacts?

The basic idea is that all of the artifacts required for deployment should follow our branching hierarchy.

For example, if we have three branches (DEV, MAIN and REL), each branch needs its own copy of the artifacts. Why? Because we want to be able to deploy and test any of those branches independently. As you can see this is a fairly obvious (and simple) solution but many teams don’t do this for various reasons and that makes the automated deployment and testing very hard (or impossible).

As I said earlier, even if we are not going to automate deployment and testing, we will still benefit from following this structure and making sure all of the artifacts under each branch are consistent and in sync with one another. Once this structure is in place, the next step is to make sure we have the capability of creating and managing the target environments as we will be deploying and testing over and over again. [I am writing the next few sentences with my “good corporate citizen” hat on…] This is where tools such as Visual Studio Lab Management can help. Many developers and testers I talk to still haven’t heard of this great product or don’t know what it does so if you fall in that category, you may want to have a look at the resources (product team blog, videos) to find out more about it. Please note that at the time of writing this post, Visual Studio Lab Management is still in pre-release status so make sure you read the pre-release license terms (rtf) before installing it.

You will be amazed when you see how easy it is to automate the process of managing the target environments (which usually consists of one or more virtual machines) as part of the build process in TFS 2010.

The above diagram shows the list of activities provided out of the box and you can use the workflow editor for the build to customise the end-to-end process.

When you use the new operator to create an instance of a class, the class constructor is called before the reference to the newly created object is returned. This also applies to the reflection scenarios where you call Activator.CreateInstance, which finds the right constructor based on the number, order and type of the parameters provided.

This is the expected behavior in most cases as you don’t want to work with an object that might be in an invalid state. However, there are cases where you don’t need to run the constructor. A good example is when you are deserializing an object from a byte stream. During deserialization, you will be using the values embedded in the incoming byte stream to initialize the member variables so there is no point in initializing those members in the constructor as they will be overwritten.

The formatters that ship with the .NET Framework (such as BinaryFormatter and DataContractSerializer) follow this approach as you can expect. There is a utility class called FormatterServices, which provides some of the underlying services to the formatters, including but not limited to object creation.

namespace System.Runtime.Serialization
{
    public sealed class FormatterServices
    {
        public static void CheckTypeSecurity(Type t, TypeFilterLevel securityLevel);
        public static object[] GetObjectData(object obj, MemberInfo[] members);
        public static object GetSafeUninitializedObject(Type type);
        public static MemberInfo[] GetSerializableMembers(Type type);
        public static MemberInfo[] GetSerializableMembers(Type type, StreamingContext context);
        public static ISerializationSurrogate GetSurrogateForCyclicalReference(ISerializationSurrogate innerSurrogate);
        public static Type GetTypeFromAssembly(Assembly assem, string name);
        public static object GetUninitializedObject(Type type);
        public static object PopulateObjectMembers(object obj, MemberInfo[] members, object[] data);
    }
}

When you call GetUninitializedObject or GetSafeUninitializedObject, an instance of the class is created and returned without calling the constructor of the class. But note that the member variables are set to null and zero. FormatterServices is a public class so you can use its published functionality inside your application but you need to be very careful when using Get(Safe)UninitializedObject. As explained in the MSDN documentation, when you use these methods you should immediately populate all fields to make sure the object is not left in an invalid state.

There are some other interesting operations in FormatterServices. For example, GetSerializableMembers returns a list of members that will be serialized by the classic formatters.

One final point on this topic. If you are deserializing an object from a byte stream and want to run a piece of code to initialize the object before it is used (for example to calculate some of the fields that were not included in the serialized format), you will need to use the serialization attributes. See my earlier post on this subject for more information.

[Note: In this post, I am assuming that you are familiar with the TFS Branching Guidance. If you haven't read the guide, I strongly recommend that you do so! I will be referring to the branch names suggested for the basic branch plan.]

Scenario: After an application is released to the production environment, a defect has been raised by the users. The production support team decides to fix this defect before the next version is released so they start by modifying the appropriate files in the REL branch to fix the defect. When the developer working on the defect finishes his job, he shelves those changes and asks the support team lead to review those changes before they are checked in. The support team lead analyses the changes and because of the risks involved, she talks to the users and they agree to wait until the release of the next version to get the fix. 

Challenge: When you create a shelveset, the server path of each shelved file is stored with the shelveset so when you try to unshelve that shelveset, the files in the same branch are modified. You cannot unshelve the shelveset to a different branch such as DEV so you will need to do this manually by unshelving to the original branch and then manually copying the modified files across, which is time consuming and error prone. 

Solution: TFS 2008 Power Tools includes a command line tool (TFPT.exe), which allows you to perform advanced operations in TFS. One of these operations is the unshelve command, which allows you to change the server path of the files in a shelveset during the unshelve operation. This means you can use this command to unshelve any shelveset to a different branch. In our scenario, the shelveset was originally created in the REL branch so the development team can use the TFPT tool to unshelve the shelveset to the DEV branch. 
So let’s assume that the support team have changed the source files in the REL branch and we want to unshelve this shelveset to the DEV branch. In order to do this, we need to: 
- Make sure the TFPT command line tool is installed!
- Go to the command prompt and change the folder to a location that is mapped in the desired workspace.
- Run the following command:

    TFPT.exe unshelve ReleaseHotFix1 /migrate /source:$/MyProject/REL /target:$/MyProject/DEV          

Where ReleaseHotFix1 is the original shelveset name and $/MyProject/REL and $/MyProject/DEV are the server paths to the REL and DEV branches respectively. 

When you run the unshelve command with the migrate option, the tool will prompt you to choose which version of the file (in origin or target) you want to keep. It will also allow you to perform a file merge if needed, which will be done in the merge tool.

The following diagram shows the sequence of events that need to happen across multiple branches in our scenario: 

  
Events: 

• S1: Production support team shelve their changes in the REL branch.
• U1: Development team unshelve the shelveset to the DEV branch by running the TFPT tool and using the shelveset migrate option.
• C1: After resolving any potential conflicts in the DEV branch, the modified files are checked in.
• RI1: At some point, the checked-in changeset will be reverse integrated (merged) back into the MAIN branch.

Windows Azure and some other components of the Azure Services Platform are expected to be commercially available later in 2009. Clearly, one of the key components required for reaching this milestone is the physical hardware, networking equipment and the supporting services such as power generation and cooling, something that we refer to as “Global Foundation Services”.

On Wednesday last week (1 July 2009), a Generation 3 data centre outside the US was brought online in Dublin. This data centre will be used to host various services including Azure service hosting and additional live and online services. There is also another data centre in Chicago, which is planned to go online on 20 July. You can read more about this new development on Microsoft Data Centers Blog.

Last week, I had a enquiry from a customer about the Spanish locale for the United States. As he mentioned, it does make sense to have this locale, especially for the users based in the southern regions of the US. If you are running Windows XP or Windows Server 2003, there is no such locale supported by the operating system so if you try to access this locale in a managed application by writing this line of code:

CultureInfo cultureInfo = new CultureInfo(“es-US”);

you will get an ArgumentException, which says: “Culture name ‘es-US’ is not supported”. As a workaround, you can use the “es-MX” locale or create your own locale.

But the good news is that the support for the “es-US” locale was added in Windows Vista and Windows Server 2008 so if you are running one of these operating systems, you can now use the es-US locale (LCID = 21514) and the same line of code we used above will work fine.

This page has a complete list of the locale IDs assigned by Microsoft so if you thinking of creating your own locale, check this list first to make sure there is no conflict with the existing locales.

As planned, VSTS 2010 Team Suite Beta 1, .NET Framework 4 Beta 1, TFS 2010 Beta 1 and VSTS 2010 Test Load Agent and Controller Beta 1 are now available for download from MSDN Subscribers area.

We now have 32-bit and 64-bit support for TFS and the TFS client OM. All of the client tools such as Visual Studio will continue to run in 32-bit environment though. For more information on the supported environments for TFS and the improvements in setup, administration and operations refer to Brian Harry’s post.

Primarily for my own reference in the future but you may find this useful too. If you are running Windows Vista/7 64-bit and want to use the VSTS 2008 Web Test Recorder plug-in, the web test recorder bar may be unavailable due to the way Windows caches the list of available explorer bars. In order to resolve this, you can apply some tweaks to the registry. Michael Taute has a useful post which explains the problem and solution in more detail and covers some other issues too.

Most .NET developers and architects are familiar with various approaches to Visual Studio solution design: 

Most of the development projects contain more than one Visual Studio project. As soon as a second project is added to the solution, as a developer/architect, you start thinking about how the dependencies should look like. 

Meet “Camp P”
Members of camp P believe file references are evil as they cause all sorts of versioning and build issues when the dependencies go out-of-date. The members of this camp try to put everything in the same solution and life is great as long as you can count the number of projects with your fingers. There has been a debate out there as to whether Visual Studio should become quicker when it comes to handling large solutions. This issue was partly addressed by release of Visual Studio 2005 SP1 where we introduced some performance improvements in the IDE. These improvements are also rolled forward to Visual Studio 2008. However, you need to take into account all the work the IDE and the compiler need to do while you are typing in the IDE to provide you with Intellisense and compiler warnings/errors. So at some point, you will hit a practical limit on the number of projects you can have open in Visual Studio.

Meet “Camp F”
This is where the members of “Camp F” are coming from. They want to avoid the project references in order to speed up the IDE but they usually end up spending the time they saved on resolving the versioning issues. 

Now Meet Both!
None of these solutions are perfect but as long as the development team is consistent in following one of these approaches, then the team can live with it. This becomes a major problem when you have members from both camps in the same development team. This has been the case in most of the development teams I have worked with and I bet most of you have had similar experiences. So what is a typical day on such project? Somebody from “Camp F” is not happy with performance of the IDE while he is working on the “Order Processing” feature of the application. As a result, he decides to replace all of the references to the “UI Framework” projects with file references. But guess that? The dependency type is defined in the project file, so this change results in modification of the project file and the changes are checked back into the source control at some point. Few hours later, somebody from “Camp P” who is working on one of those “UI Framework” projects is trying to perform an integration test to validate a change she made to the UI Framework and finds out that the dependencies are out of date. She checks the source control as it was working fine until that morning and notices the change. She then reverts all of those changes as file references will make her job harder. And the team will soon end up in changing the references types back and forth between project and file references and and this changes cause further problems in the build process. Add to this all of the discussions/arguments as to which approach is better and you will see how much time is wasted as developers and build engineers keep changing the dependencies to suit their needs. 

What is the solution then? I am going to start with re-iterating what you probably know already but bear with me – I’ll try to keep it short! 

The Team Development with Visual Studio Team Foundation Server guide does a pretty good job at providing high level guidelines for choosing the right approach. It provides three options for managing the solutions: Single Solution, Partitioned Solution and Multiple Solutions. I am going to skip over the Single Solution. If you can manage everything in the same solution, then you are not worried much about the solution design, right? 

Partitioned Solution
If you work on a large system, consider using multiple solutions, each representing a sub-system in your application. These solutions can be used by developers in order to work on smaller parts of the system without having to load all code across all projects. Design your solution structure so any projects that have dependencies are grouped together. This enables you to use project references rather than file references. Also consider creating a master solution file that contains all of the projects. You can use this to build your entire application. 

Instead, you must use a script that understands the order in which the solutions need to be built. One of the maintenance tasks associated with a multiple-solution structure is ensuring that developers do not inadvertently create circular references between solutions. This structure requires complex build scripts and explicit mapping of dependency relationships. In this structure it is not possible to build the application in its entirety within Visual Studio. Instead, you can use TFS Team Build or MSBuild directly. 

The Partitioned Solution approach doesn’t scale well for very large solutions since it will make the IDE very slow and this sends you down the Multiple Solutions approach. So you group related projects and create a solution for each group. You use project references for dependencies inside the solution and file references for those dependencies that cross the solution boundaries. This is where the problems start: 

1- You cannot create a master solution that contains all of the projects. This is because you have created file references for cross-solution dependencies, which will cause problems when you try to build the master solution. 

2- Depending on which area of the system you are working on, you may want to have a different slice of the system in your solution. For example, if you are a member of a team working on a specific feature such as Order Processing, you want to have a vertical slice that goes across application layers (presentation, business and data). But if you are working on the UI Framework for the application, you want to have a horizontal slice that includes the presentation layer from all functional areas as well as the UI Framework. 

So even if all of the team members agree on the concept of Multiple Solutions, there are decision to be made on how the projects are going to be grouped into multiple solutions and as you can see in the above diagram, these categorisations don’t necessarily play well with each other. 

You always need to consider various factors such as the solution size and simplicity of the build process and choose one of these approaches (none of which are perfect): 

• Single Solution
• Partitioned Solution
• Multiple Solutions 

Multiple Partitioned Solutions 
Some of you may know about this already but there is another approach too. One can argue that it is the same as Partitioned Solution but in order to avoid confusion(!), I usually call it “Multiple Partitioned Solutions” or “MPS” (as you know we love TLAs). So what is it and how does it work? 

MSBuild is a great build engine. One of its features is the ability to follow and find the project references in a project even if those referenced projects are not inside the solution you are trying to build. This means you no longer need to have all of the project references inside the solution. Believe it or not, but you can successfully build the solution and the projects inside it even if the referenced projects are not in the solution. This will work both from the IDE and the command line as they both rely on the underlying MSBuild functionality. The only condition here is that the referenced projects need to have been built already. MSBuild is able to follow and find those referenced projects but it won’t build them for you. Is this a major problem? Probably not because when you get the latest version from the source control, you usually do a full build and then start working on your specific area. 

Let’s see how the system will be structured then. The dependencies between the projects are created as project references wherever possible. Depending on your requirements, you can create a solution that contains only a subset of projects you need to work on. Because the dependencies are project references, you can now have cross-cutting solutions that go across functional areas and architectural layers. You can also have a master solution because the dependencies are project references. Clearly, you don’t want to open the master solution in the IDE often as it will take a long time to load, it will be slow and you don’t usually work on all projects at the same time. The master solution will primarily be used for build purposes. 

Why do I call it “Multiple Partitioned Solutions”? Because the master solution is partitioned multiple times based on various perspectives. For example, the master solution can be partitioned into three solutions, one for each logical layer (such as presentation, business and data). At the same time, it can also be partitioned into another three solutions, one for each functional area (such as Order Processing, User Management and Call Centre). 

Like the other approaches, The MPS approach is not perfect and it has its own challenges. I personally prefer this approach though. It addresses the concerns around solution size and versioning and is very flexible as you can have cross-cutting solutions. But from a tooling perspective, we can provide you with better support so this is an area for improvement. As an example, if you try to remove a project from a solution in Visual Studio, it removes all of the dependencies on that project too. If you are following the MPS approach, this is not necessarily what we want. This is not as bad as it looks as in most projects I have seen, the solutions are created and maintained by dev leads, architects and/or build engineers so as long as they know what they are doing, it will be fine but it would be nice to have a proper solution here. 

I have found a workaround for this, which is to unload the projects that depend on the project we are trying to remove before removing the project. This will prevent the dependencies from being dropped automatically. Another approach is to manually edit the solution file and remove the project(s). 

As I said, these are workarounds not solutions. When we remove a project, we ideally want to be able to specify whether we want the project to be removed from the dependencies or we are just trying to exclude that project from the solution. So maybe we could add an “Exclude” option to the project context menu in the Solution Explorer?  

So… have you used the MPS approach in your development projects? If yes, can you share your experiences? What are the challenges you have faced and how have you addressed them? What kind of support do you expect from the IDE in this area?