CC.NET - NAnt integration problem

Today I was working on migration of our continuous integration system from old test server to a new one. A year ago the system was built upon:
1. CruiseControl.NET-1.5.7256.1
2. NAnt 0.90
3. Silk SVN 1.5.2
4. Windows 2003 x86

Today I downloaded and installed the latest versions of CC.NET (CruiseControl.NET-1.6.7981.1) and tried to set everything up and running on Windows 2008 R2 x64.

After I've finally made CC.NET dashboard working: it needs integrated mode for the app pool and updated Handler mappings for the application (by default I received all of them disabled) using 'Edit Feature Permissions -> Read, Script, Execute' option on the corresponding page - I've forced build and received the following message in the build log:

...
<LastModificationDate>17.07.2011 14:34:52</LastModificationDate>
</integrationProperties>
<build date="2011-07-18 14:34:45" buildtime="00:00:07" error="true"
buildcondition="ForceBuild">NAnt 0.90 (Build 0.90.3780.0; release; 08.05.2010)
Copyright (C) 2001-2010 Gerry Shaw
http://nant.sourceforge.net

Try 'nant -help' for more information

Duplicate value '
' for command-line argument '-target'.

</build>
</cruisecontrol>

By the way, NAnt was executed in the following way:

<nant>
   <executable>$(nant.executable)</executable>
   <buildFile>$(nant_build_file).build</buildFile>
   <buildArgs>
        -D:svn.exe="$(svn.executable)"
        -D:svn.base.path="$(svn_base_path)"
        -D:svn.path="$(svn_base_path)$(svn_project_path)"      
        -D:result.file.postfix="$(output_file_postfix)"
        -D:output.dir="$(project_output_dir)"    
        -D:publish.dir="$(project_publish_dir)"
        -D:MSBuild-2.0="$(msbuild_2.0.executable)"
        -D:MSBuild-3.5="$(msbuild_3.5.executable)"
        -D:MSBuild-4.0="$(msbuild_4.0.executable)"
        $(nant_parameters)
   </buildArgs>
   <buildTimeoutSeconds>$(nant_time_out)</buildTimeoutSeconds>
</nant>

Playing with CC.NET config files, NAnt build script and googling for around 2 hours didn't end up with any result.

Finally I stopped CC.NET service, opened the installation directory and replaced all the contents with the files of CruiseControl.NET-1.5.7256.1 from old machine. No CC.NET configs or build files where changed after, I started the service and forced build via dashboard. NAnt execution went fine, a build was successfully published.

To summarize, seems like the latest CC.NET version has some integration issue with NAnt or there's some change to CC.NET command which is not reflcted in the on-line documentation.

DropDownCheckBoxes: Using control

In this post I'll give more details about the DropDownCheckBoxes control.
After you've referenced the assembly and registered the control within application you may add its' markup to .aspx pages (see introduction post about how to add the control to the project).

From development point of view programming the control is no way different from using standard CheckBoxList control. It has same events (in fact only one is of interest - OnSelectedIndexChanged) and members (except those relevant to new rendering and client side behavior).

Example to start with

Let's have a look at an example and comment it. Bellow is a markup snippet from .aspx file:

...
<asp:DropDownCheckBoxes ID="yearsDropDownCheckBoxes" runat="server"
OnSelectedIndexChanged="checkBoxes_SelcetedIndexChanged"
AddJQueryReference="True" UseButtons="True" UseSelectAllNode="True">
     <Style SelectBoxWidth="160" DropDownBoxBoxWidth="160" DropDownBoxBoxHeight="80" />
     <Texts SelectBoxCaption="Year" />
</asp:DropDownCheckBoxes>
...
<div style="padding-top: 20px;">
    <h1>
         Selected items (updated whenever postback from any of the controls occurs)
    </h1>
    <asp:Panel ID="selectedItemsPanel" runat="server">
    </asp:Panel>
</div>

Code snippet from the corresponding code-behind file:

protected void Page_Load(object sender, EventArgs e)
{
    if (!IsPostBack)
    {
         var years = new int[20];
         var currentYear = DateTime.Now.Year;

         for (int i = 0; i < years.Length; i++)
             years[i] = currentYear--;

         yearsDropDownCheckBoxes.DataSource = years;
     yearsDropDownCheckBoxes.DataBind();
    }
}

protected void checkBoxes_SelcetedIndexChanged(object sender, EventArgs e)
{
    selectedItemsPanel.Controls.Clear();

    foreach (ListItem item in (sender as ListControl).Items)
    {
         if (item.Selected)
             selectedItemsPanel.Controls.Add(
                 new Literal()
                     { Text = item.Text + " : " + item.Value + newLineConst } // HTML line break
             );
    }
}

The page contains a drop down check boxes for selecting years and a panel bellow which displays checked items whenever a postback from the control occurs (code for other controls from the picture was cut above):


From the example we see that a control yearsDropDownCheckBoxes is added to a page, the event handler checkBoxes_SelcetedIndexChanged for SelcetedIndexChanged event is assigned and the page load handler performs data binding for the control. Whenever a user clicks 'OK' button in the drop down list a postback goes to server and panel in the bottom of the page is filled with all checked items. Everything is straightforward and no different from using controls inherited from ListControl(including CheckBoxList).

Postback options & behavior

Properties DropDownCheckBoxes.AutoPostBack (default value is 'false') and DropDownCheckBoxes.UseButtons (default value is 'true') control the way postbacks from the control arise:

1. AutoPostBack == false && UseButtons == false - in this case no 'OK/Cancel' buttons are displayed in the bottom of the drop down box, no postback occurs when a user clicks outside control and drop down gets hidden. The server side event will be fired (if there's a handler specified) during postback from any other control from the page (in case selection change is detected). The behavior in this case is similar to other ASP.NET controls with AutoPostBack set to false;

2. AutoPostBack == true && UseButtons == false - no action buttons ( 'OK/Cancel' ) are displayed, postback occurs whenever a user leaves the expanded drop down by clicking outside it.

3. UseButtons == true (AutoPostBack value doesn't matter) - in this case there're action buttons displayed in the bottom of the drop down box. Postback is triggered only when a user clicks 'OK' button. Clicking 'Cancel' or outside the control collapses the drop down and restores the check box selection prior to opening the control.

JQuery dependency

JQuery library is delivered within the control assembly and the reference to it is automatically added to the page by the control. In case you have JQuery referenced on the page (e.g. in a master page) script errors may occur.

In order to turn off automatic reference adding set DropDownCheckBoxes.AddJQueryReference to 'false'.

Other control properties

• DropDownCheckBoxes.UseSelectAllNode - if 'true' then there's a 'Select all' check box added on top of other check boxes in the drop down. Clicking the check box changes all other check boxes' to corresponding state.
• DropDownCheckBoxes.Texts - a complex property that allow to change default texts of the control elements (caption, button names, 'Select all' check box), e.g. for localization.
• DropDownCheckBoxes.CssClass - specifies CSS class name applied to the control (specifically the DIV element which wraps all the contents). Setting this peoperty has the same effect as using DropDownCheckBoxes.Style.SelectBoxCssClass property.
• DropDownCheckBoxes.Style - a complex property that allows applying CSS classes to the control and also directly specify sizes of the control elements (height of the select box, width and height of drop down box).
  

NOTE: The control can work within update panel without any adjustments.

The project page at Codeplex can be found here.

DropDownCheckBoxes: Introduction

DropDownCheckBoxes is a server control for ASP.NET web forms which provides the functionality of a standard CheckBoxList control in a form of a drop down list (the standard DropDownList control doesn't have any multi-select option). The key benefit of such approach is that it keeps space on a web page by dynamically displaying check boxes. The image bellow shows a sample use of the control:


About control

Browser compatibility: FireFox, Safari, Chrome, Opera (latest versions, May 2011), Internet Explorer 8.0

Server side: ASP.NET Web Forms 3.5, 4.0, can work with both synchronous postbacks and asynchronous when the control is placed within UpdatePanel control

The control is directly inherited from CheckBoxList control and thus can be used in place of it (easy migration is possible):

public class DropDownCheckBoxes : CheckBoxList, IPostBackEventHandler
{
...

The assembly containing control class also has embedded web resources used by the control which are automatically streamed to the browser:

• Client side JavaScript
• CSS file
• Button images
• jQuery script

Note: The jQuery JavaScript library is required for control proper functioning. The library is delivered within the assembly and is automatically referenced whenever the control is added to a page. To avoid duplicate references this behavior can be turned off by setting 'IncludeJqueryScript' property.

Getting started

The process of adding the control to project is absolutely the same as for other server controls.

1. In order to start using the control in you ASP.NET application get the control assembly from project's download page.

2. After you have DropDownCheckBoxes.dll downloaded you should add references to the assembly. Right click the web application project node in the solution explorer, choose 'Add reference' and in the dialog appeared find path to the downloaded .DLL file.


3. Register the control by adding web.config setting (if there's already <system.web><pages><controls> section - update it and don't create a duplicate):

<system.web>
   <pages>
       <controls>
           <add tagPrefix="asp" namespace="Saplin.Controls" assembly="DropDownCheckBoxes"/>
       </controls>
   </pages>
</system.web>

4. Put the control to .aspx page

<asp:DropDownCheckBoxes ID="yearsDropDownCheckBoxes" runat="server">
</asp:DropDownCheckBoxes>

The project page at Codeplex can be found here.

LinqToSql. ASP.NET GridView with paging and sorting.

ObjectDataSource - DataSource class - LinqToSql - Dynamiq LINQ - GridView: almost painless grid with sorting and paging out of the box (first time there's always much work with configuration).

Let's have a look at the class consumed by ObjectDataSource control:

public class ListProductsDataSource 
{
...
    // The core method
    public IQueryable<OrderProductsProjection> Select(string orderBy, IQueryable products, int startIndex, int rowsNumber)
    {
        LastCount = products.Count(); // save count for further use

        if (!string.IsNullOrEmpty(orderBy))
               products = products.OrderBy(orderBy).ThenBy(p => p.GUID).TakeIndex(startIndex, rowsNumber);
        else
            products = products.OrderBy(p => p.Site).ThenBy(p => p.GUID).TakeIndex(startIndex, rowsNumber);

        // return projections - only fields needed and preprocessed
        return ProjectProducts(products); 
    }

...

    // Different methods called by ObjectDataSource for cases with different parameters
    // e.g. when showing all reciords in the grid, when filtering by a string from text box
    // or when using some complex conditions
    public IQueryable<ListProductsProjection> SelectAll(int startIndex, int rowsNumber, string orderBy)
    {
        var products = ProductFacade.GetProducts();

        return Select(orderBy, products, startIndex, rowsNumber);
    }

    public IQueryable<ListProductsProjection> SelectAll(int startIndex, int rowsNumber, SearchFields searchFields, string orderBy)
    {
        var products = ProductFacade.GetProducts(searchFields);

        return Select(orderBy, products, startIndex, rowsNumber);
    }

    public IQueryable<ListProductsProjection> SelectAll(int startIndex, int rowsNumber, string searchString, string orderBy)
    {
        var products = ProductFacade.GetProducts(searchString);

        return Select(orderBy, products, startIndex, rowsNumber);
    }

...

    // Return count for the ObjectDataSource control
    public int GetCount()
    {
        return LastCount;
    }

    public int GetCount(SearchFields searchFields)
    {
        return LastCount;
    }

    public int GetCount(string searchString)
    {
        return LastCount;
    }
}

Simply put OdjectDataSource control at the page and init it with data source class name, select method name, parameters if needed etc. Set SortExpression's for columns in GridView control and enable paging. Assign it's DataSourceID and all is done. As practice showed the approach is reusable, extensible and worked fine for quite big ASP.NET application with about 50 pages with different grids (sometimes absolutely different content, sometimes not). But there we had everything a bit complicated and tuned for our needs (Interfaces and abstract classes for data source objectm IoC container etc. - and that's a different question).

Points to highlight

1. Ordering
Ordering is implemented via Dynamic LINQ. Dynamic LINQ (http://msdn.microsoft.com/en-us/vcsharp/bb894665.aspx) is a cool lib that allows creating IQueryable expressions from directly strings. The case it's very convenient - sorting by column name against LinqToSql query. That makes possible passing orderBy param directly to IQueryable.OrderBy() method.

2. Paging
Simple extension method to IQueryable interface and it also works great with LinqToSql:

public static class LinqExtensions
{
    public static IQueryable<T> TakePage<T<(this IQueryable<T> obj, int pageNum, int pageSize)
    {
        return obj.Skip(pageSize * --pageNum).Take(pageSize);
    }

    public static IQueryable<T> TakeIndex<T>(this IQueryable<T> obj, int startIndex, int rowsNumber)
    {
        return obj.Skip(startIndex).Take(rowsNumber);
    }
}

NOTE 1
Due to a bug in LinqToSql shipped with .NET 3.5 you may get a mess in the records you get for the first and for the second page if no default ordering is specified. So ALWAYS USE DEFAULT ORDERING WHEN WORKING WITH LINQ2SQL if different ordering is not specified.

NOTE 2
The problem I faced when worked with Dynamiq LINQ is that if the field name passed to OrderBy method is the name of some data type (e.g. GUID) - Dynamic LINQ fails

NOTE 3
In order to be capable of adding additional ordering fields when working with the dynamic version of OrderBy method, change this method this way:

public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string ordering, params object[] values)
{
    return (IOrderedQueryable<T>)OrderBy((IQueryable)source, ordering, values);
}

ASP.NET UpdatePanel, jQuery and Async postback

It's common to have jQuery plug-in (or any other JS plug-in) to attach it's behavior to some DOM objects upon document load. It's usual for many plug-ins to subscribe to events of the elements of interest. A simple example, attach multiply row values behavior via JS plug-in to ASP.NET GridView control (which is rendered as a <table> element):

After page is loaded a new behavior is available to the grid veiw (in this case the plug-in simply recalculates values for row cells upon user input).

The problem may arise when the GridView control is inside an UpdatePanel and an sync postback occurs. In this case the old table is deleted and new markup is rendered in it's place. This means that old DOM elements are deleted and no active event subscriptions exist anymore. After postback plug-in behavior is not active and should be reattached once again. To this MS Ajax client library can be used to handle async postback event when reposnse is received and rendered:

Note: if the init script creates any DOM elements outside the update panel then duplicate elements can be created after async postbacks. This should be handled separately.

ValidationSummary and displaying error message for CustomValidator on ServerValidate

Validation in ASP.NET WebForms is a rather solid block. Various validator controls can be easily fitted into page validation pipeline with absolutely no pain. One of the controls that perfectly fits ASP.NET validation system is ValidationSummary. It's aim is to collect error messages for all validators of a specific validation group when they signalize there's an error in user input.

The standard control works perfectly when dealing with client side validation. But it has a significant bug when doing server side validation with CustomValidator: no messages are displayed for failed controls.

On the picture below you may see a screen shot of a Wizard page that shows user input errors being validated client side:

In this sample mandatory fields can be easily validated client side. But what if you need to do server request to check something against DB? In this sample I'd like to check whether there're prices persisted to DB for all sites when a checkbox 'Same prices for all sites' is checked. The easiest way is to use the standard CustomValidator control and add necessary checks to CustomValidator.ServerValidate event (the control also perfectly works with async postbacks and UpdatePanels):

protected void sitesCustomValidator_ServerValidate(object sender, ServerValidateEventArgs e)
{
    if (samePriceCheckBox.Enabled)
    {
        e.IsValid = ..; // validation result 
    }
}

But even though when custom server validation fails (there're no prices for all sites) in code behind the Page.Valid property is 'false' and there's an opportunity to block next wizard step - validation summary doesn't show the error message for the CustomValidator failed:

The problem here is that ValidationSummary is only updated on postback to server. In the case when server side validation failed validators and their results are rendered to the response and NO ValidationSummary update occurs.

To override this drawback the following script can be used (jQuery is required):

$(document).ready(function () {
    var displayAlert = function () {
        if (typeof Page_Validators == 'undefined') return;

        var groups = [];

        for (i = 0; i < Page_Validators.length; i++){
            var validationGroup = typeof Page_Validators[i].validationGroup == "undefined" ? "": Page_Validators[i].validationGroup;
            if (!Page_Validators[i].isvalid) {
                if (!groups[validationGroup]) {
                    ValidationSummaryOnSubmit(validationGroup);
                    groups[validationGroup] = true;
                }
            }
        }
    };

    displayAlert();
    
    Sys.WebForms.PageRequestManager.getInstance().add_endRequest(function () {
        displayAlert();
    });
});

What it does is checking all validation controls on page load or async postback completion and triggering ValidationSummary updates for those validation groups that have failed validators.

The script reference can be added at a master page:

<body>
    <form runat="server">
        <asp:ScriptManager ID="scriptManager" runat="server" EnableScriptLocalization="true" 
        EnableScriptGlobalization="true" >
            <Scripts>
                <asp:ScriptReference Path="~/Scripts/jquery-1.4.2.min.js" />
                <asp:ScriptReference Path="~/Scripts/validationsummary.fix.js" />
            </Scripts>
        </asp:ScriptManager>
    ...

NOTE: I've tested this solution only for a case of a single custom validator with a server side validation event. ValidationSummary was placed at the bottom of a content page. Validation controls resided in a child user control of the page.

Entity Framework 4.0 and DB versioning

I've recently tried VS 2010 on a new commercial project (no more labs and investigating spherical horse in vacuum:). The application uses DB extensively, there're many various business entities and EF 4.0 was chosen to be used as an ORM solution and DAL for it .

In fact EF 4.0 introduces various new features (either comparing to previous version of EF or to other OR Mappers). That makes development easier and code cleaner:

• T4 Code generation templates
  
• POCO and AOP approach for tracking entities
  
• Lazy Loading (EF 3.5 didn't have it)
  
• Referential constraints for associations for lowers of FKs directly in the entity (to tell the truth I wanted to use this feature but the roadblock was with zero or one to many relations - looks likes there's a bug in the framework :)
• Better testability

Having many improvements in EF 4.0 I still didn't find a feature that could be very convenient in enterprise development: the capability to version the DB, generated from entity model. Regarding code generation capabilities of EF 4.0 they are the same as in EF 3.5: generate DB scripts from model OR generate model using an existing DB.



On most of my projects (so as on the current) there were no full understanding of how the data model will look in the end. A convenient work flow is that we have a set of user stories for the current milestone and create all the data structures for them. When a new milestone comes - new user stories, changes to the existing data model, new entities etc. When you have a DB for a great application with many data in the DB you can't simply drop and recreate it every time there're changes to data model. That's why DB versioning approach involving bunches of SQL scripts is usually used. The traditional workflow for DB versioning with ORM used is the following (I used it with nHibernate, LinqToSql and some other ORM):

1. Maintain DB via a set of change and create scripts
2. When data model is to be changed - add a new change script (when a table or relation is to be changed) or a create scripts (some tables are added)
3. Run scripts against DB server (either use some special versioning tool that may track which scripts are to be run or a simple approach with DBVersion table)
4. If necessary - also manually run some migration scripts if changes require that
   
5. Sync DB changes with O/R mapping in the code, adjust entity classes

As you may see, the approach desribed above is technically 'DB first' (I used Visio to draw the schema and generate scripts) . But with the EF I've decided to use the Entity Model designer to create the data model and use it for DB creation (model first approach). The question was how may I version the DB in this case. After I googled for a while, played with VS I didn't find any feature in EF 4.0 that could provided me with the necessary functionality in place. On the other hand VS 2010 presented a new and very powerful DB project.

Just a few words about VS 2010 DB projects. In a new version the approach to maintaing DB's has significantly changed. At a first glance you may see numerous SQL Script files that are spread over numerous folders under the project and decide that all of that is just a repository for scripts that are then directly executed against a DB:

But the fact is that all these scripts are not for executing against DB. They serve to describe internal DB structure within the project only. Each table, each key, any DB object has it's own script file with simple SQL create statement. You may also use a graphical schema editor to change the DB structure - all scripts will be changed accordingly. And here comes the power of the DB project: when you build the DB project against a specific DB on a server the build system creates the internal representation of the DB in the project, compares it with the target DB and dynamically creates a scripts with all necessary SQL statements . Executing this script leads to creating and changing the target DB and databases in the project and on a server are synchronized. You may find more info on new DB project here.

To cut long story short I've decided to use EF generation tool and VS 2010 DB project together and that's what I came to:

At first

1. Create the first version of Entity Model and generate SQL script for it
2. Run the script and create DB
3. Create a new DB project and generate it's contents using existing DB (created at previous step)

During further development

1. Change data model via EF designer (add/delete entities, associations, new fields etc.)
2. Generate scripts for the entire model
3. Manually sync changes with the DB project: you may either extract some parts of the generated scripts and copy past them, manually fix scripts within DB project, use Schema editor
4. Build and deploy the DB project to the used DB (Deploy action for DB project generates script and automatically executes it against DB if you check the corresponding option in the DB project properties window)
   

On the one hand this approach is not fully automated (you can't just change the model and click a button to have DB updated), on the other hand it gives good versioning and deployment capabilities (no more need to dig in hundreds of SQL scripts), also very simple rules for keeping the DB and Data Model synchronized.

NOTE #1. For continuous integration scenario:
I've faced a problem with building and publishing VS 2010 DB projects on a build server with no VS installed. The bunch was the following: SVN, CC.NET, nAnt, msbuild. The problem is that you can't do that. Even having .NET 4.0 SDK (I've use the latest available version in July) you'll not be able to publish and deploy the VS 2010 DB project without having VS 2010 installed.

NOTE #2. For those who's trying to change column name in a table via script in DB project :)
Use schema editor, this will not lead to dropping the column and creating a new one, it'll be only renamed.

Hope this post may help others.