navigation
 Saturday, April 28, 2007

Behind this dropdownlist is some of the coolest code I have written for ages!

 

 

How can that be? Well, I learnt a lot about generics and anonymous delegates in the process… if you want to see some totally geeky code, read on,

Of course, there is a chance that for most of you this is all old hat, in which case you can go back to sleep…it was new to me for sure.

 

Still awake?

 

So, first of all, I had a lot of fun retrieving the data that populates this list.

The data that drives this DropDownList is a list of what I call FeatureGroups (the list has 2 elements, Defects and  Manage Projects), which has a Master Detail relationship to Features (a FeatureGroup contains 0 or more Features). It is in fact a List<AFFeatureGroup>, where each FeatureGroup contains a List<Feature>. Feature references back to its containing FeatureGroup through a FeatureGroupID member.

 

When I go to populate this list of FeatureGroups, I want to do it in one database call, so what I fetch is a flattened out Result Set that contains { FeatureGroup.ID, FeatureGroup.Name, Feature.ID, Feature.Name }.

I then need to iterate through that result set and for each FeatureGroup.ID I find, see if I already have created a FeatureGroup for that ID, if not, create it, then add the Feature to that new or existing feature group.

 

This is a standard pattern, you would have a list of your FeatureGroups in one hand, and be adding to it and searching in it as you go while looping through the flattened result set. Well here is how to spice it up!

 

It turns out that the generic List<T> type has a Find<T>( Predicate<T> match ). The predicate method is a method that takes a Predicate<T> and returns a bool. This method passed in as this parameter is called once on each item in the list, and the first time it returns true, that will be the item returned by Find. And, when invoking Find, you can pass in an anonymous delegate. In other words, you don’t have to pass in a new Predicate<T>( myCompareMethod ), you can just pass in the implementation of myCompareMethod…

This is how it looks:

 

private static List<AFFeatureGroup> GetFeatureGroupsFromFeatures( List<AFFeature> features )
{
  List<AFFeatureGroup> result = new List<AFFeatureGroup>();
  foreach (AFFeature f in features)
  {
    AFFeatureGroup fg = result.Find(
      delegate(AFFeatureGroup featureGroup) { return featureGroup.ID == f.FeatureGroupID; });
    if (fg == null)
    {
      fg = new AFFeatureGroup(f.FeatureGroupID, f.FeatureGroupName);
      result.Add(fg);
    }
    fg.Features.Add(f);
  }
  return result;
}

 

Let me break this down:

 

-          The method receives a List of AFFeatures, this is the flattened out result set containing one Feature in each row with its FeatureGroup as well.

-          The first line, result = new List<…> just creates an empty list of AFFeatureGroups that I will be returning once populated

-          Then,  I loop through all the features that I have fetched

-          In the loop, I call the Find method of List<AFFeatureGroup>, passing  my find implementation. Note two things:

o   a) I declare that the anonymous delegate accepts a AFFeatureGroup, which is mandated by the signature of Predicate<T>, as this code will be called once for each AFFeatureGroup in the list until a match is found

o   b) Inside the implementation curly braces, I can compare the featureGroup.ID of this passed in AFFeatureGroup to the value of f.FeatureGroupID. Think about that some! This is the powerful part! It’s kind of like in Delphi when you could pass in the address of a local nested method and that method would have access to any local variables or parameters in the enclosing method scope. The variable f is in the loop that encloses the call to Find, so in effect, what the compiler does is it captures the current value of f when creating the anonymous method, and makes it available inside the method body! Basically you have a dynamically created nested local method inside of the for loop. This is awesome compiler magic!

-          OK, so if a match is not found, I create a new AFFeatureGroup, and I then add it to the result list

-          If a match was found, or if I just created one, I can now add the feature itself to the feature group

 

And that is it: with this simple set of code I am doing a pretty complex construction of objects! This is what the language features of C# 3.0 will be taking one step further, where I will be able to use type inference to get rid of even more code above, for instance the declaration delegate(AFFeatureGroup featureGroup) can go away and I will just have a lambda expression there instead.

 

So, now for geek part 2…. I now have a method that will take a flattened list and build my two dimensional structure, but now I want to cache that list. Well, first I started with the classic approach, I look in the Cache object for a certain key, and if I don’t find it, I go fetch the data from the database (using the method in part 1), and store it in the cache. But this gets so repetitive, and I also wanted to centralize some of the aspects of the caching, like for how long the cached data is kept.

 

I started off along the lines that my colleague Adam Markowitz pioneered in the Velocity project, where you basically can create a CacheManager class that uses generics to do a type safe cache search. But then I figured if List<T> can accept a Predicate<T> in its Find method, why can’t I write a Cache.Get<T> that also accepts a T Fetcher<T> that will be invoked if there is no cache hit? Of course, Fetcher<T> might need some parameters, so I decided I would need two types of delegate, one that takes no parameters, and one that takes an instance of some arbitrary type D as a parameter (I could have been lazy and used an object instead of D, but what I love about Generics is the type safety it gives you without the performance drawbacks) :

 

public delegate T Fetcher<T, D> ( D data );
public delegate T ParameterlessFetcher<T>();
 

 

Now, my CacheManager can accept one of these 2 delegate types as a parameter in its GetFromCache method, and invoke it if the Cache doesn’t have the sought after data. For instance using the parameterless delegate:

 

public static T GetFromCache<T>(string key, ParameterlessFetcher<T> fetcher)
{
  T res = GetFromCache<T>(key);
  if (res == null)
  {
    res = fetcher();
    AddToCache(key, res);
  }
  return res;
}

 

The GetFromCache and AddToCache are the actual Cachce accessors:

 

public static void AddToCache(string key, object dataToCache)
{
  HttpContext.Current.Cache.Insert(key, dataToCache, null, DateTime.MaxValue, m_defaultTimeSpan);
}

public static T GetFromCache<T>(string key)
{
  return (T) HttpContext.Current.Cache[key];
 

 

Note that I centralize the timespan handling so that this is consistent:

 

private static TimeSpan m_defaultTimeSpan = new TimeSpan(0, 20, 0);

 

Here is how the parameter version looks:

 

public static T GetFromCache<T,D>(string key, Fetcher<T,D> fetcher, D data )
{
  T res = GetFromCache<T>(key);
  if (res == null)
  {
    res = fetcher(data);
    AddToCache(key, res);
  }
  return res;
}

 

And, putting it all together:

 

using System;
using System.Collections.Generic;
using System.Text;
using System.Web;
using System.Web.Caching;

namespace Falafel.ActiveFocus.Common
{
 public class AFCacheManager
 {
  private static TimeSpan m_defaultTimeSpan = new TimeSpan(0, 20, 0);

  public delegate T Fetcher<T, D> ( D data );
  public delegate T ParameterlessFetcher<T>();

  public static T GetFromCache<T,D>(string key, Fetcher<T,D> fetcher, D data )
  {
    T res = GetFromCache<T>(key);
    if (res == null)
    {
      res = fetcher(data);
      AddToCache(key, res);
    }
    return res;
  }

  public static T GetFromCache<T>(string key, ParameterlessFetcher<T> fetcher)
  {
    T res = GetFromCache<T>(key);
    if (res == null)
    {
      res = fetcher();
      AddToCache(key, res);
    }
    return res;
  }

  public static void AddToCache(string key, object dataToCache)
  {
    HttpContext.Current.Cache.Insert(key, dataToCache, null, DateTime.MaxValue, m_defaultTimeSpan);
  }

  public static T GetFromCache<T>(string key)
  {
    return (T) HttpContext.Current.Cache[key];
  }
 }
}
 

 

Now, what is exciting here is that I have a completely generic cache manager that can store and retrieve anything in a typesafe manner, using a delegate to do any fetching needed! Here is how I now use it to fetch that cached list of FeatureGroups:

 

public static List<AFFeatureGroup> CacheAllFeatureGroupsAndFeatures()
{
  return AFCacheManager.GetFromCache<List<AFFeatureGroup>>(
    CACHEKEY_FEATUREGROUPS,
    delegate() { return GetFeatureGroupsFromFeatures(new AFFeatureData().GetFeatures()); });
}

 

This is just amazingly concise to me! All I am supplying is :

 

-          the type of the cached object (a List<AFFeatureGroup>)

-          the Cache key to use (the constant CACHEKEY_FEATUREGROUPS)

-          the code to call when missing the data in the cache

 

Note that the code is now another anonymous delegate, it actually calls the method I explained in step 1 above that parses out the flat data. Here you also see how that method in turn gets the data by calling another method, GetFeatures. That in turn actually uses a whole similar approach of generics, that I wrote, which gives as a typesafe  way to convert result sets to lists of objects, but I won’t go there now!

 

And finally, to get down to that dropdownlist:

 

private void PopulateFeatureGroups()
{
  List<AFFeatureGroup> fglist = AFFeatureGroupData.CacheAllFeatureGroupsAndFeatures();
  ddlFeatureGroups.Items.Add(new ListItem("(All)", AFConsts.EMPTY_ITEM_VALUE.ToString()));
  foreach (AFFeatureGroup fg in fglist)
    ddlFeatureGroups.Items.Add(new ListItem(fg.Name, fg.ID.ToString()));
}

And it’s a wrap!

 

And what did I gain from all this convoluted programming asides from elegance and brevity?

I gained a reusable cache that can be tested and trusted and will save me a bunch of time caching other objects in a consistent and type safe manner. Now wasn’t that a geeky way to fill a drop down list?

posted on April 28, 2007  #    by John Waters  Comments [0]
 Tuesday, April 24, 2007
Yesterday I ran into an exception trying to read a non null DateTime value into a nullable type (DateTime?), using a helper method that was calling Convert.ChangeType. It turns out that this method, although very useful, doesn't support nullable types. However, I soon came across a useful blog that has a wrapper class that takes care of that particular problem, courtesy of Peter Johnsson.
posted on April 24, 2007  #    by John Waters  Comments [0]
 Tuesday, April 17, 2007

You might overlook the JavaScript debugging utility that's already built in to Visual Studio 2005: the Script Explorer window.  The Script Explorer can take care of the usual debugging tasks like stepping through code, adding watches and evaluating variables. 

To use the debugger in Internet Explorer navigate to the browser Tools | Internet Options | Advanced tab and make sure that "Disable script debugging" is turned off. 

Run your web application in Visual Studio 2005.  Then select the menu option for Debug | Windows | Script Explorer.  Notice in the background the tags for telerik RadEditor controls...

The first thing you notice in the Script Explorer window is a series of JavaScript and resource files that are currently loaded.  Double click on the aspx file you're currently working with and you will see the evaluated HTML returned from the server.  The RadEditor control now shows as its computed HTML, CSS and JavaScript that will actually be functioning in the browser. 

You can also navigate up to the script for the page and set breakpoints and watches.  When the JavaScript executes and hits your breakpoint you get all the usual Visual Studio debugging capabilities for free.

Next blog I'll show the excellent "Firebug" debugging utility for Firefox.  Firebug doesn't stop at just JavaScript but works with the entire stack of AJAX related technologies (and has a high cool-factor).

posted on April 17, 2007  #    by Noel Rice  Comments [0]

The Firebug debugger add-in for Firefox handles the entire stack of AJAX related technologies.  Ever wanted to tweak the margins in your style sheet while you watch the changes?  Profile a web page and see a visual representation of when scripts are loading and how big they are?  Watch the XmlHttpRequest (i.e. AJAX) requests move over the wire in real time?  You can do all this in Firebug, and of course you can step through your JavaScript code.  At Falafel we use this tool in our consulting work and telerik recommends Firebug for use in web applications using their RadControl suite.

Firebug is an innovative tool that handles usual tasks you would expect from a combined DOM explorer, AJAX/JavaScript profiler, and JavaScript debugger.  But it combines technologies in a new way that is definitely cool and a lot of fun to use.  This will take a few blogs to talk about in depth, but this should get you started.

Firebug only installs and runs in Firefox.  Get Firefox at http://www.mozilla.com/en-US/firefox if you don't already have it installed.  In Firefox download and install Firebug from http://www.getfirebug.com/Now run any page in Firefox and notice the green checkbox in the lower right hand corner.  Click it to start up Firebug for the page you're on. 

The Console tab is used for logging output.  The logging statements can be embedded in your script or run interactively on the Firebug interactive JavaScript command line.  The image below shows the special "dir" command line API dumping the contents of the "<body>" tag to the console.  A series of "console" commands output with visually helpful icons.  There's more on this tab that will wait for another blog (or jump ahead by checking out the API documentation at http://www.getfirebug.com/docs.html).

If you couldn't wait and are running Firebug right now, try clicking the Inspect button, then move your mouse on a web page.  The HTML and Style tabs will display the corresponding markup in real time as you move.  Click once on the page to stop inspecting.  Notice the crossed out items in the Style window?  The window is showing how styles are cascading and what styles are not in effect.  Try double clicking a style value -- you can edit the value and see the results immediately!  Also notice when your mouse cursor passes over a color or image tag that a thumbnail pops up.  Very smooth...

Try clicking on the Edit button (next to the Inspect button).  You can edit the HTML and see the results.  Feel free to add a completely different tag like the image below reading "Modify HTML on-the-fly!!".

By the way, when you click the refresh button, all changes go away.  The author of this tool, Joe Hewitt, mentions in his talk about the advanced features of Firebug (http://yuiblog.com/blog/2007/01/26/video-hewitt-firebug/) that this version of Firebug is not intended to be an editor, but more of an exploring and auditioning tool. 

Speaking of auditioning new settings, what about style layout settings?  The Layout tab shows the offset, margin, border and padding for each element you select.  In the browser you will see rules and other visual metric devices overlaying the web page.  Now try clicking one of the settings, say top padding for an image as shown below.  You don't have to enter a number off the keyboard.  Instead try the arrow keys to raise and lower values.  That way you can keep your eye on the layout until it's just right.

Firebugs profiling features show you the JavaScript and XHR requests going over the wire.  The example below is a demo using a set of telerik date controls using a RadAjaxManager to AJAX enable the whole process (thanks to John Waters for letting me steal the example).  In the Net tab we can select to see all or only certain traffic.  The image below shows all the JavaScript traffic; when it loads and how big each piece is.  For the web resources that contain images you can pass the mouse over to see thumbnails.  Click the plus sign to get the details like HTTP headers, requests and responses.

If we click the XHR tab we see only traffic initiated by the XMLHttpRequest object.  XMLHttpRequest is a major component of AJAX, so this feature is very important for evaluating web site performance with and without AJAX, tweaking AJAX performance, and even checking XHR traffic for security vulnerabilities. 

Firebug is after all a debugger.  All the capabilities you expect like step over, step into, run to line, step out, conditional breakpoints, watches, and automatic local variable display are there.  The conditional breakpoint window is a nice piece of UI programming in itself (see below).

Note: Thanks to Ramesh Theivendran for letting me steal the code for this XMLHttpRequest demo (I see a pattern forming here). 

This has been the briefest look at a tool that is sure to set the bar for all web debuggers.  In coming blogs I'll show Firebug in more depth, but until then I hope you try it yourself.

 |  |  | 
posted on April 17, 2007  #    by Noel Rice  Comments [0]
 Thursday, April 12, 2007

Ever need to show something in a databound gridview that isn't in your datasource? Don't worry, relax, forget about that datasource. Just create that extra row dynamically.

Lets assume I have a GridView with 2 columns. The first column shows the items on an order,  the second column shows the price of that item and the footer shows the order total.  How would I inject a row that shows the tax?

Something like this in the RowDataBound EventHandler would do the trick:

decimal orderTotal = 0.0m;
decimal tax = 0.0m;
protected void gvOrderDetail_RowDataBound(object sender, GridViewRowEventArgs e)
{
    //TODO: Calculate the orderTotal and the tax when RowType = DataRow
    if (e.Row.RowType == DataControlRowType.Footer)
    {
        //Create the Row
        GridViewRow row = new GridViewRow(-1, -1, DataControlRowType.DataRow, DataControlRowState.Normal);
        //Add the two Columns
        row.Cells.AddRange(CreateCells());
        //get a reference to the table that holds this row
        Table tbl = (e.Row.Parent as Table);
        //Add the row at the end of the list, but before the footer.
        tbl.Rows.AddAt(gvOrderDetail.Rows.Count + 1, row);

        //Don't forget to account for any changes in the footer. Since we added a row to show the tax,
        //that tax must also be accounted for in our footer. Calculating the orderTotal and the tax
        //is an exercise for the reader.
        Label lbl;
        lbl = (Label)e.Row.FindControl("lblTotal");
        lbl.Text = String.Format("{0:C}", (orderTotal + tax));
    }
}

private TableCell[] CreateCells()
{

    TableCell[] cells = new TableCell[2];
    TableCell cell;
    Label lbl;

    //The order item column
    cell = new TableCell();
    lbl = new Label();
    lbl.Text = "Sales Tax";
    cell.Controls.Add(lbl);
    cells[0] = cell;

    //The price column
    cell = new TableCell();
    lbl = new Label();
    lbl.Font.Bold = true;
    lbl.Text = tax.ToString("C");
    cell.HorizontalAlign = HorizontalAlign.Right;
    cell.Controls.Add(lbl);
    cells[1] = cell;

    return cells;
}

posted on April 12, 2007  #    by Mike Dugan  Comments [0]
 Wednesday, April 11, 2007
A nice blog post highlighting some aspects of Haskell and the new functional elements of C# 3.0 http://themechanicalbride.blogspot.com/2007/04/haskell-for-c-3-programmers.html
posted on April 11, 2007  #    by Adam Anderson  Comments [0]
 Monday, April 09, 2007

I love the new features of ADO.Net, the Object Datasources and the table adapters that the XSD designer can generate. But they kind of have a will of their own! They open and close connections as they see fit, and dont really care about transactions. If you have a set of related tables that you make changes to, and then want to write all of the changes to the database in one transaction, you need to do things a certain way to make it work...

What about the TransactionScope class?

My first attempt was to use the new TransactionScope functionality in ADO.Net 2.0. You can do something like this:

using (TransactionScope ts = new TransactionScope(TransactionScopeOption.Required))
{
   // do stuff that manipulates data
   ts.Complete();
}

While this is a very elegant solution, it is very tricky to get to work. The mechanism uses a Local Transaction Coordinator, but as soon as you execute your second database update command, it escalates the transaction to a distributed transaction (even if you are just doing sequential operations in the same SqlConnection), and tries to do 2 way communication with the DTC (Distributed Transaction Coordinator) running on the SqlServer. This communication turns out to be a major headache to get working. Asides from opening all kinds of firewall holes and configuring lots of complicated DTC options on both the server machine and your local machine, the server needs to be able to do a reverse DNS lookup to call back to your client machine, which proves to be unfeasable if you are developing from home across a VPN or have a dynamic IP address that the server cant lookup. It really only works when your client and the SQL Server are part of the same AD domain.

Long story short, TransactionScope is too complicated right now to actually use in normal development. So I wanted to stick with the traditional transaction support.

Getting the TableAdapters to use the correct Connection object

First of all, you need to make sure all of your TableAdapters use the same Connection object. TableAdapters are not an actual class in ADO.Net, they are code generated classes, kind of like typed DataSets, but unlike typed DataSets they dont inherit from some common TableAdapter class, they just inherit from Component. However, they do expose a Connection property, so I use that to hook up all my TableAdapters to the same Connection object. Something like this:

void InitTableAdapters( SqlConnection conn )
{
  taReportSetGroupAccountNotAssigned.Connection = conn;
  taReportSetGroupAccount.Connection = conn;
  taReportSetGroup.Connection = conn;
  taReportSet.Connection = conn;
}

Setting up the transaction

Now, you can encapsuluate your database operations in a transaction as follows:

SqlConnection conn = new SqlConnection( connStr );

InitTableAdapters( conn );

conn.Open();
SqlTransaction tran = conn.BeginTransaction();
try
{
  // manipulate data

  tran.Commit();
}
catch()
{
  tran.Rollback();
}
finally
{
  conn.Close();
}

However, there is still a problem with this. The SqlCommands inside the TableAdapter need to use the transaction of the Connection object in the Table Adapter, but the generated code does not do that for you. And you don't want to start messing with the generated code, or your changes will be wiped out next time it is regenerated. What to do?

Making the SqlCommand enlist in your transaction

First I tried subclassing the generated TabeAdapter, as I saw there was a protected CommandCollection property. However, it turned out that this collection only contains the Select Command, and I wanted the Insert, Update and Delete commands to participate in the transaction. These are part of the SqlDataAdapter member of the TableAdapter, which is private. Duh!

Well, partial classes come to the rescue! The TableAdapter class is generated as a partial class, which allows you to extend it by adding in additional methods in your own partial class. One gotcha is that you need to use the same namespace for your partial class as for the generated partial class, and that namespace is actually not the same as that of the typed dataset. It has .XXXTableAdapters added on to the end of it, where XXX is the name of your typed dataset. For instance, if your dataset namespace is Falafel.Data, and its name is FalafelDataset, then your TableAdapter namespace will be Falafel.Data.FalafelDatasetTableAdapters.

Armed with this knowledge, we can extend the partial class of each TableAdapter as follows:

namespace Falafel.Data.FalafelDataSetTableAdapters
{
  public partial class ReportSetGroupAccountTableAdapter
  {
    public void JoinTransaction( SqlTransaction tran )
    {  
      TransactionHelper.JoinTransaction(_adapter, tran);
    }
  }
  public partial class ReportSetGroupTableAdapter
  {
    public void JoinTransaction( SqlTransaction tran )
    { 
      TransactionHelper.JoinTransaction(_adapter, tran);
    }
  }
}

Here I am using my own helper class:

public class TransactionHelper
{
  public static void JoinTransaction( SqlAdapter adapter, SqlTransaction tran)
  {
      CheckTransaction(tran, adapter.InsertCommand);
      CheckTransaction(tran, adapter.DeleteCommand);
      CheckTransaction(tran, adapter.UpdateCommand);
      CheckTransaction(tran, adapter.SelectCommand);
  }
  public static void CheckTransaction( SqlAdapter adapter, SqlCommand cmd)
  {
    if (cmd != null)
      cmd.Transaction = tran;
  }
}

Putting it all together

Now that I have injected a public JoinTransaction method into each TableAdapter, I can get them to enlist in the transaction by surreptitiously calling the trojan horse helper:

void EnlistTableAdaptersInTransaction( SqlTransaction tran )
{
  taReportSetGroupAccountNotAssigned.JoinTransaction( tran );
  taReportSetGroupAccount.JoinTransaction( tran );
  taReportSetGroup.JoinTransaction( tran );
  taReportSet.JoinTransaction( tran );   
}

Adding a call to this in my update logic, this is the final sequence of events:

SqlConnection conn = new SqlConnection( connStr );
InitTableAdapters( conn );
conn.Open();
SqlTransaction tran = conn.BeginTransaction();
try
{
  EnlistTableAdaptersInTransaction( tran );
  // manipulate data
  tran.Commit();
}
catch()
{
  tran.Rollback();
}
finally
{
  conn.Close();
}

It's not the most elegant or compact code, but it gets the job done! I hope this will save you some time and allow you to productively use TableAdapters in transactions until the TransactionScope class becomes more development environment friendly!

posted on April 9, 2007  #    by John Waters  Comments [0]
 Tuesday, April 03, 2007

When I first installed Windows Vista I was a little worried that I would not be able to use all my development tools, like Microsoft Visual Studio 2005 and Microsoft SQL Server Management Studio, so I wanted to install an XP virtual machine just in case. I soon found out that the only version of Microsoft Virtual PC that would run on Vista was a beta of Virtual PC 2007, so I installed that, and it actually worked nicely.

Well, now the release version of Virtual PC 2007 is here. You can download the full version for free from the Microsoft Download Center. I downloaded it and installed it (you need to uninstall the beta version first – don’t worry, your virtual machines are left intact).

VPC 2007 runs on most host Windows versions, all the way back to Windows 98. It can of course host Vista, so this is a great way to start testing Vista and application compatibility in a sandbox environment.

posted on April 3, 2007  #    by John Waters  Comments [0]