Versioning and Pains - 2

Earlier I wrote about the issues with versioning assemblies when you decide to Strong Name your assemblies. While those pains are for real, I came across some additional patterns which can ease that pain considerably. Example1, Example2, Example3

Essentially, You want to do the following as much as your management team allows you to do that (yeah, its true that sometimes they decide the version numbers :D):

1. Change the AssemblyVersion value during internal builds every time so that your internal test team does not use an older version even by mistake. This works fine if you are working on an application which needs to go through future iterations as well. Once one of the versions is signed off by test team, that particular version becomes the de-facto support version and from thereon, you depend on AssemblyFileVersion for updates/patches. This will ensure that a new assembly does not break hell.

2. If any of your changes are going to necessitate significant changes, push it to next iteration :). Since the changes are significant, it is better to have a new version associated with those changes than to have one version reflect significant differences across its file versions. It may not be possible to avoid every time though but then I guess you are not in luck.

3. Use AssemblyInformationalVersion to tag the metadata information that you want to use for conversations. e.g. "1.0.-Iteration1".

For example, I can have following in my assembly.

[assembly: AssemblyVersion("1.3.6.1")]
[assembly: AssemblyFileVersion("1.3.2015.12121")]
[assembly: AssemblyInformationalVersion("1.3 iteration3")]

Notice that I have used a custom naming pattern in AssemblyFileVersion. It uses Year, Month, Day, Build count. You can use your own e.g. attaching changeset number.

Lastly, the sad part is that "*" is NOT supported in AssemblyFileVersion.

Tracing should be inbuilt in .NET framework

Microsoft's .NET platform has an amazingly performant and extremely useful class called Trace. I use this every time, I need to find some piece of information in the application for debugging/diagnosis purpose. Good thing is that you can leave the statements and have Trace turned off through configuration without worrying too much about its impact on the performance of the application.

Unfortunate part is that .NET framework still leaves you with a lot of work to do for basic TraceIn/TraceOut type of functionality. Either you end up wrapping your implementations through some known patterns like Decorator pattern or you end up making your classes have a complex class hierarchy where implementations are controlled through Base class. There are third party tools like PostSharp available in the market which let you implement this elegantly through AspectAttribute which essentially rewrites your IL code by adding the plumbing code that you would have added manually otherwise. While it is good, it seems a little intrusive to me because it makes your test cases dependent on the "additional rewritten" code and forces you to wade through that to test the actual code.

This brings me to a question - shouldn't .NET framework provide TraceIn/TraceOut functionality by default? I would suggest that it should be in-built and it should let users configure it through configuration file - much like IIS logs. Yes, there would be related security concerns but I am sure there must be clever ways to take care of that too.

Just a thought.

ConcurrentDictionary or Dictionary with lock

You may already know that there are 2 types of dictionary classes available in .NET framework - Dictionary and ConcurrentDictionary.

ConcurrentDictionary is possibly the right choice in cases when you have to maintain a static dictionary in an application where parallel access to the dictionary is possible. Surely you can do the same with a Dictionary instance but then you have to write lock statements around dictionary operation statements.

How about comparing the performance overhead between the two approaches. I wrote a simple utility that performs random read/write/delete operations on the two types and measure any difference. To my surprise, I did not see any big difference. Both approaches showed similar numbers :)

        private static bool shouldStop = false;
        private static object lockObj = new object();
        private static Dictionary dict = new Dictionary();
        private static ConcurrentDictionary dict2 = new ConcurrentDictionary();
        private static Random r = new Random();
        private static string[] OperationsList = new string[11]
        {
            "R", "W", "R", "W", "D", "R", "R", "R", "R", "W", "D"
        };
        static void Main(string[] args)
        {
            dict.Add("A", "B");
            dict2.GetOrAdd("A", "B");
            Console.WriteLine("Starting...");
            Task t = new Task(RunDictionaryTest);
            t.Start();
            Console.ReadKey();
            shouldStop = true;
            Console.WriteLine("Stopping...");
        }
        private static void RunDictionaryTest()
        {
            Action a = () =>
            {
                Console.WriteLine("Running...");
                while (!shouldStop)
                {
                    var number = r.Next(0, 10);
                    ////DoRandomDictionaryTryGetOperation(OperationsList[number]);
                    DoRandomDictionaryOperationWithContainsKey(OperationsList[number]);
                    ////DoRandomConcurrentDictionaryOperation(OperationsList[number]);
                    ////Thread.Sleep(10);
                }
            };
            Task[] tasks = new Task[100];
            for (int i = 0; i < 100; i++)
            {
                tasks[i] = new Task(a);
            }
            for (int i = 0; i < 100; i++)
            {
                tasks[i].Start();
            }
            Task.WaitAll(tasks);
        }
        private static void DoRandomConcurrentDictionaryOperation(string operationName)
        {
            string val;
            Stopwatch sw = new Stopwatch();
            sw.Start();
            switch (operationName)
            {
                case "R":
                    if (dict2.TryGetValue("Key", out val))
                    {
                        Console.WriteLine("Read:" + val);
                    }
                    else
                    {
                        Console.WriteLine("No Read");
                    }
                    break;
                case "W":
                    val = Guid.NewGuid().ToString();
                    string val2 = dict2.AddOrUpdate("Key", val, (k, v) => val);
                    if (!val.Equals(val2))
                    {
                        Console.WriteLine("no write");
                    }
                    break;
                case "D":
                    if(!dict2.TryRemove("Key", out val))
                    {
                        Console.WriteLine("no delete");
                    }
                    break;
            }
            sw.Stop();
            Console.WriteLine("Time taken" + sw.ElapsedMilliseconds.ToString());
        }
        private static void DoRandomDictionaryTryGetOperation(string operationName)
        {
            string val;
            Stopwatch sw = new Stopwatch();
            sw.Start();
            switch (operationName)
            {
                case "R":
                    lock (lockObj)
                    {
                        if (dict.TryGetValue("Key", out val))
                        {
                            Console.WriteLine("Read:" + val);
                        }
                        else
                        {
                            Console.WriteLine("No Read");
                        }
                    }
                    break;
                case "W":
                    lock (lockObj)
                    {
                        if (dict.TryGetValue("Key", out val))
                        {
                            Console.WriteLine("no write");
                        }
                        else
                        {
                            dict.Add("Key", Guid.NewGuid().ToString());
                        }
                    }
                    break;
                case "D":
                    lock (lockObj)
                    {
                        if (dict.TryGetValue("Key", out val))
                        {
                            dict.Remove("Key");
                        }
                        else
                        {
                            Console.WriteLine("no delete");
                        }
                    }
                    break;
            }
            sw.Stop();
            Console.WriteLine("Time taken" + sw.ElapsedMilliseconds.ToString());
        }
        private static void DoRandomDictionaryOperationWithContainsKey(string operationName)
        {
            Stopwatch sw = new Stopwatch();
            sw.Start();
            switch (operationName)
            {
                case "R":
                    lock (lockObj)
                    {
                        if (dict.ContainsKey("Key"))
                        {
                            Console.WriteLine("Read:" + dict["Key"]);
                        }
                        else
                        {
                            Console.WriteLine("No Read");
                        }
                    }
                    break;
                case "W":
                    lock (lockObj)
                    {
                        if (dict.ContainsKey("Key"))
                        {
                            Console.WriteLine("no write");
                        }
                        else
                        {
                            dict.Add("Key", Guid.NewGuid().ToString());
                        }
                    }
                    break;
                case "D":
                    lock (lockObj)
                    {
                        if (dict.ContainsKey("Key"))
                        {
                            dict.Remove("Key");
                        }
                        else
                        {
                            Console.WriteLine("no delete");
                        }
                    }
                    break;
            }
            sw.Stop();
            Console.WriteLine("Time taken" + sw.ElapsedMilliseconds.ToString());
        }

.NET applications and HTTP proxies

Accessing web resources using .NET's WebRequest class is such a common task that it is almost a given that it should qualify "write once and run everywhere". Yet, it does not. A simple code block to make HTTP requests start to fail as soon as it is moved to a non-dev machine :).

Almost all of the cases end up being related to the way HTTP proxies have been configured in a different environment and that the machines can not be make HTTP requests without going to proxy. The situation becomes grimmer when there are tricky configurations like URL1 is allowed to be accessed without proxy and URL2 is not allowed to be accessed without proxy. The mire!!

to the rescue then. The Internet is full with solutions that talk about this holy .NET configuration element at length and at the same time complain how Microsoft missed a trick by not enabling proxy as a default setting.

So what do we do in such cases?

1. If all the traffic is allowed through proxy then you can simply update the config file of the application with defaultProxy/Proxy element.

2. If there is a list of URLs that need to be bypassed, then additional element needs to be added.

Where do we add this entries?

Go in this order:

1. If it is for just one application, use configuration file of the application.

2. If it is to be done for all applications on the machine

• If it is a desktop application, update machine.config of the appropriate .NET version folder (there are pitfalls like Framework and Framework64 :) )
• If it is a web application hosted in IIS, update the web.config located at the same location where machine.config is generally found. I learned it the hard way.

Let us hope, no one runs into proxy issues again. Ever!!

Why do you have multiple X-Frame-Option values?

X-Frame-Options are extremely useful and are so good that ASP.NET MVC emits this header automatically unless explicitly specified. While it is a must for any enterprise grade Web application, ASP.NET MVC based implementation has an interesting issue - multiple values for the header which can conflict with the headers configured through web.config file and cause unwanted behavior.

When does ASP.NET MVC produce multiple values for this header? Well, the secret lies in the implementation of "AntiForgeryToken" method of HtmlHelper class. If you open up the implementation in reflector/ILSpy, you will notice the implementation of GetFormInputElement.

Essentially, if the page hierarchy has multiple instances of Html.AntiforgeryToken() (whether in same page or through multiple partial views), then ASP.NET MVC adds one header value for each instance, thus resulting in multiple values for same header. Of course, as you know, you can suppress that behavior but it is a painful discovery for the uninitiated.


For example: I created two partial views _First and _Second and each had this line added. The result :(



   X-Frame-Options: SAMEORIGIN, SAMEORIGIN

Bottom line: try to have just one reusable AntiforgeryToken per page. Technically speaking, one instance should be enough too. You should probably think again if you need more than one instance.

Dictionary.Keys.ElementAt method is slow (C# fun)

So, we were extending one the earlier implementations and required to loop through the Keys of a dictionary and perform operations that could be run in parallel. The code was added like following:

sw.Reset();
            sw.Start();
            Parallel.For(0, dictA.Keys.Count, (i) => {
                int x = dictA.Keys.ElementAt(i);
            });
            sw.Stop();
            Console.WriteLine("Time elapsed:" + sw.ElapsedMilliseconds);

On the surface this does not look all that bad. If you run this against a dictionary that has 100000 keys, this takes about 6-7 seconds on my machine. Clearly, something is amiss.

Looking through the implementation of ElementAt provides insight into why it is slow.

If you change the method to following, you can see why I say ElementAt is slow in this case.

sw.Reset();
            sw.Start();
            Parallel.ForEach(dictA.Keys.ToList(), (i) => {
                int x = i;
            });

            sw.Stop();
            Console.WriteLine("Time elapsed:" + sw.ElapsedMilliseconds);

Here are the results.

Time elapsed:6218
Time elapsed:6

“ and " are not the same

Here is a nifty bit that can help folks who build user interfaces (web, desktop or mobile). We seldom build UI that does not have text boxes. Text boxes, single line or multi line, allow users to enter details that are best suited as free text.

If you are a .NET developer then you would most probably be using DataAnnotation attributes on model properties that are bound with the UI elements. In case of free text, we end up using a RegularExpression Validators to enable automatic validation of user input.

In this particular case, Regular Expression is a white list of characters. And here is a catch. We end up creating regular expression using the characters typed on Visual Studio Text Editor which leaves out certain similar valid characters, mainly because they are not supported on the editor. Microsoft Word is one such case where the typed content is not same as similar content typed on notepad and therefore can cause unwanted behavior in system.

My colleague (Amit) helped me in figuring out the difference in character codes for the characters which look identical on surface.

Char Code Of – : 8211

Char Code Of ` : 8216

Char Code Of ’ : 8217

Char Code Of “ : 8220

Char Code Of ” : 8221

Char Code Of - : 45

Char Code Of ' : 39

Char Code Of ' : 39

Char Code Of " : 34

Char Code Of " : 34

So, next time onwards, when you create regular expression, think if you need to support content typed on or copied from MS word :)

C# fun

All of us know that look ups are faster than search. OK, but how much? Let us measure.

Stopwatch sw = new Stopwatch();

int count = 100000;

int[] arrayA = new int[count];

int[] arrayB = new int[count];

Dictionary dictA = new Dictionary();

Dictionary dictB = new Dictionary();

Hashtable hashA = new Hashtable();

Hashtable hashB = new Hashtable();

bool[] leftOnlyMatch = new bool[count];

for(int i = 0; i < count; i ++)

{

arrayA[i] = i;

dictA.Add(i, null);

hashA.Add(i, null);

arrayB[i] = i + 5000;

dictB.Add(i + 5000, null);

hashB.Add(i + 5000, null);

}

sw.Start();

for (int i = 0; i < count; i++)

{

var item = arrayA[i];

if(!arrayB.Any(x => x == item))

{

leftOnlyMatch[i] = true;

}

}

sw.Stop();

Console.WriteLine("Time elapsed:" + sw.ElapsedMilliseconds);

sw.Reset();

sw.Start();

for (int i = 0; i < count; i++)

{

var item = arrayA[i];

if (!dictB.ContainsKey(item))

{

leftOnlyMatch[i] = true;

}

}

sw.Stop();

Console.WriteLine("Time elapsed:" + sw.ElapsedMilliseconds);

sw.Reset();

sw.Start();

for (int i = 0; i < count; i++)

{

var item = arrayA[i];

if (!hashB.ContainsKey(item))

{

leftOnlyMatch[i] = true;

}

}

sw.Stop();

Console.WriteLine("Time elapsed:" + sw.ElapsedMilliseconds);

The output is (release mode, 32 GB RAM, 4 dual core processors)

Time elapsed:36244
Time elapsed:1
Time elapsed:2
Press any key to continue . . .

There you go. Dictionary seems slightly faster than Hashtable and it seems only logical.
Fun point

What's wrong with this code?

var testResult = test.Where(x => x == "1111").Select(y => y.Length).First();

There are 2 issues:

1. It assumes that Where clause will always return results.

2. It uses 3 operations for something that is possible with just 1.

var testResult = test.FirstOrDefault(x => x == "1111");

var length = testResult != null ? testResult.Length : 0;

Configuration and Settings: where do you keep these?

This has been a contentious point and I am sure everyone finds a suitable explanation to justify why they have a certain setting in the store of their choice. Generally in .NET world, we have 2-3 options to store application specific configurations/settings.

1. App.Config/Web.Config: These are the most used files to store any setting value whether it be through appSettings section or a custom section. It is super easy to encrypt the file and there is no overhead attached to dealing with encrypted configuration file. In case of ASP.NET deployments, it is slightly more cumbersome as the files needs to be copied across all servers and you need to ensure that all servers have the same file content.

2. External config/files: Less used. It is fairly useful in cases where a common setting value needs to be applied to multiple applications. It is also helpful in multi server deployments if you don't want to have one file replicated across servers. One of the downsides of this approach is that it can not be seamlessly used if it is encrypted. You need to apply hacks.

Both 1 & 2 do not allow strong typing in settings by default. You end up writing a custom configuration section to ensure it is strongly typed. If you don't, then your appSettings will be of keys that have comma (or semicolon or pipe) separated strings :)

3. Remote Store (Database/WebService): This is where things start to get interesting. The moment you decide to start moving configuration settings to a remote store like database, you start asking - is app.config required for storing database connection string only? In my opinion, that is a good enough use. In fact, I am of the opinion that some senior person (may be a guy who knows business side of things) reviews all of your appSettings to identify what are actual Business Configurations and what are simple Technical Configurations. Once you are through with that exercise, you would find it easy to decide what configuration needs to be moved to a remote store and what needs to reside in app.config. One crude way (but not the most effective way) to identify that is to identify the settings that would require you to restart process (appPool or service). Those settings should remain in app.config/web.config. One of the downside that you may find is that you need to find a custom encryption implementation to safeguard the settings that have been moved to a remote store (provided that those settings are actually required to be encrypted).


I personally prefer the usage of remote storage where I can store my complex configurations (and make those versioned) in JSON format. The only challenge that I have noticed in this case is that application needs to have some custom implementation to detect and refresh the configuration changes. I bet that it is not too hard to write one.


I have been part of some big teams and one of the things that I have learnt is that you need to learn to control the usage of app.config for storing configurations as developers may end up filling this file with just too many settings. In fact, I would argue that you need to question the modularity of design if the application's appSettings shows more than 5 settings (yeah, it is harsh). And the debate rages on...

Strange "method not found". Or common sense?

Imagine a situation. You have a big solution and it is already deployed. Then one day, you get to hear about an issue with one of the function implementation in one of the DLLs which required you to change type of a property. The change is not as drastic as changing an integer to string or vice versa. Rather it requires you to change type of a property from integer to double or double to integer.

We will consider change from integer to double for now. The solution compiles just fine and it looks like you are good to deploy the modified dll on running environment as there is no change in the other components that use this property. What happens next :)? For illustration, I created a Class Library named "ClassLibrary1" which has a class "Class1".

public class Class1

    {

        public int MyProperty { get; set; }

    }

I have another component "NewConsoleApplication" which has following method:

class Program

    {

        static void Main(string[] args)

        {

            double doubleValue = 0.4f;

            var c = new ClassLibrary1.Class1() { MyProperty = 100 };

            Console.WriteLine(doubleValue * c.MyProperty);

        }

}

It works just fine. Now you can change the type of property "MyProperty" to double and drop the compiled dll in the bin of "NewConsoleApplication". If you run the exe, you start to encounter weird error:

Method not found. Void set_MyProperty (Int32).

But you already changed the DLL. There is no property of type integer anymore. Why is someone trying to look for integer type?

The reason is simple. The IL code generated in NewConsoleApplication assumes that the property is still Integer and has its own IL code generated like that e.g. the code required to multiply casts the MyProperty to double before multiplying.

double num = 0.40000000596046448;
    Class1 @class = new Class1();
    @class.set_MyProperty(100);
    Class1 class2 = @class;
    Console.WriteLine(num * (double)class2.get_MyProperty());

So, there is the mystery.

Lesson: if there is a type change in one component, calling assemblies should be regenerated and replaced even if they have no change in their own code.

Reading encrypted content in external configuration file

.NET framework has a great configuration feature - external configuration files/external configuration source. It allows you to move certain configuration sections or parts of appSettings to an external file and then mention the path of external configuration file in the original configuration file. .NET framework merges the two configuration files and its ConfigurationManager API works seamlessly. Till you encrypt the external configuration file :).

 

So what is the solution? Custom code, anyone? Under the hood, .NET framework uses DpapiProtectedConfigurationProvider to some extent to read/write encrypted data. We can use the same along with ConfigXmlDocument. Here is one function that I wrote to read appSettings value from external configuration file that may or may not be encrypted:

 

public static class CommonConfigurationProvider

    {

        public static string GetAppSettingsValue(string key, string configurationFilePath)

        {

            ConfigXmlDocument configXmlDocument = new ConfigXmlDocument();

            configXmlDocument.Load(configurationFilePath);

 

            XmlNode appSettingsNode = null;

            if (configXmlDocument.DocumentElement.LocalName == "appSettings")

            {

                appSettingsNode = configXmlDocument.DocumentElement;

            }

            else

            {

                appSettingsNode = configXmlDocument.DocumentElement.SelectSingleNode("appSettings");

            }

 

            if (appSettingsNode == null)

            {

                throw new Exception("No appSettings section found!!");

            }

 

            XmlNode settingNode = null;

            if (appSettingsNode.Attributes["configProtectionProvider"] != null)

            {

                DpapiProtectedConfigurationProvider p = new DpapiProtectedConfigurationProvider();

                settingNode = p.Decrypt(appSettingsNode.SelectSingleNode("EncryptedData"));

            }

            else

            {

                settingNode = appSettingsNode;

            }

 

            if (settingNode != null)

            {

                var valueNode = settingNode.SelectNodes("add");

                if (valueNode != null)

                {

                    for (int i = 0; i < valueNode.Count; i++)

                    {

                        if (valueNode[i] != null && valueNode[i].Attributes["key"].InnerText == key)

                        {

                            return valueNode[i].Attributes["value"].InnerText;

                        }

                    }

                }

            }

 

            return null;

        }

    }

 

 

You can use both relative or absolute file path. Using this method is quite straightforward.

 

Console.WriteLine(ConfigurationManager.AppSettings["TestKey1"]);

            Console.WriteLine(CommonConfigurationProvider.GetAppSettingsValue("TestKey2", @"ExternalConfig\app.config"));

            Console.WriteLine(CommonConfigurationProvider.GetAppSettingsValue("TestKey3", @"ExternalConfig\app.config"));

The only caveat that I found was that it did not behave seamlessly. Of course, we can write a wrapper on ConfigurationManager class that takes care of that and makes usage seamless. Shouldn't be that hard.