Sunday 28 April 2013

Semantic Logging Application Block in Enterprise Library 6.0

Enterprise Library 6.0 has been released. You can find most of the stuff you need to know about it here. One of the most exciting feature of pack is Semantic Logging Application Block. It was available in its CTP form till now but is now an official release as part of EntLib.
 
At its core, it presents pretty powerful set of classes which are nicely decoupled from each other and allow LOB applications to log information (semantically, no pun intended) in 2 ways - in-process and out-of-process. In-Process logging uses ObservableEventListener to efficiently log information while Out-Of-Process logging uses combination of ETW infrastructure and Semantic Logging Service (available as a separate download here).
 
So, I decided to create a simple sample to experience its capabilities. Below is the code I used -
 
A simple Business Class -
public class BusinessClass
    {
        public void DoBusinessEvent(int token)
        {   
            if (token % 2 == 0) { 
                BusinessClassEventSource.Instance.EvenTokenEventOccurred(token); 
            }
            else { 
                BusinessClassEventSource.Instance.OddTokenEventOccurred(token); 
            }
            // Do something interesting like counting to 10.
        }
    }
 
A Custom Event Source -
 
public class BusinessClassEventSource : EventSource
    {
        private const int Even_Token_EventId = 1;
        private const int Odd_Token_EventId = 2;
        private static BusinessClassEventSource businessClassEventSource = new BusinessClassEventSource();
 
        private BusinessClassEventSource()
        {
           //nothig
        }
 
        public static BusinessClassEventSource Instance
        {
            get
            {
                return businessClassEventSource;
            }
        }
 
        public void EvenTokenEventOccurred(int token)
        {
            this.WriteEvent(Even_Token_EventId, token.ToString());
        }
 
        public void OddTokenEventOccurred(int token)
        {
            this.WriteEvent(Odd_Token_EventId, token.ToString());
        }
    }
 
A custom EventListener -
 
public class BusinessClassEventListener : EventListenerIObservable<EventEntry>
    {
        public IDisposable Subscribe(IObserver<EventEntry> observer)
        {
            return null;
        }
 
        protected override void OnEventWritten(EventWrittenEventArgs eventData)
        {
            Console.WriteLine(eventData.ToString());
        }
    }
 
And the main program -
 
class Program
    {
        static void Main(string[] args)
        {
            var listener1 = Microsoft.Practices.EnterpriseLibrary.SemanticLogging.ConsoleLog.CreateListener();
            listener1.EnableEvents(BusinessClassEventSource.Instance, EventLevel.LogAlways);
 
            var listener2 = new BusinessClassEventListener();
            listener2.EnableEvents(BusinessClassEventSource.Instance, EventLevel.LogAlways);
 
            BusinessClass businessClass = new BusinessClass();
            businessClass.DoBusinessEvent(100);
            businessClass.DoBusinessEvent(101);
            
            listener1.DisableEvents(BusinessClassEventSource.Instance);
            listener2.DisableEvents(BusinessClassEventSource.Instance);
 
            Console.ReadLine();
        }
    }
 
The good thing about SLAB is that we can write custom Formatters, Event sinks etc if out of the box capability does not suffice. SLAB provides default wiring to leverage SQL Server, Flat File, Console, Rolling Flat File, Windows Azure Table and others as storage for logged information.

Sunday 7 April 2013

Insufficient resources - MSMQ: Misleading error message

My team was recently performing performance tests on an integration solution which pushed messages on MSMQ queues hosted on a remote server via MSMQ send adapter of BizTalk Server. The functionality used the most plain vanilla implementation possible for transactional queues and it worked without a hitch on development and test environment.

It was a different story on the load test environment though. We were in for a surprise. After we had completed the first round of load testing of the system, we noticed quite a few warnings and errors in the event viewer and all of those pointed to "insufficient resources". The exact error message was -

There are insufficient resources to complete the send operation.
 For example, this could happen when you are trying to send a large message (message larger than 4095 KB) to a non-transactional queue. Large messages can be sent only to transactional queues.


This is actually a very misleading message, something we established soon after we inspected the messages in the suspended messages queue of BizTalk Server. Most of the messages were in the range of 2 KB - 20 KB. So size of the message was not an issue at all.

More intrigue awaited us as we were not able to send even a single message even after we cleared all the suspended messages from BizTalk Server. We checked the size of message log and directory of MSMQ on the destination server - everything was within the bounds of quota limit set of MSMQ and Queue respectively. Changing the destination queue did not help either - we got the same error message but that helped in establishing that the issue was not with the destination server. Somehow the servers that were running the send handler of MSMQ were choked as far as MSMQ transport was concerned.

So we did what every sane person does - turn to search engines. One of the top search results was "Insufficient Resources? Run away, run away!". It has 11 possible reasons listed in it. In our case, it turned out to be 3rd reason. When we checked the folders at "%windir%\system32\msmq\" on the machines that were running host instances of MSMQ send ports, it turned out that there were lot of MSMQ messages that could not be sent (because of our system's issues) and the cumulative size of MSMQ messages had reached beyond 1.7 GB. Once we cleared these folders on the client machines, the process started working again.

Learning - beware of misleading error messages.

Monday 1 April 2013

MSMQ vs SSSB vs Azure Service Bus

There are so many queuing solutions available, both commercial and open source, that it is not funny. Some of the queue solutions come in-built with Microsoft products such as MSMQ, SQL Server Service Broker, Azure Service Bus, Service Bus for Windows.
 
Like all other technology spaces that are crowded with similar solutions, each product/solution has its strength and weaknesses in this case too. Well, that makes things easier for us, the confused souls, in choosing the most appropriate option for ourselves. An interesting point to note is that all of these solutions provide functionalities that are similar in nature e.g. reliability of message delivery, transactional sanctity, poison message handling, journal. Here is my understanding of the three very viable options for a queue based implementation -
 
MSMQ - Possibly one of the oldest queuing solutions available on Windows ecosystem. It has evolved over the years and is perfect for cases when you want to maintain reliability of message delivery across servers that perform different jobs specially when one (or all) system falls under "occasionally offline" category. There is even a feature that allows MSMQ to be exposed over HTTP/HTTPS.
 
SQL Server Service Broker - SQL Server Service Broker is perfect choice for the cases where you need messages to be processed in a sequence inside SQL Server itself. Of course it can be used otherwise too.
 
Azure Service Bus/Service Bus for Windows - Both are great for the cases where two systems need to talk despite the network challenges e.g. Firewall across organizations etc. Both Azure Service Bus and Service Bus for Windows allow communication over HTTP and TCP. Also, they have Fabric Controller as their core component and therefore can easily scale with very little downtime. Not to forget that 1) it supports AMQP standard and therefore can easily be consumed from applications that run on other platforms 2) it supports Topics (publish/subscribe) in addition to the regular queues to allow multiple subscribers for an item 3) generally supports many messaging patterns 4) Supports better security mechanism via ACS/STS.