Revelation: Why my diagnostic logging was not working using log4net TraceAppender

I thought I better share with people some information that I found which has shed some light on why I've had such a hard time getting native Azure diagnostic logging working.  Hopefully, if I hit enough key words here, someone will find this discussion in the future and may save you some time and heart ache.

Our application is a legacy ASP.Net application which I am porting to Azure.  The application makes extensive use of log4net and I decided that I would use the log4net TraceAppender to forward the logs to the Azure diagnostic listener.  Within my worker role, I was configuring my azure logs like so:

privatestaticvoid ConfigureLogging() {
	CrashDumps.EnableCollection(true);var diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();var directories = diagConfig.Directories;var infrastructureDiagnostics = diagConfig.DiagnosticInfrastructureLogs;var applicationLogs = diagConfig.Logs;var eventLogs = diagConfig.WindowsEventLog;

	SetTransferPeriod(applicationLogs, 1);
	SetTransferPeriod(directories, 1);
	SetTransferPeriod(infrastructureDiagnostics, 1);
	SetTransferPeriod(eventLogs, 1);

	SetFilterLevel(applicationLogs, LogLevel.Information);
	SetFilterLevel(eventLogs, LogLevel.Information);
	SetFilterLevel(infrastructureDiagnostics, LogLevel.Warning);

	DiagnosticMonitor.Start("DiagnosticsConnectionString", diagConfig);

	Log4NetHelper.ConfigureLog4Net();
}

privatestaticvoid SetFilterLevel(WindowsEventLogsBufferConfiguration eventLogs, LogLevel logLevel)
{
	eventLogs.DataSources.Add( "Application!*" );
	eventLogs.DataSources.Add( "System!*" );
	eventLogs.ScheduledTransferLogLevelFilter = logLevel;
}privatestaticvoid SetFilterLevel(BasicLogsBufferConfiguration infrastructureDiagnostics, LogLevel logLevel)
{
	infrastructureDiagnostics.ScheduledTransferLogLevelFilter = logLevel;
}privatestaticvoid SetTransferPeriod(DiagnosticDataBufferConfiguration directories, int minutes)
{var period = TimeSpan.FromMinutes(minutes);
	directories.ScheduledTransferPeriod = period;
}

Log4NetHelper.Configure uses the log4net programattic API to setup and configure a TraceAppender that captured all LogLevels (DEBUG or higher).  I also made sure that the follwing appeared in my Web.Config

<system.diagnostics><traceautoflush="false"indentsize="4"><listeners><clear/><addtype="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"name="AzureDiagnostics"><filter type="" /></add></listeners></trace></system.diagnostics>

However, this never worked.  I saw a number of forum posts that suggested that the Azure filter level should be "Undefined" - namely that I should replace

SetFilterLevel(applicationLogs, LogLevel.Information);

with

SetFilterLevel(applicationLogs, LogLevel.Undefined);

I had set my filter level to Information, cos our app does so much Debug level logging, that I thought I'd save money in storage by really only capturing the INFO or higher messages.  But in the interest of getting the blasted thing to work, I set to Undefined, and sure enough all the log message from all levels came through. Setting back to Information and NO messages come though.

I then noticed that when my logging was working (filter = Undefined) that ALL my log messages were in fact being logged at the Verbose level - even log4net calls to log.Error/log.Warn (exceptions messages being logged at Verbose is not a good sign!)

You have to realize, that my thought process was not structured at this point as I had no idea what was going on.  Until I found this article:

http://www.dotnetsolutions.co.uk/blog/archive/2010/02/22/windows-azure-diagnostics-%E2%80%93-why-the-trace-writeline-method-only-sends-verbose-messages/

While not related to log4net, it was an eye opener, and I used reflector to look at the TraceAppender, and I see they are using Trace.Write to write messages.  

In my mind, this explains why I was seeing the behaviour I was seeing:  AzureDiagnostic listener is converting all Trace.Write messags to Verbose (details in the above article) and log4net TraceAppender is converting all log messages to Trace.Write messages.  Setting the filter to Information was thus stripping all the calls.

My solution is to inherit TraceAppender and change their impl of Append to use a switch on the LogLevel:

using System.Diagnostics;
using log4net.Appender;
using log4net.Core;

namespace XXX.Azure
{
	public class AzureTraceAppender : TraceAppender
	{
		protected override void Append(LoggingEvent loggingEvent)
		{
			var level = loggingEvent.Level;
			var message = RenderLoggingEvent(loggingEvent);

			if (level >= Level.Error)
				Trace.TraceError(message);
			else if (level >= Level.Warn)
				Trace.TraceWarning(message);
			else if (level >= Level.Info)
				Trace.TraceInformation(message);
			else
				Trace.Write(message);

			if (ImmediateFlush)
				Trace.Flush();
		}
	}
}

With log4net configured to use this appender, then log messages appearing in the WADLogsTable are all appearing at the correct (or nearest appropriate) level

YMMV

Pete