基於Log4Net簡單實現KafkaAppender
摘要：
背景
基於之前基於Log4Net本地日誌服務簡單實現
實現本地日誌服務，但是隨著專案開發演進，本地日誌服務滿足不了需求，譬如在預釋出環境或者生產環境，不可能讓開發人員登入檢視本地日誌檔案分析。
Kafka+ELK日誌服務套件，可以線上日誌服務可以解決...
背景
- 基於之前基於Log4Net本地日誌服務簡單實現 實現本地日誌服務，但是隨著專案開發演進，本地日誌服務滿足不了需求，譬如在預釋出環境或者生產環境，不可能讓開發人員登入檢視本地日誌檔案分析。
- Kafka+ELK日誌服務套件，可以線上日誌服務可以解決上述問題，並且提供豐富報表分析等等；
- 具體原始碼：MasterChief
- Nuget：Install-Package MasterChief.DotNet.Core.KafkaLog
- 歡迎Star，歡迎Issues；
原始碼
-
基於Log4Net來實現與kafka通訊Appender
public class KafkaAppender : AppenderSkeleton { #region Fields /// <summary> ///Kafka 生產者 /// </summary> private Producer _kafkaProducer; #endregion Fields #region Properties /// <summary> ///Brokers /// </summary> public string Brokers { get; set; } /// <summary> ///Topic /// </summary> public string Topic { get; set; } #endregion Properties #region Methods /// <summary> ///Initialize the appender based on the options set /// </summary> /// <remarks> ///<para> ///This is part of the <see cref="T:log4net.Core.IOptionHandler" /> delayed object ///activation scheme. The <see cref="M:log4net.Appender.AppenderSkeleton.ActivateOptions" /> method must ///be called on this object after the configuration properties have ///been set. Until <see cref="M:log4net.Appender.AppenderSkeleton.ActivateOptions" /> is called this ///object is in an undefined state and must not be used. ///</para> ///<para> ///If any of the configuration properties are modified then ///<see cref="M:log4net.Appender.AppenderSkeleton.ActivateOptions" /> must be called again. ///</para> /// </remarks> public override void ActivateOptions() { base.ActivateOptions(); InitKafkaProducer(); } /// <summary> ///Subclasses of <see cref="T:log4net.Appender.AppenderSkeleton" /> should implement this method ///to perform actual logging. /// </summary> /// <param name="loggingEvent">The event to append.</param> /// <remarks> ///<para> ///A subclass must implement this method to perform ///logging of the <paramref name="loggingEvent" />. ///</para> ///<para> ///This method will be called by <see cref="M:DoAppend(LoggingEvent)" /> ///if all the conditions listed for that method are met. ///</para> ///<para> ///To restrict the logging of events in the appender ///override the <see cref="M:PreAppendCheck()" /> method. ///</para> /// </remarks> protected override void Append(LoggingEvent loggingEvent) { try { var message = GetLogMessage(loggingEvent); var topic = GetTopic(loggingEvent); _ = _kafkaProducer.SendMessageAsync(topic, new[] {new Message(message)}); } catch (Exception ex) { ErrorHandler.Error("KafkaProducer SendMessageAsync", ex); } } /// <summary> ///Raises the Close event. /// </summary> /// <remarks> ///<para> ///Releases any resources allocated within the appender such as file handles, ///network connections, etc. ///</para> ///<para> ///It is a programming error to append to a closed appender. ///</para> /// </remarks> protected override void OnClose() { base.OnClose(); StopKafkaProducer(); } private string GetLogMessage(LoggingEvent loggingEvent) { var builder = new StringBuilder(); using (var writer = new StringWriter(builder)) { Layout.Format(writer, loggingEvent); if (Layout.IgnoresException && loggingEvent.ExceptionObject != null) writer.Write(loggingEvent.GetExceptionString()); return writer.ToString(); } } private string GetTopic(LoggingEvent loggingEvent) { return string.IsNullOrEmpty(Topic) ? Path.GetFileNameWithoutExtension(loggingEvent.Domain) : Topic; } /// <summary> ///初始化Kafka 生產者 /// </summary> private void InitKafkaProducer() { try { if (string.IsNullOrEmpty(Brokers)) Brokers = "http://localhost:9200"; if (_kafkaProducer == null) { var brokers = new Uri(Brokers); var kafkaOptions = new KafkaOptions(brokers) { Log = new KafkaLog() }; _kafkaProducer = new Producer(new BrokerRouter(kafkaOptions)); } } catch (Exception ex) { ErrorHandler.Error("InitKafkaProducer", ex); } } /// <summary> ///停止生產者 /// </summary> private void StopKafkaProducer() { try { _kafkaProducer?.Stop(); } catch (Exception ex) { ErrorHandler.Error("StopKafkaProducer", ex); } }#endregion Methods }
-
基於之前定義介面，來實現kafkaLogService
public sealed class KafkaLogService : ILogService { #region Constructors /// <summary> ///Initializes the <see cref="FileLogService" /> class. /// </summary> static KafkaLogService() { KafkaLogger = LogManager.GetLogger(KafkaLoggerName); } #endregion Constructors #region Fields /// <summary> ///Kafka logger name /// </summary> public const string KafkaLoggerName = "KafkaLogger"; /// <summary> ///Kafka logger /// </summary> public static readonly ILog KafkaLogger; #endregion Fields #region Methods /// <summary> ///Debug記錄 /// </summary> /// <param name="message">日誌資訊</param> public void Debug(string message) { if (KafkaLogger.IsDebugEnabled) KafkaLogger.Debug(message); } /// <summary> ///Debug記錄 /// </summary> /// <param name="message">日誌資訊</param> /// <param name="ex">異常資訊</param> public void Debug(string message, Exception ex) { if (KafkaLogger.IsDebugEnabled) KafkaLogger.Debug(message, ex); } /// <summary> ///Error記錄 /// </summary> /// <param name="message">日誌資訊</param> public void Error(string message) { if (KafkaLogger.IsErrorEnabled) KafkaLogger.Error(message); } /// <summary> ///Error記錄 /// </summary> /// <param name="message">日誌資訊</param> /// <param name="ex">異常資訊</param> public void Error(string message, Exception ex) { if (KafkaLogger.IsErrorEnabled) KafkaLogger.Error(message, ex); } /// <summary> ///Fatal記錄 /// </summary> /// <param name="message">日誌資訊</param> public void Fatal(string message) { if (KafkaLogger.IsFatalEnabled) KafkaLogger.Fatal(message); } /// <summary> ///Fatal記錄 /// </summary> /// <param name="message">日誌資訊</param> /// <param name="ex">異常資訊</param> public void Fatal(string message, Exception ex) { if (KafkaLogger.IsFatalEnabled) KafkaLogger.Fatal(message, ex); } /// <summary> ///Info記錄 /// </summary> /// <param name="message">日誌資訊</param> public void Info(string message) { if (KafkaLogger.IsInfoEnabled) KafkaLogger.Info(message); } /// <summary> ///Info記錄 /// </summary> /// <param name="message">日誌資訊</param> /// <param name="ex">異常資訊</param> public void Info(string message, Exception ex) { if (KafkaLogger.IsInfoEnabled) KafkaLogger.Info(message, ex); } /// <summary> ///Warn記錄 /// </summary> /// <param name="message">日誌資訊</param> public void Warn(string message) { if (KafkaLogger.IsWarnEnabled) KafkaLogger.Warn(message); } /// <summary> ///Warn記錄 /// </summary> /// <param name="message">日誌資訊</param> /// <param name="ex">異常資訊</param> public void Warn(string message, Exception ex) { if (KafkaLogger.IsWarnEnabled) KafkaLogger.Warn(message, ex); }#endregion Methods }
-
修改Log4Net.Config，定義Kafka的Topic以及Brokers
<appender name="KafkaAppender" type="MasterChief.DotNet.Core.KafkaLog.KafkaAppender, MasterChief.DotNet.Core.KafkaLog"> <param name="Topic" value="beats" /> <param name="Brokers" value="http://localhost:9092" /> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="發生時間：%date %newline事件級別：%-5level %newline事件來源：%logger%newline日誌內容：%message%newline" /> </layout> </appender>
使用
- 由於基於上篇說的日誌介面，所以可以通過Ioc切換，而且不影響在業務程式碼呼叫；
- 基於業務需求，您可以同時落地本地日誌，保證網路抖動或者不正常的時候能夠正常記錄日誌；
結語
- 小弟不才，大佬輕拍；