1. 程式人生 > >Java日誌信息存庫(logback篇)

Java日誌信息存庫(logback篇)

values deb extends 數據連接 trigger 日誌系統 bst update array

一、Logback簡介

  Logback是由log4j創始人設計的又一個開源日誌組件。logback當前分成三個模塊:logback-core,logback- classic和logback-access。logback-core是其它兩個模塊的基礎模塊。logback-classic是log4j的一個 改良版本。此外logback-classic完整實現SLF4J API使你可以很方便地更換成其它日誌系統如log4j或JDK14 Logging。logback-access訪問模塊與Servlet容器集成提供通過Http來訪問日誌的功能。

二、編寫背景

  很不明白客戶為什麽要求將日誌信息寫入到數據庫中去,還要提供個頁面給系統使用人員查看相應日誌。作為一個業務人員真的就能看懂系統日誌上報錯信息是啥意思麽,個人深表懷疑。沒辦法,作為一枚屌絲程序猿,需求下來了只能硬著頭皮去開發。

三、編寫目的

  只相信一句話:好記性不如爛筆頭,何況我記性差到前一周寫的代碼到現在竟毫無印象的境地呢。

四、Java日誌信息存庫詳細解決方案

1.開發環境說明

  Eclipse+Tomcat6+JDK1.6+Oracle+logback1.1

2.Java日誌存庫實現方案

(1)使用logback組件默認的DBAppender類實現

  最初需求下來的時候想著logback應該有自己的寫數據庫的解決辦法,於是乎結合源碼及度娘終究還是找到了。在logback-classic-1.1.3.jar的ch/qos/logback/classic/db/script/路徑下找到了Oracle數據庫對應的建表語句腳本oracle.sql,其建表語句如下所示:

-- Logback: the reliable, generic, fast and flexible logging framework.
-- Copyright (C) 1999-2010, QOS.ch. All rights reserved.
--
-- See http://logback.qos.ch/license.html for the applicable licensing 
-- conditions.


-- This SQL script creates the required tables by ch.qos.logback.classic.db.DBAppender
-- -- It is intended for Oracle 9i, 10g and 11g databases. Tested on version 9.2, -- 10g and 11g. -- The following lines are useful in cleaning any previously existing tables --drop TRIGGER logging_event_id_seq_trig; --drop SEQUENCE logging_event_id_seq; --drop table logging_event_property; --drop table logging_event_exception; --drop table logging_event; CREATE SEQUENCE logging_event_id_seq MINVALUE 1 START WITH 1; CREATE TABLE logging_event ( timestmp NUMBER(20) NOT NULL, formatted_message VARCHAR2(4000) NOT NULL, logger_name VARCHAR(254) NOT NULL, level_string VARCHAR(254) NOT NULL, thread_name VARCHAR(254), reference_flag SMALLINT, arg0 VARCHAR(254), arg1 VARCHAR(254), arg2 VARCHAR(254), arg3 VARCHAR(254), caller_filename VARCHAR(254) NOT NULL, caller_class VARCHAR(254) NOT NULL, caller_method VARCHAR(254) NOT NULL, caller_line CHAR(4) NOT NULL, event_id NUMBER(10) PRIMARY KEY ); -- the / suffix may or may not be needed depending on your SQL Client -- Some SQL Clients, e.g. SQuirrel SQL has trouble with the following -- trigger creation command, while SQLPlus (the basic SQL Client which -- ships with Oracle) has no trouble at all. CREATE TRIGGER logging_event_id_seq_trig BEFORE INSERT ON logging_event FOR EACH ROW BEGIN SELECT logging_event_id_seq.NEXTVAL INTO :NEW.event_id FROM DUAL; END; / CREATE TABLE logging_event_property ( event_id NUMBER(10) NOT NULL, mapped_key VARCHAR2(254) NOT NULL, mapped_value VARCHAR2(1024), PRIMARY KEY(event_id, mapped_key), FOREIGN KEY (event_id) REFERENCES logging_event(event_id) ); CREATE TABLE logging_event_exception ( event_id NUMBER(10) NOT NULL, i SMALLINT NOT NULL, trace_line VARCHAR2(254) NOT NULL, PRIMARY KEY(event_id, i), FOREIGN KEY (event_id) REFERENCES logging_event(event_id) );

  該sql腳本共創建了3張表,一個序列和一個觸發器。其中主要日誌信息記錄在logging_event中,觸發器是在logging_event數據新增時,將創建的序列值賦值給logging_event表的event_id。

1)JDBC連接池方式

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>
<configuration>
    
    <!-- JDBC方式將日誌信息存入數據庫-->
    <appender name="DB" class="ch.qos.logback.classic.db.DBAppender">  
    <connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">
        <!--此處使用的是阿裏的數據連接池,也換成常用的C3P0數據連接池-->
        <dataSource class="com.alibaba.druid.pool.DruidDataSource">  
        <driverClass>oracle.jdbc.driver.OracleDriver</driverClass>  
        <url>jdbc:oracle:thin:@127.0.0.1:1521:orcl</url>  
        <user>tiger</user>  
        <password>123456</password>  
        </dataSource>  
    </connectionSource>  
    </appender>  
    
    <!--在logger標簽指定使用的Appender-->
    <logger name="dblog" level="info">
        <appender-ref ref="DB" />
    </logger>
  
    <root level="debug" >
    </root>
</configuration>
    

2) JNDI方式

A.Tomcat服務器安裝目錄/conf/server.xml配置JNDI信息

<Context debug="0" docBase="E:\prj_abic\src\trunk\fundats\ats-modules-webservice\target\ats-modules-webservice" path="/webservice" reloadable="true">
    <Resource auth="Container" driverClassName="oracle.jdbc.driver.OracleDriver" maxActive="30" maxIdle="30" name="jdbc/logging" password="123456" type="javax.sql.DataSource" url="jdbc:oracle:thin:@127.0.0.1:1521:orcl" username="tiger"/>
</Context>

B.Spring配置文件applicationContext.xml配置JNDI信息

<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName">                    
    <!--此處引用Tomcat服務器名稱為"jdbc/logging"的JNDI,WebLogic服務器可不填前綴直接寫jdbc/logging即可-->
    <value>java:comp/env/jdbc/logging</value>          
    </property>
</bean>

C.logback.xml文件配置JNDI信息

?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration>
<configuration>
    
    <!-- JNDI方式將日誌信息存入數據庫-->
   <appender name="DB" class="ch.qos.logback.classic.db.DBAppender">
    <connectionSource class="ch.qos.logback.core.db.JNDIConnectionSource">
        <!--此處引用Tomcat服務器名稱為"jdbc/logging"的JNDI,WebLogic服務器可不填前綴直接寫jdbc/logging即可-->
        <jndiLocation>java:comp/env/jdbc/logging</jndiLocation>
    </connectionSource>
   </appender>
    
    <!--在logger標簽指定使用的Appender-->
    <logger name="dblog" level="info">
        <appender-ref ref="DB" />
    </logger>
  
    <root level="debug" >
    </root>
</configuration>

  註:對於JNDI的配置不熟悉的,可以去找度娘幫忙或參考JNDI官方文檔,本文不做細究。

(2)使用自定義DBAppender類實現

  當初按上述的配置輕輕松松實現了日誌信息存庫,但天有不測風雲,完成後卻被告知客戶那邊任務觸發器不安全,禁用觸發器。得知此消息後心裏仿佛有千萬只草泥馬飛奔而過,沒辦法只有繼續想著怎麽去改造了。於是就只能去自定義個DBAppender了,既然是自定義就幹脆把其他兩張表直接刪了,僅使用logging_event表,其表結構如下:

字段名

中文說明

類型

為空

TIMESTMP

記錄時間

NUMBER(20)

N

FORMATTED_MESSAGE

格式化後的日誌信息

CLOB

N

LOGGER_NAME

執行記錄請求的logger

VARCHAR2(256)

N

LEVEL_STRING

日誌級別

VARCHAR2(256)

N

THREAD_NAME

日誌線程名

VARCHAR2(256)

Y

REFERENCE_FLAG

包含標識:1-MDC或上下文屬性;2-異常;3-均包含

INTEGER

Y

ARG0

參數1

VARCHAR2(256)

Y

ARG1

參數2

VARCHAR2(256)

Y

ARG2

參數3

VARCHAR2(256)

Y

ARG3

參數4

VARCHAR2(256)

Y

CALLER_FILENAME

文件名

VARCHAR2(256)

N

CALLER_CLASS

VARCHAR2(256)

N

CALLER_METHOD

方法名

VARCHAR2(256)

N

CALLER_LINE

行號

VARCHAR2(256)

N

EVENT_ID

主鍵ID

NUMBER(10)

N

  註:這個表結構與logback提供的默認表結構有細微差別,主要體現在字段類型上,由於FORMATTED_MESSAGE存儲的是詳細的日誌信息,字段類型VARCHAR2(4000)無法存儲大文本信息,所以直接改造成了CLOB類型。

1)自定義DBAppender類(ATSDBAppender.java)

package com.hundsun.fund.ats.core.system.loggingevent.dao;

import java.lang.reflect.Method;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.HashMap;import java.util.Map;
import static ch.qos.logback.core.db.DBHelper.closeStatement;
import static ch.qos.logback.core.db.DBHelper.closeConnection;
import ch.qos.logback.classic.db.DBHelper;
import ch.qos.logback.classic.db.names.DBNameResolver;
import ch.qos.logback.classic.db.names.DefaultDBNameResolver;
import ch.qos.logback.classic.spi.CallerData;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.db.DBAppenderBase;

public class ATSDBAppender extends DBAppenderBase<ILoggingEvent>{
      protected String insertSQL;
      protected static final Method GET_GENERATED_KEYS_METHOD;

      private DBNameResolver dbNameResolver;

      static final int TIMESTMP_INDEX = 1;
      static final int  FORMATTED_MESSAGE_INDEX  = 2;
      static final int  LOGGER_NAME_INDEX = 3;
      static final int  LEVEL_STRING_INDEX = 4;
      static final int  THREAD_NAME_INDEX = 5;
      static final int  REFERENCE_FLAG_INDEX = 6;
      static final int  ARG0_INDEX = 7;
      static final int  ARG1_INDEX = 8;
      static final int  ARG2_INDEX = 9;
      static final int  ARG3_INDEX = 10;
      static final int  CALLER_FILENAME_INDEX = 11;
      static final int  CALLER_CLASS_INDEX = 12;
      static final int  CALLER_METHOD_INDEX = 13;
      static final int  CALLER_LINE_INDEX = 14;
      static final int  EVENT_ID_INDEX  = 15;

      static final StackTraceElement EMPTY_CALLER_DATA = CallerData.naInstance();

      static {
          Method getGeneratedKeysMethod;
        try {
            getGeneratedKeysMethod = PreparedStatement.class.getMethod("getGeneratedKeys", (Class[]) null);
        } catch (Exception ex) {
            getGeneratedKeysMethod = null;
        }
        GET_GENERATED_KEYS_METHOD = getGeneratedKeysMethod;
      }

      public void setDbNameResolver(DBNameResolver dbNameResolver) {
        this.dbNameResolver = dbNameResolver;
      }

      @Override
      public void start() {
        if (dbNameResolver == null)
            dbNameResolver = new DefaultDBNameResolver();
        insertSQL = ATSSQLBuilder.buildInsertSQL(dbNameResolver);
        super.start();
      }
      
      public void append(ILoggingEvent eventObject) {
            Connection connection = null;
            PreparedStatement insertStatement = null;
            try {
              connection = connectionSource.getConnection();
              connection.setAutoCommit(false);

              insertStatement = connection.prepareStatement(getInsertSQL());

              synchronized (this) {
                  subAppend(eventObject, connection, insertStatement);
              }
              connection.commit();
            } catch (Throwable sqle) {
                addError("problem appending event", sqle);
            } finally {
                closeStatement(insertStatement);
                closeConnection(connection);
            }
      }
      
      @Override
      protected void subAppend(ILoggingEvent event, Connection connection,PreparedStatement insertStatement) throws Throwable {

          bindLoggingEventWithInsertStatement(insertStatement, event);
          bindLoggingEventArgumentsWithPreparedStatement(insertStatement, event.getArgumentArray());

          bindCallerDataWithPreparedStatement(insertStatement, event.getCallerData());

          int updateCount = insertStatement.executeUpdate();
          if (updateCount != 1) {
              addWarn("Failed to insert loggingEvent");
          }
      }

      protected void secondarySubAppend(ILoggingEvent event, 
              Connection connection,long eventId) throws Throwable {
      }

      void bindLoggingEventWithInsertStatement(PreparedStatement stmt,
              ILoggingEvent event) throws SQLException {
          stmt.setLong(TIMESTMP_INDEX, event.getTimeStamp());
          stmt.setString(FORMATTED_MESSAGE_INDEX, event.getFormattedMessage());
          stmt.setString(LOGGER_NAME_INDEX, event.getLoggerName());
          stmt.setString(LEVEL_STRING_INDEX, event.getLevel().toString());
          stmt.setString(THREAD_NAME_INDEX, event.getThreadName());
          stmt.setShort(REFERENCE_FLAG_INDEX, DBHelper.computeReferenceMask(event));
      }

      void bindLoggingEventArgumentsWithPreparedStatement(PreparedStatement stmt,
          Object[] argArray) throws SQLException {

          int arrayLen = argArray != null ? argArray.length : 0;

          for(int i = 0; i < arrayLen && i < 4; i++) {
              stmt.setString(ARG0_INDEX+i, asStringTruncatedTo254(argArray[i]));
          }
          if(arrayLen < 4) {
              for(int i = arrayLen; i < 4; i++) {
                  stmt.setString(ARG0_INDEX+i, null);
              }
          }
      }

      String asStringTruncatedTo254(Object o) {
         String s = null;
         if(o != null) {
             s= o.toString();
         }

        if(s == null) {
          return null;
        }
        if(s.length() <= 254) {
          return s;
        } else {
          return s.substring(0, 254);
        }
      }

      void bindCallerDataWithPreparedStatement(PreparedStatement stmt,
          StackTraceElement[] callerDataArray) throws SQLException {

        StackTraceElement caller = extractFirstCaller(callerDataArray);

        stmt.setString(CALLER_FILENAME_INDEX, caller.getFileName());
        stmt.setString(CALLER_CLASS_INDEX, caller.getClassName());
        stmt.setString(CALLER_METHOD_INDEX, caller.getMethodName());
        stmt.setString(CALLER_LINE_INDEX, Integer.toString(caller.getLineNumber()));
      }

      private StackTraceElement extractFirstCaller(StackTraceElement[] callerDataArray) {
        StackTraceElement caller = EMPTY_CALLER_DATA;
        if(hasAtLeastOneNonNullElement(callerDataArray))
          caller = callerDataArray[0];
        return caller;
      }

      private boolean hasAtLeastOneNonNullElement(StackTraceElement[] callerDataArray) {
        return callerDataArray != null && callerDataArray.length > 0 && callerDataArray[0] != null;
      }

      Map<String, String> mergePropertyMaps(ILoggingEvent event) {
          Map<String, String> mergedMap = new HashMap<String, String>();
        Map<String, String> loggerContextMap = event.getLoggerContextVO().getPropertyMap();
        Map<String, String> mdcMap = event.getMDCPropertyMap();
        if (loggerContextMap != null) {
            mergedMap.putAll(loggerContextMap);
        }
        if (mdcMap != null) {
            mergedMap.putAll(mdcMap);
        }

        return mergedMap;
      }

      @Override
      protected Method getGeneratedKeysMethod() {
        return GET_GENERATED_KEYS_METHOD;
      }

      @Override
      protected String getInsertSQL() {
        return insertSQL;
      }
}

2)自定義SQLBuilder類(ATSSQLBuilder.java)

package com.hundsun.fund.ats.core.system.loggingevent.dao;

import ch.qos.logback.classic.db.names.ColumnName;
import ch.qos.logback.classic.db.names.DBNameResolver;
import ch.qos.logback.classic.db.names.TableName;

public class ATSSQLBuilder {
      static String buildInsertSQL(DBNameResolver dbNameResolver) {
        StringBuilder sqlBuilder = new StringBuilder("INSERT INTO ");
        sqlBuilder.append(dbNameResolver.getTableName(TableName.LOGGING_EVENT)).append(" (");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.TIMESTMP)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.FORMATTED_MESSAGE)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.LOGGER_NAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.LEVEL_STRING)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.THREAD_NAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.REFERENCE_FLAG)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG0)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG1)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG2)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.ARG3)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_FILENAME)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_CLASS)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_METHOD)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.CALLER_LINE)).append(", ");
        sqlBuilder.append(dbNameResolver.getColumnName(ColumnName.EVENT_ID)).append(") ");
        sqlBuilder.append("VALUES (?, ?, ? ,?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,LOGGING_EVENT_ID_SEQ.nextval)");
        return sqlBuilder.toString();
    }
}

  註:類中LOGGING_EVENT_ID_SEQ序列需自行創建,logback默認建表語句中有該序列的創建語句,直接拿來使用即可。

3)logback.xml配置

  此時JDBC與JNDI方式配置只需要將原name為DB的appender標簽class屬性的值指向自定義DBAppender即可,其他配置不變,下述代碼為JDBC配置示例:

<!-- JDBC方式將日誌信息存入數據庫-->
<appender name="DB" class="com.hundsun.fund.ats.core.system.loggingevent.dao.ATSDBAppender">  
    <connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">  
    <dataSource class="com.alibaba.druid.pool.DruidDataSource">  
        <driverClass>oracle.jdbc.driver.OracleDriver</driverClass>  
        <url>jdbc:oracle:thin:@127.0.0.1:1521:orcl</url>  
        <user>tiger</user>  
        <password>123456</password>  
    </dataSource>  
    </connectionSource>  
</appender>

(3)測試類的編寫

package com.hundsun.fund.ats.modules.server.test;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LogbackTest {
    public static void main(String[] args) {
        //dblog為logback.xml中logger標簽name屬性的值
        Logger logger= LoggerFactory.getLogger("dblog");
        logger.debug("DEBUG級別信息");
        logger.warn("WARN級別信息");
        logger.info("INFO級別信息");
        logger.error("ERROR級別信息");
    }
}

五、總結

  文中涉及到的Logback日誌信息存庫的處理只是在源代碼的基礎上做了點小小的改動而已,並非完整地介紹該組件的功能。想要全面學習Logback日誌組件,請參考官方提供的源碼和相應的API幫助文檔。

Java日誌信息存庫(logback篇)