關於實現log4j2日誌脫敏的一種方案
阿新 • • 發佈：2018-12-30
情景
最近公司嚴格要求日誌脫敏，對於敏感欄位，諸如身份證號、手機號、銀行卡號等使用者資訊進行掩碼，保證日誌中沒有明文。
專案程式碼中列印日誌的地方形如：
logger.info("idCard:{},phone:{},mobile:{},name:{}", idCard, phone, mobile, name);相信很多javaer都是這麼做的，現在要對日誌進行掩碼，怎麼做？難道一行一行去改？
當然不行！
這種情況當然是…debug了！
DEBUG的重要性
第一次遇上這種問題，很多人可能手足無措，那麼是時候祭出終極大招：DEBUG。
從logger.info()進去，發現呼叫了AbstractLogger.logIfEnabled()方法，繼續往下走到AbstractLogger.logMessage()，這一方法有下面兩行程式碼：
final Message msg = messageFactory.newMessage(message, p0, p1, p2, p3);
logMessageSafely(fqcn, level, marker, msg, msg.getThrowable());首先看下AbstractLogger的私有屬性messageFactory是怎麼來的:
public AbstractLogger() {
this.name = getClass().getName();
this.messageFactory = createDefaultMessageFactory();
this 其三個構造方法，對於messageFactory都使用了createDefaultMessageFactory()方法
private static MessageFactory2 createDefaultMessageFactory() {
try {
final MessageFactory result = DEFAULT_MESSAGE_FACTORY_CLASS.newInstance();
return narrow(result);
} catch (final InstantiationException | IllegalAccessException e) {
throw new IllegalStateException(e);
}
}預設來自靜態屬性DEFAULT_MESSAGE_FACTORY_CLASS的例項：
public static final Class<? extends MessageFactory> DEFAULT_MESSAGE_FACTORY_CLASS =
createClassForProperty("log4j2.messageFactory", ReusableMessageFactory.class,
ParameterizedMessageFactory.class);仔細看看這個靜態方法：
private static Class<? extends MessageFactory> createClassForProperty(final String property,
final Class<ReusableMessageFactory> reusableParameterizedMessageFactoryClass,
final Class<ParameterizedMessageFactory> parameterizedMessageFactoryClass) {
try {
final String fallback = Constants.ENABLE_THREADLOCALS ? reusableParameterizedMessageFactoryClass.getName()
: parameterizedMessageFactoryClass.getName();
final String clsName = PropertiesUtil.getProperties().getStringProperty(property, fallback);
return LoaderUtil.loadClass(clsName).asSubclass(MessageFactory.class);
} catch (final Throwable t) {
return parameterizedMessageFactoryClass;
}
}直接從系統屬性log4j2.messageFactory獲取了工廠類全名（真的如此嗎？）
看到這裡以為要大功告成了，目前看上去是需要實現工廠類的介面，再將其配置給log4j2.messageFactory屬性即可。於是就有了工廠類如下：
public class DesensitizedParameterizedMessageFactory implements MessageFactory {
@Override
public Message newMessage(Object message) {
return null;
}
@Override
public Message newMessage(String message) {
return null;
}
@Override
public Message newMessage(String message, Object... params) {
return null;
}
}為啥非要叫XXParameterizedMessageFactory，具體細節請自行debug。
現在工廠有了，產品呢？
回到AbstractLogger.logMessage()方法，工廠類messageFactory產生的產品是Message，開啟看下，(肯定)是個介面。這裡我們只看由ParameterizedMessageFactory產生的ParameterizedMessage。
關注其幾個重要私有屬性：
//訊息模板，本例子中是idCard:{},phone:{},mobile:{},name:{}
private String messagePattern;
//引數陣列，本例子中是陣列{idCard, phone, mobile, name}
private transient Object[] argArray;
//格式化後的具體訊息
private String formattedMessage;
//特殊處理異常物件
private transient Throwable throwable;
//這個很重要，是標識位{}在messagePattern中的位置
private int[] indices;
//當前indices的位置，表示使用到第幾個標識位了
private int usedCount;我們是希望在格式化的時候，就對敏感欄位進行掩碼，因此重點關注org.apache.logging.log4j.message.ParameterizedMessage#formatTo這個方法，debug進去後執行了org.apache.logging.log4j.message.ParameterFormatter#formatMessage2這個方法：
static void formatMessage2(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
recursiveDeepToString(arguments[i], buffer, null);
}
buffer.append(messagePattern, previous, messagePattern.length());
}如上，我們看到在這裡對arguments陣列進行了迴圈，將其值最終append到目標StringBuilder，如果我們在這裡能判斷模板包含了什麼敏感字串，就可以在這裡使用不同的掩碼方式對欄位進行處理。
怎麼獲取當前是哪個字讀在append了呢？
前面提到的indices這個時候就可以用了，那麼我們可以從模板messagePattern截取出當前要append這段字串：
String word = messagePattern.substring(previous, indices[i]);為了能達到掩碼的目的，我們加了一個列舉類來完成對不同欄位進行掩碼操作：
public enum DesensitizedWords {
idCard("idCard", 6, 3),
phone("phone", 3, 4),
mobile("mobile", 3, 4),
name("name", 0, 1),
;
private String word;
private int front;
private int tail;
DesensitizedWords(String word, int front, int tail) {
this.word = word;
this.front = front;
this.tail = tail;
}
public static String desensitize(String word, String val) {
for (DesensitizedWords item : DesensitizedWords.values()) {
if (word.contains(item.word)) {
return hide(val, item.front, item.tail, '*');
}
}
return val;
}
public static String hide(String src, int front, int tail, char replace) {
if (null == src)
return src;
int len = src.length();
if (front > len || tail > len) {
return src;
}
StringBuilder builder = new StringBuilder();
if (front > 0) {
builder.append(src.substring(0, front));
} else {
front = 0;
}
String tailStr = "";
if (tail > 0) {
tailStr = src.substring(src.length() - tail, src.length());
} else {
tail = 0;
}
int padding = len - front - tail;
if (padding > 0) {
for (int i = 0; i < padding; i++) {
builder.append(replace);
}
}
builder.append(tailStr);
return builder.toString();
}
}然後將org.apache.logging.log4j.message.ParameterFormatter#formatMessage2方法重寫（實際上由於ParameterFormatter是final並且是package可見的，因此我們將org.apache.logging.log4j.message.ParameterizedMessage整個類copy一份為DesensitizedParameterizedMessage，並且將ParameterFormatter作為DesensitizedParameterizedMessage的內部類，並重寫formatMessage2方法如下）：
static void formatMessage2(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
String word = messagePattern.substring(previous, indices[i]);
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
StringBuilder builder = new StringBuilder();
recursiveDeepToString(arguments[i], builder, null);
buffer.append(DesensitizedWords.desensitize(word, builder.toString()));
}
buffer.append(messagePattern, previous, messagePattern.length());
}通過上面的改寫（copy），我們已經將最基本的類創建出來，現在要考慮如何使用他們，前文有講到使用系統屬性log4j2.messageFactory來配置工廠類，實際上並沒有起作用，再次debug後發現，我們需要在resources下加一個log4j2.component.properties檔案來配置這個屬性才能生效，檔案配置如下：
log4j2.messageFactory=com.X.Xx.DesensitizedParameterizedMessageFactory記得將工廠類產生的產品改為DesensitizedParameterizedMessage喲。
效果
測試用例
String idCard = "210002197812129527";
String name = "周星星";
String phone = "14700000000";
String mobile = "14700000000";
logger.info("明文資料:{},{},{},{}", idCard, phone, mobile, name);
logger.info("idCard:{},phone:{},mobile:{},name:{}", idCard, phone, mobile, name);
logger.error("error-idCard:{},error-phone:{},error-mobile:{},error-name:{}", idCard, phone, mobile, name);
logger.warn("warn-idCard:{},warn-phone:{},warn-mobile:{},warn-name:{}", idCard, phone, mobile, name);
logger.debug("debug-idCard:{},debug-phone:{},debug-mobile:{},debug-name:{}", idCard, phone, mobile, name);效果如下
2017-12-06 10:48:39.490 [http-bio-8080-exec-1:563179] [DemoController.java:83] - [INFO] 明文資料:210002188012120359,14700000000,14700000000,周星星
2017-12-06 10:48:39.491 [http-bio-8080-exec-1:563180] [DemoController.java:84] - [INFO] idCard:210002*********359,phone:147****0000,mobile:147****0000,name:**星
2017-12-06 10:48:39.491 [http-bio-8080-exec-1:563180] [DemoController.java:85] - [ERROR] error-idCard:210002*********359,error-phone:147****0000,error-mobile:147****0000,error-name:**星
2017-12-06 10:48:39.492 [http-bio-8080-exec-1:563181] [DemoController.java:86] - [WARN] warn-idCard:210002*********359,warn-phone:147****0000,warn-mobile:147****0000,warn-name:**星
2017-12-06 10:48:39.493 [http-bio-8080-exec-1:563182] [DemoController.java:87] - [DEBUG] debug-idCard:210002*********359,debug-phone:147****0000,debug-mobile:147****0000,debug-name:**星
原始碼
終於到了大家喜歡的環節（沒有做裁剪，自行操作）。
DesensitizedParameterizedMessage.java
import org.apache.logging.log4j.message.Message;
import org.apache.logging.log4j.util.StringBuilderFormattable;
import java.text.SimpleDateFormat;
import java.util.*;
public class DesensitizedParameterizedMessage implements Message, StringBuilderFormattable {
// Should this be configurable?
private static final int DEFAULT_STRING_BUILDER_SIZE = 255;
/**
* Prefix for recursion.
*/
public static final String RECURSION_PREFIX = ParameterFormatter.RECURSION_PREFIX;
/**
* Suffix for recursion.
*/
public static final String RECURSION_SUFFIX = ParameterFormatter.RECURSION_SUFFIX;
/**
* Prefix for errors.
*/
public static final String ERROR_PREFIX = ParameterFormatter.ERROR_PREFIX;
/**
* Separator for errors.
*/
public static final String ERROR_SEPARATOR = ParameterFormatter.ERROR_SEPARATOR;
/**
* Separator for error messages.
*/
public static final String ERROR_MSG_SEPARATOR = ParameterFormatter.ERROR_MSG_SEPARATOR;
/**
* Suffix for errors.
*/
public static final String ERROR_SUFFIX = ParameterFormatter.ERROR_SUFFIX;
private static final long serialVersionUID = -665975803997290697L;
private static final int HASHVAL = 31;
// storing JDK classes in ThreadLocals does not cause memory leaks in web apps, so this is okay
private static ThreadLocal<StringBuilder> threadLocalStringBuilder = new ThreadLocal<>();
private String messagePattern;
private transient Object[] argArray;
private String formattedMessage;
private transient Throwable throwable;
private int[] indices;
private int usedCount;
public DesensitizedParameterizedMessage(final String messagePattern, final Object[] arguments, final Throwable throwable) {
this.argArray = arguments;
this.throwable = throwable;
init(messagePattern);
}
public DesensitizedParameterizedMessage(final String messagePattern, final Object... arguments) {
this.argArray = arguments;
init(messagePattern);
}
/**
* Constructor with a pattern and a single parameter.
*
* @param messagePattern The message pattern.
* @param arg The parameter.
*/
public DesensitizedParameterizedMessage(final String messagePattern, final Object arg) {
this(messagePattern, new Object[]{arg});
}
public DesensitizedParameterizedMessage(final String messagePattern, final Object arg0, final Object arg1) {
this(messagePattern, new Object[]{arg0, arg1});
}
private void init(final String messagePattern) {
this.messagePattern = messagePattern;
final int len = Math.max(1, messagePattern == null ? 0 : messagePattern.length() >> 1); // divide by 2
this.indices = new int[len]; // LOG4J2-1542 ensure non-zero array length
final int placeholders = ParameterFormatter.countArgumentPlaceholders2(messagePattern, indices);
initThrowable(argArray, placeholders);
this.usedCount = Math.min(placeholders, argArray == null ? 0 : argArray.length);
}
private void initThrowable(final Object[] params, final int usedParams) {
if (params != null) {
final int argCount = params.length;
if (usedParams < argCount && this.throwable == null && params[argCount - 1] instanceof Throwable) {
this.throwable = (Throwable) params[argCount - 1];
}
}
}
@Override
public String getFormat() {
return messagePattern;
}
@Override
public Object[] getParameters() {
return argArray;
}
@Override
public Throwable getThrowable() {
return throwable;
}
@Override
public String getFormattedMessage() {
if (formattedMessage == null) {
final StringBuilder buffer = getThreadLocalStringBuilder();
formatTo(buffer);
formattedMessage = buffer.toString();
}
return formattedMessage;
}
private static StringBuilder getThreadLocalStringBuilder() {
StringBuilder buffer = threadLocalStringBuilder.get();
if (buffer == null) {
buffer = new StringBuilder(DEFAULT_STRING_BUILDER_SIZE);
threadLocalStringBuilder.set(buffer);
}
buffer.setLength(0);
return buffer;
}
@Override
public void formatTo(final StringBuilder buffer) {
if (formattedMessage != null) {
buffer.append(formattedMessage);
} else {
if (indices[0] < 0) {
ParameterFormatter.formatMessage(buffer, messagePattern, argArray, usedCount);
} else {
ParameterFormatter.formatMessage2(buffer, messagePattern, argArray, usedCount, indices);
}
}
}
public static String format(final String messagePattern, final Object[] arguments) {
return ParameterFormatter.format(messagePattern, arguments);
}
@Override
public boolean equals(final Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
final DesensitizedParameterizedMessage that = (DesensitizedParameterizedMessage) o;
if (messagePattern != null ? !messagePattern.equals(that.messagePattern) : that.messagePattern != null) {
return false;
}
if (!Arrays.equals(this.argArray, this.argArray)) {
return false;
}
//if (throwable != null ? !throwable.equals(that.throwable) : that.throwable != null) return false;
return true;
}
@Override
public int hashCode() {
int result = messagePattern != null ? messagePattern.hashCode() : 0;
result = HASHVAL * result + (argArray != null ? Arrays.hashCode(argArray) : 0);
return result;
}
public static int countArgumentPlaceholders(final String messagePattern) {
return ParameterFormatter.countArgumentPlaceholders(messagePattern);
}
public static String deepToString(final Object o) {
return ParameterFormatter.deepToString(o);
}
public static String identityToString(final Object obj) {
return ParameterFormatter.identityToString(obj);
}
@Override
public String toString() {
return "ParameterizedMessage[messagePattern=" + messagePattern + ", stringArgs=" +
Arrays.toString(argArray) + ", throwable=" + throwable + ']';
}
static class ParameterFormatter {
/**
* Prefix for recursion.
*/
static final String RECURSION_PREFIX = "[...";
/**
* Suffix for recursion.
*/
static final String RECURSION_SUFFIX = "...]";
/**
* Prefix for errors.
*/
static final String ERROR_PREFIX = "[!!!";
/**
* Separator for errors.
*/
static final String ERROR_SEPARATOR = "=>";
/**
* Separator for error messages.
*/
static final String ERROR_MSG_SEPARATOR = ":";
/**
* Suffix for errors.
*/
static final String ERROR_SUFFIX = "!!!]";
private static final char DELIM_START = '{';
private static final char DELIM_STOP = '}';
private static final char ESCAPE_CHAR = '\\';
private static ThreadLocal<SimpleDateFormat> threadLocalSimpleDateFormat = new ThreadLocal<>();
private ParameterFormatter() {
}
static int countArgumentPlaceholders(final String messagePattern) {
if (messagePattern == null) {
return 0;
}
final int length = messagePattern.length();
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern.charAt(i + 1) == DELIM_STOP) {
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static int countArgumentPlaceholders2(final String messagePattern, final int[] indices) {
if (messagePattern == null) {
return 0;
}
final int length = messagePattern.length();
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
indices[0] = -1; // escaping means fast path is not available...
result++;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern.charAt(i + 1) == DELIM_STOP) {
indices[result] = i;
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static int countArgumentPlaceholders3(final char[] messagePattern, final int length, final int[] indices) {
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern[i];
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern[i + 1] == DELIM_STOP) {
indices[result] = i;
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static String format(final String messagePattern, final Object[] arguments) {
final StringBuilder result = new StringBuilder();
final int argCount = arguments == null ? 0 : arguments.length;
formatMessage(result, messagePattern, arguments, argCount);
return result.toString();
}
static void formatMessage2(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
String word = messagePattern.substring(previous, indices[i]);
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
StringBuilder builder = new StringBuilder();
recursiveDeepToString(arguments[i], builder, null);
buffer.append(DesensitizedWords.desensitize(word, builder.toString()));
}
buffer.append(messagePattern, previous, messagePattern.length());
}
static void formatMessage3(final StringBuilder buffer, final char[] messagePattern, final int patternLength,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null) {
return;
}
if (arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
recursiveDeepToString(arguments[i], buffer, null);
}
buffer.append(messagePattern, previous, patternLength);
}
static void formatMessage(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int escapeCounter = 0;
int currentArgument = 0;
int i = 0;
final int len = messagePattern.length();
for (; i < len - 1; i++) { // last char is excluded from the loop
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
escapeCounter++;
} else {
if (isDelimPair(curChar, messagePattern, i)) { // looks ahead one char
i++;
// write escaped escape chars
writeEscapedEscapeChars(escapeCounter, buffer);
if (isOdd(escapeCounter)) {
// i.e. escaped: write escaped escape chars
writeDelimPair(buffer);
} else {
// unescaped
writeArgOrDelimPair(arguments, argCount, currentArgument, buffer);
currentArgument++;
}
} else {
handleLiteralChar(buffer, escapeCounter, curChar);
}
escapeCounter = 0;
}
}
handleRemainingCharIfAny(messagePattern, len, buffer, escapeCounter, i);
}
private static boolean isDelimPair(final char curChar, final String messagePattern, final int curCharIndex) {
return curChar == DELIM_START && messagePattern.charAt(curCharIndex + 1) == DELIM_STOP;
}
private static void handleRemainingCharIfAny(final String messagePattern, final int len,
final StringBuilder buffer, final int escapeCounter, final int i) {
if (i == len - 1) {
final char curChar = messagePattern.charAt(i);
handleLastChar(buffer, escapeCounter, curChar);
}
}
private static void handleLastChar(final StringBuilder buffer, final int escapeCounter, final char curChar) {
if (curChar == ESCAPE_CHAR) {
writeUnescapedEscapeChars(escapeCounter + 1, buffer);
} else {
handleLiteralChar(buffer, escapeCounter, curChar);
}
}
private static void handleLiteralChar(final StringBuilder buffer, final int escapeCounter, final char curChar) {
writeUnescapedEscapeChars(escapeCounter, buffer);
buffer.append(curChar);
}
private static void writeDelimPair(final StringBuilder buffer) {
buffer.append(DELIM_START);
buffer.append(DELIM_STOP);
}
private static boolean isOdd(final int number) {
return (number & 1) == 1;
}
private static void writeEscapedEscapeChars(final int escapeCounter, final StringBuilder buffer) {
final int escapedEscapes = escapeCounter >> 1; // divide by two
writeUnescapedEscapeChars(escapedEscapes, buffer);
}
private static void writeUnescapedEscapeChars(int escapeCounter, final StringBuilder buffer) {
while (escapeCounter > 0) {
buffer.append(ESCAPE_CHAR);
escapeCounter--;
}
}
private static void writeArgOrDelimPair(final Object[] arguments, final int argCount, final int currentArgument,
final StringBuilder buffer) {
if (currentArgument < argCount) {
recursiveDeepToString(arguments[currentArgument], buffer, null);
} else {
writeDelimPair(buffer);
}
}
static String deepToString(final Object o) {
if (o == null) {
return null;
}
if (o instanceof String) {
return (String) o;
}
final StringBuilder str = new StringBuilder();
final Set<String> dejaVu = new HashSet<>(); // that's actually a neat name ;)
recursiveDeepToString(o, str, dejaVu);
return str.toString();
}
private static void recursiveDeepToString(final Object o, final StringBuilder str, final Set<String> dejaVu) {
if (appendSpecialTypes(o, str)) {
return;
}
if (isMaybeRecursive(o)) {
appendPotentiallyRecursiveValue(o, str, dejaVu);
} else {
tryObjectToString(o, str);
}
}
private static boolean appendSpecialTypes(final Object o, final StringBuilder str) {
if (o == null || o instanceof String) {
str.append((String) o);
return true;
} else if (o instanceof CharSequence) {
str.append((CharSequence) o);
return true;
} else if (o instanceof StringBuilderFormattable) {
((StringBuilderFormattable) o).formatTo(str);
return true;
} else if (o instanceof Integer) { // LOG4J2-1415 unbox auto-boxed primitives to avoid calling toString()
str.append(((Integer) o).intValue());
return true;
} else if (o instanceof Long) {
str.append(((Long) o).longValue());
return true;
} else if (o instanceof Double) {
str.append(((Double) o).doubleValue());
return true;
} else if (o instanceof Boolean) {
str.append(((Boolean) o).booleanValue());
return true;
} else if (o instanceof Character) {
str.append(((Character) o).charValue());
return true;
} else if (o instanceof Short) {
str.append(((Short) o).shortValue());
return true;
} else if (o instanceof Float) {
str.append(((Float) o).floatValue());
return true;
}
return appendDate(o, str);
}
private static boolean appendDate(final Object o, final StringBuilder str) {
if (!(o instanceof Date)) {
return false;
}
final Date date = (Date) o;
final SimpleDateFormat format = getSimpleDateFormat();
str.append(format.format(date));
return true;
}
private static SimpleDateFormat getSimpleDateFormat() {
SimpleDateFormat result = threadLocalSimpleDateFormat.get();
if (result == null) {
result = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ"