1. 程式人生 > >Netty Pipeline原始碼分析(2)

Netty Pipeline原始碼分析(2)

原文連結:wangwei.one/posts/netty…

前面 ,我們分析了Netty Pipeline的初始化及節點新增與刪除邏輯。接下來,我們將來分析Pipeline的事件傳播機制。

Netty版本:4.1.30

inBound事件傳播

示例

我們通過下面這個例子來演示Netty Pipeline的事件傳播機制。

public class NettyPipelineInboundExample {

    public static void main(String[] args) {
        EventLoopGroup group = new NioEventLoopGroup(1
); ServerBootstrap strap = new ServerBootstrap(); strap.group(group) .channel(NioServerSocketChannel.class) .localAddress(new InetSocketAddress(8888)) .childOption(ChannelOption.TCP_NODELAY, true) .childHandler(new ChannelInitializer<SocketChannel>() { @Override
protected void initChannel(SocketChannel ch) throws Exception { ch.pipeline().addLast(new InboundHandlerA()); ch.pipeline().addLast(new InboundHandlerB()); ch.pipeline().addLast(new InboundHandlerC()); } }); try
{ ChannelFuture future = strap.bind().sync(); future.channel().closeFuture().sync(); } catch (InterruptedException e) { e.printStackTrace(); } finally { group.shutdownGracefully(); } } } class InboundHandlerA extends ChannelInboundHandlerAdapter { @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { System.out.println("InboundHandler A : " + msg); // 傳播read事件到下一個channelhandler ctx.fireChannelRead(msg); } } class InboundHandlerB extends ChannelInboundHandlerAdapter { @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { System.out.println("InboundHandler B : " + msg); // 傳播read事件到下一個channelhandler ctx.fireChannelRead(msg); } @Override public void channelActive(ChannelHandlerContext ctx) throws Exception { // channel啟用,觸發channelRead事件,從pipeline的heandContext節點開始往下傳播 ctx.channel().pipeline().fireChannelRead("Hello world"); } } class InboundHandlerC extends ChannelInboundHandlerAdapter { @Override public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception { System.out.println("InboundHandler C : " + msg); // 傳播read事件到下一個channelhandler ctx.fireChannelRead(msg); } } 複製程式碼

原始碼

通過 telnet 來連線上面啟動好的netty服務,觸發channel active事件:

$ telnet 127.0.0.1 8888
複製程式碼

按照InboundHandlerA、InboundHandlerB、InboundHandlerC的新增順序,控制檯輸出如下資訊:

InboundHandler A : Hello world
InboundHandler B : Hello world
InboundHandler C : Hello world
複製程式碼

若是呼叫它們的新增順序,則會輸出對應順序的資訊,e.g:

...

ch.pipeline().addLast(new InboundHandlerB());
ch.pipeline().addLast(new InboundHandlerA());
ch.pipeline().addLast(new InboundHandlerC());

...
複製程式碼

輸出如下資訊:

InboundHandler B : Hello world
InboundHandler A : Hello world
InboundHandler C : Hello world
複製程式碼

原始碼分析

強烈建議 下面的流程,自己通過IDE的Debug模式來分析

待netty啟動成功,通過telnet連線到netty,然後通過telnet終端輸入任意字元(這一步才開啟Debug模式),進入Debug模式。

觸發channel read事件,從下面的入口開始呼叫

public class DefaultChannelPipeline implements ChannelPipeline {
	
	...
	
    // 出發channel read事件
	@Override
    public final ChannelPipeline fireChannelRead(Object msg) {
        // 從head節點開始往下傳播read事件
        AbstractChannelHandlerContext.invokeChannelRead(head, msg);
        return this;
    }
	
	...

}
複製程式碼

呼叫 AbstractChannelHandlerContext 中的 invokeChannelRead(head, msg) 介面:

abstract class AbstractChannelHandlerContext extends DefaultAttributeMap
        implements ChannelHandlerContext, ResourceLeakHint {

	...

	// 呼叫channel read
    static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
        // 獲取訊息
        final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
        // 獲取 EventExecutor
        EventExecutor executor = next.executor();
        // true
        if (executor.inEventLoop()) {
        	// 呼叫下面的invokeChannelRead介面:invokeChannelRead(Object msg)
            next.invokeChannelRead(m);
        } else {
            executor.execute(new Runnable() {
                @Override
                public void run() {
                    next.invokeChannelRead(m);
                }
            });
        }
    }
    
    private void invokeChannelRead(Object msg) {
        if (invokeHandler()) {
            try {
            	// handler():獲取當前遍歷到的channelHandler,第一個為HeandContext,最後為TailContext
            	// 呼叫channel handler的channelRead介面
                ((ChannelInboundHandler) handler()).channelRead(this, msg);
            } catch (Throwable t) {
                notifyHandlerException(t);
            }
        } else {
            fireChannelRead(msg);
        }
    }
    
    ...
	
    @Override
    public ChannelHandlerContext fireChannelRead(final Object msg) {
        // 調回到上面的 invokeChannelRead(final AbstractChannelHandlerContext next, Object msg)
        invokeChannelRead(findContextInbound(), msg);
        return this;
    }
    
    ...
    
    // 遍歷出下一個ChannelHandler
    private AbstractChannelHandlerContext findContextInbound() {
        AbstractChannelHandlerContext ctx = this;
        do {
        	//獲取下一個inbound型別的節點
            ctx = ctx.next;
            // 必須為inbound型別
        } while (!ctx.inbound);
        return ctx;
    }
    
    ...
}    

複製程式碼

Pipeline中的第一個節點為HeadContext,它對於channelRead事件的處理,是直接往下傳播,程式碼如下:

final class HeadContext extends AbstractChannelHandlerContext
	
	...
	
	@Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
    	// HeadContext往下傳播channelRead事件,
   		// 呼叫HeandlerContext中的介面:fireChannelRead(final Object msg)
    	ctx.fireChannelRead(msg);
    }
	
	...
}
複製程式碼

就這樣一直迴圈下去,依次會呼叫到 InboundHandlerA、InboundHandlerB、InboundHandlerC 中的 channelRead(ChannelHandlerContext ctx, Object msg) 介面。

到最後一個TailContext節點,它對channelRead事件的處理如下:

public class DefaultChannelPipeline implements ChannelPipeline {
	    
    final class TailContext extends AbstractChannelHandlerContext implements ChannelInboundHandler {
		
        ...
		        
        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
            // 呼叫onUnhandledInboundMessage介面
            onUnhandledInboundMessage(msg);
        }      

        ...

    }
	
	...
    
    // 對未處理inbound訊息做最後的處理
	protected void onUnhandledInboundMessage(Object msg) {
        try {
            logger.debug("Discarded inbound message {} that reached at the tail of the pipeline. Please check your pipeline configuration.", msg);
        } finally {
            // 對msg物件的引用數減1,當msg物件的引用數為0時,釋放該物件的記憶體
            ReferenceCountUtil.release(msg);
        }
    }
    
    ...
   
}
複製程式碼

以上就是pipeline對inBound訊息的處理流程。

SimpleChannelInboundHandler

在前面的例子中,假如中間有一個ChannelHandler未對channelRead事件進行傳播,就會導致訊息物件無法得到釋放,最終導致記憶體洩露。

我們還可以繼承 SimpleChannelInboundHandler 來自定義ChannelHandler,它的channelRead方法,對訊息物件做了msg處理,防止記憶體洩露。

public abstract class SimpleChannelInboundHandler<I> extends ChannelInboundHandlerAdapter {
	...

	@Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        boolean release = true;
        try {
            if (acceptInboundMessage(msg)) {
                @SuppressWarnings("unchecked")
                I imsg = (I) msg;
                channelRead0(ctx, imsg);
            } else {
                release = false;
                ctx.fireChannelRead(msg);
            }
        } finally {
            if (autoRelease && release) {
                // 對msg物件的引用數減1,當msg物件的引用數為0時,釋放該物件的記憶體
                ReferenceCountUtil.release(msg);
            }
        }
    }
    
    ...
    
}    
複製程式碼

outBound事件傳播

接下來,我們來分析Pipeline的outBound事件傳播機制。程式碼示例如下:

示例

public class NettyPipelineOutboundExample {

    public static void main(String[] args) {
        EventLoopGroup group = new NioEventLoopGroup(1);
        ServerBootstrap strap = new ServerBootstrap();
        strap.group(group)
                .channel(NioServerSocketChannel.class)
                .localAddress(new InetSocketAddress(8888))
                .childOption(ChannelOption.TCP_NODELAY, true)
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel ch) throws Exception {
                        ch.pipeline().addLast(new OutboundHandlerA());
                        ch.pipeline().addLast(new OutboundHandlerB());
                        ch.pipeline().addLast(new OutboundHandlerC());
                    }
                });
        try {
            ChannelFuture future = strap.bind().sync();
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            group.shutdownGracefully();
        }
    }
}

class OutboundHandlerA extends ChannelOutboundHandlerAdapter {

    @Override
    public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
        // 輸出訊息
        System.out.println("OutboundHandlerA: " + msg);
        // 傳播write事件到下一個節點
        ctx.write(msg, promise);
    }
}

class OutboundHandlerB extends ChannelOutboundHandlerAdapter {

    @Override
    public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
        // 輸出訊息
        System.out.println("OutboundHandlerB: " + msg);
        // 傳播write事件到下一個節點
        ctx.write(msg, promise);
    }
	
    
    @Override
    public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
        // 待handlerAdded事件觸發3s後,模擬觸發一個
        ctx.executor().schedule(() -> {
//            ctx.write("Hello world ! ");
            ctx.channel().write("Hello world ! ");
        }, 3, TimeUnit.SECONDS);
    }
}

class OutboundHandlerC extends ChannelOutboundHandlerAdapter {

    @Override
    public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
        // 輸出訊息
        System.out.println("OutboundHandlerC: " + msg);
        // 傳播write事件到下一個節點
        ctx.write(msg, promise);
    }
}
複製程式碼

原始碼

通過 telnet 來連線上面啟動好的netty服務,觸發channel added事件:

$ telnet 127.0.0.1 8888
複製程式碼

按照OutboundHandlerA、OutboundHandlerB、OutboundHandlerC的新增順序,控制檯輸出如下資訊:

OutboundHandlerC: Hello world ! 
OutboundHandlerB: Hello world ! 
OutboundHandlerA: Hello world ! 
複製程式碼

輸出的順序正好與ChannelHandler的新增順序相反。

若是呼叫它們的新增順序,則會輸出對應順序的資訊,e.g:

...

ch.pipeline().addLast(new InboundHandlerB());
ch.pipeline().addLast(new InboundHandlerA());
ch.pipeline().addLast(new InboundHandlerC());

...
複製程式碼

輸出如下資訊:

OutboundHandlerC: Hello world ! 
OutboundHandlerA: Hello world ! 
OutboundHandlerB: Hello world ! 
複製程式碼

原始碼分析

強烈建議 下面的流程,自己通過IDE的Debug模式來分析

從channel的write方法開始,往下傳播write事件:

public abstract class AbstractChannel extends DefaultAttributeMap implements Channel {

	...

    @Override
    public ChannelFuture write(Object msg) {
        // 呼叫pipeline往下傳播wirte事件
        return pipeline.write(msg);
    }
    
    ...

}
複製程式碼

接著來看看Pipeline中的write介面:

public class DefaultChannelPipeline implements ChannelPipeline {
    
    ...
    
    @Override
    public final ChannelFuture write(Object msg) {
        // 從tail節點開始傳播
        return tail.write(msg);
    }
    
    ...    

}
複製程式碼

呼叫ChannelHandlerContext中的write介面:

abstract class AbstractChannelHandlerContext extends DefaultAttributeMap
        implements ChannelHandlerContext, ResourceLeakHint {
        
    ...
    
    @Override
    public ChannelFuture write(Object msg) {
   	 	// 往下呼叫write介面
        return write(msg, newPromise());
    }

    @Override
    public ChannelFuture write(final Object msg, final ChannelPromise promise) {
        if (msg == null) {
            throw new NullPointerException("msg");
        }

        try {
            if (isNotValidPromise(promise, true)) {
                ReferenceCountUtil.release(msg);
                // cancelled
                return promise;
            }
        } catch (RuntimeException e) {
            ReferenceCountUtil.release(msg);
            throw e;
        }
        // 往下呼叫write介面
        write(msg, false, promise);

        return promise;
    }    
     
    ...
    
    private void write(Object msg, boolean flush, ChannelPromise promise) {
    	// 尋找下一個outbound型別的channelHandlerContext
        AbstractChannelHandlerContext next = findContextOutbound();
        final Object m = pipeline.touch(msg, next);
        EventExecutor executor = next.executor();
        if (executor.inEventLoop()) {
            if (flush) {
                next.invokeWriteAndFlush(m, promise);
            } else {
            	// 呼叫介面 invokeWrite(Object msg, ChannelPromise promise)
                next.invokeWrite(m, promise);
            }
        } else {
            AbstractWriteTask task;
            if (flush) {
                task = WriteAndFlushTask.newInstance(next, m, promise);
            }  else {
                task = WriteTask.newInstance(next, m, promise);
            }
            safeExecute(executor, task, promise, m);
        }
    }
    
    // 尋找下一個outbound型別的channelHandlerContext
    private AbstractChannelHandlerContext findContextOutbound() {
        AbstractChannelHandlerContext ctx = this;
        do {
            ctx = ctx.prev;
        } while (!ctx.outbound);
        return ctx;
    }
    
    private void invokeWrite(Object msg, ChannelPromise promise) {
        if (invokeHandler()) {
        	// 繼續往下呼叫
            invokeWrite0(msg, promise);
        } else {
            write(msg, promise);
        }
    }
	
    private void invokeWrite0(Object msg, ChannelPromise promise) {
        try {
        	// 獲取當前的channelHandler,呼叫其write介面
        	// handler()依次會返回 OutboundHandlerC OutboundHandlerB OutboundHandlerA
            ((ChannelOutboundHandler) handler()).write(this, msg, promise);
        } catch (Throwable t) {
            notifyOutboundHandlerException(t, promise);
        }
    }
    
    ...    
        
}        
複製程式碼

最終會呼叫到HeadContext的write介面:

@Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
    // 呼叫unsafe進行寫資料操作
    unsafe.write(msg, promise);
}
複製程式碼

異常傳播

瞭解了Pipeline的入站與出站事件的機制之後,我們再來看看Pipeline的異常處理機制。

示例

public class NettyPipelineExceptionCaughtExample {

    public static void main(String[] args) {
        EventLoopGroup group = new NioEventLoopGroup(1);
        ServerBootstrap strap = new ServerBootstrap();
        strap.group(group)
                .channel(NioServerSocketChannel.class)
                .localAddress(new InetSocketAddress(8888))
                .childOption(ChannelOption.TCP_NODELAY, true)
                .childHandler(new ChannelInitializer<SocketChannel>() {
                    @Override
                    protected void initChannel(SocketChannel ch) throws Exception {
                        ch.pipeline().addLast(new InboundHandlerA());
                        ch.pipeline().addLast(new InboundHandlerB());
                        ch.pipeline().addLast(new InboundHandlerC());
                        ch.pipeline().addLast(new OutboundHandlerA());
                        ch.pipeline().addLast(new OutboundHandlerB());
                        ch.pipeline().addLast(new OutboundHandlerC());
                    }
                });
        try {
            ChannelFuture future = strap.bind().sync();
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            group.shutdownGracefully();
        }
    }

    static class InboundHandlerA extends ChannelInboundHandlerAdapter {

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("InboundHandlerA.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }
    }

    static class InboundHandlerB extends ChannelInboundHandlerAdapter {

        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
            throw new Exception("ERROR !!!");
        }

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("InboundHandlerB.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }
    }

    static class InboundHandlerC extends ChannelInboundHandlerAdapter {

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("InboundHandlerC.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }
    }


    static class OutboundHandlerA extends ChannelOutboundHandlerAdapter {

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("OutboundHandlerA.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }

    }

    static class OutboundHandlerB extends ChannelOutboundHandlerAdapter {

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("OutboundHandlerB.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }
    }

    static class OutboundHandlerC extends ChannelOutboundHandlerAdapter {

        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            System.out.println("OutboundHandlerC.exceptionCaught:" + cause.getMessage());
            ctx.fireExceptionCaught(cause);
        }
    }

}

複製程式碼

原始碼

通過 telnet 來連線上面啟動好的netty服務,並在控制檯傳送任意字元:

$ telnet 127.0.0.1 8888
複製程式碼

觸發channel read事件並丟擲異常,控制檯輸出如下資訊:

InboundHandlerB.exceptionCaught:ERROR !!!
InboundHandlerC.exceptionCaught:ERROR !!!
OutboundHandlerA.exceptionCaught:ERROR !!!
OutboundHandlerB.exceptionCaught:ERROR !!!
OutboundHandlerC.exceptionCaught:ERROR !!!
複製程式碼

可以看到異常的捕獲與我們新增的ChannelHandler順序相同。

原始碼分析

在我們的示例中,InboundHandlerB的ChannelRead介面丟擲異常,導致從InboundHandlerA將ChannelRead事件傳播到InboundHandlerB的過程中出現異常,異常被捕獲。

abstract class AbstractChannelHandlerContext extends DefaultAttributeMap
        implements ChannelHandlerContext, ResourceLeakHint {
	
    ...
	
	@Override
    public ChannelHandlerContext fireExceptionCaught(final Throwable cause) {
    	//呼叫invokeExceptionCaught介面
        invokeExceptionCaught(next, cause);
        return this;
    }

    static void invokeExceptionCaught(final AbstractChannelHandlerContext next, final Throwable cause) {
        ObjectUtil.checkNotNull(cause, "cause");
        EventExecutor executor = next.executor();
        if (executor.inEventLoop()) {
        	// 呼叫下一個節點的invokeExceptionCaught介面
            next.invokeExceptionCaught(cause);
        } else {
            try {
                executor.execute(new Runnable() {
                    @Override
                    public void run() {
                        next.invokeExceptionCaught(cause);
                    }
                });
            } catch (Throwable t) {
                if (logger.isWarnEnabled()) {
                    logger.warn("Failed to submit an exceptionCaught() event.", t);
                    logger.warn("The exceptionCaught() event that was failed to submit was:", cause);
                }
            }
        }
    }
    
    ...
    
	private void invokeChannelRead(Object msg) {
        if (invokeHandler()) {
            try {
                // 丟擲異常
                ((ChannelInboundHandler) handler()).channelRead(this, msg);
            } catch (Throwable t) {
                // 異常捕獲,往下傳播
                notifyHandlerException(t);
            }
        } else {
            fireChannelRead(msg);
        }
    }
	
	// 通知Handler發生異常事件
    private void notifyHandlerException(Throwable cause) {
        if (inExceptionCaught(cause)) {
            if (logger.isWarnEnabled()) {
                logger.warn(
                    "An exception was thrown by a user handler " +
                    "while handling an exceptionCaught event", cause);
            }
            return;
        }
		// 往下呼叫invokeExceptionCaught介面
        invokeExceptionCaught(cause);
    }

	
    private void invokeExceptionCaught(final Throwable cause) {
        if (invokeHandler()) {
            try {
                // 呼叫當前ChannelHandler的exceptionCaught介面
                // 在我們的案例中,依次會呼叫InboundHandlerB、InboundHandlerC、
                // OutboundHandlerA、OutboundHandlerB、OutboundHandlC
                handler().exceptionCaught(this, cause);
            } catch (Throwable error) {
                if (logger.isDebugEnabled()) {
                    logger.debug(
                        "An exception {}" +
                        "was thrown by a user handler's exceptionCaught() " +
                        "method while handling the following exception:",
                        ThrowableUtil.stackTraceToString(error), cause);
                } else if (logger.isWarnEnabled()) {
                    logger.warn(
                        "An exception '{}' [enable DEBUG level for full stacktrace] " +
                        "was thrown by a user handler's exceptionCaught() " +
                        "method while handling the following exception:", error, cause);
                }
            }
        } else {
            fireExceptionCaught(cause);
        }
    }
	
	...
		
}
複製程式碼

最終會呼叫到TailContext節點的exceptionCaught介面,如果我們中途沒有對異常進行攔截處理,做會打印出一段警告資訊!

public class DefaultChannelPipeline implements ChannelPipeline {

    ...
    
    final class TailContext extends AbstractChannelHandlerContext implements ChannelInboundHandler {
       	
        ...
            
        @Override
        public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
            onUnhandledInboundException(cause);
        }
        
        ...
        
        protected void onUnhandledInboundException(Throwable cause) {
            try {
                logger.warn(
                    "An exceptionCaught() event was fired, and it reached at the tail of the pipeline. " +
                    "It usually means the last handler in the pipeline did not handle the exception.",
                    cause);
            } finally {
                ReferenceCountUtil.release(cause);
            }
        }
    }    
    
    ...
    
}

複製程式碼

在實際的應用中,一般會定一個ChannelHandler,放置Pipeline末尾,專門用來處理中途出現的各種異常。

最佳異常處理實踐

單獨定義ExceptionCaughtHandler來處理異常:


...
   
class ExceptionCaughtHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        if (cause instanceof Exception) {
            // TODO
            System.out.println("Successfully caught exception ! ");
        } else {
            // TODO
        }
    }
}

...
    
ch.pipeline().addLast(new ExceptionCaughtHandler());

...
複製程式碼

輸出:

InboundHandlerB.exceptionCaught:ERROR !!!
InboundHandlerC.exceptionCaught:ERROR !!!
OutboundHandlerA.exceptionCaught:ERROR !!!
OutboundHandlerB.exceptionCaught:ERROR !!!
OutboundHandlerC.exceptionCaught:ERROR !!!
Successfully caught exception !  // 成功捕獲日誌
複製程式碼

Pipeline回顧與總結

至此,我們對Pipeline的原理的解析就完成了。

  • Pipeline是在什麼時候建立的?
  • Pipeline新增與刪除節點的邏輯是怎麼樣的?
  • netty是如何判斷ChannelHandler型別的?
  • 如何處理ChannelHandler中丟擲的異常?
  • 對於ChannelHandler的新增應遵循什麼樣的順序?

參考資料