【.NET Core專案實戰-統一認證平臺】第二章閘道器篇-重構Ocelot來滿足需求

摘要： 【.NET Core專案實戰-統一認證平臺】開篇及目錄索引 這篇文章，我們將從Ocelot的中介軟體原始碼分析，目前Ocelot已經實現那些功能，還有那些功能在我們實際專案中暫時還未實現，如果我們要使用這些功能，應該如何改造等方面來說明。 一、Ocelot原始碼解讀 在使用...

ofollow,noindex" target="_blank"> 【.NET Core專案實戰-統一認證平臺】開篇及目錄索引

這篇文章，我們將從Ocelot的中介軟體原始碼分析，目前Ocelot已經實現那些功能，還有那些功能在我們實際專案中暫時還未實現，如果我們要使用這些功能，應該如何改造等方面來說明。

一、Ocelot原始碼解讀

在使用一個元件前，最好我們要了解其中的一些原理，否則在使用過程中遇到問題，也無從下手，今天我帶著大家一起來解讀下Ocelot原始碼，並梳理出具體實現的原理和流程，便於我們根據需求擴充套件應用。

Ocelot原始碼地址[ https://github.com/ThreeMammals/Ocelot ],

Ocelot文件地址[ https://ocelot.readthedocs.io/en/latest/ ]

然後繼續順藤摸瓜，檢視詳細的實現，我們會發現如下程式碼

public static async Task<IApplicationBuilder> UseOcelot(this IApplicationBuilder builder, OcelotPipelineConfiguration pipelineConfiguration)
{
//建立配置資訊
var configuration = await CreateConfiguration(builder);
//監聽配置資訊
ConfigureDiagnosticListener(builder);
//建立執行管道
return CreateOcelotPipeline(builder, pipelineConfiguration);
}

然後我們繼續跟蹤到建立管道方法，可以發現Ocelot的執行流程已經被找到，現在問題變的簡單了，直接檢視

private static IApplicationBuilder CreateOcelotPipeline(IApplicationBuilder builder, OcelotPipelineConfiguration pipelineConfiguration)
{
var pipelineBuilder = new OcelotPipelineBuilder(builder.ApplicationServices);
//詳細建立的管道順序在此方法
pipelineBuilder.BuildOcelotPipeline(pipelineConfiguration);

var firstDelegate = pipelineBuilder.Build();

/*
inject first delegate into first piece of asp.net middleware..maybe not like this
then because we are updating the http context in ocelot it comes out correct for
rest of asp.net..
*/

builder.Properties["analysis.NextMiddlewareName"] = "TransitionToOcelotMiddleware";

builder.Use(async (context, task) =>
{
var downstreamContext = new DownstreamContext(context);
await firstDelegate.Invoke(downstreamContext);
});

return builder;
}

管道建立流程及實現，會不會感覺到摸到大動脈了，核心的功能及原理基本找到了，那以後動手術也就可以避開一些坑了，我們可以對著這個執行順序，再檢視詳細的原始碼，按照這個執行順序檢視原始碼，您就會發現整個思路非常清晰，每一步的實現一目瞭然。為了更直觀的介紹原始碼的解讀方式，這裡我們就拿我們後續要操刀的中介軟體來講解下中介軟體的具體實現。

public static class OcelotPipelineExtensions
{
public static OcelotRequestDelegate BuildOcelotPipeline(this IOcelotPipelineBuilder builder,
OcelotPipelineConfiguration pipelineConfiguration)
{
// This is registered to catch any global exceptions that are not handled
// It also sets the Request Id if anything is set globally
builder.UseExceptionHandlerMiddleware();

// If the request is for websockets upgrade we fork into a different pipeline
builder.MapWhen(context => context.HttpContext.Socket/">WebSockets.IsWebSocketRequest,
app =>
{
app.UseDownstreamRouteFinderMiddleware();
app.UseDownstreamRequestInitialiser();
app.UseLoadBalancingMiddleware();
app.UseDownstreamUrlCreatorMiddleware();
app.UseWebSocketsProxyMiddleware();
});

// Allow the user to respond with absolutely anything they want.
builder.UseIfNotNull(pipelineConfiguration.PreErrorResponderMiddleware);

// This is registered first so it can catch any errors and issue an appropriate response
builder.UseResponderMiddleware();

// Then we get the downstream route information
builder.UseDownstreamRouteFinderMiddleware();

// This security module, IP whitelist blacklist, extended security mechanism
builder.UseSecurityMiddleware();

//Expand other branch pipes
if (pipelineConfiguration.MapWhenOcelotPipeline != null)
{
foreach (var pipeline in pipelineConfiguration.MapWhenOcelotPipeline)
{
builder.MapWhen(pipeline);
}
}

// Now we have the ds route we can transform headers and stuff?
builder.UseHttpHeadersTransformationMiddleware();

// Initialises downstream request
builder.UseDownstreamRequestInitialiser();

// We check whether the request is ratelimit, and if there is no continue processing
builder.UseRateLimiting();

// This adds or updates the request id (initally we try and set this based on global config in the error handling middleware)
// If anything was set at global level and we have a different setting at re route level the global stuff will be overwritten
// This means you can get a scenario where you have a different request id from the first piece of middleware to the request id middleware.
builder.UseRequestIdMiddleware();

// Allow pre authentication logic. The idea being people might want to run something custom before what is built in.
builder.UseIfNotNull(pipelineConfiguration.PreAuthenticationMiddleware);

// Now we know where the client is going to go we can authenticate them.
// We allow the ocelot middleware to be overriden by whatever the
// user wants
if (pipelineConfiguration.AuthenticationMiddleware == null)
{
builder.UseAuthenticationMiddleware();
}
else
{
builder.Use(pipelineConfiguration.AuthenticationMiddleware);
}

// The next thing we do is look at any claims transforms in case this is important for authorisation
builder.UseClaimsToClaimsMiddleware();

// Allow pre authorisation logic. The idea being people might want to run something custom before what is built in.
builder.UseIfNotNull(pipelineConfiguration.PreAuthorisationMiddleware);

// Now we have authenticated and done any claims transformation we 
// can authorise the request
// We allow the ocelot middleware to be overriden by whatever the
// user wants
if (pipelineConfiguration.AuthorisationMiddleware == null)
{//使用自定義認證，移除預設的認證方式
//builder.UseAuthorisationMiddleware();
}
else
{
builder.Use(pipelineConfiguration.AuthorisationMiddleware);
}

// Now we can run the claims to headers transformation middleware
builder.UseClaimsToHeadersMiddleware();

// Allow the user to implement their own query string manipulation logic
builder.UseIfNotNull(pipelineConfiguration.PreQueryStringBuilderMiddleware);

// Now we can run any claims to query string transformation middleware
builder.UseClaimsToQueryStringMiddleware();

// Get the load balancer for this request
builder.UseLoadBalancingMiddleware();

// This takes the downstream route we retrieved earlier and replaces any placeholders with the variables that should be used
builder.UseDownstreamUrlCreatorMiddleware();

// Not sure if this is the best place for this but we use the downstream url 
// as the basis for our cache key.
builder.UseOutputCacheMiddleware();

//We fire off the request and set the response on the scoped data repo
builder.UseHttpRequesterMiddleware();

return builder.Build();
}
private static void UseIfNotNull(this IOcelotPipelineBuilder builder,
Func<DownstreamContext, Func<Task>, Task> middleware)
{
if (middleware != null)
{
builder.Use(middleware);
}
}
}

限流中介軟體實現解析

實現程式碼如下 builder.UseRateLimiting(); ，我們轉到定義，得到如下程式碼，詳細的實現邏輯在 ClientRateLimitMiddleware 方法裡，繼續轉定義到這個方法，我把方法裡用到的內容註釋了下。

public static class RateLimitMiddlewareExtensions
{
public static IOcelotPipelineBuilder UseRateLimiting(this IOcelotPipelineBuilder builder)
{
return builder.UseMiddleware<ClientRateLimitMiddleware>();
}
}

public class ClientRateLimitMiddleware : OcelotMiddleware
{
private readonly OcelotRequestDelegate _next;
private readonly IRateLimitCounterHandler _counterHandler;
private readonly ClientRateLimitProcessor _processor;

public ClientRateLimitMiddleware(OcelotRequestDelegate next,
IOcelotLoggerFactory loggerFactory,
IRateLimitCounterHandler counterHandler)
:base(loggerFactory.CreateLogger<ClientRateLimitMiddleware>())
{
_next = next;
_counterHandler = counterHandler;
_processor = new ClientRateLimitProcessor(counterHandler);
}
//熟悉的Tnvoke方法，所有的邏輯都在此方法裡。
public async Task Invoke(DownstreamContext context)
{
var options = context.DownstreamReRoute.RateLimitOptions;

// 校驗是否啟用限流配置
if (!context.DownstreamReRoute.EnableEndpointEndpointRateLimiting)
{//未啟用直接進入下一個中介軟體
Logger.LogInformation($"EndpointRateLimiting is not enabled for {context.DownstreamReRoute.DownstreamPathTemplate.Value}");
await _next.Invoke(context);
return;
}

// 獲取配置的校驗客戶端的方式
var identity = SetIdentity(context.HttpContext, options);

// 校驗是否為白名單
if (IsWhitelisted(identity, options))
{//白名單直接放行
Logger.LogInformation($"{context.DownstreamReRoute.DownstreamPathTemplate.Value} is white listed from rate limiting");
await _next.Invoke(context);
return;
}

var rule = options.RateLimitRule;
if (rule.Limit > 0)
{//限流數是否大於0
// 獲取當前客戶端請求情況，這裡需要注意_processor是從哪裡注入的，後續重
var counter = _processor.ProcessRequest(identity, options);

// 校驗請求數是否大於限流數
if (counter.TotalRequests > rule.Limit)
{
//獲取下次有效請求的時間，就是避免每次請求，都校驗一次
var retryAfter = _processor.RetryAfterFrom(counter.Timestamp, rule);

// 寫入日誌
LogBlockedRequest(context.HttpContext, identity, counter, rule, context.DownstreamReRoute);

var retrystring = retryAfter.ToString(System.Globalization.CultureInfo.InvariantCulture);

// 丟擲超出限流異常並把下次可請求時間寫入header裡。
await ReturnQuotaExceededResponse(context.HttpContext, options, retrystring);

return;
}
}

//如果啟用了限流頭部
if (!options.DisableRateLimitHeaders)
{
var headers = _processor.GetRateLimitHeaders(context.HttpContext, identity, options);
context.HttpContext.Response.OnStarting(SetRateLimitHeaders, state: headers);
}
//進入下一個中介軟體
await _next.Invoke(context);
}

public virtual ClientRequestIdentity SetIdentity(HttpContext httpContext, RateLimitOptions option)
{
var clientId = "client";
if (httpContext.Request.Headers.Keys.Contains(option.ClientIdHeader))
{
clientId = httpContext.Request.Headers[option.ClientIdHeader].First();
}

return new ClientRequestIdentity(
clientId,
httpContext.Request.Path.ToString().ToLowerInvariant(),
httpContext.Request.Method.ToLowerInvariant()
);
}

public bool IsWhitelisted(ClientRequestIdentity requestIdentity, RateLimitOptions option)
{
if (option.ClientWhitelist.Contains(requestIdentity.ClientId))
{
return true;
}

return false;
}

public virtual void LogBlockedRequest(HttpContext httpContext, ClientRequestIdentity identity, RateLimitCounter counter, RateLimitRule rule, DownstreamReRoute downstreamReRoute)
{
Logger.LogInformation(
$"Request {identity.HttpVerb}:{identity.Path} from ClientId {identity.ClientId} has been blocked, quota {rule.Limit}/{rule.Period} exceeded by {counter.TotalRequests}. Blocked by rule { downstreamReRoute.UpstreamPathTemplate.OriginalValue }, TraceIdentifier {httpContext.TraceIdentifier}.");
}

public virtual Task ReturnQuotaExceededResponse(HttpContext httpContext, RateLimitOptions option, string retryAfter)
{
var message = string.IsNullOrEmpty(option.QuotaExceededMessage) ? $"API calls quota exceeded! maximum admitted {option.RateLimitRule.Limit} per {option.RateLimitRule.Period}." : option.QuotaExceededMessage;

if (!option.DisableRateLimitHeaders)
{
httpContext.Response.Headers["Retry-After"] = retryAfter;
}

httpContext.Response.StatusCode = option.HttpStatusCode;
return httpContext.Response.WriteAsync(message);
}

private Task SetRateLimitHeaders(object rateLimitHeaders)
{
var headers = (RateLimitHeaders)rateLimitHeaders;

headers.Context.Response.Headers["X-Rate-Limit-Limit"] = headers.Limit;
headers.Context.Response.Headers["X-Rate-Limit-Remaining"] = headers.Remaining;
headers.Context.Response.Headers["X-Rate-Limit-Reset"] = headers.Reset;

return Task.CompletedTask;
}
}

通過原始碼解析，發現實現一個限流還是很簡單的嗎！再進一步解析， IRateLimitCounterHandler ClientRateLimitProcessor裡的相關介面 又是怎麼實現的呢？這時候我們就需要了解下.NETCORE 的執行原理，其中 ConfigureServices 方法實現了 依賴注入(DI)的配置 。這時候我們看下 Ocelot 是在哪裡進行注入的呢？

services.AddOcelot() 是不是印象深刻呢？原來所有的注入資訊都寫在這裡，那麼問題簡單了， Ctrl+F 查詢 AddOcelot 方法，馬上就能定位到 ServiceCollectionExtensions 方法，然後再轉到定義 OcelotBuilder

public static class ServiceCollectionExtensions
{
public static IOcelotBuilder AddOcelot(this IServiceCollection services)
{
var service = services.First(x => x.ServiceType == typeof(IConfiguration));
var configuration = (IConfiguration)service.ImplementationInstance;
return new OcelotBuilder(services, configuration);
}

public static IOcelotBuilder AddOcelot(this IServiceCollection services, IConfiguration configuration)
{
return new OcelotBuilder(services, configuration);
}
}

又摸到大動脈啦，現在問題迎刃而解，原來所有的注入都寫在這裡，從這裡可以找下我們熟悉的幾個介面注入。

public OcelotBuilder(IServiceCollection services, IConfiguration configurationRoot)
{
Configuration = configurationRoot;
Services = services;
Services.Configure<FileConfiguration>(configurationRoot);

Services.TryAddSingleton<IOcelotCache<FileConfiguration>, InMemoryCache<FileConfiguration>>();
Services.TryAddSingleton<IOcelotCache<CachedResponse>, InMemoryCache<CachedResponse>>();
Services.TryAddSingleton<IHttpResponseHeaderReplacer, HttpResponseHeaderReplacer>();
Services.TryAddSingleton<IHttpContextRequestHeaderReplacer, HttpContextRequestHeaderReplacer>();
Services.TryAddSingleton<IHeaderFindAndReplaceCreator, HeaderFindAndReplaceCreator>();
Services.TryAddSingleton<IInternalConfigurationCreator, FileInternalConfigurationCreator>();
Services.TryAddSingleton<IInternalConfigurationRepository, InMemoryInternalConfigurationRepository>();
Services.TryAddSingleton<IConfigurationValidator, FileConfigurationFluentValidator>();
Services.TryAddSingleton<HostAndPortValidator>();
Services.TryAddSingleton<IReRoutesCreator, ReRoutesCreator>();
Services.TryAddSingleton<IAggregatesCreator, AggregatesCreator>();
Services.TryAddSingleton<IReRouteKeyCreator, ReRouteKeyCreator>();
Services.TryAddSingleton<IConfigurationCreator, ConfigurationCreator>();
Services.TryAddSingleton<IDynamicsCreator, DynamicsCreator>();
Services.TryAddSingleton<ILoadBalancerOptionsCreator, LoadBalancerOptionsCreator>();
Services.TryAddSingleton<ReRouteFluentValidator>();
Services.TryAddSingleton<FileGlobalConfigurationFluentValidator>();
Services.TryAddSingleton<FileQoSOptionsFluentValidator>();
Services.TryAddSingleton<IClaimsToThingCreator, ClaimsToThingCreator>();
Services.TryAddSingleton<IAuthenticationOptionsCreator, AuthenticationOptionsCreator>();
Services.TryAddSingleton<IUpstreamTemplatePatternCreator, UpstreamTemplatePatternCreator>();
Services.TryAddSingleton<IRequestIdKeyCreator, RequestIdKeyCreator>();
Services.TryAddSingleton<IServiceProviderConfigurationCreator,ServiceProviderConfigurationCreator>();
Services.TryAddSingleton<IQoSOptionsCreator, QoSOptionsCreator>();
Services.TryAddSingleton<IReRouteOptionsCreator, ReRouteOptionsCreator>();
Services.TryAddSingleton<IRateLimitOptionsCreator, RateLimitOptionsCreator>();
Services.TryAddSingleton<IBaseUrlFinder, BaseUrlFinder>();
Services.TryAddSingleton<IRegionCreator, RegionCreator>();
Services.TryAddSingleton<IFileConfigurationRepository, DiskFileConfigurationRepository>();
Services.TryAddSingleton<IFileConfigurationSetter, FileAndInternalConfigurationSetter>();
Services.TryAddSingleton<IServiceDiscoveryProviderFactory, ServiceDiscoveryProviderFactory>();
Services.TryAddSingleton<ILoadBalancerFactory, LoadBalancerFactory>();
Services.TryAddSingleton<ILoadBalancerHouse, LoadBalancerHouse>();
Services.TryAddSingleton<IOcelotLoggerFactory, AspDotNetLoggerFactory>();
Services.TryAddSingleton<IRemoveOutputHeaders, RemoveOutputHeaders>();
Services.TryAddSingleton<IClaimToThingConfigurationParser, ClaimToThingConfigurationParser>();
Services.TryAddSingleton<IClaimsAuthoriser, ClaimsAuthoriser>();
Services.TryAddSingleton<IScopesAuthoriser, ScopesAuthoriser>();
Services.TryAddSingleton<IAddClaimsToRequest, AddClaimsToRequest>();
Services.TryAddSingleton<IAddHeadersToRequest, AddHeadersToRequest>();
Services.TryAddSingleton<IAddQueriesToRequest, AddQueriesToRequest>();
Services.TryAddSingleton<IClaimsParser, ClaimsParser>();
Services.TryAddSingleton<IUrlPathToUrlTemplateMatcher, RegExUrlMatcher>();
Services.TryAddSingleton<IPlaceholderNameAndValueFinder, UrlPathPlaceholderNameAndValueFinder>();
Services.TryAddSingleton<IDownstreamPathPlaceholderReplacer, DownstreamTemplatePathPlaceholderReplacer>();
Services.TryAddSingleton<IDownstreamRouteProvider, DownstreamRouteFinder>();
Services.TryAddSingleton<IDownstreamRouteProvider, DownstreamRouteCreator>();
Services.TryAddSingleton<IDownstreamRouteProviderFactory, DownstreamRouteProviderFactory>();
Services.TryAddSingleton<IHttpRequester, HttpClientHttpRequester>();
Services.TryAddSingleton<IHttpResponder, HttpContextResponder>();
Services.TryAddSingleton<IErrorsToHttpStatusCodeMapper, ErrorsToHttpStatusCodeMapper>();
Services.TryAddSingleton<IRateLimitCounterHandler, MemoryCacheRateLimitCounterHandler>();
Services.TryAddSingleton<IHttpClientCache, MemoryHttpClientCache>();
Services.TryAddSingleton<IRequestMapper, RequestMapper>();
Services.TryAddSingleton<IHttpHandlerOptionsCreator, HttpHandlerOptionsCreator>();
Services.TryAddSingleton<IDownstreamAddressesCreator, DownstreamAddressesCreator>();
Services.TryAddSingleton<IDelegatingHandlerHandlerFactory, DelegatingHandlerHandlerFactory>();
Services.TryAddSingleton<IHttpRequester, HttpClientHttpRequester>();

// see this for why we register this as singleton http://stackoverflow.com/questions/37371264/invalidoperationexception-unable-to-resolve-service-for-type-microsoft-aspnetc
// could maybe use a scoped data repository
Services.TryAddSingleton<IHttpContextAccessor, HttpContextAccessor>();
Services.TryAddSingleton<IRequestScopedDataRepository, HttpDataRepository>();
Services.AddMemoryCache();
Services.TryAddSingleton<OcelotDiagnosticListener>();
Services.TryAddSingleton<IMultiplexer, Multiplexer>();
Services.TryAddSingleton<IResponseAggregator, SimpleJsonResponseAggregator>();
Services.TryAddSingleton<ITracingHandlerFactory, TracingHandlerFactory>();
Services.TryAddSingleton<IFileConfigurationPollerOptions, InMemoryFileConfigurationPollerOptions>();
Services.TryAddSingleton<IAddHeadersToResponse, AddHeadersToResponse>();
Services.TryAddSingleton<IPlaceholders, Placeholders>();
Services.TryAddSingleton<IResponseAggregatorFactory, InMemoryResponseAggregatorFactory>();
Services.TryAddSingleton<IDefinedAggregatorProvider, ServiceLocatorDefinedAggregatorProvider>();
Services.TryAddSingleton<IDownstreamRequestCreator, DownstreamRequestCreator>();
Services.TryAddSingleton<IFrameworkDescription, FrameworkDescription>();
Services.TryAddSingleton<IQoSFactory, QoSFactory>();
Services.TryAddSingleton<IExceptionToErrorMapper, HttpExeptionToErrorMapper>();

//add security 
this.AddSecurity();

//add asp.net services..
var assembly = typeof(FileConfigurationController).GetTypeInfo().Assembly;

Services.AddMvcCore()
.AddApplicationPart(assembly)
.AddControllersAsServices()
.AddAuthorization()
.AddJsonFormatters();

Services.AddLogging();
Services.AddMiddlewareAnalysis();
Services.AddWebEncoders();
}

至此 Ocelot 原始碼解析就到這裡了，其他的具體實現程式碼就根據流程一個一個檢視即可，這裡就不詳細講解了，因為我們已經掌握整個Ocelot程式碼的執行原理和實現方式及流程，專案裡其他的一大堆的程式碼都是圍繞這個流程去一步一步實現的。

有沒有感覺新增一箇中間件不是很複雜呢，是不是都躍躍欲試，準備嘗試開發自己的自定義中介軟體啦，本篇就不介紹中介軟體的具體開發流程了，後續實戰中會包含部分專案中需要用到的中介軟體，到時候會詳細講解如何規劃和開發一個滿足自己專案需求的中介軟體。

二、結合專案梳理功能

在完整學習完Ocelot文件和原始碼後，我們基本掌握了Ocelot目前已經實現的功能，再結合我們實際專案需求，我們梳理下還有哪些功能可能需要自己擴充套件實現。

專案設計閘道器基本需求包括路由、認證、授權、限流、快取，仔細學習文件和原始碼後發現功能都已經存在，那是不是我們就可以直接拿來使用呢？這時候我們需要拿出一些複雜業務場景來對號入座，看能否實現複雜場景的一些應用。

1、授權

能否為每一個客戶端設定獨立的訪問許可權，如果客戶端A可以訪問服務A、服務B，客戶端B只能訪問服務A，從閘道器層面直接授權，不滿足需求不路由到具體服務。從文件和程式碼分析後發現暫時未實現。

2、限流

能否為每一個客戶端設定不能限流規則，例如客戶端A為我們內容應用，我希望對服務A不啟用限流，客戶端B為第三方接入應用，我需要B訪問服務A訪問進行單獨限流（30次/分鐘），看能否通過配置實現自定義限流。從文件和程式碼分析後發現暫時未實現。

3、快取

通過程式碼發現目前快取實現的只是Dictionary方式實現的快取，不能實現分散式結構的應用。

通過分析我們發現列舉的5個基本需求，盡然有3個在我們實際專案應用中可能會存在問題，如果不解決這些問題，很難直接拿這個完美的閘道器專案應用到正式專案，所以我們到通過擴充套件Ocelot方法來實現我們的目的。

如何擴充套件呢

為了滿足我們專案應用的需要，我們需要為每一個路由進行單獨設定，如果還採用配置檔案的方式，肯定無法滿足需求，且後續閘道器動態增加路由、授權、限流等無法控制，所以我們需要把閘道器配置資訊從配置檔案中移到資料庫中，由資料庫中的路由表、限流表、授權表等方式記錄當前閘道器的應用，且後續擴充套件直接在資料庫中增加或減少相關配置，然後動態更新閘道器配置實現閘道器的高可用。

想一想是不是有點小激動，原來只要稍微改造下寶駿瞬間變寶馬，那接下來的課程就是閘道器改造之旅，我會從設計、思想、編碼等方面講解下如何實現我們的第一輛寶馬。

本系列文章我也是邊想邊寫邊實現，如果發現中間有任何描述或實現不當的地方，也請各位大神批評指正，我會第一時間整理並修正，避免讓後續學習的人走彎路。