1. 程式人生 > >Magicodes.IE 在100萬資料量下匯入匯出效能測試

Magicodes.IE 在100萬資料量下匯入匯出效能測試

原文作者:HueiFeng

前言

目前Magicodes.IE更新到了2.2.3,感謝大家的支援,同時建議大家在使用過程中如果遇到一些問題或者說需要一些額外的功能可以直接提issues,當然更建議大家提PR。

近期更新

2020.05.24

  • 【Nuget】版本更新到2.2.2

  • 【Excel匯入】增加了stream匯入擴充套件方法

  • 【Excel匯出】增加了內容居中(單列居中、整表居中)

  • 【匯出】對一些中介軟體程式碼進行了修復及優化

2020.05.16

  • 【Nuget】版本更新到2.2.1

  • 【PDF匯出】對模板引擎進行升級更新

2020.05.12

  • 【Nuget】版本更新到2.2.0

  • 【Excel模板匯出】支援匯出位元組

  • 【文件】Magicodes.IE Csv匯入匯出

  • 【Excel匯入匯出】修復標註的新增問題

  • 【匯出】ASP.NET Core Web API 中使用自定義格式化程式匯出Excel、Pdf、Csv等內容 #64

  • 【匯入匯出】支援使用System.ComponentModel.DataAnnotations名稱空間下的部分特性來控制匯入匯出 #63

效能測試

電腦配置以及環境如下所示:



BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18363.836 (1909/November2018Update/19H2)
AMD Ryzen 5 3600X, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-preview.4.20258.7
  [Host]     : .NET Core 3.1.4 (CoreCLR 4.700.20.20201, CoreFX 4.700.20.22101), X64 RyuJIT
  Job-OONFAJ : .NET Framework 4.8 (4.8.4180.0), X64 RyuJIT
  Job-YIUEXF : .NET Core 2.2.8 (CoreCLR 4.6.28207.03, CoreFX 4.6.28208.02), X64 RyuJIT
  Job-LZHMKS : .NET Core 3.1.4 (CoreCLR 4.700.20.20201, CoreFX 4.700.20.22101), X64 RyuJIT
​
IterationCount=5  LaunchCount=1  WarmupCount=1  

Excel & Csv 匯出

Excel匯出如下所示:

MethodJobRuntimeRowsCountMeanErrorStdDev
ExportExcelAsByteArrayTest Job-OONFAJ .NET 4.6.1 10000 153.1 ms 5.52 ms 0.85 ms
ExportExcelAsByteArrayTest Job-YIUEXF .NET Core 2.2 10000 138.0 ms 1.53 ms 0.40 ms
ExportExcelAsByteArrayTest Job-LZHMKS .NET Core 3.1 10000 143.0 ms 9.86 ms 1.53 ms
ExportExcelAsByteArrayTest Job-OONFAJ .NET 4.6.1 120000 1,904.2 ms 98.37 ms 25.55 ms
ExportExcelAsByteArrayTest Job-YIUEXF .NET Core 2.2 120000 1,662.8 ms 132.04 ms 20.43 ms
ExportExcelAsByteArrayTest Job-LZHMKS .NET Core 3.1 120000 1,636.1 ms 99.64 ms 25.88 ms
ExportExcelAsByteArrayTest Job-OONFAJ .NET 4.6.1 240000 3,688.3 ms 58.52 ms 15.20 ms
ExportExcelAsByteArrayTest Job-YIUEXF .NET Core 2.2 240000 3,268.1 ms 138.16 ms 21.38 ms
ExportExcelAsByteArrayTest Job-LZHMKS .NET Core 3.1 240000 3,223.9 ms 70.93 ms 10.98 ms
ExportExcelAsByteArrayTest Job-OONFAJ .NET 4.6.1 500000 7,723.4 ms 179.06 ms 46.50 ms
ExportExcelAsByteArrayTest Job-YIUEXF .NET Core 2.2 500000 6,959.7 ms 481.61 ms 125.07 ms
ExportExcelAsByteArrayTest Job-LZHMKS .NET Core 3.1 500000 6,833.2 ms 331.74 ms 86.15 ms
ExportExcelAsByteArrayTest Job-OONFAJ .NET 4.6.1 1000000 15,443.7 ms 582.64 ms 151.31 ms
ExportExcelAsByteArrayTest Job-YIUEXF .NET Core 2.2 1000000 13,798.9 ms 207.09 ms 32.05 ms
ExportExcelAsByteArrayTest Job-LZHMKS .NET Core 3.1 1000000 13,484.1 ms 424.85 ms 110.33 ms

Csv匯出如下所示:

MethodJobRuntimeRowsCountMeanErrorStdDev
ExportCsvAsByteArrayTest Job-SRSOYE .NET 4.6.1 10000 30.43 ms 2.493 ms 0.647 ms
ExportCsvAsByteArrayTest Job-WLDFBY .NET Core 2.2 10000 30.12 ms 5.981 ms 1.553 ms
ExportCsvAsByteArrayTest Job-JSEPRQ .NET Core 3.1 10000 24.53 ms 0.142 ms 0.022 ms
ExportCsvAsByteArrayTest Job-SRSOYE .NET 4.6.1 120000 345.51 ms 16.385 ms 4.255 ms
ExportCsvAsByteArrayTest Job-WLDFBY .NET Core 2.2 120000 330.03 ms 16.025 ms 4.162 ms
ExportCsvAsByteArrayTest Job-JSEPRQ .NET Core 3.1 120000 287.98 ms 11.898 ms 3.090 ms
ExportCsvAsByteArrayTest Job-SRSOYE .NET 4.6.1 240000 687.57 ms 10.379 ms 2.695 ms
ExportCsvAsByteArrayTest Job-WLDFBY .NET Core 2.2 240000 656.00 ms 13.741 ms 2.126 ms
ExportCsvAsByteArrayTest Job-JSEPRQ .NET Core 3.1 240000 560.43 ms 12.721 ms 3.304 ms
ExportCsvAsByteArrayTest Job-SRSOYE .NET 4.6.1 500000 1,478.88 ms 31.074 ms 8.070 ms
ExportCsvAsByteArrayTest Job-WLDFBY .NET Core 2.2 500000 1,379.52 ms 20.652 ms 5.363 ms
ExportCsvAsByteArrayTest Job-JSEPRQ .NET Core 3.1 500000 1,182.48 ms 39.358 ms 10.221 ms
ExportCsvAsByteArrayTest Job-SRSOYE .NET 4.6.1 1000000 2,918.99 ms 43.023 ms 6.658 ms
ExportCsvAsByteArrayTest Job-WLDFBY .NET Core 2.2 1000000 2,751.29 ms 19.970 ms 5.186 ms
ExportCsvAsByteArrayTest Job-JSEPRQ .NET Core 3.1 1000000 2,332.39 ms 57.988 ms 8.974 ms

Excel & Csv 匯入

Excel匯入如下所示:

MethodJobRuntimeRowsCountMeanErrorStdDev
ImportByStreamTest Job-URKTYJ .NET 4.6.1 10000 711.4 μs 136.2 μs 35.37 μs
ImportByStreamTest Job-AFBGUZ .NET Core 2.2 10000 746.2 μs 131.8 μs 20.40 μs
ImportByStreamTest Job-DUXAFG .NET Core 3.1 10000 792.4 μs 186.5 μs 48.42 μs
ImportByStreamTest Job-URKTYJ .NET 4.6.1 120000 2,297.6 μs 181.3 μs 47.08 μs
ImportByStreamTest Job-AFBGUZ .NET Core 2.2 120000 2,139.5 μs 204.5 μs 53.12 μs
ImportByStreamTest Job-DUXAFG .NET Core 3.1 120000 2,035.8 μs 304.8 μs 47.17 μs
ImportByStreamTest Job-URKTYJ .NET 4.6.1 240000 5,378.2 μs 887.9 μs 230.59 μs
ImportByStreamTest Job-AFBGUZ .NET Core 2.2 240000 5,345.6 μs 989.9 μs 257.08 μs
ImportByStreamTest Job-DUXAFG .NET Core 3.1 240000 4,672.5 μs 1,004.5 μs 260.86 μs
ImportByStreamTest Job-URKTYJ .NET 4.6.1 500000 11,336.6 μs 657.3 μs 170.69 μs
ImportByStreamTest Job-AFBGUZ .NET Core 2.2 500000 10,833.4 μs 952.9 μs 247.45 μs
ImportByStreamTest Job-DUXAFG .NET Core 3.1 500000 10,525.7 μs 561.0 μs 145.69 μs
ImportByStreamTest Job-URKTYJ .NET 4.6.1 1000000 21,965.7 μs 1,058.5 μs 274.88 μs
ImportByStreamTest Job-AFBGUZ .NET Core 2.2 1000000 20,612.1 μs 628.2 μs 163.14 μs
ImportByStreamTest Job-DUXAFG .NET Core 3.1 1000000 20,451.1 μs 3,807.7 μs 988.84 μs

Csv匯入如下所示:

MethodJobRuntimeRowsCountMeanErrorStdDev
ImportByStreamTest Job-OPUXWE .NET 4.6.1 10000 294.1 μs 1.99 μs 0.31 μs
ImportByStreamTest Job-OVPAAE .NET Core 2.2 10000 283.7 μs 3.54 μs 0.55 μs
ImportByStreamTest Job-AMXXYD .NET Core 3.1 10000 284.5 μs 1.77 μs 0.46 μs
ImportByStreamTest Job-OPUXWE .NET 4.6.1 120000 2,116.1 μs 13.28 μs 2.05 μs
ImportByStreamTest Job-OVPAAE .NET Core 2.2 120000 1,855.8 μs 80.59 μs 20.93 μs
ImportByStreamTest Job-AMXXYD .NET Core 3.1 120000 1,849.3 μs 186.63 μs 48.47 μs
ImportByStreamTest Job-OPUXWE .NET 4.6.1 240000 5,121.9 μs 270.99 μs 70.37 μs
ImportByStreamTest Job-OVPAAE .NET Core 2.2 240000 4,448.3 μs 84.91 μs 13.14 μs
ImportByStreamTest Job-AMXXYD .NET Core 3.1 240000 4,301.9 μs 78.77 μs 20.46 μs
ImportByStreamTest Job-OPUXWE .NET 4.6.1 500000 11,245.2 μs 120.46 μs 18.64 μs
ImportByStreamTest Job-OVPAAE .NET Core 2.2 500000 10,214.1 μs 113.20 μs 29.40 μs
ImportByStreamTest Job-AMXXYD .NET Core 3.1 500000 10,460.1 μs 90.90 μs 23.61 μs
ImportByStreamTest Job-OPUXWE .NET 4.6.1 1000000 21,807.0 μs 299.03 μs 46.28 μs
ImportByStreamTest Job-OVPAAE .NET Core 2.2 1000000 19,830.9 μs 186.01 μs 48.30 μs
ImportByStreamTest Job-AMXXYD .NET Core 3.1 1000000 20,051.3 μs 395.98 μs 102.83 μs

Reference

https://github.com/dotnetcore/Magicodes.IE

 

原文作者:HueiFeng