一、官方介紹
Apache nifi is an easy to use, powerful, and reliable system to process and distribute data.
Apache NiFi是由美國過國家安全局(NSA)貢獻給Apache基金會的開源項目，其設計目標是自動化系統間的數據流。基於其工作流式的編程理念，NiFi非常易於使用，強大，可靠及高可配置。兩個最重要的特性是其強大的用戶界面及良好的數據回溯工具。
Put simply NiFi was built to automate the flow of data between systems. While the term dataflow is used in a variety of contexts, we’ll use it here to mean the automated and managed flow of information between systems. This problem space has been around ever since enterprises had more than one system, where some of the systems created data and some of the systems consumed data. The problems and solution patterns that emerged have been discussed and articulated extensively. A comprehensive and readily consumed form is found in the Enterprise Integration Patterns [eip].
Some of the high-level challenges of dataflow include:
Networks fail, disks fail, software crashes, people make mistakes.
Sometimes a given data source can outpace some part of the processing or delivery chain - it only takes one weak-link to have an issue.
You will invariably get data that is too big, too small, too fast, too slow, corrupt, wrong, or in the wrong format.
Priorities of an organization change - rapidly. Enabling new flows and changing existing ones must be fast.
The protocols and formats used by a given system can change anytime and often irrespective of the systems around them. Dataflow exists to connect what is essentially a massively distributed system of components that are loosely or not-at-all designed to work together.
Laws, regulations, and policies change. Business to business agreements change. System to system and system to user interactions must be secure, trusted, accountable.
It is often not possible to come even close to replicating production environments in the lab.
Over the years dataflow has been one of those necessary evils in an architecture. Now though there are a number of active and rapidly evolving movements making dataflow a lot more interesting and a lot more vital to the success of a given enterprise. These include things like; Service Oriented Architecture [soa], the rise of the API [api][api2], Internet of Things [iot], and Big Data [bigdata]. In addition, the level of rigor necessary for compliance, privacy, and security is constantly on the rise. Even still with all of these new concepts coming about, the patterns and needs of dataflow are still largely the same. The primary differences then are the scope of complexity, the rate of change necessary to adapt, and that at scale the edge case becomes common occurrence. NiFi is built to help tackle these modern dataflow challenges.
二、特點
關鍵特性包括:
-
基於web的用戶界面
-
無縫體驗設計、控制和監視
-
高度可配置的
-
數據丟失容錯和保證交付
-
低延遲和高吞吐量
-
動態優先級
-
流可以在運行時修改
-
背壓 Back presure
-
數據來源
-
從始至終跟蹤數據流
-
為擴展設計
-
構建自己數據處理器
-
支持快速開發和有效的測試
-
安全
-
SSL,SSH,HTTPS加密內容,等等……
-
可插拔的基於角色的驗證/授權
- 查看圖片附件
Tags: software solution capacity contexts problems
文章來源：