威脅獵殺實戰（三）：基於Wazuh, Snort/Suricata和Elastic Stack的SOC

摘要： 整合HIDS、NIDS和Elastic Stack，在此基礎上實現SOC The Elastic Stack delivers security analytics capabilities that are widely used for threat detection, visibi...

整合HIDS、NIDS和Elastic Stack，在此基礎上實現SOC

The Elastic Stack delivers security analytics capabilities that are widely used for threat detection, visibility, and incident response. The speed and scale at which Elasticsearch can index and search security-related information enable security analysts to work more efficiently, while Kibana dashboards provide wide visibility and enable interactive threat hunting. And the machine learning engine can automate the analysis of complex datasets, making it possible to spot intruders that otherwise would’ve gone unnoticed.

Popular Intrusion Detection Systems (IDS), such as Wazuh or Snort/Suricata, use a signature-basedapproach to threat detection. That is, they compare patterns found in files, logs, and network traffic against a database of patterns known to be associated with malicious activity, alerting when a match is found. They provide useful rulesets to analyze and correlate data, usually generating thousands or millions of alerts per day in a production environment.

Casting a wide net can ensure that all potential security events are caught, but it also adds the work of sifting through thousands (or millions) of alerts every day. Elastic machine learning features help reduce the noise by automatically identifying unusual behaviors. This is a clear use case where anomaly-based and signature-based technologies complement each other, making threat detection easier and investigations more efficient.

目錄：

1.部署 Snort/Suricata

Ubuntu
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata
RHEL/CentOS
yum install epel-release
yum install suricata
參考：
Suricata
https://github.com/tianyulab/dalton/blob/master/dalton-agent/Dockerfiles/Dockerfile_suricata
https://suricata.readthedocs.io/en/suricata-4.0.5/install.html
Snort
https://github.com/tianyulab/SnortCP/blob/master/Scripts/Snort_Wireshark.sh

2.配置Suricata Eve JSON Output

# 配置舉例：
vi /etc/suricata/suricata.yaml
outputs:
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
types:
- alert:
metadata: yes
tagged-packets: yes
xff:
enabled: yes
mode: extra-data
- http:
extended: yes
- dns:
query: yes# enable logging of DNS queries
answer: yes# enable logging of DNS answers
- tls:
extended: yes# enable this for extended logging information
- files:
force-magic: no# force logging magic on all logged files
- smtp:
extended: yes # enable this for extended logging information
- ssh
- flow
參考：
https://suricata.readthedocs.io/en/suricata-4.0.5/configuration/suricata-yaml.html#eve-extensible-event-format

3.部署 Wazuh Stack

Wazuh stack包含3個元件：
1.Wazuh server: 包含Wazuh manager，API 和 Filebeat（Filebeat僅在分散式架構下使用）
2.Elastic Stack: 包含Elasticsearch，Logstash，Kibana 和 Wazuh Kibana app，讀取，解析，索引和儲存Wazuh伺服器生成的警報資料。
3.Wazuh agent
# 分散式架構：在不同主機上執行Wazuh伺服器和Elastic Stack叢集（一個或多個伺服器）
https://documentation.wazuh.com/current/_images/installing_wazuh2.png
# 單主機架構：在同一系統上執行Wazuh伺服器和Elastic Stack
https://documentation.wazuh.com/current/_images/installing_wazuh_singlehost2.png
# 本文采用分散式架構，分別在Ubuntu 16.04上部署Wazuh server，CentOS 7.x上部署Elastic Stack
Ubuntu 16.04
1.部署Wazuh server
# 1.新增Wazuh Repositories
## 1.1）安裝依賴軟體包
apt-get update
apt-get -y install curl apt-transport-https lsb-release
# if [ ! -f /usr/bin/python ]; then ln -s /usr/bin/python3 /usr/bin/python; fi # 可選
curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add -
echo "deb https://packages.wazuh.com/3.x/apt/ stable main" | tee -a /etc/apt/sources.list.d/wazuh.list
# 2.安裝wazuh-manager
apt-get update
apt-get -y install wazuh-manager
systemctl status wazuh-manager
systemctl enable wazuh-manager
# 3.安裝Wazuh API
## 3.1）安裝依賴軟體包，NodeJS >= 4.6.1，Python >= 2.7
curl -sL https://deb.nodesource.com/setup_8.x | bash -
apt-get -y install nodejs
apt-get -y install wazuh-api
systemctl status wazuh-api
systemctl enable wazuh-api
# 4.配置Wazuh Kibana app
參考：
https://documentation.wazuh.com/current/user-manual/kibana-app/connect-kibana-app.html
# 5.安裝Filebeat（分散式架構）
curl -s https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-6.x.list
apt-get update
apt-get -y install filebeat=6.4.2
curl -so /etc/filebeat/filebeat.yml https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/filebeat/filebeat.yml
修改/etc/filebeat/filebeat.yml中ELASTIC_SERVER_IP的值為Elastic Stack伺服器IP
systemctl daemon-reload
systemctl enable filebeat.service
systemctl start filebeat.service
# 6.驗證
systemctl status wazuh-manager
systemctl status wazuh-api
systemctl status filebeat.service
# 7.一鍵部署指令碼
https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Wazuh_server.sh
2.部署Elastic Stack
CentOS 7.x
# 1.安裝依賴軟體包，JRE
curl -Lo jre-8-linux-x64.rpm --header "Cookie: oraclelicense=accept-securebackup-cookie" "https://download.oracle.com/otn-pub/java/jdk/8u191-b12/2787e4a523244c269598db4e85c51e0c/jre-8u191-linux-x64.rpm"
rpm -qlp jre-8-linux-x64.rpm > /dev/null 2>&1 && echo "Java package downloaded successfully" || echo "Java package did not download successfully"
yum -y install jre-8-linux-x64.rpm
rm -f jre-8-linux-x64.rpm
# 2.安裝elasticsearch、logstash、kibana
rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch
cat > /etc/yum.repos.d/elastic.repo << EOF
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
yum install elasticsearch-6.4.2
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service
curl "localhost:9200/?pretty"
# Load the Wazuh template for Elasticsearch:
curl https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/elasticsearch/wazuh-elastic6-template-alerts.json | curl -XPUT 'http://localhost:9200/_template/wazuh' -H 'Content-Type: application/json' -d @-
yum install logstash-6.4.2
# Download the Wazuh configuration file for Logstash:
curl -so /etc/logstash/conf.d/01-wazuh.conf https://raw.githubusercontent.com/wazuh/wazuh/3.6/extensions/logstash/01-wazuh-remote.conf
systemctl daemon-reload
systemctl enable logstash.service
systemctl start logstash.service
yum install kibana-6.4.2
export NODE_OPTIONS="--max-old-space-size=3072"
sudo -u kibana /usr/share/kibana/bin/kibana-plugin install https://packages.wazuh.com/wazuhapp/wazuhapp-3.6.1_6.4.2.zip
# 參考：https://github.com/wazuh/wazuh-kibana-app#installation
# /etc/kibana/kibana.yml # 可選
# server.host: "0.0.0.0"
systemctl daemon-reload
systemctl enable kibana.service
systemctl start kibana.service
# 3.驗證
curl "localhost:9200/?pretty"
systemctl status logstash.service
systemctl status kibana.service
# 4.一鍵部署指令碼
https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Elastic_Stack.sh

4.在Suricata伺服器上安裝Wazuh Agent

Ubuntu 16.04 
# 1.部署Wazuh Agent
apt-get -y install curl apt-transport-https lsb-release
curl -s https://packages.wazuh.com/key/GPG-KEY-WAZUH | apt-key add -
echo "deb https://packages.wazuh.com/3.x/apt/ stable main" | tee /etc/apt/sources.list.d/wazuh.list
apt-get update
apt-get -y install wazuh-agent
# 2.註冊Wazuh Agent
# Wazuh Manager 上執行：
openssl req -x509 -batch -nodes -days 365 -newkey rsa:2048 -keyout /var/ossec/etc/sslmanager.key -out /var/ossec/etc/sslmanager.cert
/var/ossec/bin/ossec-authd -i
# Wazuh Agent 上執行：
sed -i "s/MANAGER_IP/8.8.8.8/"/var/ossec/etc/ossec.conf
/var/ossec/bin/agent-auth -m 8.8.8.8
systemctl restart wazuh-agent
# 參考：
https://documentation.wazuh.com/current/user-manual/registering/index.html
# 3.驗證
systemctl status wazuh-agent
# 4.一鍵部署指令碼
https://github.com/tianyulab/Threat_Hunting_with_ELK/tree/master/HIDS_NIDS_ELK/Deploy_Wazuh_agent.sh
# 注：此指令碼為互動模式

5.在Wazuh Manager伺服器上配置Wazuh rules處理Suricata日誌

sed -i 's/id="86600" level="0"/id="86600" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml
sed -i 's/id="86602" level="0"/id="86602" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml
sed -i 's/id="86603" level="0"/id="86603" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml
sed -i 's/id="86604" level="0"/id="86604" level="4"/g' /var/ossec/ruleset/rules/0475-suricata_rules.xml
systemctl restart wazuh-manager.service
# /var/ossec/bin/ossec-control restart

6.在Suricata伺服器上配置Wazuh Agent讀取Suricata的eve.json檔案

vi /var/ossec/etc/ossec.conf
# 在<ossec_config> tag裡新增如下內容
# Modify ossec.conf - read localfile suricata EVE json log
<localfile>
<log_format>syslog</log_format>
<location>/var/log/suricata/eve.json</location>
</localfile>
# 驗證
systemctl restart wazuh-agent
systemctl status wazuh-agent

7.在Elastic Stack上配置wazuh logstash filter

# 1.wazuh logstash filter 配置
在Elastic Stack伺服器上執行，
vi /etc/logstash/conf.d/01-wazuh.conf
# 新增以下內容
filter {
if [data][src_ip] {
mutate{
add_field => [ "[data][srcip]","%{[data][src_ip]}"]
remove_field => [ "[data][src_ip]" ]
}
}
if [data][dest_ip] {
mutate{
add_field => [ "[data][dstip]","%{[data][dest_ip]}"]
remove_field => [ "[data][dest_ip]" ]
}
}
if [data][dest_port] {
mutate{
add_field => [ "[data][dstport]","%{[data][dest_port]}"]
remove_field => [ "[data][dest_port]" ]
}
}
if [data][src_port] {
mutate{
add_field => [ "[data][srcport]","%{[data][src_port]}"]
remove_field => [ "[data][src_port]" ]
}
}
}
# 配置驗證
/usr/share/logstash/bin/logstash -f 01-wazuh.conf --config.test_and_exit
# 重啟Logstash服務
systemctl restart logstash.service

8.Wazuh + Snort/Suricata 聯動（active response）

# 1.Snort
# 修改snort輸出為alert_fast：
vi /etc/snort/snort.conf
output alert_fast: snort.log 128M
systemctl restart snort
# 配置Wazuh agent
vi /var/ossec/etc/ossec.conf
<localfile>
<log_format>snort-full</log_format>
<location>/var/log/snort/snort.log</location>
</localfile>
systemctl restart wazuh-agent
# 配置Wazuh Manager
vi /var/ossec/etc/ossec.conf
# 新增如下內容：
<!-- Active response -->
<global>
<white_list>127.0.0.1</white_list>
<white_list>^localhost.localdomain$</white_list>
<white_list>8.8.8.8</white_list>
</global>
<command>
<name>disable-account</name>
<executable>disable-account.sh</executable>
<expect>user</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>restart-ossec</name>
<executable>restart-ossec.sh</executable>
<expect></expect>
</command>
<command>
<name>firewall-drop</name>
<executable>default-firewall-drop.sh</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>host-deny</name>
<executable>host-deny.sh</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>route-null</name>
<executable>route-null.sh</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>win_route-null</name>
<executable>route-null.cmd</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>win_route-null-2012</name>
<executable>route-null-2012.cmd</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>netsh</name>
<executable>netsh.cmd</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<command>
<name>netsh-win-2016</name>
<executable>netsh-win-2016.cmd</executable>
<expect>srcip</expect>
<timeout_allowed>yes</timeout_allowed>
</command>
<!-- Active Response Config -->
<active-response>
<!-- This response is going to execute the host-deny
- command for every event that fires a rule with
- level (severity) >= 6.
- The IP is going to be blocked for600 seconds.
-->
<command>host-deny</command>
<location>local</location>
<level>12</level>
<timeout>600</timeout>
</active-response>
<active-response>
<!-- Firewall Drop response. Block the IP for
- 600 seconds on the firewall (iptables,
- ipfilter, etc).
-->
<command>firewall-drop</command>
<location>local</location>
<level>12</level>
<timeout>600</timeout>
</active-response>
<!-- Snort active response 配置 -->
<active-response>
<command>firewall-drop</command>
<location>local</location>
<rules_id>20101</rules_id> 
<timeout>600</timeout> 
</active-response>
<active-response>
<command>host-deny</command>
<location>local</location>
<rules_id>20101</rules_id> 
<timeout>600</timeout> 
</active-response>
# 重啟服務，
systemctl restart wazuh-manager.service
參考：
https://groups.google.com/forum/#!msg/wazuh/8cu1hZ9PHCM/RiPK41gWAgAJ
https://documentation.wazuh.com/current/user-manual/reference/ossec-conf/localfile.html?highlight=localfile#log-format
/var/ossec/ruleset/decoders/0285-snort_decoders.xml
/var/ossec/ruleset/rules/0240-ids_rules.xml
https://github.com/wazuh/wazuh-ruleset/blob/master/decoders/0285-snort_decoders.xml
# 2.Suricata
省略
參考：
https://github.com/wazuh/wazuh/issues/202
https://documentation.wazuh.com/current/user-manual/ruleset/custom.html
https://documentation.wazuh.com/current/user-manual/ruleset/json-decoder.html
除錯/測試工具：
/var/ossec/bin/ossec-logtest

效果圖：

9.未完待續

1.機器學習例項
2.Wazuh和Snort/Suricata事件關聯
3.下期我們會講一講Wazuh