Spark的配置項分為三部分

• Spark properties control most application parameters and can be set by using a SparkConf object, or through Java system properties.
• Environment variables can be used to set per-machine settings, such as the IP address, through the conf/spark-env.sh script on each node.
• Logging can be configured through log4j.properties
  .

Spark properities

Note that we run with local[2], meaning two threads - which represents “minimal” parallelism, which can help detect bugs that only exist when we run in a distributed context.

Spark Configuration (2.2.0) 筆記