1. 程式人生 > >error:value reduceByKey is not a member of Array[(String, Int)]

error:value reduceByKey is not a member of Array[(String, Int)]

Scala報錯value reduceByKey is not a member of Array[(String, Int)]

1.原因在於在spark中,使用scala程式設計時,沒有使用pairRDD

2.錯誤程式碼示例:

val book = Array(("spark",2),("hadoop",6),("hadoop",4),("spark",6))
val result = book.reduceByKey(_+_)

3.修正如下:

val book = Array(("spark",2),("hadoop",6),("hadoop",4),("spark",6
)) val rdd = sc.parallelize(book) val result = book.reduceByKey(_+_)

4.原理分析:【待整理】

  • 什麼是pairRDD
  • 為什麼需要sc.parallelize()?