1. 程式人生 > >大資料之scala(三) --- 類的檢查、轉換、繼承,檔案,特質trait,操作符,apply,update,unapply,高階函式,柯里化,控制抽象,集合

大資料之scala(三) --- 類的檢查、轉換、繼承,檔案,特質trait,操作符,apply,update,unapply,高階函式,柯里化,控制抽象,集合

一、類的檢查和轉換
--------------------------------------------------------
    1.類的檢查 isInstanceOf -- 包括子類
        if( p.isInstanceOf[Employee])
        {
            println("是");
        }

    2.精確類的檢查 classOf -- 不包括子類
        if(p.getClass == classOf[Employee])
        {

        }

    3.類的轉換 asInstanceOf[強轉]
        if( p.isInstanceOf[Employee])
        {
            //s 的 型別是Employee
            val s = p.asInstanceOf[Employee];
        }


二、類的繼承
----------------------------------------------------------
    1.帶主構造繼承
        scala> class Animal(val name:String){}
        scala> class Dog(name:String,val age:Int) extends Animal(name){}

    2.抽象類的繼承
        a.抽象類
            scala> abstract class Animal(val name:String){
                        //抽象欄位,沒有初始化。
                        val id:Int  ;
                        //抽象方法,沒有方法體,不需要抽象關鍵字修飾。
                        def run() ;
                    }

        b.繼承
            scala> class Dog(name:String,val age:Int) extends Animal(name){
                override def run()
                {
                    ...
                }
            }

    3.scala繼承樹

                AnyVal[值型別] <-- Int| Byte | Unit
        Any
                AnyRef[引用型別] <-- class | Nothing


三、檔案
------------------------------------------------------------------
    1.讀取行
        package test.demo1

        import scala.io.Source

        object FileDemo {
          def main(args: Array[String]): Unit = {
            //第一個引數可以是字串或者java.io.File
            val s = Source.fromFile("d:/calllog.log","UTF-8") ;
            val lines = s.getLines();
            for(line <- lines){
              println(line)
            }
          }
        }

    2.讀取字元
         package test.demo1

         import scala.io.Source

         object FileDemo {

           def main(args: Array[String]): Unit = {
               val s = Source.fromFile("d:/calllog.log") ;
               //列印每個字元
               for(c <- s)
               {
                   print(c);
               }
           }
         }

    3.檔案轉換成字串
        var s = Source.fromFile("d:/calllog.log").mkString;
        println(s);

    4.使用正則表示式讀取檔案 -- 按照空白字元分割檔案
        package test.demo1

        import scala.io.Source

        object FileDemo {

          def main(args: Array[String]): Unit = {
            var s = Source.fromFile("d:/calllog1.log").mkString
            //匹配所有空白字元
            var ss = s.split("\\s+");
            for(sss <- ss)
            {
              println(sss);
            }
          }
        }

    5.從URL或者其他源中讀取檔案
        val source1 = Source.fromURL("http://xxxxxx");
        val source1 = Source.fromString("hello world");

    6.通過正則實現爬蟲 -- 正則href解析
        val str = Source.fromFile("d:/scala/1.html").mkString
        val p = Pattern.compile("<a\\s*href=\"([\u0000-\uffff&&[^\u005c\u0022]]*)\"");
        val m = p.matcher(str);
        while (m.find()) {
            val s = m.group(1);
            System.out.println(s);
        }

    7.讀取二進位制檔案
        val file  = new File(filename);
        val in = new FileInputStream(file)
        val bytes = new Array[Byte](file.length.toInt)
        in.read(bytes)
        in.close()

    8.寫入文字檔案
        val out = new PrintWriter("a.txt");
        for(i <- 1 to 100) {
            out.println(i)};
        }
        out.close();

    9.遞迴遍歷某目錄下的所有子目錄
        object FileDemo {

          def subdirs(dir:File):Iterator[File] = {
            val child = dir.listFiles.filter(_.isDirectory);
            child.toIterator ++ child.toIterator.flatMap(subdirs _)
          }

          def main(args: Array[String]): Unit = {

            for(d <- subdirs(new File("D:\\Downloads")))
            {
              println(d)
            }

          }
        }


四、特質trait[介面]
--------------------------------------------------------
    1.普通特質:特質可以同時擁有抽象和具體方法
        trait Logger{
            def log(msg:String);    //抽象方法,沒有方法體
        }

        //用extends而不是implements
        class ConsoleLogger extends Logger
        {
            //不需要寫override
            def log(msg : String)
            {
                println(msg);
            }
        }

    2.自帶實現方法的特質

        trait ConsoleLogger{
            def log(msg:String){
                println(msg);
            };
        }

        class A extends B with ConsoleLogger{
            def func(d : Double){
                if(d > 10) log("大於10");
            }
        }

    3.trait的多繼承
        //如果只有一個trait使用extends進行擴充套件,如果多個,使用with對剩餘的trait進行擴充套件。
        trait logger1{
            def log1() = println("hello log1");
        }

        trait logger2{
            def log2() = println("hello log2");
        }

        trait logger3{
            def log3() = println("hello log3");
        }

        class Dog extends logger1 with logger2 with logger3{

        }

    4.trait之間的繼承
       trait logger1{
            def log1() = println("hello log1");
        }
        trait logger2 {
            def log2() = println("hello log2");
        }
        trait logger3 extends logger2 with logger1{

        }

    5.自身型別 [this:Dog =>] -- 限定trait的子類型別,就是隻有屬於限定型別的子類,才可以繼承這個trait
        class Dog {

        }

        trait logger{
            this:Dog =>
            def run() = println("run....")
        }

        //可以繼承logger,因為Jing8 extends Dog
        class Jing8 extends Dog with logger{

        }

        //不可以繼承logger,因為貓和狗沒有關係
        class Cat extends logger{

        }


五、操作符
--------------------------------------------------------
    1.中置操作符:操作符在中間
        a 中置識別符號 b       // 1->2  1+2

    2.一元操作符,只操作一個引數
        a 一元識別符號         //1.toString  +1 -1 ~1[按位取反] !true[布林取反]

    3.賦值操作符
        a 操作符= b        //a += b

    4.結合性[左結合:從左向右計算    右結合:從右向左計算]
        a.scala中所有的操作符都是左結合的,除了
            1)以冒號(:)結尾的操作符
            2)賦值操作符

        b.構造列表操作符[::],是右結合的
            1)建立一個空的集合
                scala> val a = Nil
                a: scala.collection.immutable.Nil.type = List()

            2)給空集合增加單個元素
                scala> 1::Nil
                res1: List[Int] = List(1)

            3)給空集合增加多個元素
                scala> 1::2::Nil
                res1: List[Int] = List(1,2)

                相當於
                scala> val list = 2::Nil
                list: List[Int] = List(2)

                scala> 1::list
                res4: List[Int] = List(1, 2)

                因為如果是從左計算的,結果應該如下
                scala> val list = 1::Nil
                list: List[Int] = List(1)

                scala> 2::list
                res5: List[Int] = List(2, 1)


六、apply() / update() /unapply()
--------------------------------------------------------
    1.apply/update
        val scores = new scala.collection.mutable.HashMap[String,Int]
        scores("bob") = 100     ==> scores.update("bob",100)
        val bobs = scores("bob") ==> scores.apply("bob")

        scala> Array.apply(100)
        res34: Array[Int] = Array(100)

        scala> Array(100)
        res35: Array[Int] = Array(100)

        scala> f.update(0,100)
        scala> f(0) = 200

    2.unapply
        //定義類
        class Fraction(val n:Int,val d:Int){
        }

        object Fraction{
            //通過
            def apply(n : Int,d:Int)= new Fraction(n,d)
            //逆向過程
            def unapply(f:Fraction) = Some(f.n,f.d)
        }

      scala> val f = Fraction(1,2)
      f: Fraction = 
[email protected]
scala> f.n res41: Int = 1 scala> f.d res42: Int = 2 scala> val Fraction(a,b) = f a: Int = 1 b: Int = 2 七、高階函式 ----------------------------------------------- 1.函式型別的變數,將函式賦值給一個變數 scala> def add(a:Int,b:Int) = a + b add: (a: Int, b: Int)Int scala> add(1,2) res43: Int = 3 scala> val f = add _ f: (Int, Int) => Int = <function2> scala> f(1,2) res44: Int = 3 2.函式作為引數 scala> def add(a:Int) = a + 1 add: (a: Int)Int scala> val f = add _ f: Int => Int = <function1> //Array().map() ==> map方法接受一個函式作為引數,對集合中的所有值進行函式運算,並返回新的集合 scala> Array(1,2,3).map(add); res2: Array[Int] = Array(2, 3, 4) scala> Array(1,2,3).map(f) res3: Array[Int] = Array(2, 3, 4) 3.匿名函式[(n:Double) => n *3] scala> val f = (n:Double) => n *3 //等價於 def f(n:Double) = n *3 f: Double => Double = <function1> scala> f(3) res5: Double = 9.0 scala> Array(1,2,3).map((n:Int) => n * 3) res7: Array[Int] = Array(3, 6, 9) 4.遍歷陣列輸出元素值,每個元素平方返回 def fun(n:Array[Int]) = { for(e <- n) println(e) val f = for(e <- n) yield e * e f } scala> val f = fun(Array(1,2,3)) 1 2 3 f: Array[Int] = Array(1, 4, 9) 5.函式作為函式的引數,並且在函式體中呼叫引數函式 val f1 = (a:Int,b:Int) => a + b val f2 = (a:Int,b:Int) => a - b def call(a:Int,b:Int,f1:(Int,Int)=>Int,f2:(Int,Int)=>Int)={ if(a > 0) f1(a,b); else f2(a,b); } scala> call(1,2,f1,f2) res23: Int = 3 scala> call(-1,2,f1,f2) res24: Int = -3 6.引數和返回值都是函式 val f1 = (a:Int,b:Int) => a + b val f2 = (a:Int,b:Int) => a - b def call(a:Int,b:Int,f1:(Int,Int)=>Int,f2:(Int,Int)=>Int):(Int)=>Int={ var res = 0; if(a > 0){ res = f1(a,b); } else { res = f2(a,b); } var f = (n:Int) => n * res return f } scala> val f = call(1,2,f1,f2) f: Int => Int = <function1> scala> f(2) res26: Int = 6 7.高階函式的簡寫(1) def valueAt(f:Double => Double) = { f(0.25); } scala> import scala.math._ import scala.math._ scala> valueAt(ceil _) res28: Double = 1.0 scala> valueAt(sqrt _) res29: Double = 0.5 8.高階函式的簡寫(2) def mulby(factor : Double) = { (x:Double) => x * factor; } scala> val f = mulby(2.0) f: Double => Double = <function1> scala> f(2) res33: Double = 4.0 9.函式推斷 def valueAt(f:(Double)=>Double) = f(0.25) scala> valueAt(x => x * 3) //等價於 valueAt((x:Double) => x * 3),scala自動推斷引數型別 scala> valueAt(3 * _) //等價於 valueAt((x:Double) => x * 3),引數在右側只出現一次,使用下劃線代替 res0: Double = 0.75 scala> val arr = Array(1,2,3,4) arr: Array[Int] = Array(1, 2, 3, 4) scala> arr.map(_ * 3) //引數在右側只出現一次,使用下劃線代替 scala> arr.map((e:Int) => e * 3) //等價 scala> arr.map(e => e * 3) //等價 res4: Array[Int] = Array(3, 6, 9, 12) //先列印陣列,後將陣列方法三倍 scala> arr.map( e=> {println(e); e * 3} ) 1 2 3 4 res7: Array[Int] = Array(3, 6, 9, 12) 10.陣列過濾arr.filter //取陣列中所有的偶數值 scala> val arr = Array(1,2,3,4) arr: Array[Int] = Array(1, 2, 3, 4) scala> arr.filter(e=> e % 2 == 0) res8: Array[Int] = Array(2, 4) //鏈式程式設計 -- 先擴大三倍然後過濾取偶數 scala> arr.map(_ * 3).filter( _ % 2 == 0 ) res9: Array[Int] = Array(6, 12) 11.一些有用的高階函式 a. "*" ,將一個字元重複n次 scala> "a" * 3 res10: String = aaa scala> "ab" * 3 res11: String = ababab b.foreach(),遍歷集合,並對每個元素應用函式 scala> (1 to 9).map("*" * _ ).foreach(println(_)) * ** *** **** ***** ****** ******* ******** ********* scala> (1 to 9).map(e => "*" * e ).foreach(e => println(e)) * ** *** **** ***** ****** ******* ******** ********* c.reduceLeft 方法接受一個二元的函式 -- 即一個帶有兩個引數的函式,並將函式應用到序列中的所有元素中 scala> (1 to 9).reduceLeft(_ * _) res21: Int = 362880 //相當於 9! scala> (1 to 9).reduceLeft((x,y)=> x-y) res24: Int = -43 //相當於 1-2-3-4-5-6-7-8-9 d.reduceRight/reduceLeft //從右側開始化簡 -- (1 - (2 - (3 - 4))) scala> (1 to 4).reduceRight(_ - _) res29: Int = -2 //從左側開始化簡 -- (((1 - 2) - 3) - 4) scala> (1 to 4).reduceRight(_ - _) res29: Int = -8 d.sortWith 方法接受一個二元的函式,然後進行排序,輸出一個數組 scala> val a = "Hello World Tom HAHAHAHA" a: String = Hello World Tom HAHAHAHA scala> a.split(" ") res32: Array[String] = Array(Hello, World, Tom, HAHAHAHA) scala> a.split(" ").sortWith((x,y)=> x.length < y.length) scala> a.split(" ").sortWith(_.length < _.length) res33: Array[String] = Array(Tom, Hello, World, HAHAHAHA) e.閉包:函式中傳遞的引數,然後將函式賦值給一個變數,那麼這個變數就永久的持有這個引數了 def mulBy(fac : Double) = { (x :Double) => fac * x; } val f1 = mulBy(3); //f1 就永久的持有3這個引數因子,即使MulBy已經結束 val f2 = mulBy(0.5); //f2 就永久的持有0.5這個引數因子,即使MulBy已經結束 scala> println(f1(3) + ":" + f2(3)) 9.0:1.5 八、柯里化 ---------------------------------------------------------------- 1.方法鏈式化 將多引數的一個函式,變成只有一個引數的多函式,達到每個函式僅有一個引數,每個函式僅完成一個任務,達到簡化的目的 2.兩個引數的函式的另外一種寫法[為了將引數單個的隔離出來] :一個新的函式,以原來的一個引數為引數,返回值是一個函式,並且返回的函式是,以原來的函式的另外一個引數為引數,的函式 scala> def mul(x:Int,y:Int) = { x * y } mul: (x: Int, y: Int)Int scala> def mulOne(x :Int) = { ( y:Int ) => x * y } scala> def mulOne(x :Int)(y :Int) = x * y mulOne: (x: Int)Int => Int scala> mulOne(5)(6) res37: Int = 30 scala> mul(5,6) res38: Int = 30 九、控制抽象 ----------------------------------------------------------- 1.定義過程,啟動分執行緒執行block程式碼,將執行緒執行的程式碼抽離成引數 def newThread( block : () => Unit ){ new Thread(){ override def run() { block(); } }.start(); } newThread( () =>{ (1 to 10).foreach( (e) => { val tname = Thread.currentThread.getName(); println(tname + " " + e) } ) } ); scala> Thread-2 1 Thread-2 2 Thread-2 3 Thread-2 4 Thread-2 5 Thread-2 6 Thread-2 7 Thread-2 8 Thread-2 9 Thread-2 10 2.換名呼叫表示法() => 在引數宣告和呼叫的時候都省略(),但保留=> def newThread( block : => Unit ){ new Thread(){ override def run() { block; } }.start(); } newThread( (1 to 10).foreach( e => { val tname = Thread.currentThread.getName(); println(tname + " " + e) } ) ); 十、集合 ----------------------------------------------------------- 1.不可變:List //Nil空集合 scala> val l = List(1,2,3,4,5) l: List[Int] = List(1, 2, 3, 4, 5) scala> l.head res50: Int = 1 scala> l.tail res51: List[Int] = List(2, 3, 4, 5) scala> 9::l res52: List[Int] = List(9, 1, 2, 3, 4, 5) 2.遞迴計算集合中所有元素的和 def sum(lst:List[Int]) : Int = { if(lst == Nil) 0 else lst.head + sum(lst.tail) } 3.模式匹配,實現sum求和 def sum(lst : List[Int]) : Int = { lst match{ case Nil => 0 case h :: t => h + sum(t) //:: 折構 ,將lst.head表示成h ,將 lst.tail表示成t } } 4.集Set(儲存不重複元素) a.不重複 scala> val set = scala.collection.mutable.Set(1,2,3) set: scala.collection.mutable.Set[Int] = Set(1, 2, 3) scala> set.add(2) res56: Boolean = false scala> set res57: scala.collection.mutable.Set[Int] = Set(1, 2, 3) scala> set.add(4) res58: Boolean = true scala> set res59: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) 5.集合的新增和刪除元素操作符 a. [:+] / [+:] 向集合中新增元素,返回新的集合,原集合不變[List可以,Set不行] scala> val list = List(1,2,3) list: List[Int] = List(1, 2, 3) scala> list:+4 res67: List[Int] = List(1, 2, 3, 4) scala> 4 +: list res68: List[Int] = List(4, 1, 2, 3) b. [+] / [-] 集合中新增/移除集合,返回新的集合,無序 適用於Set和Map,不適用於List scala> set res75: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> set + (9,10) res79: scala.collection.mutable.Set[Int] = Set(9, 1, 2, 3, 10, 4) scala> set -(1,2,5) res83: scala.collection.mutable.Set[Int] = Set(3, 4) c. [++] / [++:] 集合中新增相同型別的集合,返回新的包含兩個集合元素的集合,適用List和Set scala> set res89: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> val set1 = set + (5,6) - (1,2) set1: scala.collection.mutable.Set[Int] = Set(5, 6, 3, 4) scala> set ++ set1 res87: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 6, 3, 4) scala> set ++: set1 res88: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 6, 3, 4) scala> list ++ list1 res91: List[Int] = List(1, 2, 3, 4, 5, 6) scala> list1 ++ list res92: List[Int] = List(4, 5, 6, 1, 2, 3) scala> list1 ++: list res93: List[Int] = List(4, 5, 6, 1, 2, 3) d.[--] 操作兩個集合,移除左側集合中所有包含於右側集合中的元素,返回新的集合。適用於Set,不適用List scala> set res94: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> set1 res95: scala.collection.mutable.Set[Int] = Set(5, 6, 3, 4) scala> set -- set1 res96: scala.collection.mutable.Set[Int] = Set(1, 2) e.[::] / [:::] 向集合中新增元素或者集合。返回新的集合。適用於List,不適用於Set scala> list res98: List[Int] = List(1, 2, 3) scala> 4 :: list res101: List[Int] = List(4, 1, 2, 3) scala> list1 res102: List[Int] = List(4, 5, 6) scala> list res103: List[Int] = List(1, 2, 3) scala> list1 ::: list res104: List[Int] = List(4, 5, 6, 1, 2, 3) scala> list ::: list1 res105: List[Int] = List(1, 2, 3, 4, 5, 6) f.[|] / [&] / [&~] 去兩個集合的並集,交集和差集,返回新的集合。適用於set不適用List scala> set res106: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> set1 res107: scala.collection.mutable.Set[Int] = Set(5, 6, 3, 4) scala> set | set1 res108: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 6, 3, 4) scala> set1 | set res109: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 6, 3, 4) scala> set & set1 res110: scala.collection.mutable.Set[Int] = Set(3, 4) scala> set &~ set1 res111: scala.collection.mutable.Set[Int] = Set(1, 2) scala> set1 &~ set res112: scala.collection.mutable.Set[Int] = Set(5, 6) g. +=, ++=, -=, --= 帶賦值=,改變原有集合,不產生新集合,適用於Set,不適用於List scala> set res113: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> set += 5 res114: set.type = Set(1, 5, 2, 3, 4) scala> set -= 5 res115: set.type = Set(1, 2, 3, 4) scala> set += (56,67) res123: set.type = Set(1, 67, 2, 56, 3, 4) scala> set -= (67,56) res127: set.type = Set(1, 2, 3, 4) scala> set res128: scala.collection.mutable.Set[Int] = Set(1, 2, 3, 4) scala> set1 res129: scala.collection.mutable.Set[Int] = Set(5, 6, 3, 4) scala> set ++= set1 res130: set.type = Set(1, 5, 2, 6, 3, 4) scala> set --= set1 res131: set.type = Set(1, 2) 6.常用方法 scala> set res143: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 3, 4) scala> list res147: List[Int] = List(1, 2, 3) //isEmpty scala> set.isEmpty res134: Boolean = false //tail scala> set.tail res140: scala.collection.mutable.Set[Int] = Set(5, 2, 3, 4) //head scala> set.head res141: Int = 1 //init scala> set.init res142: scala.collection.mutable.Set[Int] = Set(1, 5, 2, 3) //length scala> list.length res146: Int = 3 //take(n) scala> list.take(2) res152: List[Int] = List(1, 2) scala> set.take(2) res153: scala.collection.mutable.Set[Int] = Set(1, 5) //drop(n) scala> set.drop(2) res154: scala.collection.mutable.Set[Int] = Set(2, 3, 4) //splitAt(n) scala> list.splitAt(2) res157: (List[Int], List[Int]) = (List(1, 2),List(3)) //zip scala> import scala.collection.mutable.ArrayBuffer import scala.collection.mutable.ArrayBuffer scala> val b1 = ArrayBuffer(1,2,3) b1: scala.collection.mutable.ArrayBuffer[Int] = ArrayBuffer(1, 2, 3) scala> val b2 = ArrayBuffer(3,4,5,6) b2: scala.collection.mutable.ArrayBuffer[Int] = ArrayBuffer(3, 4, 5, 6) scala> b1.zip(b2) res158: scala.collection.mutable.ArrayBuffer[(Int, Int)] = ArrayBuffer((1,3), (2,4), (3,5)) scala> b2.zip(b1) res159: scala.collection.mutable.ArrayBuffer[(Int, Int)] = ArrayBuffer((3,1), (4,2), (5,3)) //zipAll scala> b1.zipAll(b2,-1,-2) res160: scala.collection.mutable.ArrayBuffer[(Int, Int)] = ArrayBuffer((1,3), (2,4), (3,5), (-1,6)) scala> b2.zipAll(b1,-1,-2) res161: scala.collection.mutable.ArrayBuffer[(Int, Int)] = ArrayBuffer((3,1), (4,2), (5,3), (6,-2)) //zipWithIndex scala> b1.zipWithIndex res163: scala.collection.mutable.ArrayBuffer[(Int, Int)] = ArrayBuffer((1,0), (2,1), (3,2)) //folderLeft scala> List(1,7,2,9).foldLeft(0)(_ - _) //相當於 ((((0 - 1) - 7) - 2) - 9) = -19 res167: Int = -19 //folderRight scala> List(1,7,2,9).foldRight(0)(_ - _) //相當於 (1 - (7 - (2 - (9 - 0)))) = -13 res168: Int = -13