1、 在hadoop中所有的key/value都必須實現Writable介面，有兩個方法，分別用於讀（反序列化）和寫（序列化）操作。

參考程式碼：

  1 package org.dragon.hadoop.mapreduce.app;
  2 
  3 import java.io.DataInput;
  4 import java.io.DataOutput;
  5 import java.io.IOException;
  6 
  7 import org.apache.hadoop.io.Writable;
  8 
  9 /**
 10  * 
 11  * @author ZhuXY
 
 12  * @time 2016-3-10 下午3:49:55
 13  * 
 14  */
 15 public class DataWritable implements Writable {
 16 
 17     // telsphone
 18 
 19     // upload
 20     private int upPackNum;
 21     private int upPayLoad;
 22 
 23     // download
 24     private int downPackNum;
 25     private int downPayLoad;
 26 
 27
 
     public DataWritable() {
 28 
 29     }
 30 
 31     public void set(int upPackNum, int upPayLoad, int downPackNum,
 32             int downPayload) {
 33         this.upPackNum = upPackNum;
 34         this.upPayLoad = upPayLoad;
 35         this.downPackNum = downPackNum;
 36         this.downPayLoad = downPayload;
 
 37 
 38     }
 39 
 40     public int getUpPackNum() {
 41         return upPackNum;
 42     }
 43 
 44     public int getUpPayLoas() {
 45         return upPayLoad;
 46     }
 47 
 48     public int getDownPackNum() {
 49         return downPackNum;
 50     }
 51 
 52     public int getDownPayload() {
 53         return downPayLoad;
 54     }
 55 
 56     @Override
 57     public void write(DataOutput out) throws IOException {
 58         out.writeInt(upPackNum);
 59         out.writeInt(upPayLoad);
 60         out.writeInt(downPackNum);        
 61         out.writeInt(downPayLoad);
 62     }
 63 
 64     /**
 65      * 讀出的順序要和寫入的順序相同
 66      */
 67     @Override
 68     public void readFields(DataInput in) throws IOException {
 69         // TODO Auto-generated method stub
 70         this.upPackNum = in.readInt();
 71         this.upPayLoad = in.readInt();
 72         this.downPackNum = in.readInt();
 73         this.downPayLoad = in.readInt();
 74     }
 75 
 76     @Override
 77     public String toString() {
 78         return upPackNum + "\t" + upPayLoad + "\t" + downPackNum + "\t"
 79                 + downPayLoad;
 80     }
 81 
 82     @Override
 83     public int hashCode() {
 84         final int prime = 31;
 85         int result = 1;
 86         result = prime * result + downPackNum;
 87         result = prime * result + downPayLoad;
 88         result = prime * result + upPackNum;
 89         result = prime * result + upPayLoad;
 90         return result;
 91     }
 92 
 93     @Override
 94     public boolean equals(Object obj) {
 95         if (this == obj)
 96             return true;
 97         if (obj == null)
 98             return false;
 99         if (getClass() != obj.getClass())
100             return false;
101         DataWritable other = (DataWritable) obj;
102         if (downPackNum != other.downPackNum)
103             return false;
104         if (downPayLoad != other.downPayLoad)
105             return false;
106         if (upPackNum != other.upPackNum)
107             return false;
108         if (upPayLoad != other.upPayLoad)
109             return false;
110         return true;
111     }
112 
113 }

簡單繼承Writable例子 Code

2、所有的key必須實現Comparable介面，在MapReduce過程中需要對Key/Value對進行反覆的排序。預設情況下依據Key進行排序的，要實現comparaTo()方法。所以通過Key既要實現Writable介面又要實現Comparable介面，Hadoop中提供了一個公共的介面，叫做WritableComparable介面：

3、由於需要序列化反序列化和進行比較，對java物件需要重寫一下幾個方法：

①    equals();

②    hashCode();

③    toString()方法

如IntWritable型別的實現：

 1 package org.apache.hadoop.io;
 2 
 3 import java.io.*;
 4 
 5 /** A WritableComparable for ints. */
 6 public class IntWritable implements WritableComparable {
 7   private int value;
 8 
 9   public IntWritable() {}
10 
11   public IntWritable(int value) { set(value); }
12 
13   /** Set the value of this IntWritable. */
14   public void set(int value) { this.value = value; }
15 
16   /** Return the value of this IntWritable. */
17   public int get() { return value; }
18 
19   public void readFields(DataInput in) throws IOException {
20     value = in.readInt();
21   }
22 
23   public void write(DataOutput out) throws IOException {
24     out.writeInt(value);
25   }
26 
27   /** Returns true iff <code>o</code> is a IntWritable with the same value. */
28   public boolean equals(Object o) {
29     if (!(o instanceof IntWritable))
30       return false;
31     IntWritable other = (IntWritable)o;
32     return this.value == other.value;
33   }
34 
35   public int hashCode() {
36     return value;
37   }
38 
39   /** Compares two IntWritables. */
40   public int compareTo(Object o) {
41     int thisValue = this.value;
42     int thatValue = ((IntWritable)o).value;
43     return (thisValue<thatValue ? -1 : (thisValue==thatValue ? 0 : 1));
44   }
45 
46   public String toString() {
47     return Integer.toString(value);
48   }

View Code

4、資料型別，必須有一個無參的構造方法，為了方便反射建立物件。

在自定義資料型別中，建議使用java原生資料型別，最好不要使用hadoop對原生型別封裝好的資料型別，即如下樣例程式碼：

推薦使用：

相關推薦