hive 求月訪問記錄，累計訪問記錄

阿新 • • 發佈：2018-12-14

求出當月記錄和累計記錄，如2月累計記錄=1月+2月記錄，3月累計記錄=1月+2月+3月記錄

A,2015-01,1
A,2015-01,11
A,2015-01,12
A,2015-01,13
B,2015-01,10
B,2015-01,20
B,2015-01,30
B,2015-01,40
A,2015-01,1
A,2015-01,11
A,2015-01,12
A,2015-02,13
B,2015-01,10
B,2015-02,20
B,2015-02,30
B,2015-01,40

[[email protected] apps]# beeline
Beeline version 1.2.2 by Apache Hive
beeline>  !connect jdbc:hive2://hdp-nn-01:10000
Connecting to jdbc:hive2://hdp-nn-01:10000
Enter username for jdbc:hive2://hdp-nn-01:10000: root
Enter password for jdbc:hive2://hdp-nn-01:10000:
Connected to: Apache Hive (version 1.2.2)
Driver: Hive JDBC (version 1.2.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hdp-nn-01:10000> select * from t_access_time;
+-------------------------+----------------------+-----------------------+--+
| t_access_time.username  | t_access_time.month  | t_access_time.salary  |
+-------------------------+----------------------+-----------------------+--+
| A                       | 2015-01              | 1                     |
| A                       | 2015-01              | 11                    |
| A                       | 2015-01              | 12                    |
| A                       | 2015-01              | 13                    |
| B                       | 2015-01              | 10                    |
| B                       | 2015-01              | 20                    |
| B                       | 2015-01              | 30                    |
| B                       | 2015-01              | 40                    |
+-------------------------+----------------------+-----------------------+--+
8 rows selected (1.512 seconds)
0: jdbc:hive2://hdp-nn-01:10000> [ 
[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[ 
[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]#
[[email protected] apps]# beeline                      
Beeline version 1.2.2 by Apache Hive
beeline>  !connect jdbc:hive2://hdp-nn-01:10000
Connecting to jdbc:hive2://hdp-nn-01:10000
Enter username for jdbc:hive2://hdp-nn-01:10000: root
Enter password for jdbc:hive2://hdp-nn-01:10000:
Connected to: Apache Hive (version 1.2.2)
Driver: Hive JDBC (version 1.2.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> select username ,month ,sum(salary) salary from t_access_time group by username,month;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0001
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0001/
INFO  : Starting Job = job_1539182373092_0001, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0001/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0001
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2018-10-10 23:45:45,273 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-10-10 23:45:56,000 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 5.34 sec
INFO  : 2018-10-10 23:46:10,379 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 10.51 sec
INFO  : MapReduce Total cumulative CPU time: 10 seconds 510 msec
INFO  : Ended Job = job_1539182373092_0001
+-----------+----------+---------+--+
| username  |  month   | salary  |
+-----------+----------+---------+--+
| A         | 2015-01  | 37      |
| B         | 2015-01  | 100     |
+-----------+----------+---------+--+
2 rows selected (46.304 seconds)
0: jdbc:hive2://hdp-nn-01:10000> delete from t_accces_time;
Error: Error while compiling statement: FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations. (state=42000,code=10294)
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> select * from t_access_time;
+-------------------------+----------------------+-----------------------+--+
| t_access_time.username  | t_access_time.month  | t_access_time.salary  |
+-------------------------+----------------------+-----------------------+--+
| A                       | 2015-01              | 1                     |
| A                       | 2015-01              | 11                    |
| A                       | 2015-01              | 12                    |
| A                       | 2015-01              | 13                    |
| B                       | 2015-01              | 10                    |
| B                       | 2015-01              | 20                    |
| B                       | 2015-01              | 30                    |
| B                       | 2015-01              | 40                    |
+-------------------------+----------------------+-----------------------+--+
8 rows selected (0.187 seconds)
0: jdbc:hive2://hdp-nn-01:10000> load data local inpath '/root/apps/hive1.txt'  into table t_access_time;
INFO  : Loading data to table default.t_access_time from file:/root/apps/hive1.txt
INFO  : Table default.t_access_time stats: [numFiles=2, totalSize=206]
No rows affected (0.741 seconds)
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> select * from t_access_time;
+-------------------------+----------------------+-----------------------+--+
| t_access_time.username  | t_access_time.month  | t_access_time.salary  |
+-------------------------+----------------------+-----------------------+--+
| A                       | 2015-01              | 1                     |
| A                       | 2015-01              | 11                    |
| A                       | 2015-01              | 12                    |
| A                       | 2015-01              | 13                    |
| B                       | 2015-01              | 10                    |
| B                       | 2015-01              | 20                    |
| B                       | 2015-01              | 30                    |
| B                       | 2015-01              | 40                    |
| A                       | 2015-01              | 1                     |
| A                       | 2015-01              | 11                    |
| A                       | 2015-02              | 12                    |
| A                       | 2015-01              | 13                    |
| B                       | 2015-01              | 10                    |
| B                       | 2015-01              | 20                    |
| B                       | 2015-02              | 30                    |
| B                       | 2015-01              | 40                    |
+-------------------------+----------------------+-----------------------+--+
16 rows selected (0.173 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select username ,month ,sum(salary) salary from t_access_time group by username,month;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0002
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0002/
INFO  : Starting Job = job_1539182373092_0002, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0002/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0002
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2018-10-10 23:48:31,221 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-10-10 23:48:36,391 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.95 sec
INFO  : 2018-10-10 23:48:41,516 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 5.15 sec
INFO  : MapReduce Total cumulative CPU time: 5 seconds 150 msec
INFO  : Ended Job = job_1539182373092_0002
+-----------+----------+---------+--+
| username  |  month   | salary  |
+-----------+----------+---------+--+
| A         | 2015-01  | 62      |
| A         | 2015-02  | 12      |
| B         | 2015-01  | 170     |
| B         | 2015-02  | 30      |
+-----------+----------+---------+--+
4 rows selected (16.206 seconds)
[ 
[email protected] apps]# beeline
Beeline version 1.2.2 by Apache Hive
beeline>  !connect jdbc:hive2://hdp-nn-01:10000
Connecting to jdbc:hive2://hdp-nn-01:10000
Enter username for jdbc:hive2://hdp-nn-01:10000: root
Enter password for jdbc:hive2://hdp-nn-01:10000:
Connected to: Apache Hive (version 1.2.2)
Driver: Hive JDBC (version 1.2.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> set hive.exec.mode.local.auto=true;
No rows affected (0.057 seconds)
0: jdbc:hive2://hdp-nn-01:10000> create table t_temp1 as select username ,month ,sum(salary) salary from t_access_time group by username,month;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_local164879396_0001
INFO  : The url to track the job: http://localhost:8080/
INFO  : Job running in-process (local Hadoop)
INFO  : 2018-10-10 23:52:25,322 Stage-1 map = 100%,  reduce = 100%
INFO  : Ended Job = job_local164879396_0001
INFO  : Moving data to: hdfs://hdp-nn-01:9000/user/hive/warehouse/t_temp1 from hdfs://hdp-nn-01:9000/user/hive/warehouse/.hive-staging_hive_2018-10-10_23-52-23_803_4672116048929793587-5/-ext-10001
INFO  : Table default.t_temp1 stats: [numFiles=1, numRows=4, totalSize=53, rawDataSize=49]
No rows affected (2.014 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select * from t_temp1;
+-------------------+----------------+-----------------+--+
| t_temp1.username  | t_temp1.month  | t_temp1.salary  |
+-------------------+----------------+-----------------+--+
| A                 | 2015-01        | 62              |
| A                 | 2015-02        | 12              |
| B                 | 2015-01        | 170             |
| B                 | 2015-02        | 30              |
+-------------------+----------------+-----------------+--+
4 rows selected (0.179 seconds)
0: jdbc:hive2://hdp-nn-01:10000> [[email protected] apps]# beeline
Beeline version 1.2.2 by Apache Hive
beeline>  !connect jdbc:hive2://hdp-nn-01:10000
Connecting to jdbc:hive2://hdp-nn-01:10000
Enter username for jdbc:hive2://hdp-nn-01:10000: root
Enter password for jdbc:hive2://hdp-nn-01:10000:
Connected to: Apache Hive (version 1.2.2)
Driver: Hive JDBC (version 1.2.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> select * from t_temp1 a  join t_temp1 b  on a.username=b.username;
INFO  : Execution completed successfully
INFO  : MapredLocal task succeeded
INFO  : Number of reduce tasks is set to 0 since there's no reduce operator
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0003
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0003/
INFO  : Starting Job = job_1539182373092_0003, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0003/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0003
INFO  : Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0
INFO  : 2018-10-10 23:56:53,180 Stage-3 map = 0%,  reduce = 0%
INFO  : 2018-10-10 23:56:58,333 Stage-3 map = 100%,  reduce = 0%, Cumulative CPU 1.17 sec
INFO  : MapReduce Total cumulative CPU time: 1 seconds 170 msec
INFO  : Ended Job = job_1539182373092_0003
+-------------+----------+-----------+-------------+----------+-----------+--+
| a.username  | a.month  | a.salary  | b.username  | b.month  | b.salary  |
+-------------+----------+-----------+-------------+----------+-----------+--+
| A           | 2015-01  | 62        | A           | 2015-01  | 62        |
| A           | 2015-02  | 12        | A           | 2015-01  | 62        |
| A           | 2015-01  | 62        | A           | 2015-02  | 12        |
| A           | 2015-02  | 12        | A           | 2015-02  | 12        |
| B           | 2015-01  | 170       | B           | 2015-01  | 170       |
| B           | 2015-02  | 30        | B           | 2015-01  | 170       |
| B           | 2015-01  | 170       | B           | 2015-02  | 30        |
| B           | 2015-02  | 30        | B           | 2015-02  | 30        |
+-------------+----------+-----------+-------------+----------+-----------+--+
8 rows selected (14.906 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select * from t_temp1 a  join t_temp1 b  on a.username=b.username where a.month <=b.month ;
INFO  : Execution completed successfully
INFO  : MapredLocal task succeeded
INFO  : Number of reduce tasks is set to 0 since there's no reduce operator
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0004
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0004/
INFO  : Starting Job = job_1539182373092_0004, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0004/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0004
INFO  : Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0
INFO  : 2018-10-11 00:01:42,367 Stage-3 map = 0%,  reduce = 0%
INFO  : 2018-10-11 00:01:46,472 Stage-3 map = 100%,  reduce = 0%, Cumulative CPU 1.24 sec
INFO  : MapReduce Total cumulative CPU time: 1 seconds 240 msec
INFO  : Ended Job = job_1539182373092_0004
+-------------+----------+-----------+-------------+----------+-----------+--+
| a.username  | a.month  | a.salary  | b.username  | b.month  | b.salary  |
+-------------+----------+-----------+-------------+----------+-----------+--+
| A           | 2015-01  | 62        | A           | 2015-01  | 62        |
| A           | 2015-01  | 62        | A           | 2015-02  | 12        |
| A           | 2015-02  | 12        | A           | 2015-02  | 12        |
| B           | 2015-01  | 170       | B           | 2015-01  | 170       |
| B           | 2015-01  | 170       | B           | 2015-02  | 30        |
| B           | 2015-02  | 30        | B           | 2015-02  | 30        |
+-------------+----------+-----------+-------------+----------+-----------+--+
6 rows selected (13.334 seconds)
0: jdbc:hive2://hdp-nn-01:10000> create table t_temp2 as
0: jdbc:hive2://hdp-nn-01:10000> select  
0: jdbc:hive2://hdp-nn-01:10000> a.username as aname,
0: jdbc:hive2://hdp-nn-01:10000> a.month as amonth,
0: jdbc:hive2://hdp-nn-01:10000> a.salary as asalary,
0: jdbc:hive2://hdp-nn-01:10000> b.username as bname,
0: jdbc:hive2://hdp-nn-01:10000> b.month as bmonth,
0: jdbc:hive2://hdp-nn-01:10000> b.salary as bsalary
0: jdbc:hive2://hdp-nn-01:10000>
0: jdbc:hive2://hdp-nn-01:10000> from t_temp1 a  join t_temp1 b  on a.username=b.username where a.month <=b.month ;
INFO  : Execution completed successfully
INFO  : MapredLocal task succeeded
INFO  : Number of reduce tasks is set to 0 since there's no reduce operator
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0005
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0005/
INFO  : Starting Job = job_1539182373092_0005, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0005/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0005
INFO  : Hadoop job information for Stage-4: number of mappers: 1; number of reducers: 0
INFO  : 2018-10-11 00:06:07,100 Stage-4 map = 0%,  reduce = 0%
INFO  : 2018-10-11 00:06:12,213 Stage-4 map = 100%,  reduce = 0%, Cumulative CPU 1.24 sec
INFO  : MapReduce Total cumulative CPU time: 1 seconds 240 msec
INFO  : Ended Job = job_1539182373092_0005
INFO  : Moving data to: hdfs://hdp-nn-01:9000/user/hive/warehouse/t_temp2 from hdfs://hdp-nn-01:9000/user/hive/warehouse/.hive-staging_hive_2018-10-11_00-06-00_196_2056446029980412768-7/-ext-10001
INFO  : Table default.t_temp2 stats: [numFiles=1, numRows=6, totalSize=159, rawDataSize=153]
No rows affected (13.422 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select * from t_temp2;
+----------------+-----------------+------------------+----------------+-----------------+------------------+--+
| t_temp2.aname  | t_temp2.amonth  | t_temp2.asalary  | t_temp2.bname  | t_temp2.bmonth  | t_temp2.bsalary  |
+----------------+-----------------+------------------+----------------+-----------------+------------------+--+
| A              | 2015-01         | 62               | A              | 2015-01         | 62               |
| A              | 2015-01         | 62               | A              | 2015-02         | 12               |
| A              | 2015-02         | 12               | A              | 2015-02         | 12               |
| B              | 2015-01         | 170              | B              | 2015-01         | 170              |
| B              | 2015-01         | 170              | B              | 2015-02         | 30               |
| B              | 2015-02         | 30               | B              | 2015-02         | 30               |
+----------------+-----------------+------------------+----------------+-----------------+------------------+--+
6 rows selected (0.103 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select bname,bmonth,sum(asalary) from t_temp2 group by bname,bmonth, bsalary;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0006
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0006/
INFO  : Starting Job = job_1539182373092_0006, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0006/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0006
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2018-10-11 00:11:48,163 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-10-11 00:11:53,351 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.38 sec
INFO  : 2018-10-11 00:11:58,484 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.44 sec
INFO  : MapReduce Total cumulative CPU time: 3 seconds 440 msec
INFO  : Ended Job = job_1539182373092_0006
+--------+----------+------+--+
| bname  |  bmonth  | _c2  |
+--------+----------+------+--+
| A      | 2015-01  | 62   |
| A      | 2015-02  | 74   |
| B      | 2015-01  | 170  |
| B      | 2015-02  | 200  |
+--------+----------+------+--+
4 rows selected (15.245 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select bname,bmonth,bsalary,sum(asalary) as tsalary  from t_temp2 group by bname,bmonth, bsalary;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0007
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0007/
INFO  : Starting Job = job_1539182373092_0007, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0007/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0007
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2018-10-11 00:12:50,334 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-10-11 00:12:55,495 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.2 sec
INFO  : 2018-10-11 00:12:59,609 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.22 sec
INFO  : MapReduce Total cumulative CPU time: 3 seconds 220 msec
INFO  : Ended Job = job_1539182373092_0007
+--------+----------+----------+----------+--+
| bname  |  bmonth  | bsalary  | tsalary  |
+--------+----------+----------+----------+--+
| A      | 2015-01  | 62       | 62       |
| A      | 2015-02  | 12       | 74       |
| B      | 2015-01  | 170      | 170      |
| B      | 2015-02  | 30       | 200      |
+--------+----------+----------+----------+--+
4 rows selected (13.917 seconds)
0: jdbc:hive2://hdp-nn-01:10000> create table t_final as select bname,bmonth,bsalary,sum(asalary) as tsalary  from t_temp2 group by bname,bmonth, bsalary;
INFO  : Number of reduce tasks not specified. Estimated from input data size: 1
INFO  : In order to change the average load for a reducer (in bytes):
INFO  :   set hive.exec.reducers.bytes.per.reducer=<number>
INFO  : In order to limit the maximum number of reducers:
INFO  :   set hive.exec.reducers.max=<number>
INFO  : In order to set a constant number of reducers:
INFO  :   set mapreduce.job.reduces=<number>
INFO  : number of splits:1
INFO  : Submitting tokens for job: job_1539182373092_0008
INFO  : The url to track the job: http://hdp-nn-01:8088/proxy/application_1539182373092_0008/
INFO  : Starting Job = job_1539182373092_0008, Tracking URL = http://hdp-nn-01:8088/proxy/application_1539182373092_0008/
INFO  : Kill Command = /root/apps/hadoop-2.6.5/bin/hadoop job  -kill job_1539182373092_0008
INFO  : Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
INFO  : 2018-10-11 00:20:12,409 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-10-11 00:20:17,539 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.68 sec
INFO  : 2018-10-11 00:20:23,692 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.75 sec
INFO  : MapReduce Total cumulative CPU time: 3 seconds 750 msec
INFO  : Ended Job = job_1539182373092_0008
INFO  : Moving data to: hdfs://hdp-nn-01:9000/user/hive/warehouse/t_final from hdfs://hdp-nn-01:9000/user/hive/warehouse/.hive-staging_hive_2018-10-11_00-20-08_776_6311242436066492297-7/-ext-10001
INFO  : Table default.t_final stats: [numFiles=1, numRows=4, totalSize=67, rawDataSize=63]
No rows affected (16.612 seconds)
0: jdbc:hive2://hdp-nn-01:10000> select * from t_final;
+----------------+-----------------+------------------+------------------+--+
| t_final.bname  | t_final.bmonth  | t_final.bsalary  | t_final.tsalary  |
+----------------+-----------------+------------------+------------------+--+
| A              | 2015-01         | 62               | 62               |
| A              | 2015-02         | 12               | 74               |
| B              | 2015-01         | 170              | 170              |
| B              | 2015-02         | 30               | 200              |
+----------------+-----------------+------------------+------------------+--+
4 rows selected (0.138 seconds)
0: jdbc:hive2://hdp-nn-01:10000>

hive 求月訪問記錄，累計訪問記錄

求出當月記錄和累計記錄，如2月累計記錄=1月+2月記錄，3月累計記錄=1月+2月+3月記錄 A,2015-01,1 A,2015-01,11 A,2015-01,12 A,2015-01,13 B,2015-01,10 B,2015-01,20 B,2015-01,30 B

執行bash指令碼，訪問網頁，生成日誌記錄

1.編寫bash指令碼 #!/bin/bash step=1 #間隔的秒數，不能大於60 host_list=("localhost" "192.168.75.137" "master1"

通過設定ie的通過跨域訪問資料來源，來訪問本地服務

1、首先設定通過域訪問資料來源設定通過域訪問資料來源 2、javascript指令碼ajax使用本地服務登入（評價，人證的類似）介面 <html> <head> <script type="text/javascript"> f

Hive 求月銷售額和總銷售額

str rmi select limit row mina roo del ive 求月銷售額和總銷售額a,01,150a,01,200b,01,1000b,01,800c,01,250c,01,220b,01,6000a,02,2000a,02,3000b,02,1000

FileZilla上傳圖片成功，瀏覽器可以正常訪問nginx，但是訪問圖片報404

1、在/usr/local/nginx/html目錄下新建images資料夾 cd /usr/local/nginx/html mkdir images 2、新增一個location cd /usr/local/nginx/conf

kubernetes RBAC實戰 kubernetes 使用者角色訪問控制，dashboard訪問，kubectl配置生成_Kubernetes中文社群

kubernetes RBAC實戰環境準備先用kubeadm安裝好kubernetes叢集，[包地址在此](https://market.aliyun.com/products/56014009/cmxz022571.html#sku=yuncode1657100000) 好用又方便，服務

瀏覽器訪問tomcat，頁面訪問出錯404,405,500

在剛開始進行Web應用開發的時候，經常看見頁面出現404這個數字，我們一般會稱之為執行產生了404錯誤。類似於404這個數字，還有可能在頁面上看到405、500這兩個數字，他們都是伺服器執行完客戶端的請求以後，返回給客戶端的一個關於執行結果的狀態編碼說明。如果

kubernetes RBAC實戰 kubernetes 使用者角色訪問控制，dashboard訪問，kubectl配置生成

kubernetes RBAC實戰環境準備先用kubeadm安裝好kubernetes叢集，[包地址在此](https://market.aliyun.com/products/56014009/cmxz022571.html#sku=yuncode1657100

微信小程式，安卓訪問正常，蘋果訪問不執行wx.request

今天遇到個奇怪的問題，客戶打電話來說小程式出問題了，安卓手機訪問是正常的，可是蘋果手機卻不能訪問。開啟開發工具查了一下，沒有問題，用蘋果手機進入除錯模組看輸入結果，發現每次都停在request begin這裡，後臺也無法獲取到前臺傳送過來的資料，所以斷定小程式沒有執行re

目前研究生二年級，還有兩個月找工作，開貼記錄自己學習經歷

本科普通211高校，研究生末流985，光學工程專業就讀，由於老師的專案很坑比，靠專案找不到工作，而且老師還push的厲害。但是沒有辦法，找工作要緊，準備自學演算法。買了C語言，資料結構還有演算法書，現在開始自學。不知道有沒有高手指點下，學習這些夠不夠，目前還不確定自己就業方向

oracle查詢重複記錄，去除重複記錄

轉載自http://blog.163.com/aner_rui/blog/static/12131232820105901451809/ SELECT * FROM t_info a WHERE ((SELECT COUNT(*) FROM t_i

c語言每隔1秒向檔案中寫入一行記錄，類似日誌記錄

讀寫一個檔案test.txt，每隔1秒向檔案中寫入一行記錄，直到按Ctrl-C終止。下次啟動程式時在test.txt檔案末尾追加記錄，並且序號能夠接續上次的序號。 #include <stdio

spring aop實現（使用者操作記錄，一場記錄）

第一步定義兩個註解： Java程式碼複製程式碼 package com.annotation; import java.lang.annotation.*; /** *自定義註解攔截Controller */ @Target({ElementType.PA

記一次基於雲服務開發文件線上編輯系統的開發記錄，支援版本記錄、可增加批註。

　　從工作實習的時候我就參與了一個專案叫做“雲文件管理系統”，說白了就是檔案的上傳、下載、預覽、分享、回收站等等一些操作。上傳下載以及分享都很Easy，複雜的就在文件預覽上，圖片、視訊、音訊都有現成的外掛可以使用，Office文件的線上預覽相對來說還是比較複雜的，當時也是看好多把Off

基於歷史資料的使用者訪問次數，每天新老使用者，日活，周活，月活的hive計算

最近有一個需求，統計每天的新老使用者，日活，周活，月活。我們每天的增量資料會加入到hive歷史資料表中，包含使用者訪問網站的一些資訊，欄位有很多，包括使用者唯一標識guid。當然了日活，周活，月活就是一個count(distinct(guid))語句，非常常用的sql。但是

LNMP架構（nginx訪問日誌，Nginx日誌切割，靜態文件不記錄訪問日誌）

tab ane pac 12月計劃 asdasd sans min ref 一、nginx訪問日誌[root@lnmp ~]# vim /usr/local/nginx/conf/nginx.conf （修改Nginx的配置文件）搜索/log_form

Nginx配置：訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

Nginx配置訪問日誌一、訪問日誌 1、查看Nginx日誌格式 [root@zhulinux-02 ~]# grep -A2 log_format /usr/local/nginx/conf/nginx.conf log_format combined_realip ‘$remote_ad

nginx訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

nginx訪問日誌日誌切割靜態文件不記錄日誌靜態文件不記錄過期時間 nginx訪問日誌日誌格式vim /usr/local/nginx/conf/nginx.conf //搜索log_format 編輯虛擬主機配置文件，配置日誌格式，（註意日誌後面的名稱與主配置文件保持一致）驗證訪問其

91.Nginx配置：訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

Nginx配置：訪問日誌日誌切割靜態一、訪問日誌 1、查看Nginx日誌格式 [root@sdwaqw ~]# grep -A2 log_format /usr/local/nginx/conf/nginx.conflog_format combined_realip ‘$remote_addr $http_

12.10-12.12 Nginx的訪問日誌，日誌切割，靜態文件不記錄日誌和過期時間

nginx訪問日誌12.10 Nginx訪問日誌12.11 Nginx日誌切割12.12 靜態文件不記錄日誌和過期時間12.10 Nginx訪問日誌1 打開配置文件，搜索/log_format，查看日誌文件格式。或者直接grep過濾出來[root@AliKvn vhost]# grep -A2 log_for

hive 求月訪問記錄，累計訪問記錄

相關推薦