rand()函式埋的一個坑，大家注意了
一、背景 在開發部落格網站隨機文章的時候，剛開始沒有注意到這個問題，隨便加了一個rand()函式，資料量小的時候效能還可以，一旦資料達到幾十萬的時候，就會產生效能問題。 二、問題分析 剛開始的程式碼如下：
if(!StringUtils.isEmpty(postParam.getSortType())){
if(postParam.getSortType().equals(PostConstant.SORTTYPE_COMMMENT)){
example.setOrderByClause(" comment_count desc ");
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_DATE)){
example.setOrderByClause(" post_date desc ");
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_RANDOM)){
example.setOrderByClause(" RAND() ");
log.info("開始載入隨機文章列表。。。。");
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_VIEW)){
example.setOrderByClause(" post_date desc ");
}
}else{
example.setOrderByClause(" post_date desc ");
}
Page
啟動程式，檢視隨機文章，後臺日誌報錯
INFO | 2018-11-19 18:43:53,040 | JWordpres-v2.0 | [http-nio-80-exec-17-47] (o.s.b.f.x.XmlBeanDefinitionReader:317) | Loading XML bean definitions from class path resource [org/springframework/jdbc/support/sql-error-codes.xml]
INFO | 2018-11-19 18:43:53,202 | JWordpres-v2.0 | [http-nio-80-exec-17-47] (o.s.j.s.SQLErrorCodesFactory:126) | SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase, Hana]
ERROR | 2018-11-19 18:43:53,231 | JWordpres-v2.0 | [http-nio-80-exec-17-47] (o.a.c.c.C.[.[.[.[dispatcherServlet]:181) | Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.jdbc.UncategorizedSQLException:
Error querying database. Cause: java.sql.SQLException: Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it
The error may exist in cn/liuhaihua/web/mapper/WpPostsMapper.java (best guess)
The error may involve cn.liuhaihua.web.mapper.WpPostsMapper.selectByExample-Inline
The error occurred while setting parameters
SQL: SELECT id,post_author,post_date,post_date_gmt,post_content,post_title,post_excerpt,post_status,comment_status,ping_status,post_password,post_name,to_ping,pinged,post_modified,post_modified_gmt,post_content_filtered,post_parent,guid,menu_order,post_type,post_mime_type,comment_count FROM wp_posts WHERE ( post_type = ? and post_status = ? ) order by RAND() LIMIT 10
Cause: java.sql.SQLException: Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it
; uncategorized SQLException for SQL []; SQL state [HY000]; error code [126]; Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it; nested exception is java.sql.SQLException: Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it] with root cause
java.sql.SQLException: Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it
檢視sql 發現如下程式碼：
SELECT
id,
post_author,
post_date,
post_date_gmt,
post_content,
post_title,
post_excerpt,
post_status,
comment_status,
ping_status,
post_password,
post_name,
to_ping,
pinged,
post_modified,
post_modified_gmt,
post_content_filtered,
post_parent,
guid,
menu_order,
post_type,
post_mime_type,
comment_count
FROM
wp_posts
WHERE
(
post_type = "post"
AND post_status = "publish"
)
ORDER BY
RAND()
LIMIT 10
這個sql會造成嚴重的效能問題，rand()造成在系統檔案上來回排序。非常損耗效能
[Err] 126 - Incorrect key file for table '/tmp/#sql_7c6d_0.MYI'; try to repair it
三、優化方案 知道問題所在，優化方案其實也蠻簡單的。原理如下
1首先 select count(*) from test where $where; （計算所需要的資料的總條數）
2然後 $id=rand($a[0],$a[1]); 產生一個隨機數;
3最後 SELECT * FROM tablename WHERE id>='$id' LIMIT 1 將上面產生的隨機數帶入查詢;
修改程式碼如下：
if(!StringUtils.isEmpty(postParam.getSortType())){
if(postParam.getSortType().equals(PostConstant.SORTTYPE_COMMMENT)){
example.setOrderByClause(" comment_count desc ");
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_DATE)){
example.setOrderByClause(" post_date desc ");
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_RANDOM)){
//example.setOrderByClause(" RAND() ");
/***
- 首先 select count(*) from test where $where; （計算所需要的資料的總條數）
*然後 $id=rand($a[0],$a[1]); 產生一個隨機數;
*最後 SELECT * FROM tablename WHERE id>='$id' LIMIT 1 將上面產生的隨機數帶入查詢;
*/
log.info("開始載入隨機文章列表。。。。");
Random random = new Random();
int randId =random.nextInt(count);
criteria.andGreaterThan("id", randId);
}else if(postParam.getSortType().equals(PostConstant.SORTTYPE_VIEW)){
example.setOrderByClause(" post_date desc ");
}
}else{
example.setOrderByClause(" post_date desc ");
}
Page
這樣修改後之後，系統完美執行，sql查詢時間縮短到0.058秒 四、總結 其實mysql官網也說明這種情況了，意思是說當記錄超過30萬，rand這種方法就不可用,需要更換方案。
works for small tables, but once the tables grow larger than 300,000 records or
so this will be very slow because MySQL will have to process ALL the entries from the table,
order them randomly and then return the first row of the ordered result,
and this sorting takes long time.
Instead you can do it like this (atleast if you have an auto_increment PK):
官方建議修改成這樣
SELECT MIN(id), MAX(id) FROM tablename;
Fetch the result into $a
$id=rand($a[0],$a[1]);
SELECT * FROM tablename WHERE id>='$id' LIMIT 1
原文釋出時間為：2018-11-20 本文作者：HARRIES 本文來自雲棲社群合作伙伴“Java雜記”，瞭解相關資訊可以關注“Java雜記”。