1. 程式人生 > >[樂意黎]MySQL使用事務及 PDO 插入大批量資料

[樂意黎]MySQL使用事務及 PDO 插入大批量資料

<?php
$serverIP = "127.0.0.1";
$userName= "user";
$passWord= "pass";
$dbname = "aerchi";
 
try {
    $pdo = new PDO("mysql:host=$serverIP;dbname=$dbName", $userName, $passWord);
    // set the PDO error mode to exception
    $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

    $pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
 
    // begin transaction
    $pdo->beginTransaction();
    // SQL 
    for($n=0; $n<($multi_items_size+1); $n++){
        $pdo->exec("INSERT INTO userInfo(name, phone, email) VALUES ('zhangsan', '13808710001', '
[email protected]
')"); //$pdo->exec("INSERT INTO userInfo(name, phone, email) VALUES ('lisi', '13808710002', '[email protected]')"); //$pdo->exec("INSERT INTO userInfo(name, phone, email) VALUES ('wanger', '13808710003', '[email protected]')"); //... } //commit $pdo->commit(); echo "insert records successful"; //return true; } catch(PDOException $e) { // failure, rollback $pdo->rollback(); echo $sql . "::<br>\r\n" . $e->getMessage(); //return false; } $pdo = null; ?>

不開啟事務,是每個for迴圈就是個事務,1w個迴圈,就要開關1w個事務。當然慢了。聲明瞭事務,就只用一個事務,所以快。

You can change this value on the server by setting the max_allowed_packet’ variable.

出現上面的錯誤是因為資料庫表的 max_allowed_packet這個配置沒配置足夠大,因為預設的為4M的,後來我調為100M就沒報錯了

set global max_allowed_packet = 100*1024*1024*
1.setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
When you receive error like this:
General error: 2014 Cannot execute queries while other unbuffered queries are active.
This means that you need to uncomment the following:
On line 137 of “lib/Varien/Db/Adapter/Pdo/Mysql.php”, find:
#$this->_connection->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);
and then uncomment it out, so it should be
$this->_connection->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, true);


2.將查詢的結果集fetchAll()以釋放PDOStatement:
PDO::query裡的一段話:

If you do not fetch all of the data in a result set before issuing your next call to PDO::query(), your call may fail. Call PDOStatement::closeCursor() to release the database resources associated with the PDOStatement object before issuing your next call to PDO::query().

在執行的過程中,報出如下的錯誤:
.1. SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'cover' at row 479601 ...
原因是:插入欄位長度超過設定的長度
cover 欄位是用來存取 封面圖片Url的.

 `cover` varchar(160) DEFAULT NULL,改成 `cover` varchar(180) DEFAULT NULL,也就是把欄位長度改成20即可。
或者是
原因是:
在my.ini裡找到sql-mode=”STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION”
把其中的STRICT_TRANS_TABLES,去掉,或者把sqlmode=STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
註釋掉,然後重啟mysql就ok了 


2. SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '2015-10-15 17:41:44', '', 'http://mp.weixin.qq.com/s?__biz=MzU2ODU0OTk4MA==&mid=' at line 2

解決方法: 出現此種錯誤,一般是SQL 語句有問題,目前只有先嚐試列印錯誤來看。

$query = "INSERT INTO $useTable(artID, gzh_en, title, cover, summary, oriDate, content, ori_url, read_url)
        SELECT '$artID', '$gzh_en', '$title', '$cover', '$summary', '$lastModified', '$content', '$ori_url', '$read_url'
        FROM dual
        WHERE NOT EXISTS (".$existSqlStr_suffix.")";

$result = $bdd->exec($query);
if ($result === FALSE) {
    print_r($bdd->errorInfo());
    die("Query: " . $query);
}

 樂意黎

2018-12-10