python 64式: 第20式、sqlalchemy進行資料庫操作與alembic進行資料庫升級
文章目錄編排如下:
1 引言
2 使用sqlalchemy實現資料庫增刪改查
3 使用alembic進行資料庫升級
4 總結
1 引言
sqlalchemy是python專案採用的ORM(物件關係對映)，主要用於資料庫相關的操作。
而alembic是與sqlalchemy搭配使用的資料庫升級工具，主要用於資料庫相關表結構的修改升級。
基於這篇文章:
python 64式: 第18式、python專案通用rest api框架搭建與編寫
的基礎上，介紹編寫sqlalchemy與alembic部分。
前提:已經建立了專案myproject
2 使用sqlalchemy實現資料庫增刪改查
2.1 在myproject下面建立db目錄
2.2 db目錄下建立__init__.py檔案，檔案內容如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import retrying
import six.moves.urllib.parse as urlparse
from stevedore import driver
G_NAMESPACE = "myproject.storage"
def getConnectionFromConfig(conf):
url = conf.database.connection
connectionScheme = urlparse.urlparse(url).scheme
mgr = driver.DriverManager(G_NAMESPACE, connectionScheme)
retries = conf.database.max_retries
@retrying.retry(wait_fixed=conf.database.retry_interval * 1000,
stop_max_attempt_number=retries if retries >= 0 else None)
def getConnection():
return mgr.driver(conf)
connection = getConnection()
return connection
分析:
1) 這裡最重要的程式碼如下:
mgr = driver.DriverManager(G_NAMESPACE, connectionScheme)
主要是採用stevedore.driver.DriverManager來獲取對應的資料庫外掛。
stevedore.driver.DriverManager :一個名字對應一個entry point。根據外掛名稱空間和名字，定位到單獨外掛
stevedore.driver.DriverManager(namespace, name, invoke_on_load, invoke_args=(), invoke_kwds={})
namespace: 名稱空間
name: 外掛名稱
invoke_on_load:如果為True，表示會例項化該外掛的類
invoke_args:呼叫外掛物件時傳入的位置引數
invoke_kwds:傳入的字典引數
2.3 設定資料庫外掛
修改myproject/setup.cfg中的內容為如下內容:
[metadata]
name = myproject
version = 1.0
summary = myproject
description-file =
README.rst
author = me
author-email =
classifier =
Intended Audience :: Developers
Programming Language :: Python :: 2.7
[global]
setup-hooks =
pbr.hooks.setup_hook
[files]
packages =
myproject
data_files =
/etc/myproject = etc/myproject/*
/var/www/myproject = etc/apache2/app.wsgi
/etc/httpd/conf.d = etc/apache2/myproject.conf
[entry_points]
wsgi_scripts =
myproject-api = myproject.api.app:build_wsgi_app
console_scripts =
myproject-dbsync = myproject.cmd.storage:dbsync
oslo.config.opts =
myproject = myproject.opts:list_opts
myproject.storage =
mysql = myproject.db.mariadb.impl_mariadb:Connection
mysql+pymysql = myproject.db.mariadb.impl_mariadb:Connection
分析:
1) 其中最重要的內容是在[entry_points]下面新增了:
myproject.storage =
mysql = myproject.db.mariadb.impl_mariadb:Connection
mysql+pymysql = myproject.db.mariadb.impl_mariadb:Connection
來表示myproject.storage的名稱空間下，mysql或者mysql+pymysql這個外掛名稱對應的
處理物件是: myproject.db.mariadb.impl_mariadb下面的Connection類
2) 外掛格式
名稱空間 =
外掛名稱=模組:可匯入物件
2.4 定義資料庫模型
在db目錄下新建models.py,具體內容如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sqlalchemy import Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Index
from sqlalchemy import Integer
from sqlalchemy import PrimaryKeyConstraint
from sqlalchemy import String
Base = declarative_base()
class Student(Base):
__tablename__ = 'student'
id = Column(Integer(), nullable=False)
userId = Column(String(256), nullable=False)
name = Column(String(256), nullable=False)
age = Column(Integer(), nullable=True)
email = Column(String(256), nullable=True)
__table_args__ = (
Index('ix_student_userId', 'userId'),
PrimaryKeyConstraint('id')
)
TRANSFORMED_FIELDS = ['id', 'userId', 'name', 'age', 'email']
# from database model to dict
def as_dict(self):
result = dict()
for field in Student.TRANSFORMED_FIELDS:
result[field] = getattr(self, field, '')
return result
分析:
1) as_dict方法主要是用於將models.Student物件轉換為字典
2.5 為請求繫結database hook, config hook
2.5.1 在myproject的api目錄下新建hooks.py
具體內容如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from pecan import hooks
class ConfigHook(hooks.PecanHook):
"""Attach config information to the request
"""
def __init__(self, conf):
self.conf = conf
def before(self, state):
state.request.cfg = self.conf
class DatabaseHook(hooks.PecanHook):
"""Attach database information to the request
"""
def __init__(self, conn):
self.storage = conn
def before(self, state):
state.request.storage = self.storage
分析:
1) 這個hook在每個請求進來的時候例項化一個db的Connection物件，然後在controller程式碼中我們可以直接使用這個Connection例項
具體參考:
https://pecan.readthedocs.io/en/latest/hooks.html
hook的作用:
pecan hooks是一種可以與框架互動的方式，不需要編寫單獨的中介軟體。
Hooks允許你在請求生命週期的關鍵時間點去執行程式碼:
on_route(): called before Pecan attempts to route a request to a controller
before(): called after routing, but before controller code is run
after(): called after controller code has been run
on_error(): called when a request generates an exception
參考:
https://www.sqlalchemy.org/library.html#tutorials
https://segmentfault.com/a/1190000004466246
2)修改myproject的api目錄下app.py
內容為如下內容:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
from oslo_config import cfg
from oslo_log import log
from paste import deploy
import pecan
from myproject.api import hooks
from myproject import service
from myproject import db
PECAN_CONFIG = {
'app': {
'root': 'myproject.api.controllers.root.RootController',
'modules': ['myproject.api'],
},
}
LOG = log.getLogger(__name__)
def app_factory(global_config, **local_config):
print "######### enter app_factory"
conf = service.prepareService()
# NOTE, add config and databse information to the request
# by using pecan.hooks.PecanHook
configHook = hooks.ConfigHook(conf)
conn = db.getConnectionFromConfig(conf)
databaseHook = hooks.DatabaseHook(conn)
appHooks = [configHook, databaseHook]
# NOTE, it needs add the line below
pecan.configuration.set_config(dict(PECAN_CONFIG), overwrite=True)
app = pecan.make_app(
PECAN_CONFIG['app']['root'],
hooks=appHooks
)
return app
def getUri(conf):
# TODO(), it needs to get real path of api-paste.ini
# the path is setted by the data_files under [files] in setup.config
# configPath = "/etc/myproject/api-paste.ini"
# find the absolute path of api-paste.ini
cfgFile = None
apiPastePath = conf.api.paste_config
if not os.path.isabs(apiPastePath):
cfgFile = conf.find_file(apiPastePath)
elif os.path.exists(apiPastePath):
cfgFile = apiPastePath
if not cfgFile:
raise cfg.ConfigFilesNotFoundError([conf.api.paste_config])
LOG.info("The wsgi config file path is: %s" % (cfgFile))
result = "config:" + cfgFile
return result
def getAppName():
return "main"
def build_wsgi_app():
print "######### enter build_wsgi_app"
conf = service.prepareService()
uri = getUri(conf)
appName = getAppName()
app = deploy.loadapp(uri, name=appName)
return app
分析:
1) 在app_factory方法中，設定了
configHook = hooks.ConfigHook(conf)
conn = db.getConnectionFromConfig(conf)
databaseHook = hooks.DatabaseHook(conn)
appHooks = [configHook, databaseHook]
# NOTE, it needs add the line below
pecan.configuration.set_config(dict(PECAN_CONFIG), overwrite=True)
app = pecan.make_app(
PECAN_CONFIG['app']['root'],
hooks=appHooks
)
主要的目的就是讓請求進入的時候可以繫結資料庫連線物件和配置物件，方便後面進行處理
2.6 編寫資料庫連線的基類
在db目錄下新建base.py，具體內容如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import abc
import six
@six.add_metaclass(abc.ABCMeta)
class Connection(object):
def __init__(self, conf, url):
pass
def upgrade(self):
""" Migrate database """
def getStudents(self):
""" Get student list """
def createStudent(self):
""" Create a student"""
def updateStudent(self):
""" Update a student"""
def deleteStudent(self):
""" Delere a student """
2.7 實現資料庫連線的子類
在db目錄下新建mariadb目錄，
2.7.1 在mariadb目錄下新建__init__.py
內容為空
2.7.2 在mariadb目錄下新建impl_mariadb.py
內容具體如下所示:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
from alembic import command
from alembic import config
from alembic import migration
from oslo_db.sqlalchemy import session as dbSession
from myproject.db import base
from myproject.db import models
class Connection(base.Connection):
def __init__(self, conf):
self.conf = conf
options = dict(conf.database)
options['max_retries'] = 0
self.engineFacade = dbSession.EngineFacade(
conf.database.connection,
**options)
def getAlembicConfig(self):
dirName = os.path.dirname(__file__)
realDirName = os.path.dirname(dirName)
path = "%s/migrate/alembic.ini" % (realDirName)
cfg = config.Config(path)
cfg.set_main_option(
'sqlalchemy.url', self.conf.database.connection)
return cfg
def upgrade(self, noCreate=False):
# import pdb;pdb.set_trace()
cfg = self.getAlembicConfig()
cfg.conf = self.conf
if noCreate:
command.upgrade(cfg, "head")
else:
engine = self.engineFacade.get_engine()
ctxt = migration.MigrationContext.configure(engine.connect())
currentVersion = ctxt.get_current_revision()
if currentVersion is None:
models.Base.metadata.create_all(engine, checkfirst=False)
command.stamp(cfg, "head")
else:
command.upgrade(cfg, "head")
@staticmethod
def rowToStudentModel(row):
result = models.Student(
id=row.id,
userId=row.userId,
name=row.name,
age=row.age,
email=row.email
)
return result
def retrieveStudents(self, query):
return (self.rowToStudentModel(obj) for obj in query.all())
def getStudents(self, id=None, userId=None, name=None,
age=None, email=None, pagination=None):
# NOTE, pagination is needed
pagination = pagination or {}
session = self.engineFacade.get_session()
query = session.query(models.Student)
if id is not None:
query = query.filter(models.Student.id == id)
if userId is not None:
query = query.filter(models.Student.userId == userId)
if name is not None:
query = query.filter(models.Student.name == name)
if age is not None:
query = query.filter(models.Student.age == age)
if email is not None:
query = query.filter(models.Student.email == email)
# TODO(), add pagination query
students = self.retrieveStudents(query)
return students
def createStudent(self, obj):
session = self.engineFacade.get_session()
row = models.Student(
userId=obj.userId,
name=obj.name,
age=obj.age,
email=obj.email
)
with session.begin():
session.add(row)
return row
def updateStudent(self, obj):
session = self.engineFacade.get_session()
with session.begin():
count = session.query(models.Student).filter(
models.Student.id == obj.id).update(
obj.as_dict()
)
if not count:
raise "Not found student with id: %s" % obj.id
return obj
def deleteStudent(self, id):
session = self.engineFacade.get_session()
with session.begin():
session.query(models.Student).filter(
models.Student.id == id).delete()
分析:
1) upgrade(self, nocreate=False):
作用: 通過alembic判斷是否有版本，如果沒有版本就建立所有的表，並升級到最新版本；
如果有版本就直接升級到最新版本
處理過程:
步驟1: 獲取alembic的配置物件，設定alembic.ini中sqlalchemy.url為當前資料庫連線串
步驟2: 如果不建立表，就直接升級到最新版本；否則，執行步驟3
步驟3: 升級資料庫版本，具體如下:
步驟3.1: 根據 oslo_db.sqlalchemy.session.EngineFacade物件獲取engine
步驟3.2: 獲取enine.connect()做為引數傳遞給alembic.migration.MigrationContext.configure來得到
資料庫遷移上下文物件
步驟3.3: 獲取資料庫遷移上下文物件的當前版本
步驟3.3.1: 如果當前版本不存在，就通過models.Base.metadata.create_all(engine)建立所有表，
並通過alembic.command.stamp(cfg, 'head')給當前版本打上標記
否則，執行3.3.2
步驟3.3.2: 直接執行alembic.command.upgrade(cfg, 'head')升級到最新版本
2) _get_alembic_config(self):
作用: 初始化alembic.ini的配置物件，並設定alembic.ini中sqlalchemy.url的值為配置檔案中[database]下面
的connection對應的值，返回該配置物件
3) oslo_db.sqlalchemy.session.EngineFacade
作用:
1 類似SQLAlchemy Engine和Session物件的閘道器(外觀模式，類似一個代理)。
得到該物件後
2 管理資料庫連線，會話，事務
本質: 單例，具體參見C++如何獲取一個單例的寫法
用法:
1 init函式初始化獲得
_FACADE = None
def _create_facade_lazily():
global _FACADE
if _FACADE is None:
_FACADE = db_session.EngineFacade(
CONF.database.connection,
**dict(CONF.database.iteritems())
)
return _FACADE
2 從配置檔案初始化獲得
_ENGINE_FACADE = None
_LOCK = threading.Lock()
def _create_facade_lazily():
global _LOCK, _ENGINE_FACADE
if _ENGINE_FACADE is None:
with _LOCK:
if _ENGINE_FACADE is None:
_ENGINE_FACADE = db_session.EngineFacade.from_config(CONF)
return _ENGINE_FACADE
參考:
https://specs.openstack.org/openstack/oslo-specs/specs/kilo/make-enginefacade-a-facade.html
https://blog.csdn.net/bill_xiang_/article/details/78592389
3 使用alembic進行資料庫升級
3.1 在myproject下新建cmd目錄
cmd目錄存放各個指令碼或者服務的啟動程式碼。
3.1.1 在cmd目錄下新建__init__.py檔案
內容設定為空
3.1.2 在cmd目錄下新建storage.py檔案
內容設定為如下內容:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from myproject import service
from myproject import db
def dbsync():
conf = service.prepareService()
db.getConnectionFromConfig(conf).upgrade()
分析:
1) service.prepareService()是獲取了oslo.config物件
2) db.getConnectionFromConfig(conf).upgrade()
這句話就是讀取資料庫連線串根據呼叫對應的資料庫外掛物件的upgrade方法來
進行資料庫升級
3) 資料庫升級的程式碼實際就是myproject/db/mariadb/impl_mariadb.py
的Connection類的upgrade方法，具體如下:
def upgrade(self, noCreate=False):
# import pdb;pdb.set_trace()
cfg = self.getAlembicConfig()
cfg.conf = self.conf
if noCreate:
command.upgrade(cfg, "head")
else:
engine = self.engineFacade.get_engine()
ctxt = migration.MigrationContext.configure(engine.connect())
currentVersion = ctxt.get_current_revision()
if currentVersion is None:
models.Base.metadata.create_all(engine, checkfirst=False)
command.stamp(cfg, "head")
else:
command.upgrade(cfg, "head")
3.2 設定資料庫升級指令碼
具體參見myproject下的setup.cfg中
[entry_points]
下有如下一行內容
console_scripts =
myproject-dbsync = myproject.cmd.storage:dbsync
這表示會生成一個myproject-dbsync的指令碼，呼叫
myproject/cmd/storage.py中的dbsync方法，
該方法的具體定義請參見3.1.2
3.3 建立alembic配置
3.3.1 alembic生成資料庫遷移環境
進入到db/sqlalchemy目錄下執行
alembic init alembic
3.3.2 進入到db/sqlalchemy目錄下執行
alembic revision -m "Create student table"
在db/sqlalchemy/alembic/versions 目錄下生成一個檔案
xxx_create_student_table.py
然後向該檔案中填入建立表操作的資訊，具體樣例資訊內容如下:
"""Create student table
Revision ID: 677de41920e9
Revises:
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '677de41920e9'
down_revision = None
branch_labels = None
depends_on = None
def upgrade():
op.create_table(
'student',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('userId', sa.String(256), nullable=False),
sa.Column('name', sa.String(128), nullable=False),
sa.Column('age', sa.Integer(), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_index(
'ix_student_userId', 'student', ['userId'], unique=True)
def downgrade():
op.drop_table('student')
解釋:
alembic init :建立一個alembic倉庫
alembic revision -m '提交資訊': 建立一個新的版本檔案
alembic upgrade: 升級命令
參考:
https://www.baidu.com/link?url=Aad8bvURqUkaCJxNXjmIRKdzKE5XZrP02j_m9nndmJdhW6NE-p5jRAWYU5xY3HW5gQu9LlI52mSwJCFvniNWwq&wd=&eqid=da2ccd990001d02d000000055bfe26a4
3.4 生成myproject-dbsync檔案
具體在myproject目錄下執行:
python setup.py install
如果是centos環境，那麼會在/usr/bin目錄下生成myproject-dbsync檔案
3.5 修改配置檔案中的資料庫連線串
3.5.1 生成配置檔案(如果有，就不要執行這一步)
找到myproject-config-generator.conf檔案，該檔案內容如下:
[DEFAULT]
output_file = /etc/myproject/myproject.conf
wrap_width = 79
namespace = myproject
namespace = oslo.db
namespace = oslo.log
namespace = oslo.messaging
namespace = oslo.policy
執行命令:
oslo-config-generator --config-file=myproject-config-generator.conf
生成myproject.conf檔案，
3.5.2 找到myproject.conf裡面的
[database]
下面的
# The SQLAlchemy connection string to use to connect to the database. (string
# value)
# Deprecated group/name - [DEFAULT]/sql_connection
# Deprecated group/name - [DATABASE]/sql_connection
# Deprecated group/name - [sql]/connection
#connection = <None>
新增如下一行內容:
connection = mysql+pymysql://myproject:[email protected]/myproject?charset=utf
3.6 修改alembic配置檔案
修改alembic相關檔案如下:
3.6.1 alembic/env.py中的內容:
新增:
from myproject.db import models
修改
target_metadata = None
為
target_metadata = models.Base.metadata
解釋:
這樣做的原因是資料庫遷移需要知道具體對應資料庫的資訊，
這裡是設定資料庫的元資訊為當前應用的資料庫資訊
參考:
https://alembic.sqlalchemy.org/en/latest/autogenerate.html
3.6.2 alembic.ini中內容:
將
[alembic]
# path to migration scripts
# TODO(), edit it as real path
script_location = alembic
修改為:
[alembic]
# path to migration scripts
# TODO(), edit it as real path
script_location = myproject.db.migrate:alembic
解釋:
這裡主要是指定當前專案遷移指令碼的路徑，需要根據具體的專案路徑進行替換
參考:
https://alembic.sqlalchemy.org/en/latest/tutorial.html
將預設的sqlalchemy.url修改為如下內容
sqlalchemy.url =
解釋:
這裡原本主要是配置資料庫連線串的，因為在程式碼中配置了升級方法upgrade()
中通過如下方法中的set_main_option重新設定了sqlalchemy.url，所以
這裡替換成空值。
3.6.3 請確保alembic目錄和alembic.ini在同級目錄
樣例如下:
[[email protected] migrate]# tree
.
|-- alembic
| |-- env.py
| |-- env.pyc
| |-- __init__.py
| |-- README
| |-- script.py.mako
| `-- versions
| |-- 677de41920e9_create_student_table.py
| |-- 677de41920e9_create_student_table.pyc
| |-- 77cca51ca78e_add_email_to_student.py
| `-- __init__.py
|-- alembic.ini
`-- __init__.py
然後確保:
myproject/db/mariadb/impl_mariadb.py中Connection類的getAlembicConfig
方法中內容如下:
def getAlembicConfig(self):
dirName = os.path.dirname(__file__)
realDirName = os.path.dirname(dirName)
path = "%s/migrate/alembic.ini" % (realDirName)
cfg = config.Config(path)
cfg.set_main_option(
'sqlalchemy.url', self.conf.database.connection)
return cfg
注意:
請確保:
path = "%s/migrate/alembic.ini" % (realDirName)
這個路徑是正確的，請根據實際路徑進行修改
3.7 建立資料庫
執行如下命令來建立資料庫:
mysql -e "create database IF NOT EXISTS myproject"
mysql -e "GRANT ALL PRIVILEGES ON myproject.* TO 'myproject'@'localhost' IDENTIFIED BY 'password';"
mysql -e "GRANT ALL PRIVILEGES ON myproject.* TO 'myproject'@'%' IDENTIFIED BY 'password';"
3.8 第一次資料庫同步
3.8.1 修改myproject/db/models.py中內容為如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sqlalchemy import Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Index
from sqlalchemy import Integer
from sqlalchemy import PrimaryKeyConstraint
from sqlalchemy import String
Base = declarative_base()
class Student(Base):
__tablename__ = 'student'
id = Column(Integer(), nullable=False)
userId = Column(String(256), nullable=False)
name = Column(String(256), nullable=False)
age = Column(Integer(), nullable=True)
# email = Column(String(256), nullable=True)
__table_args__ = (
Index('ix_student_userId', 'userId'),
PrimaryKeyConstraint('id')
)
TRANSFORMED_FIELDS = ['id', 'userId', 'name', 'age', 'email']
# from database model to dict
def as_dict(self):
result = dict()
for field in Student.TRANSFORMED_FIELDS:
result[field] = getattr(self, field, '')
return result
注意: 這裡先註釋了email欄位，後續以升級形式新增email欄位到student表中
3.8.2 資料庫同步
執行:
/usr/bin/myproject-dbsync
注意: 不同環境下生成的yproject-dbsync路徑不同，請根據實際情況進行處理，這裡是centos環境的
進入mysql,執行如下命令:
use myproject;
show tables;
可以看到:
MariaDB [myproject]> select * from alembic_version;
+--------------+
| version_num |
+--------------+
| 677de41920e9 |
MariaDB [myproject]> desc student;
+--------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| userId | varchar(256) | NO | MUL | NULL | |
| name | varchar(256) | NO | | NULL | |
| age | int(11) | YES | | NULL | |
+--------+--------------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
3.9 資料庫升級
3.9.1 修改myproject/db/models.py中內容為如下:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from sqlalchemy import Column
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Index
from sqlalchemy import Integer
from sqlalchemy import PrimaryKeyConstraint
from sqlalchemy import String
Base = declarative_base()
class Student(Base):
__tablename__ = 'student'
id = Column(Integer(), nullable=False)
userId = Column(String(256), nullable=False)
name = Column(String(256), nullable=False)
age = Column(Integer(), nullable=True)
email = Column(String(256), nullable=True)
__table_args__ = (
Index('ix_student_userId', 'userId'),
PrimaryKeyConstraint('id')
)
TRANSFORMED_FIELDS = ['id', 'userId', 'name', 'age', 'email']
# from database model to dict
def as_dict(self):
result = dict()
for field in Student.TRANSFORMED_FIELDS:
result[field] = getattr(self, field, '')
return result
注意: 這裡取消註釋email欄位，驗證升級後email欄位會新增到student表中
3.9.2 生成版本遷移檔案
這裡在myproject/db/migrate/alembic/versions目錄下生成 77cca51ca78e_add_email_to_student.py 檔案
其中: 77cca51ca78e 就是版本號
具體內容如下:
"""add email to student
Revision ID: 77cca51ca78e
Revises: 677de41920e9
"""
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision = '77cca51ca78e'
down_revision = '677de41920e9'
branch_labels = None
depends_on = None
def upgrade():
op.add_column('student', sa.Column('email', sa.String(256), nullable=True))
def downgrade():
op.drop_column('student', 'email')
分析:
1) down_revision 就是上一個版本的版本號，請根據實際情況修改
2) upgrade()定義了資料庫升級的操作，這裡是新增email欄位到student表中
3) upgrade()定義了資料庫降級的操作，這裡是從student表中刪除email欄位
3.9.3 再次執行/usr/bin/myproject-dbsync
結果如下:
MariaDB [myproject]> desc student;
+--------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| userId | varchar(256) | NO | MUL | NULL | |
| name | varchar(256) | NO | | NULL | |
| age | int(11) | YES | | NULL | |
| email | varchar(256) | YES | | NULL | |
+--------+--------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)
MariaDB [myproject]> select * from alembic_version;
+--------------+
| version_num |
+--------------+
| 77cca51ca78e |
+--------------+
驗證升級成功，且版本更新到最新
3.10 alembic常用api
alembic命令解釋
3.10.1
alembic.op.create_table(table_name, *columns, **kws)
作用: 使用當前遷移上下文來執行一個建立表的命令
引數:
table_name: 表名
columns: 陣列， 每個元素都是一個sqlalchemy.Column列物件，
可能還包括sqlalchemy.Index物件和sqlalchemy.PrimaryKeyConstraint物件等
kws: 字典引數，例如: mysql_engine='InnoDB', mysql_charset='utf8'等
3.10.2
alembic.op.create_index(index_name, table_name, columns, schema=None, unique=False, **kw)
作用: 使用當前遷移上下文來建立一個索引
引數:
index_name: 索引名稱
table_name: 表名
columns: 列名的陣列
unique: 如果為True,建立一個唯一的索引
3.10.3
sqlalchemy.PrimaryKeyConstraint(*columns, **kw)
作用: 設定表的主鍵
引數:
columns: 列名陣列，即要設定為主鍵的一個或多個列名組成的陣列
kw: 字典引數
3.10.4
sqlalchemy.UniqueConstraint(*columns, **kw)
作用: 設定表的唯一性索引。即確保某個列或某幾個列的值在表中是唯一的。
往往用於列去重。
引數:
columns: 列名陣列，即要設定為主鍵的一個或多個列名組成的陣列
kw: 字典引數
用法示例:
sqlalchemy.UniqueConstraint('name','addr',name='unique_mytable_name_addr')
3.10.5
sqlalchemy.Column(*args, **kwargs)
作用: 表的列名及相關屬性設定
引數:
args: 陣列引數，例如: 列名，列欄位型別
kwargs: 字典引數，例如: 是否可以為空等。
nullable: 為True表示該列可為空；為False表示該列不可以為空
3.10.6
alembic.op.add_column(table_name, column, schema=None)
作用: 根據當前遷移上下文環境新增某個列
引數:
table_name:表名
column: sqlalchemy.Column列物件
alembic.op.drop_column(table_name, column_name, schema=None, **kw)
作用: 根據當前遷移上下文環境刪除某列
引數:
table_name: 表名
column_name: 列名，字串型別
alembic.op.alter_column(table_name, column_name, nullable=None,
server_default=False, new_column_name=None, type_=None,
existing_type=None, existing_server_default=False,
existing_nullable=None, schema=None, **kw)
參考:
https://alembic.sqlalchemy.org/en/latest/ops.html#alembic.operations.Operations.create_table
http://alembic.zzzcomputing.com/en/latest/ops.html
https://alembic.sqlalchemy.org/en/latest/tutorial.html
3.11 其他問題
3.11.1 丟失alembic錯誤
執行:
/usr/bin/myproject-dbsync
報錯:
2018-11-28 14:52:26.357 31559 ERROR myproject File "/usr/lib/python2.7/site-packages/myproject/db/mariadb/impl_mariadb.py", line 12, in <module>
2018-11-28 14:52:26.357 31559 ERROR myproject from myproject.db import models
2018-11-28 14:52:26.357 31559 ERROR myproject File "/usr/lib/python2.7/site-packages/myproject/db/models.py", line 4, in <module>
2018-11-28 14:52:26.357 31559 ERROR myproject from sqlalchemy import Column
2018-11-28 14:52:26.357 31559 ERROR myproject ImportError: cannot import name Column
分析:
丟失amebic檔案
嘗試:
package_data={
'package': ['./path/*.xsd'],
},
字典的鍵必須是你的真實包名，值必須是要包含的模式的列表。要包含Package 2/http/api/api.yaml：
package_data={
'package2': ['http/api/*.yaml'],
},
列出所有非python檔案和模式。
另一種方法是建立MANIFEST.in(通常用於源分發)和https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files
include_package_data=True,
setup()。
參考:
https://cloud.tencent.com/developer/ask/143280
https://blog.csdn.net/s1234567_89/article/details/53008444
https://docs.python.org/2/distutils/sourcedist.html#the-manifest-in-template
setup.cfg設定package_data
最終解決辦法:
在setup.cfg中修改為如下內容:
import setuptools
setuptools.setup(
# FIXME(), if not add package_data and include_package_data
# some files which ends with .ini or other will be missed
package_data={
# If any package contains *.ini or *.txt files, include them:
'': ['*.ini', '*.mako', '*.yaml', '*.txt', '*.py', 'README', '*.json', '*.wsgi'],
},
include_package_data=True,
setup_requires=['pbr'],
pbr=True)
3.11.2 資料庫連線失敗
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory")
解決方法:
[[email protected] alembic]# systemctl status mariadb
● mariadb.service - MariaDB 10.1 database server
Loaded: loaded (/usr/lib/systemd/system/mariadb.service; disabled; vendor preset: disabled)
Active: inactive (dead)
[[email protected] alembic]# systemctl restart mariadb
解決
4 總結
python專案中通過sqlalchemy來操作資料庫，進行增刪改查等；通過alembic進行資料庫的升級