1. 程式人生 > >Python判斷字串、檔案字元編碼

Python判斷字串、檔案字元編碼

本段工具程式碼用於判斷字串或者文字檔案的字元編碼型別,可以識別常用的UTF-8,UTF-8-SIG,UTF-16,GBK,GB2312 ,GB18030 ,ASCII字元編碼格式,如果有特殊字符集需求,可以擴充字元編碼列表。

程式碼如下:

[charset.py]

# this util is used to detect file or string bytes encoding
# author: lixk
# email: [email protected]

# info: UTF includes ISO8859-1,GB18030 includes GBK,GBK includes GB2312,GB2312 includes ASCII
CODES = ['UTF-8', 'UTF-16', 'GB18030', 'BIG5'] # UTF-8 BOM prefix UTF_8_BOM = b'\xef\xbb\xbf' def detect(s): """ get file encoding or bytes charset :param s: file path or bytes data :return: encoding charset """ if type(s) == str: with open(s, 'rb') as f: data = f.read() elif
type(s) == bytes: data = s else: raise TypeError('unsupported argument type!') # iterator charset for code in CODES: try: data.decode(encoding=code) if 'UTF-8' == code and data.startswith(UTF_8_BOM): return 'UTF-8-SIG' return
code except UnicodeDecodeError: continue raise UnicodeDecodeError('unknown charset!')

detect方法用於判斷檔案或者位元組列表編碼型別,引數為檔案路徑或者位元組列表bytes

使用示例:

import charset

print(charset.detect('abc哈哈'.encode('gbk')))
print(charset.detect('test.txt'))