架設BT伺服器和BitTorrent協議詳解
阿新 • • 發佈：2019-01-03
1.開始執行Tracker（已執行的跳過這一步）；
2.開始執行普通網路伺服器端程式，如Apache，已執行的跳過這一步；
3.在網路伺服器上將.torrent檔案關聯到Mimetype型別application/x-bittorrent（已關聯的跳過這一步）；
4.用要釋出的完整檔案和Tracker的URL建立一個元資訊檔案（.torrent檔案）；
5.將元資訊檔案放置在網路伺服器上；
6.在網頁上釋出元資訊檔案（.torrent檔案）連結；
7.原始下載者提供完整的檔案（原本）。
通過BT下載步驟如下：
1.安裝BT客戶端程式（已安裝的跳過這一步）；
2.上網；
3.點選一個鏈到.torrent檔案的連結；
4.選擇本地儲存路徑，選定需要下載的檔案（對有選擇下載功能的BT客戶端使用者）；
5.等待下載完成；
6.使用者退出下載（之前下載者不停止上傳）。
連線狀況如下：
·網站正常提供靜態檔案連線，並且啟動客戶端上的BT程式；
·Tracker即時接收所有下載者資訊，並且給每個下載者一份隨機的peer列表。通過HTTP或HTTPS協議實現；
·下載者每隔一段時間連一次Tracher，告知自己的進度，並和那些已經直接連線上的peer進行資料的上傳下載。這些連線遵循BitTorrent peer協議，通過TCP協議進行通訊。
·原始下載者只上傳不下載，他擁有整個檔案，所以很必要向網路中傳輸完檔案的所有部分。在一些人氣很旺的下載中，原始下載者經常可以在較短的時間內退出上傳，由其它已經下載到整個檔案的下載者繼續提供上傳。
元資訊檔案和Tracker的迴應資訊都以一種簡單高效可擴充套件的格式（Bencoding，B編碼）傳送。B編碼過的資訊就是以包含字串和整型資料的字典和列表的巢狀（像在Python中一樣）,可擴充套件性是指可以通過減少字典忽略的關鍵值來新增新的特性。
B編碼規則如下：
·字串表示為十進位制數的既定字串長度加冒號再跟原字串。
如4:spam就相當於'spam'。
·整型資料表示成前面加'i'後面加'e'中間是十進位制數，如i3e就相當於3，i-3e就是-3。整型資料沒有長度限制。i-0e無效，所有以'i0'開頭的除了代表0的i0e，其它都無效。
·列表編碼為一個'l'開頭後面跟它所包含的專案（已經編碼過）最後加一個'e'，比如l4:spam4:eggse就等於<'spam', 'eggs'>。
·字典編碼為一個'd'開頭後面跟一個交替關鍵值（key）及其對應值的列表最後加一個'e'。
如：d3:cow3:moo4:spam4:eggse相當於{'cow': 'moo', 'spam': 'eggs'}
d4:spaml1:a1:bee相當於{'spam': <'a', 'b'>}
關鍵值必須是處理過的字串（用原始字串編碼的，而且不是數字字母混合編碼的）。
元資訊檔案就是B編碼的有以下關鍵值的字典：
announce（宣告）
Tracker的URL。
info（資訊）
此關鍵值對應一個字典包含以下描述的關鍵值:
關鍵值name對應一個字串，代表預設的下載檔案或存成目錄的名字。它是純粹建議性的。
關鍵值piece length（塊長）對應檔案分割成的塊的位元組數。出於傳輸需要，檔案被分割成大小相等的塊，除了最後一塊通常會小一些。塊長一般來說是2的權值，大部分設塊長為256K(2的18次冪)。
關鍵值pieces（塊）對應一個字串，此字串長度是20的倍數。它可以再分成每20位元組一段的多個字串，分別對應塊在索引中的SHA1校驗碼（hash）。
還有關鍵值length（長度）和files（檔案），它們不能同時出現也不能都不出現。當length出現說明這個元資訊檔案只是單檔案下載，否則說明是多檔案的目錄結構下載。
單檔案情況下，length對應檔案長度的位元組數。
多檔案情況被看作是把許多單檔案按檔案列表中的順序連成一個大檔案下載，而關鍵值files就對應檔案列表，是一個字典的列表，其中每個字典又包含以下關鍵值：
length（長度） 檔案長度的位元組數。
path（路徑） 一個包含字串的列表，字串就是子目錄名，最後一項的字串是檔名。
（一個長度為零的length表單是錯誤的。）
在單檔案情況下，關鍵值name是檔名；多檔案情況下，它就成了目錄名。
Tracker質詢是雙向的。Tracker通過HTTP GET引數獲得資訊，然後返回一個B編碼後的資訊。儘管Tracker需要在伺服器端執行，但它執行流暢像Apache的一個模組。
Tracker的GET請求有如下關鍵值：
info_hash
20位元組長的SHA1驗證碼，來自B編碼過的元資訊檔案中的info值下，是元資訊檔案的一個支鏈。這個值是自動轉換的。
peer_id
一個20位元組長的字串，是每個使用者開始下載時隨機生成的ID。這個值也是是自動轉換的。
ip
一個可選擇的引數給出peer所在的IP（或DNS主機名），一般是和Tracker同機器的原始下載者得到後以便散發檔案。
port
監聽埠，官方預設的是從6881埠開始試，如果埠被佔用則依次向後推一個埠找空閒埠，到6889埠為止。
uploaded
目前總上傳量，編碼為十進位制ASCII碼。
downloaded
目前總下載量，編碼為十進位制ASCII碼。
left
未下載的位元組數，編碼為十進位制ASCII碼。這個數不是通過檔案長度和已下載數算出來的，因為檔案可能在被續傳，還有一些已經下載的資料不能通過完整性檢查必須重新下載。
event
這 是個選擇性的關鍵值，選項有started，completed或stopped（或empty，等同於沒有執行）。如果沒有執行，這個宣告會定期間隔一 定時間發出。開始下載時發出started值，完成下載時發出completed。當檔案完整後再開始，沒有completed發出，下載者中止下載時發 出stopped。
Tracker的迴應也是B編碼字典。如果Tracker迴應中有關鍵值failure reason（失敗原因），就會對應一個人可以讀懂的字串資訊解釋質詢失敗的原因，不需要其它關鍵值。否則，迴應必須有兩個關鍵值：interval （間隔）對應下載者定期發出請求的間隔秒數；peers，peer自選ID，IP地址或DNS主機名的字串和埠號。記住peers不會完全按照計劃的 間隔傳送請求，假如他們發生一個事件或者想要更多的peers。
如果你想對元資訊檔案或者Tracker質詢進行擴充套件，請與Bram Cohen進行協調，確保所有擴充套件都相容。
BitTorrent peer協議通過TCP協議進行操作。它不用調節任何socket選項就可以流暢執行。
peer之間的連線是對稱的。兩個方向送出的資訊要協調一致，資料可以流入任一方。
peer協議指一個peer從零開始下載，每得到元資訊檔案索引中所描述的一個塊且驗證碼一致，就向所有peer宣告已得到此塊。
連線的兩個終端有2個狀態指標，被阻塞與否，被關注與否，被阻塞（choking）是表明在恢復通暢之前資料不再發出的通知。發生阻塞的原因和技術問題稍後會提到。
數 據傳輸發生在一方關注對方且對方沒有阻塞的情況下。關注狀態必須一致保持-如果一個沒阻塞的peer沒有別人需要的資料，別人對他就會失去關注，轉而關注 那些正在阻塞的peer。完全執行這種條件需要非常慎重，但這樣的確可以讓下載者知道哪些peer在阻塞消失後可以馬上開始下載。
連線會逐漸斷開不感興趣和阻塞的peer。
當資料傳輸時，下載者要備好多份請求排成佇列，以獲得較高的TCP傳輸效率（這叫“管運請求”）。另一方面，不能被寫入TCP緩衝區的請求要被立即排入記憶體，而不是一個應用程式級的網路緩衝，一旦阻塞出現，這些請求全部丟棄。
peer連線協議包括一次握手跟著不斷的大小一致且確定的資訊流。握手的開始是字元十九（十進位制），跟著是字串'BitTorrentprotocol'。開頭的字元是長度固定的，希望其它新協議也能這樣以便區分。
此後所有送入協議的整數都編碼為4位元組大中止端。
在現有的應用中頭部資料之後是8個全部預留為0的位元組，若果你想通過改變這8個預留位元組以擴充套件協議，請與Bram Cohen協調以保證所有擴充套件相容。
然 後是來自元資訊檔案中B編碼的info值中長20位元組的SHA1驗證碼（和info_hash向Tracker宣告的值相同，但這裡是原始值那裡是引 用）。如果雙方的值不同，連線斷開。一個例外是下載者想只用一個埠進行多個連線下載，它們會先從接入連線得到一個驗證碼，然後和列表裡面的對照，有相同 的就答覆。
驗證碼之後是20位元組的peer id，它包含在Tracker迴應的peer列表中，在向Tracker的請求中被報告。如果接受方peer id不符合傳送方希望，連線斷開。
握手完畢。之後是長度固定的互動資訊流。零長度資訊用來保持連線，被忽略。這種資訊一般2分鐘發出一次，但是在等待資料期間很容易超時。
所有非保持連線用資訊開頭的位元組給出型別，可能值如下：
·0-阻塞
·1-通暢
·2-關注
·3-不關注
·4-有
·5-位元組
·6-請求
·7-塊
·8-取消
“阻塞”、“通暢”、“關注”和“不關注”類資訊沒有荷載。
“位元組”類資訊僅作為首資訊發出。它負載一個位元組，下載者有索引的設為1，其它為0。開始下載時沒有任何資料的下載者跳過“位元組”資訊。首位元組高位到低位對應索引0-7，依次類推，第二位元組對應8-15，等等。尾部的剩餘的位元位設為0。
“已有”類資訊負載一個數，即剛下載並核對完驗證碼的索引數。
“請求”類資訊包括包含一個索引，開始和長度。後兩者是位元組偏移。長度一般是2的權值除非被檔案尾截斷。現行一般是2的15次冪，並且關閉大於2的17次冪長度的連線。
“取 消”類資訊負載和“請求”類資訊有一樣的負載。它通常在下載接近完成即“最後階段”發出。當下載快要完成時，剩下幾個塊有都從同一個執行緒下載的趨向，這樣 會很慢。為了確保剩餘塊下載迅速，一旦還沒有決定剩餘塊的下載請求向誰發出，先向所有他正在從對方下載資料的連線者傳送要求所有剩餘塊的請求。為避免低 效，每當一個塊開始下載就向其他peer發出取消資訊。
“塊”類資訊包含一個索引，開始和塊。記住它和“請求”類資訊是相關的。當傳輸速度很慢或“阻塞”“通暢”類資訊高頻率交替發出或兩者同時發生，可能會載到一個不需要的塊。
下載者下載塊的順序是隨機的，這樣適當防止下載者與其他Peers僅有相同的塊子集或超集。
阻塞的發生有很多原因。TCP協議的資訊擁擠控制在即時向多連線傳送資訊的過程中表現極差。同時，阻塞的存在使下載者們能夠用以牙還牙式的演算法來確保穩定的下載速率。
下面描述的阻塞演算法是目前基礎的配置。重要的是所有新演算法不光要在包含全部擴充套件演算法的網路中執行良好，也要在主要包含這個基礎演算法的網路中執行良好。
一 個優秀的阻塞演算法有許多標準。它必須封鎖一定同時上傳的數量以獲得良好的TCP表現，還要避免頻繁的堵塞和通暢交替，即所謂“纖維化”。它應該用資料交換 報答給自己資料的peer。最後，它還應該偶爾嘗試一下與未使用過的peer端連線，找出比現有連線好的連線，這叫做嘗試性疏通。
現行的 阻塞演算法避免纖維化的手段是每10秒轉換被阻塞的名單。疏通4個自己關注且能從他們身上得到最高下載速率的peer，進行上傳和資料交換。有較高上傳速率 但是不被關注下載者的peer被疏通，一旦這些peer開始被關注，那些上傳率最低的peer的就被阻塞。如果下載者有了完整的檔案，他用自己的上傳率而 不是下載率來決定疏通誰的連線。
在嘗試性疏通中，任何一次中都有一個peer被疏通不管他的上傳率如何（如果被關注，他會成為4個提供下載的peer之一）。被嘗試性疏通的這種peer每30秒輪換一次。為了給它們一個上傳整一個塊的機會，新連線會以輪換中嘗試性疏通次數的3倍開始連線。
附原文：
BitTorrent is a protocol for distributing files. It identifies content by URL and is designed to integrate seamlessly with the web. Its advantage over plain HTTP is that when multiple downloads of the same file happen concurrently, the downloaders upload to each other, making it possible for the file source to support very large numbers of downloaders with only a modest increase in its load.
A BitTorrent file distribution consists of these entities:
An ordinary web server
A static 'metainfo' file
A BitTorrent tracker
An 'original' downloader
The end user web browsers
The end user downloaders
There are ideally many end users for a single file.
To start serving, a host goes through the following steps:
Start running a tracker (or, more likely, have one running already).
Start running an ordinary web server, such as apache, or have one already.
Associate the extension .torrent with mimetype application/x-bittorrent on their web server (or have done so already).
Generate a metainfo (.torrent) file using the complete file to be served and the URL of the tracker.
Put the metainfo file on the web server.
Link to the metainfo (.torrent) file from some other web page.
Start a downloader which already has the complete file (the 'origin').
To start downloading, a user does the following:
Install BitTorrent (or have done so already).
Surf the web.
Click on a link to a .torrent file.
Select where to save the file locally, or select a partial download to resume.
Wait for download to complete.
Tell downloader to exit (it keeps uploading until this happens).
The connectivity is as follows:
The web site is serving up static files as normal, but kicking off the BitTorrent helper app on the clients.
The tracker is receiving information from all downloaders and giving them random lists of peers. This is done over HTTP or HTTPS.
Downloaders are periodically checking in with the tracker to keep it informed of their progress, and are uploading to and downloading from each other via direct connections. These connections use the BitTorrent peer protocol, which operates over TCP.
The origin is uploading but not downloading at all, since it has the entire file. The origin is necessary to get the entire file into the network. Often for popular downloads the origin can be taken down after a while since several downloads may have completed and been left running indefinitely.
Metainfo file and tracker responses are both sent in a simple, efficient, and extensible format called bencoding (pronounced 'bee encoding'). Bencoded messages are nested dictionaries and lists (as in Python), which can contain strings and integers. Extensibility is supported by ignoring unexpected dictionary keys, so additional optional ones can be added later
Bencoding is done as follows:
Strings are length-prefixed base ten followed by a colon and the string. For example 4:spam corresponds to 'spam'.
Integers are represented by an 'i' followed by the number in base 10 followed by an 'e'. For example i3e corresponds to 3 and i-3e corresponds to -3. Integers have no size limitation. i-0e is invalid. All encodings with a leading zero, such as i03e , are invalid, other than i0e , which of course corresponds to 0.
Lists are encoded as an 'l' followed by their elements (also bencoded) followed by an 'e'. For example l4:spam4:eggse corresponds to ['spam', 'eggs'].
Dictionaries are encoded as a 'd' followed by a list of alternating keys and their corresponding values followed by an 'e'. For example, d3:cow3:moo4:spam4:eggse corresponds to {'cow': 'moo', 'spam': 'eggs'} and d4:spaml1:a1:bee corresponds to {'spam': ['a', 'b']} . Keys must be strings and appear in sorted order (sorted as raw strings, not alphanumerics).
Metainfo files are bencoded dictionaries with the following keys:
announce
The URL of the tracker.
info
This maps to a dictionary, with keys described below.
The name key maps to a string which is the suggested name to save the file (or directory) as. It is purely advisory.
piece length maps to the number of bytes in each piece the file is split into. For the purposes of transfer, files are split into fixed-size pieces which are all the same length except for possibly the last one which may be truncated. Piece length is almost always a power of two, most commonly 2 18 = 256 K (BitTorrent prior to version 3.2 uses 2 20 = 1 M as default).
pieces maps to a string whose length is a multiple of 20. It is to be subdivided into strings of length 20, each of which is the SHA1 hash of the piece at the corresponding index.
There is also a key length or a key files , but not both or neither. If length is present then the download represents a single file, otherwise it represents a set of files which go in a directory structure.
In the single file case, length maps to the length of the file in bytes.
For the purposes of the other keys, the multi-file case is treated as only having a single file by concatenating the files in the order they appear in the files list. The files list is the value files maps to, and is a list of dictionaries containing the following keys:
length The length of the file, in bytes. path A list of strings corresponding to subdirectory names, the last of which is the actual file name (a zero length list is an error case).
In the single file case, the name key is the name of a file, in the muliple file case, it's the name of a directory.
Tracker queries are two way. The tracker receives information via HTTP GET parameters and returns a bencoded message. Note that although the current tracker implementation has its own web server, the tracker could run very nicely as, for example, an apache module.
Tracker GET requests have the following keys:
info_hash
The 20 byte sha1 hash of the bencoded form of the info value from the metainfo file. Note that this is a substring of the metainfo file. This value will almost certainly have to be escaped.
peer_id
A string of length 20 which this downloader uses as its id. Each downloader generates its own id at random at the start of a new download. This value will also almost certainly have to be escaped.
ip
An optional parameter giving the IP (or dns name) which this peer is at. Generally used for the origin if it's on the same machine as the tracker.
port
The port number this peer is listening on. Common behavior is for a downloader to try to listen on port 6881 and if that port is taken try 6882, then 6883, etc. and give up after 6889.
uploaded
The total amount uploaded so far, encoded in base ten ascii.
downloaded
The total amount downloaded so far, encoded in base ten ascii.
left
The number of bytes this peer still has to download, encoded in base ten ascii. Note that this can't be computed from downloaded and the file length since it might be a resume, and there's a chance that some of the downloaded data failed an integrity check and had to be re-downloaded.
event
This is an optional key which maps to started , completed , or stopped (or empty, which is the same as not being present). If not present, this is one of the announcements done at regular intervals. An announcement using started is sent when a download first begins, and one using completed is sent when the download is complete. No completed is sent if the file was complete when started. Downloaders send an announcement using 'stopped' when they cease downloading.
Tracker responses are bencoded dictionaries. If a tracker response has a key failure reason , then that maps to a human readable string which explains why the query failed, and no other keys are required. Otherwise, it must have two keys: interval , which maps to the number of seconds the downloader should wait between regular rerequests, and peers . peers maps to a list of dictionaries corresponding to peers, each of which contains the keys peer id , ip , and port , which map to the peer's self-selected ID, IP address or dns name as a string, and port number, respectively. Note that downloaders may rerequest on nonscheduled times if an event happens or they need more peers.
If you want to make any extensions to metainfo files or tracker queries, please coordinate with Bram Cohen to make sure that all extensions are done compatibly.
BitTorrent's peer protocol operates over TCP. It performs efficiently without setting any socket options.
Peer connections are symmetrical. Messages sent in both directions look the same, and data can flow in either direction.
The peer protocol refers to pieces of the file by index as described in the metainfo file, starting at zero. When a peer finishes downloading a piece and checks that the hash matches, it announces that it has that piece to all of its peers.
Connections contain two bits of state on either end: choked or not, and interested or not. Choking is a notification that no data will be sent until unchoking happens. The reasoning and common techniques behind choking are explained later in this document.
Data transfer takes place whenever one side is interested and the other side is not choking. Interest state must be kept up to date at all times - whenever a downloader doesn't have something they currently would ask a peer for in unchoked, they must express lack of interest, despite being choked. Implementing this properly is tricky, but makes it possible for downloaders to know which peers will start downloading immediately if unchoked.
Connections start out choked and not interested.
When data is being transferred, downloaders should keep several piece requests queued up at once in order to get good TCP performance (this is called 'pipelining'.) On the other side, requests which can't be written out to the TCP buffer immediately should be queued up in memory rather than kept in an application-level network buffer, so they can all be thrown out when a choke happens.
The peer wire protocol consists of a handshake followed by a never-ending stream of length-prefixed messages. The handshake starts with character ninteen (decimal) followed by the string 'BitTorrent protocol'. The leading character is a length prefix, put there in the hope that other new protocols may do the same and thus be trivially distinguishable from each other.
All later integers sent in the protocol are encoded as four bytes big-endian.
After the fixed headers come eight reserved bytes, which are all zero in all current implementations. If you wish to extend the protocol using these bytes, please coordinate with Bram Cohen to make sure all extensions are done compatibly.
Next comes the 20 byte sha1 hash of the bencoded form of the info value from the metainfo file. (This is the same value which is announced as info_hash to the tracker, only here it's raw instead of quoted here). If both sides don't send the same value, they sever the connection. The one possible exception is if a downloader wants to do multiple downloads over a single port, they may wait for incoming connections to give a download hash first, and respond with the same one if it's in their list.
After the download hash comes the 20-byte peer id which is reported in tracker requests and contained in peer lists in tracker responses. If the receiving side's peer id doesn't match the one the initiating side expects, it severs the connection.
That's it for handshaking, next comes an alternating stream of length prefixes and messages. Messages of length zero are keepalives, and ignored. Keepalives are generally sent once every two minutes, but note that timeouts can be done much more quickly when data is expected.
All non-keepalive messages start with a single byte which gives their type. The possible values are:
0 - choke
1 - unchoke
2 - interested
3 - not interested
4 - have
5 - bitfield
6 - request
7 - piece
8 - cancel
'choke', 'unchoke', 'interested', and 'not interested' have no payload.
'bitfield' is only ever sent as the first message. Its payload is a bitfield with each index that downloader has sent set to one and the rest set to zero. Downloaders which don't have anything yet may skip the 'bitfield' message. The first byte of the bitfield corresponds to indices 0 - 7 from high bit to low bit, respectively. The next one 8-15, etc. Spare bits at the end are set to zero.
The 'have' message's payload is a single number, the index which that downloader just completed and checked the hash of.
'request' messages contain an index, begin, and length. The last two are byte offsets. Length is generally a power of two unless it gets truncated by the end of the file. All current implementations use 2 15 , and close connections which request an amount greater than 2 17 .
'cancel' messages have the same payload as request messages. They are generally only sent towards the end of a download, during what's called 'endgame mode'. When a download is almost complete, there's a tendency for the last few pieces to all be downloaded off a single hosed modem line, taking a very long time. To make sure the last few pieces come in quickly, once requests for all pieces a given downloader doesn't have yet are currently pending, it sends requests for everything to everyone it's downloading from. To keep this from becoming horribly inefficient, it sends cancels to everyone else every time a piece arrives.
'piece' messages contain an index, begin, and piece. Note that they are correlated with request messages implicitly. It's possible for an unexpected piece to arrive if choke and unchoke messages are sent in quick succession and/or transfer is going very slowly.
Downloaders generally download pieces in random order, which does a reasonably good job of keeping them from having a strict subset or superset of the pieces of any of their peers.
Choking is done for several reasons. TCP congestion control behaves very poorly when sending over many connections at once. Also, choking lets each peer use a tit-for-tat-ish algorithm to ensure that they get a consistent download rate.
The choking algorithm described below is the currently deployed one. It is very important that all new algorithms work well both in a network consisting entirely of themselves and in a network consisting mostly of this one.
There are several criteria a good choking algorithm should meet. It should cap the number of simultaneous uploads for good TCP performance. It should avoid choking and unchoking quickly, known as 'fibrillation'. It should reciprocate to peers who let it download. Finally, it should try out unused connections once in a while to find out if they might be better than the currently used ones, known as optimistic unchoking.
The currently deployed choking algorithm avoids fibrillation by only changing who's choked once every ten seconds. It does reciprocation and number of uploads capping by unchoking the four peers which it has the best download rates from and are interested. Peers which have a better upload rate but aren't interested get unchoked and if they become interested the worst uploader gets choked. If a downloader has a complete file, it uses its upload rate rather than its download rate to decide who to unchoke.
For optimistic unchoking, at any one time there is a single peer which is unchoked regardless of it's upload rate (if interested, it counts as one of the four allowed downloaders.) Which peer is optimistically unchoked rotates every 30 seconds. To give them a decent chance of getting a complete piece to upload, new connections are three times as likely to start as the current optimistic unchoke as anywhere else in the rotation.