TCP網絡通訊如何解決分包粘包問題(有模擬代碼)
阿新 • • 發佈:2017-09-06
負責 return 發送數據 pop boa target u3d swe n) TCP作為常用的網絡傳輸協議,數據流解析是網絡應用開發人員永遠繞不開的一個問題。
TCP數據傳輸是以無邊界的數據流傳輸形式,所謂無邊界是指數據發送端發送的字節數,在數據接收端接受時並不一定等於發送的字節數,可能會出現粘包情況。
TCP粘包的解決方案有很多種方法,最簡單的一種就是發送的數據協議定義發送的數據包的結構: 1. 數據頭:數據包的大小,固定長度。 2. 數據內容:數據內容,長度為數據頭定義的長度大小。 實際操作如下: a)發送端:先發送數據包的大小,再發送數據內容。 b)接收端:先解析本次數據包的大小N,在讀取N個字節,這N個字節就是一個完整的數據內容。 具體流程如下:
實現源碼 [cpp] view plain copy
服務端模擬接受數據 $ ./server.o listening... waiting for new socket accept. waiting for new socket accept. newclient sock fd: 4 receive data: 00000088 data size: 00000088, value:88 receive data: This i receive data: s a te receive data: st cas receive data: e for receive data: client receive data: send receive data: subpac receive data: kage d receive data: ata. d receive data: ata is receive data: not s receive data: end co receive data: mplete receive data: at on receive data: ce. data size: 88, text: This is a test case for client send subpackage data. data is not send complete at once. receive data: 00000007 data size: 00000007, value:7 receive data: Hello data size: 7, text: Hello receive data: 00000011 data size: 00000011, value:11 receive data: I‘m lucky. data size: 11, text: I‘m lucky. receive data: 00000016 data size: 00000016, value:16 receive data: Nice too me you data size: 16, text: Nice too me you receive data: 00000005 data size: 00000005, value:5 receive data: exit data size: 5, text: exit newclient sockfd: 4, finish. http://blog.csdn.net/sweettool/article/details/77018506
一、TCP粘包情況:
1. 發送端發送了數量比較的數據,接收端讀取數據時候數據分批到達,造成一次發送多次讀取;通常網絡路由的緩存大小有關系,一個數據段大小超過緩存大小,那麽就要拆包發送。 2. 發送端發送了幾次數據,接收端一次性讀取了所有數據,造成多次發送一次讀取;通常是網絡流量優化,把多個小的數據段集滿達到一定的數據量,從而減少網絡鏈路中的傳輸次數。TCP粘包的解決方案有很多種方法,最簡單的一種就是發送的數據協議定義發送的數據包的結構: 1. 數據頭:數據包的大小,固定長度。 2. 數據內容:數據內容,長度為數據頭定義的長度大小。 實際操作如下: a)發送端:先發送數據包的大小,再發送數據內容。 b)接收端:先解析本次數據包的大小N,在讀取N個字節,這N個字節就是一個完整的數據內容。 具體流程如下:
實現源碼 [cpp] view plain copy
- /**
- * read size of len from sock into buf.
- */
- bool readPack(int sock, char* buf, size_t len) {
- if (NULL == buf || len < 1) {
- return false;
- }
- memset(buf, 0, len); // only reset buffer len.
- ssize_t read_len = 0, readsum = 0;
- do {
- read_len = read(sock, buf + readsum, len - readsum);
- if (-1 == read_len) { // ignore error case
- return false;
- }
- printf("receive data: %s\n", buf + readsum);
- readsum += read_len;
- } while (readsum < len && 0 != read_len);
- return true;
- }
二、測試用例介紹
本篇提供的demo主要流程如下: 1. 客戶端負責模擬發送數據,服務端負責接受數據,處理粘包問題 a)emulate_subpackage 模擬情況1,一個長數據經過多次才到達目的地, 在客戶端字符串“This is a test case for client send subpackage data. data is not send complete at once.”每次只發送6個字節長度。服務端要把字符串集滿才能處理數據(打印字符串) b)emulate_adheringpackage 模擬情況2,多個數據在一次性到達目的地 在客戶端將字符串“Hello I‘m lucky. Nice too me you”切成三個數據段(都包含數據頭和數據內容),然後一次性發送,服務端讀取數據時對三個數據段逐個處理。三、源碼實現
server.cpp [cpp] view plain copy- #include <cstdio>
- #include <cstdlib>
- #include <cstring>
- #include <errno.h>
- #include <sys/socket.h>
- #include <sys/types.h>
- #include <arpa/inet.h>
- #include <unistd.h>
- void newclient(int sock);
- bool readPack(int sock, char* buf, size_t len);
- void safe_close(int &sock);
- int main(int argc, char *argv[]) {
- int sockfd = -1, newsockfd = -1;
- socklen_t c = 0;
- struct sockaddr_in serv_addr, cli_addr;
- // Create socket
- sockfd = socket(AF_INET, SOCK_STREAM, 0);
- if (-1 == sockfd) {
- printf("new socket failed. errno: %d, error: %s\n", errno, strerror(errno));
- exit(-1);
- }
- // Prepare the sockaddr_in structure
- serv_addr.sin_family = AF_INET;
- serv_addr.sin_addr.s_addr = INADDR_ANY;
- serv_addr.sin_port = htons(7890);
- // bind
- if (bind(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0) {
- printf("bind failed. errno: %d, error: %s\n", errno, strerror(errno));
- exit(-1);
- }
- // listen
- listen(sockfd, 5);
- printf("listening...\n");
- // accept new connection.
- c = sizeof(struct sockaddr_in);
- int i = 0;
- while (i++ < 3) {
- printf("waiting for new socket accept.\n");
- newsockfd = accept(sockfd, (struct sockaddr*)&cli_addr, (socklen_t*)&c);
- if (newsockfd < 0) {
- printf("accept connect failed. errno: %d, error: %s\n", errno, strerror(errno));
- safe_close(sockfd);
- exit(-1);
- }
- pid_t pid = fork();
- if (0 == pid) {
- newclient(newsockfd);
- safe_close(sockfd);
- break;
- } else if (pid > 0) {
- safe_close(newsockfd);
- }
- }
- safe_close(sockfd);
- return 0;
- }
- void newclient(int sock) {
- printf("newclient sock fd: %d\n", sock);
- int datasize = 0;
- const int HEAD_SIZE = 9;
- char buf[512] = {0};
- while (true) {
- memset(buf, 0, sizeof(buf));
- if (! readPack(sock, buf, HEAD_SIZE)) {
- printf("read head buffer failed.\n");
- safe_close(sock);
- return;
- }
- datasize = atoi(buf);
- printf("data size: %s, value:%d\n", buf, datasize);
- memset(buf, 0, sizeof(buf));
- if (! readPack(sock, buf, datasize)) {
- printf("read data buffer failed\n");
- safe_close(sock);
- return;
- }
- printf("data size: %d, text: %s\n", datasize, buf);
- if (0 == strcmp(buf, "exit")) {
- break;
- }
- }
- memset(buf, 0, sizeof(buf));
- snprintf(buf, sizeof(buf), "from server read complete.");
- write(sock, buf, strlen(buf) + 1);
- printf("newclient sockfd: %d, finish.\n", sock);
- safe_close(sock);
- }
- void safe_close(int &sock) {
- if (sock > 0) {
- close(sock);
- sock = -1;
- }
- }
- /**
- * read size of len from sock into buf.
- */
- bool readPack(int sock, char* buf, size_t len) {
- if (NULL == buf || len < 1) {
- return false;
- }
- memset(buf, 0, len); // only reset buffer len.
- ssize_t read_len = 0, readsum = 0;
- do {
- read_len = read(sock, buf + readsum, len - readsum);
- if (-1 == read_len) { // ignore error case
- return false;
- }
- printf("receive data: %s\n", buf + readsum);
- readsum += read_len;
- } while (readsum < len && 0 != read_len);
- return true;
- }
- #include <cstdio>
- #include <cstdlib>
- #include <cstring>
- #include <time.h>
- #include <errno.h>
- #include <sys/socket.h>
- #include <arpa/inet.h>
- #include <unistd.h>
- void safe_close(int &sock);
- void emulate_subpackage(int sock);
- void emulate_adheringpackage(int sock);
- int main(int argc, char *argv[]) {
- char buf[128] = {0};
- int sockfd = -1;
- struct sockaddr_in serv_addr;
- // Create sock
- sockfd = socket(AF_INET, SOCK_STREAM, 0);
- if (-1 == sockfd) {
- printf("new socket failed. errno: %d, error: %s\n", errno, strerror(errno));
- exit(-1);
- }
- serv_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
- serv_addr.sin_family = AF_INET;
- serv_addr.sin_port = htons(7890);
- // Connect to remote server
- if (connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(serv_addr)) < 0) {
- printf("connection failed. errno: %d, error: %s\n", errno, strerror(errno));
- exit(-1);
- }
- emulate_subpackage(sockfd);
- emulate_adheringpackage(sockfd);
- const int HEAD_SIZE = 9;
- const char temp[] = "exit";
- memset(buf, 0, sizeof(buf));
- snprintf(buf, sizeof(buf), "%0.*zu", HEAD_SIZE - 1, sizeof(temp));
- write(sockfd, buf, HEAD_SIZE);
- write(sockfd, temp, sizeof(temp));
- printf("send complete.\n");
- memset(buf, 0, sizeof(buf));
- read(sockfd, buf, sizeof(buf));
- printf("receive data: %s\n", buf);
- printf("client finish.\n");
- safe_close(sockfd);
- return 0;
- }
- void safe_close(int &sock) {
- if (sock > 0) {
- close(sock);
- sock = -1;
- }
- }
- /**
- * emulate socket data write multi part.
- */
- void emulate_subpackage(int sock) {
- printf("emulate_subpackage...\n");
- char text[] = "This is a test case for client send subpackage data. data is not send complete at once.";
- const size_t TEXTSIZE = sizeof(text);
- ssize_t len = 0;
- size_t sendsize = 0, sendsum = 0;
- const int HEAD_SIZE = 9;
- char buf[64] = {0};
- snprintf(buf, HEAD_SIZE, "%08zu", TEXTSIZE);
- write(sock, buf, HEAD_SIZE);
- printf("send data size: %s\n", buf);
- do {
- sendsize = 6;
- if (sendsum + sendsize > TEXTSIZE) {
- sendsize = TEXTSIZE - sendsum;
- }
- len = write(sock, text + sendsum, sendsize);
- if (-1 == len) {
- printf("send data failed. errno: %d, error: %s\n", errno, strerror(errno));
- return;
- }
- memset(buf, 0, sizeof(buf));
- snprintf(buf, len + 1, text + sendsum);
- printf("send data: %s\n", buf);
- sendsum += len;
- sleep(1);
- } while (sendsum < TEXTSIZE && 0 != len);
- }
- /**
- * emualte socket data write adhering.
- */
- void emulate_adheringpackage(int sock) {
- printf("emulate_adheringpackage...\n");
- const int HEAD_SIZE = 9;
- char buf[1024] = {0};
- char text[128] = {0};
- char *pstart = buf;
- // append text
- memset(text, 0, sizeof(text));
- snprintf(text, sizeof(text), "Hello ");
- snprintf(pstart, HEAD_SIZE, "%08zu", strlen(text) + 1);
- pstart += HEAD_SIZE;
- snprintf(pstart, strlen(text) + 1, "%s", text);
- pstart += strlen(text) + 1;
- // append text
- memset(text, 0, sizeof(text));
- snprintf(text, sizeof(text), "I‘m lucky.");
- snprintf(pstart, HEAD_SIZE, "%08zu", strlen(text) + 1);
- pstart += HEAD_SIZE;
- snprintf(pstart, strlen(text) + 1, "%s", text);
- pstart += strlen(text) + 1;
- // append text
- memset(text, 0, sizeof(text));
- snprintf(text, sizeof(text), "Nice too me you");
- snprintf(pstart, HEAD_SIZE, "%08zu", strlen(text) + 1);
- pstart += HEAD_SIZE;
- snprintf(pstart, strlen(text) + 1, "%s", text);
- pstart += strlen(text) + 1;
- write(sock, buf, pstart - buf);
- }
- CC=g++
- CFLAGS=-I
- all: server.o client.o
- server.o: server.cpp
- $(CC) -o server.o server.cpp
- client.o: client.cpp
- $(CC) -o client.o client.cpp
- clean:
- rm *.o
四、測試結果
編譯及運行 $ make g++ -o server.o server.cpp g++ -o client.o client.cpp 客戶端模擬發送數據 $ ./client.o emulate_subpackage... send data size: 00000088 send data: This i send data: s a te send data: st cas send data: e for send data: client send data: send send data: subpac send data: kage d send data: ata. d send data: ata is send data: not s send data: end co send data: mplete send data: at on send data: ce. emulate_adheringpackage... send complete. receive data: from server read complete. client finish.服務端模擬接受數據 $ ./server.o listening... waiting for new socket accept. waiting for new socket accept. newclient sock fd: 4 receive data: 00000088 data size: 00000088, value:88 receive data: This i receive data: s a te receive data: st cas receive data: e for receive data: client receive data: send receive data: subpac receive data: kage d receive data: ata. d receive data: ata is receive data: not s receive data: end co receive data: mplete receive data: at on receive data: ce. data size: 88, text: This is a test case for client send subpackage data. data is not send complete at once. receive data: 00000007 data size: 00000007, value:7 receive data: Hello data size: 7, text: Hello receive data: 00000011 data size: 00000011, value:11 receive data: I‘m lucky. data size: 11, text: I‘m lucky. receive data: 00000016 data size: 00000016, value:16 receive data: Nice too me you data size: 16, text: Nice too me you receive data: 00000005 data size: 00000005, value:5 receive data: exit data size: 5, text: exit newclient sockfd: 4, finish. http://blog.csdn.net/sweettool/article/details/77018506
TCP網絡通訊如何解決分包粘包問題(有模擬代碼)