粘包与半包
在TCP中，粘包(Sticky Packets)和半包(Unpacked Packets)比较常见。
1. 粘包与半包现象
1.1 粘包现象
java
public class HelloWorldClient {
static final Logger log = LoggerFactory.getLogger(HelloWorldClient.class);
public static void main(String[] args) {
NioEventLoopGroup worker = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap();
bootstrap.channel(NioSocketChannel.class);
bootstrap.group(worker);
bootstrap.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
log.debug("connetted...");
ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
log.debug("sending...");
Random r = new Random();
char c = 'a';
for (int i = 0; i < 10; i++) {
ByteBuf buffer = ctx.alloc().buffer();
buffer.writeBytes(new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15});
ctx.writeAndFlush(buffer);
}
}
});
}
});
ChannelFuture channelFuture = bootstrap.connect("127.0.0.1", 8080).sync();
channelFuture.channel().closeFuture().sync();
} catch (InterruptedException e) {
log.error("client error", e);
} finally {
worker.shutdownGracefully();
}
}
}java
public class HelloWorldServer {
static final Logger log = LoggerFactory.getLogger(HelloWorldServer.class);
void start() {
NioEventLoopGroup boss = new NioEventLoopGroup(1);
NioEventLoopGroup worker = new NioEventLoopGroup();
try {
ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap.channel(NioServerSocketChannel.class);
serverBootstrap.group(boss, worker);
serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
log.debug("connected {}", ctx.channel());
super.channelActive(ctx);
}
@Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
log.debug("disconnect {}", ctx.channel());
super.channelInactive(ctx);
}
});
}
});
ChannelFuture channelFuture = serverBootstrap.bind(8080);
log.debug("{} binding...", channelFuture.channel());
channelFuture.sync();
log.debug("{} bound...", channelFuture.channel());
channelFuture.channel().closeFuture().sync();
} catch (InterruptedException e) {
log.error("server error", e);
} finally {
boss.shutdownGracefully();
worker.shutdownGracefully();
log.debug("stoped");
}
}
public static void main(String[] args) {
new HelloWorldServer().start();
}
}运行结果：
客户端代码发送10次消息，每个消息是16字节，服务器端的输出，可以看到一次就接收了160个字节，而非分10次接收。
1.2 半包现象
服务端修改一下接收缓冲区，其它代码不变：
java
// 设置服务器接收缓冲区(滑动窗口大小)
serverBootstrap.option(ChannelOption.SO_RCVBUF, 10);
// 或者设置Netty的接收ByteBuf大小，值得注意的是大小必须是16的倍数
serverBootstrap.option(ChannelOption.RCVBUF_ALLOCATOR, new AdaptiveRecvByteBufAllocator(16, 16, 16));多次执行几次，有几率出现半包： 
2. 现象分析
2.1 粘包分析
现象：发送abc def，接收abcdef
原因：
- 应用层：接收方ByteBuf设置太大(Netty默认1024)
- 滑动窗口：假设发送方256bytes表示一个完整报文，但由于接收方处理不及时且窗口大小足够大，这256bytes字节就会缓冲在接收方的滑动窗口中，当滑动窗口中缓冲了多个报文就会粘包
- Nagle算法：会造成粘包。原因是网络层和传输层都有报头20+20占用40个字节，如果发送很小的字节就不划算，Nagle算法就是将琐碎的小数据包合并在发出去。
2.2 半包分析
现象：发送abcdef，接收abc def
原因：
- 应用层：接收方ByteBuf小于实际发送数据量
- 滑动窗口：假设接收方的窗口只剩了128bytes，发送方的报文大小是256bytes，这时放不下了，只能先发送前128 bytes，等待ack后才能发送剩余部分，这就造成了半包
- MSS限制(maximum segment size)：当发送的数据超过MSS限制后，会将数据切分发送，就会造成半包，它和MTU(maximum transmission unit)有关，MTU也就是链路层对一次能够发送的最大数据有限制，例如以太网的MTU是1500，光纤的MTU是4352，本地回环地址的MTU是65535。MSS=MTU-tcp头(20)-ip头(20)
本质是因为TCP是流式协议，消息无边界, 需要我们确立边界才行。
Nagle算法规则
即使发送一个字节，也需要加入tcp头和ip头，也就是总字节数会使用41bytes，非常不经济。因此为了提高网络利用率，tcp希望尽可能发送足够大的数据，这就是Nagle算法产生的缘由。
该算法是指发送端即使还有应该发送的数据，但如果这部分数据很少的话，则进行延迟发送
- 如果SO_SNDBUF的数据达到MSS，则需要发送
- 如果SO_SNDBUF中含有FIN(表示需要连接关闭)这时将剩余数据发送，再关闭
- 如果TCP_NODELAY = true，则需要发送
- 已发送的数据都收到ack时，则需要发送
- 上述条件不满足，但发生超时(一般为200ms)则需要发送
- 除上述情况，延迟发送
3. 滑动窗口
TCP以一个段(segment)为单位，每发送一个段就需要进行一次确认应答(ack)处理，但如果这么做，缺点是包的往返时间越长性能就越差。
为了解决此问题，引入了窗口概念，窗口大小即决定了无需等待应答而可以继续发送的数据最大值。
4. 解决办法-短链接
短链接就是人为的每一次发送数据后立即断开，造成每一条数据产生类似数据边界的效果，断开的时候服务端会收到-1状态。但是不能解决半包问题，因为接收方的缓冲区大小是有限的。
java
public class HelloWorldClient {
static final Logger log = LoggerFactory.getLogger(HelloWorldClient.class);
public static void main(String[] args) {
// 分 10 次发送
for (int i = 0; i < 10; i++) {
send();
}
}
private static void send() {
NioEventLoopGroup worker = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap();
bootstrap.channel(NioSocketChannel.class);
bootstrap.group(worker);
bootstrap.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
log.debug("conneted...");
ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
log.debug("sending...");
ByteBuf buffer = ctx.alloc().buffer();
buffer.writeBytes(new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15});
ctx.writeAndFlush(buffer);
// 发完即关
ctx.close();
}
});
}
});
ChannelFuture channelFuture = bootstrap.connect("localhost", 8080).sync();
channelFuture.channel().closeFuture().sync();
} catch (InterruptedException e) {
log.error("client error", e);
} finally {
worker.shutdownGracefully();
}
}
}5. 解决办法-固定长度
让所有数据包长度固定(一般考虑所有消息类型最长的长度为准，假设长度为10字节)，使用解码器类FixedLengthFrameDecoder：
java
// 加入入栈固定长度解码器，需要加在第一个位置
ch.pipeline().addLast(new FixedLengthFrameDecoder(8));运行结果：
客户端发送消息，长度不足10字节，使用下划线_补足。
6. 解决办法-固定分隔符
- 使用解码器LineBasedFrameDecoder类，默认以
\n或\r\n作为分隔符，如果超出指定长度仍未出现分隔符，则抛出异常：
java
// 服务端设置解析范围在1kb数据
ch.pipeline().addLast(new LineBasedFrameDecoder(1024));- 使用解码器DelimiterBasedFrameDecoder类，指定分隔符和长度，如果超出指定长度仍未出现该分隔符，则抛出异常：
java
// 服务端使用||作为分隔符
ByteBuf delimiter = ByteBufAllocator.DEFAULT.buffer().writeBytes("||".getBytes());
ch.pipeline().addLast(new DelimiterBasedFrameDecoder(1024, delimiter));运行效果：
sh
14:08:18 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x1282d755, L:/192.168.0.103:63641 - R:/192.168.0.103:9090] WRITE: 60B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 0a 62 62 62 0a 63 63 63 0a 64 64 0a 65 65 65 |a.bbb.ccc.dd.eee|
|00000010| 65 65 65 65 65 65 65 0a 66 66 0a 67 67 67 67 67 |eeeeeee.ff.ggggg|
|00000020| 67 67 0a 68 68 68 68 0a 69 69 69 69 69 69 69 0a |gg.hhhh.iiiiiii.|
|00000030| 6a 6a 6a 6a 6a 6a 6a 6a 6a 6a 6a 0a |jjjjjjjjjjj. |
+--------+-------------------------------------------------+----------------+sh
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 1B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 |a |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 3B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 62 62 62 |bbb |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 3B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 63 63 63 |ccc |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 2B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 64 64 |dd |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 10B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 65 65 65 65 65 65 65 65 65 65 |eeeeeeeeee |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 2B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 66 66 |ff |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 7B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 67 67 67 67 67 67 67 |ggggggg |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 4B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 68 68 68 |hhhh |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 7B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 69 69 69 69 69 69 |iiiiiii |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ: 11B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 6a 6a 6a 6a 6a 6a 6a 6a 6a 6a 6a |jjjjjjjjjjj |
+--------+-------------------------------------------------+----------------+
14:08:18 [DEBUG] [nioEventLoopGroup-3-5] i.n.h.l.LoggingHandler - [id: 0xa4b3be43, L:/192.168.0.103:9090 - R:/192.168.0.103:63641] READ COMPLETE其中客户端日志显示的.其实就是换行符，可以看到客户端最终将所有数据是一次性发送给服务端，但是服务端能够根据换行符进行拆分，读取正确的客户端每次发送过来的数据。
7. 解决办法-LTC解码器
解码器类有一个LengthFieldBasedFrameDecoder，他需要传入5个参数：
比如下图的报文含有长度，Header信息，配置LengthFieldBasedFrameDecoder的解析参数：
java
public static void main(String[] args) {
EmbeddedChannel channel = new EmbeddedChannel();
channel.pipeline().addLast(new LengthFieldBasedFrameDecoder(16, 0, 4,1,5));
channel.pipeline().addLast(new LoggingHandler(LogLevel.INFO));
ByteBuf buffer = ByteBufAllocator.DEFAULT.buffer();
generateData(buffer, "hello netty");
generateData(buffer, "Hi!");
channel.writeInbound(buffer);
}
private static void generateData(ByteBuf buffer, String str) {
byte[] bytes = str.getBytes();
// 写入长度字段
buffer.writeInt(bytes.length);
// 写入版本号
buffer.writeByte(1);
// 写入数据
buffer.writeBytes(bytes);
}运行结果：
sh
13:26:22.740 [main] INFO io.netty.handler.logging.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 11B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 20 6e 65 74 74 79 |hello netty |
+--------+-------------------------------------------------+----------------+
13:26:22.745 [main] INFO io.netty.handler.logging.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 3B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 69 21 |Hi! |
+--------+-------------------------------------------------+----------------+
13:26:22.745 [main] INFO io.netty.handler.logging.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE