佳佳的博客
Menu
首页
RedisResponseException:No more data & Zero length respose
Posted by
佳佳
on 2018-07-26
IT
Redis
运维切换了新的 *Redis* 架构之后,偶尔会报 *No more data* 或 *Zero length respose* 的错误。 ### 架构 客户端 C# 使用的 *ServiceStack.Redis 3.9.60.0* 包,通过连接池 `PooledRedisClientManager` 管理连接; *Redis* 是一主多从; 中间通过 *HAproxy* 代理,实现LB; ### 异常信息 #### 客户端 ```cs ServiceStack.Redis.RedisResponseException No more data, sPort: 53059, LastCommand: ServiceStack.Redis 在 ServiceStack.Redis.RedisNativeClient.CreateResponseError(String error) 在 ServiceStack.Redis.RedisNativeClient.ExpectSuccess() 在 ServiceStack.Redis.RedisNativeClient.Set(String key, Byte[] value) 在 ServiceStack.Redis.RedisClient.Set[T](String key, T value) 在 ServiceStack.Redis.RedisClient.Set[T](String key, T value, DateTime expiresAt) 在 Framework.Utils.RedisProvider.Set[T](String key, T value, Int32 minutes) 在 MX.OBP.WebApp.Login.InitPage() 在 System.Web.UI.Control.LoadRecursive() 在 System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) ``` ```cs ServiceStack.Redis.RedisResponseException Zero length respose, sPort: 52673, LastCommand: ServiceStack.Redis 在 ServiceStack.Redis.RedisNativeClient.CreateResponseError(String error) 在 ServiceStack.Redis.RedisNativeClient.ParseSingleLine(String r) 在 ServiceStack.Redis.RedisNativeClient.GetBytes(String key) 在 ServiceStack.Redis.RedisClient.Get[T](String key) 在 Framework.Utils.RedisProvider.Get[T](String key) ``` #### HAProxy 查看 *HAProxy* 的日志,对应上面的客户端的出错信息,日志能看到 **cD** 和 **CD** 错误状态。 > CD > The client unexpectedly aborted during data transfer. > This can be caused by a browser crash, by an intermediate equipment between the client and haproxy which decided to actively break the connection, > by network routing issues between the client and haproxy, or by a keep-alive session between the server and the client terminated first by the client. > 当传输数据时客户端异常退出 > > cD > The client did not send nor acknowledge any data for as long as the timeout client delay. > This is often caused by network failures on client side, or the client simply leaving the net uncleanly. > 在data阶段,客户端超时 ### ServiceStack.Redis 源码 下面贴出了上面错误消息中调用的部分源码,可以看出不管是保存还是获取,在返回的 `Bstream` 中没有值时,就会报上面的错误。 *RedisNativeClient.cs* ```cs namespace ServiceStack.Redis { public class RedisNativeClient : IRedisNativeClient, IDisposable { public void Set(string key, byte[] value) { if (key == null) throw new ArgumentNullException(nameof (key)); value = value ?? new byte[0]; if (value.Length > 1073741824) throw new ArgumentException("value exceeds 1G", nameof (value)); this.SendExpectSuccess(Commands.Set, key.ToUtf8Bytes(), value); } protected void SendExpectSuccess(params byte[][] cmdWithBinaryArgs) { if (!this.SendCommand(cmdWithBinaryArgs)) throw this.CreateConnectionError(); if (this.Pipeline != null) this.Pipeline.CompleteVoidQueuedCommand(new Action(this.ExpectSuccess)); else this.ExpectSuccess(); } protected void ExpectSuccess() { int num = this.SafeReadByte(); if (num == -1) throw this.CreateResponseError("No more data"); string str = this.ReadLine(); if (num == 45) throw this.CreateResponseError(!str.StartsWith("ERR") || str.Length < 4 ? str : str.Substring(4)); } public byte[] GetBytes(string key) { if (key == null) throw new ArgumentNullException(nameof (key)); return this.SendExpectData(Commands.Get, key.ToUtf8Bytes()); } protected byte[] SendExpectData(params byte[][] cmdWithBinaryArgs) { if (!this.SendCommand(cmdWithBinaryArgs)) throw this.CreateConnectionError(); if (this.Pipeline == null) return this.ReadData(); this.Pipeline.CompleteBytesQueuedCommand(new Func<byte[]>(this.ReadData)); return (byte[]) null; } private byte[] ReadData() { return this.ParseSingleLine(this.ReadLine()); } private byte[] ParseSingleLine(string r) { if (r.Length == 0) throw this.CreateResponseError("Zero length respose"); switch (r[0]) { case '$': if (r == "$-1") return (byte[]) null; int result; if (!int.TryParse(r.Substring(1), out result)) throw this.CreateResponseError("Invalid length"); byte[] buffer = new byte[result]; int offset = 0; while (result > 0) { int num = this.Bstream.Read(buffer, offset, result); if (num <= 0) throw this.CreateResponseError("Unexpected end of Stream"); offset += num; result -= num; } if (this.Bstream.ReadByte() != 13 || this.Bstream.ReadByte() != 10) throw this.CreateResponseError("Invalid termination"); return buffer; case '-': throw this.CreateResponseError(r.StartsWith("-ERR") ? r.Substring(5) : r.Substring(1)); case ':': return r.Substring(1).ToUtf8Bytes(); default: throw this.CreateResponseError("Unexpected reply: " + r); } } private int SafeReadByte() { return this.Bstream.ReadByte(); } private RedisResponseException CreateResponseError(string error) { this.HadExceptions = true; RedisResponseException responseException = new RedisResponseException(string.Format("{0}, sPort: {1}, LastCommand: {2}", (object) error, (object) this.clientPort, (object) this.lastCommand)); RedisNativeClient.log.Error((object) responseException.Message); throw responseException; } } } ``` ### 对策 o(╯□╰)o 因为更改架构之前直连 *Redis* 是正常的,所以猜测是 *HAProxy* 连接 *Redis* 超时导致的。 *Redis* 的超时时间设置的是 5min,*HAProxy* 的超时时间是 10s,所以将 *HAProxy* 的超时时间设为和 *Redis* 一样看看是否有效果。 ### 参考 - [使用ServiceStack.Redis过程中遇到的问题](https://www.cnblogs.com/magialmoon/archive/2012/12/25/2832974.html) - [RedisResponseException from BlockingDequeue](https://stackoverflow.com/questions/17453594/redisresponseexception-from-blockingdequeue) ```cs var redisFactory = new PooledRedisClientManager(redisConn); redisFactory.ConnectTimeout = 5; redisFactory.IdleTimeOutSecs = 30; ``` - [Redis Mass Insertion](https://redis.io/topics/mass-insert) - [Socket.Connect](https://msdn.microsoft.com/zh-cn/library/ych8bz3x(v=vs.110).aspx) - [Socket.BeginConnect](https://msdn.microsoft.com/zh-cn/library/tad07yt6(v=vs.110).aspx) - [BufferedStream.ReadByte](https://msdn.microsoft.com/zh-cn/library/system.io.bufferedstream.readbyte(v=vs.100).aspx) - [Protocol errors, “no more data” errors, “Zero length response” errors while using servicestack.redis in a high volume scenario](https://stackoverflow.com/questions/27929192/protocol-errors-no-more-data-errors-zero-length-response-errors-while-usin) - [haproxy状态总结(官方文档加翻译),读haproxy日志必备](https://wenku.baidu.com/view/37021f396bd97f192279e92b.html)
版权声明:原创文章,未经允许不得转载。
https://www.liujiajia.me/2018/7/26/redisresponseexception:no-more-data-zero-length-respose
“Buy me a nongfu spring”
« 【IIS】编译器失败,错误代码为 -2146232576
APP测试最常见BUG »
昵称
*
电子邮箱
*
回复内容
*
(回复审核后才会显示)
提交
目录
AUTHOR
刘佳佳
江苏 - 苏州
软件工程师