聊聊claudb的importRDB
序
本文主要研究一下claudb的importRDB
importRDB
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/DBServerState.java
public class DBServerState {
//......
public void importRDB(InputStream input) throws IOException {
RDBInputStream rdb = new RDBInputStream(input);
Map<Integer, Map<DatabaseKey, DatabaseValue>> load = rdb.parse();
for (Map.Entry<Integer, Map<DatabaseKey, DatabaseValue>> entry : load.entrySet()) {
databases.get(entry.getKey()).overrideAll(ImmutableMap.from(entry.getValue()));
}
}
//......
}
- importRDB方法创建RDBInputStream,然后执行其parse方法进行解析,之后遍历解析结果,挨个执行databases.get(entry.getKey()).overrideAll(ImmutableMap.from(entry.getValue()))
RDBInputStream
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
public class RDBInputStream {
private static final SafeString REDIS_PREAMBLE = safeString("REDIS");
private static final long TO_MILLIS = 1000L;
private static final int HASH = 0x04;
private static final int SORTED_SET = 0x03;
private static final int SET = 0x02;
private static final int LIST = 0x01;
private static final int STRING = 0x00;
private static final int TTL_MILLISECONDS = 0xFC;
private static final int TTL_SECONDS = 0xFD;
private static final int SELECT = 0xFE;
private static final int END_OF_STREAM = 0xFF;
private static final int REDIS_VERSION = 6;
private static final int VERSION_LENGTH = 4;
private static final int REDIS_LENGTH = 5;
private final CheckedInputStream in;
public RDBInputStream(InputStream in) {
this.in = new CheckedInputStream(in, new CRC64());
}
public Map<Integer, Map<DatabaseKey, DatabaseValue>> parse() throws IOException {
Map<Integer, Map<DatabaseKey, DatabaseValue>> databases = new HashMap<>();
int version = version();
if (version > REDIS_VERSION) {
throw new IOException("invalid version: " + version);
}
Long expireTime = null;
HashMap<DatabaseKey, DatabaseValue> db = null;
for (boolean end = false; !end;) {
int read = in.read();
switch (read) {
case SELECT:
db = new HashMap<>();
databases.put(readLength(), db);
break;
case TTL_SECONDS:
expireTime = parseTimeSeconds();
break;
case TTL_MILLISECONDS:
expireTime = parseTimeMillis();
break;
case STRING:
ensure(db, readKey(), readString(expireTime));
expireTime = null;
break;
case LIST:
ensure(db, readKey(), readList(expireTime));
expireTime = null;
break;
case SET:
ensure(db, readKey(), readSet(expireTime));
expireTime = null;
break;
case SORTED_SET:
ensure(db, readKey(), readSortedSet(expireTime));
expireTime = null;
break;
case HASH:
ensure(db, readKey(), readHash(expireTime));
expireTime = null;
break;
case END_OF_STREAM:
// end of stream
end = true;
db = null;
expireTime = null;
break;
default:
throw new IOException("not supported: " + read);
}
}
verifyChecksum();
return databases;
}
private int version() throws IOException {
SafeString redis = new SafeString(read(REDIS_LENGTH));
if (!redis.equals(REDIS_PREAMBLE)) {
throw new IOException("not valid stream");
}
return parseVersion(read(VERSION_LENGTH));
}
private int parseVersion(byte[] version) {
StringBuilder sb = new StringBuilder();
for (byte b : version) {
sb.append((char) b);
}
return Integer.parseInt(sb.toString());
}
private void verifyChecksum() throws IOException {
long calculated = in.getChecksum().getValue();
long readed = parseChecksum();
if (calculated != readed) {
throw new IOException("invalid checksum: " + readed);
}
}
private long parseChecksum() throws IOException {
return ByteUtils.byteArrayToLong(read(Long.BYTES));
}
//......
}
- RDBInputStream的构造器使用CheckedInputStream对InputStream进行包装;其parse方法先解析version进行校验,之后循环进行in.read(),分别解析SELECT、TTL_SECONDS、TTL_MILLISECONDS、STRING、LIST、SET、SORTED_SET、HASH、END_OF_STREAM;最后执行verifyChecksum校验checksum
readKey
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
private DatabaseKey readKey() throws IOException {
return new DatabaseKey(readSafeString());
}
private SafeString readSafeString() throws IOException {
int length = readLength();
return new SafeString(read(length));
}
private int readLength() throws IOException {
int length = in.read();
if (length < 0x40) {
// 1 byte: 00XXXXXX
return length;
} else if (length < 0x80) {
// 2 bytes: 01XXXXXX XXXXXXXX
int next = in.read();
return readLength(length, next);
} else {
// 5 bytes: 10...... XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
return byteArrayToInt(read(Integer.BYTES));
}
}
private byte[] read(int size) throws IOException {
byte[] array = new byte[size];
int read = in.read(array);
if (read != size) {
throw new IOException("error reading stream");
}
return array;
}
- readKey方法执行readSafeString,先readLength,在根据length进行read
readString
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
private DatabaseValue readString(Long expireTime) throws IOException {
return string(readSafeString()).expiredAt(expireTime != null ? ofEpochMilli(expireTime) : null);
}
- readString方法先readSafeString,然后转为DatabaseValue再设置expiredAt
readList
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
private DatabaseValue readList(Long expireTime) throws IOException {
int size = readLength();
List<SafeString> list = new LinkedList<>();
for (int i = 0; i < size; i++) {
list.add(readSafeString());
}
return list(list).expiredAt(expireTime != null ? ofEpochMilli(expireTime) : null);
}
- readList方法先通过readLength读取list元素个数,在挨个执行readSafeString,最后将list转为DatabaseValue再设置expiredAt
readSet
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
private DatabaseValue readSet(Long expireTime) throws IOException {
int size = readLength();
Set<SafeString> set = new LinkedHashSet<>();
for (int i = 0; i < size; i++) {
set.add(readSafeString());
}
return set(set).expiredAt(expireTime != null ? ofEpochMilli(expireTime) : null);
}
- readSet方法先通过readLength读取set元素个数,在挨个执行readSafeString,最后将set转为DatabaseValue再设置expiredAt
readSortedSet
claudb-1.7.1/src/main/java/com/github/tonivade/claudb/persistence/RDBInputStream.java
private DatabaseValue readSortedSet(Long expireTime) throws IOException {
int size = readLength();
Set<Entry<Double, SafeString>> entries = new LinkedHashSet<>();
for (int i = 0; i < size; i++) {
SafeString value = readSafeString();
Double score = readDouble();
entries.add(score(score, value));
}
return zset(entries).expiredAt(expireTime != null ? ofEpochMilli(expireTime) : null);
}
- readSortedSet方法先通过readLength读取set元素个数,在挨个执行readSafeString、readDouble,添加到LinkedHashSet,最后将entries转为DatabaseValue再设置expiredAt;与readSet的差别在于多读取了score
readHash
private DatabaseValue readHash(Long expireTime) throws IOException {
int size = readLength();
Set<Tuple2<SafeString, SafeString>> entries = new LinkedHashSet<>();
for (int i = 0; i < size; i++) {
entries.add(entry(readSafeString(), readSafeString()));
}
return hash(entries).expiredAt(expireTime != null ? ofEpochMilli(expireTime) : null);
}
- readHash方法先通过readLength读取hash元素个数,在挨个执行readSafeString、readSafeString,转为entry添加到LinkedHashSet,最后将entries转为DatabaseValue再设置expiredAt
小结
importRDB方法创建RDBInputStream,然后执行其parse方法进行解析,之后遍历解析结果,挨个执行databases.get(entry.getKey()).overrideAll(ImmutableMap.from(entry.getValue()))
doc
- 通过多说服务器缓存加速Gravatar 头像,解决被墙问题
- asp.net mvc脚手架代码生成工具
- Page.FindControl方法找不到指定控件的原因
- Silverlight 2 DispatcherTimer和通过XAML创建UI元素
- 腾讯移动安全实验室发布《2013年手机安全报告》
- 自定义Unity 容器的扩展 --- Unity Application Block Event Broker
- LINQ to SQL集成到应用程序中需考虑的一些问题
- WCF的追踪分析工具——SvcPerf
- 解决七牛云存储缓存加速Gravatar 头像图片路径url 参数失效的问题
- 命令行解析的规则以及Command Line Parser Library
- 简单代码让WordPress 支持电子邮箱(Email)作为登录名
- .NET Migration工具
- 如何有效监控.NET 应用程序
- 写入Ring Buffer
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- Ubuntu16.04下安装python3.6
- Mybatis 注解
- Python如何使用Matplotlib的作图
- 在tinycolinux上组建子目录引导和混合32位64位的rootfs系统
- 微服务中的负载均衡简单实现
- 3分钟短文:素未谋面,Laravel数据库模型初阶入门
- 在tinycolinux上编译seafile
- Alink漫谈(二十一) :回归评估之源码分析
- Linux环境下通过GDB调试C项目实战
- Alink漫谈(二十二) :源码分析之聚类评估
- Python3.x将代码打包成exe程序并添加图标
- 在tinycolinux上编译pypy和hippyvm
- IDEA 热部署配置 HotSwapAgent-IntelliJ-IDEA-plugin
- 在tinycolinux上编译odoo8
- 在tinycolinux上编译jupyter和rootcling组建混合cpp,python学习环境