Redis熱點(diǎn)Key獨(dú)立集群實(shí)現(xiàn)方案(核心思路)
Redis熱點(diǎn)Key獨(dú)立集群實(shí)現(xiàn)方案
1. 設(shè)計(jì)背景
在高并發(fā)場(chǎng)景下,熱點(diǎn)Key會(huì)導(dǎo)致Redis實(shí)例負(fù)載過(guò)高,影響整個(gè)系統(tǒng)的穩(wěn)定性。通過(guò)將熱點(diǎn)Key分離到獨(dú)立的Redis集群,可以實(shí)現(xiàn)資源隔離,提高系統(tǒng)的抗風(fēng)險(xiǎn)能力。
2. 實(shí)現(xiàn)方案
2.1 核心設(shè)計(jì)思路
- 多實(shí)例配置:支持配置多個(gè)Redis實(shí)例,包括普通實(shí)例和熱點(diǎn)實(shí)例
- Key路由策略:根據(jù)Key的特征或規(guī)則,將請(qǐng)求路由到不同的Redis實(shí)例
- 統(tǒng)一訪問(wèn)接口:對(duì)外提供統(tǒng)一的Redis訪問(wèn)接口,屏蔽底層實(shí)例差異
- 靈活的路由規(guī)則:支持多種路由規(guī)則,如前綴匹配、正則匹配、哈希路由等
2.2 配置擴(kuò)展
2.2.1 修改配置屬性類(lèi)
@Data
@ConfigurationProperties(prefix = "redis.sdk", ignoreInvalidFields = true)
public class RedisClientConfigProperties {
// 默認(rèn)實(shí)例配置
private DefaultConfig defaultConfig;
// 多個(gè)實(shí)例配置
private Map<String, InstanceConfig> instances = new HashMap<>();
// 路由規(guī)則配置
private Map<String, RouteRule> routeRules = new HashMap<>();
@Data
public static class DefaultConfig {
private String host;
private int port;
private String password;
private int poolSize = 64;
private int minIdleSize = 10;
private int idleTimeout = 10000;
private int connectTimeout = 10000;
private int retryAttempts = 3;
private int retryInterval = 1000;
private int pingInterval = 0;
private boolean keepAlive = true;
}
@Data
public static class InstanceConfig {
private String host;
private int port;
private String password;
private int poolSize = 64;
private int minIdleSize = 10;
private int idleTimeout = 10000;
private int connectTimeout = 10000;
private int retryAttempts = 3;
private int retryInterval = 1000;
private int pingInterval = 0;
private boolean keepAlive = true;
}
@Data
public static class RouteRule {
// 路由類(lèi)型:prefix(前綴匹配)、regex(正則匹配)、hash(哈希路由)
private String type;
// 匹配規(guī)則
private String pattern;
// 目標(biāo)實(shí)例名稱(chēng)
private String targetInstance;
}
}2.2.2 配置文件示例
redis:
sdk:
# 默認(rèn)實(shí)例配置
default-config:
host: 127.0.0.1
port: 6379
# 多個(gè)Redis實(shí)例配置
instances:
# 普通實(shí)例
normal:
host: 127.0.0.1
port: 6379
# 熱點(diǎn)Key實(shí)例
hot:
host: 127.0.0.1
port: 6380
# 活動(dòng)相關(guān)實(shí)例
activity:
host: 127.0.0.1
port: 6381
# 路由規(guī)則配置
route-rules:
# 熱點(diǎn)Key路由規(guī)則:以"hot_"開(kāi)頭的Key路由到hot實(shí)例
hot-rule:
type: prefix
pattern: hot_
target-instance: hot
# 活動(dòng)Key路由規(guī)則:以"activity_"開(kāi)頭的Key路由到activity實(shí)例
activity-rule:
type: prefix
pattern: activity_
target-instance: activity2.3 Redis客戶端配置擴(kuò)展
@Configuration
@EnableConfigurationProperties(RedisClientConfigProperties.class)
public class RedisClientConfig {
// Redis實(shí)例映射,key為實(shí)例名稱(chēng),value為RedissonClient實(shí)例
private final Map<String, RedissonClient> redisInstances = new ConcurrentHashMap<>();
// 路由規(guī)則列表
private final List<RouteRuleWrapper> routeRules = new ArrayList<>();
@PostConstruct
public void init(ConfigurableApplicationContext applicationContext, RedisClientConfigProperties properties) {
// 1. 初始化默認(rèn)實(shí)例
initDefaultInstance(applicationContext, properties);
// 2. 初始化其他實(shí)例
initOtherInstances(applicationContext, properties);
// 3. 初始化路由規(guī)則
initRouteRules(properties);
}
private void initDefaultInstance(ConfigurableApplicationContext applicationContext, RedisClientConfigProperties properties) {
RedisClientConfigProperties.DefaultConfig defaultConfig = properties.getDefaultConfig();
RedissonClient redissonClient = createRedissonClient(defaultConfig);
redisInstances.put("default", redissonClient);
}
private void initOtherInstances(ConfigurableApplicationContext applicationContext, RedisClientConfigProperties properties) {
Map<String, RedisClientConfigProperties.InstanceConfig> instances = properties.getInstances();
for (Map.Entry<String, RedisClientConfigProperties.InstanceConfig> entry : instances.entrySet()) {
String instanceName = entry.getKey();
RedisClientConfigProperties.InstanceConfig instanceConfig = entry.getValue();
RedissonClient redissonClient = createRedissonClient(instanceConfig);
redisInstances.put(instanceName, redissonClient);
}
}
private void initRouteRules(RedisClientConfigProperties properties) {
Map<String, RedisClientConfigProperties.RouteRule> routeRulesConfig = properties.getRouteRules();
for (Map.Entry<String, RedisClientConfigProperties.RouteRule> entry : routeRulesConfig.entrySet()) {
RedisClientConfigProperties.RouteRule config = entry.getValue();
RouteRuleWrapper wrapper = new RouteRuleWrapper();
wrapper.setType(config.getType());
wrapper.setPattern(config.getPattern());
wrapper.setTargetInstance(config.getTargetInstance());
routeRules.add(wrapper);
}
}
private RedissonClient createRedissonClient(Object configObj) {
Config config = new Config();
config.setCodec(JsonJacksonCodec.INSTANCE);
String host;
int port;
String password;
int poolSize;
int minIdleSize;
int idleTimeout;
int connectTimeout;
int retryAttempts;
int retryInterval;
int pingInterval;
boolean keepAlive;
// 根據(jù)配置對(duì)象類(lèi)型獲取配置屬性
if (configObj instanceof RedisClientConfigProperties.DefaultConfig) {
RedisClientConfigProperties.DefaultConfig defaultConfig = (RedisClientConfigProperties.DefaultConfig) configObj;
host = defaultConfig.getHost();
port = defaultConfig.getPort();
password = defaultConfig.getPassword();
poolSize = defaultConfig.getPoolSize();
minIdleSize = defaultConfig.getMinIdleSize();
idleTimeout = defaultConfig.getIdleTimeout();
connectTimeout = defaultConfig.getConnectTimeout();
retryAttempts = defaultConfig.getRetryAttempts();
retryInterval = defaultConfig.getRetryInterval();
pingInterval = defaultConfig.getPingInterval();
keepAlive = defaultConfig.isKeepAlive();
} else {
RedisClientConfigProperties.InstanceConfig instanceConfig = (RedisClientConfigProperties.InstanceConfig) configObj;
host = instanceConfig.getHost();
port = instanceConfig.getPort();
password = instanceConfig.getPassword();
poolSize = instanceConfig.getPoolSize();
minIdleSize = instanceConfig.getMinIdleSize();
idleTimeout = instanceConfig.getIdleTimeout();
connectTimeout = instanceConfig.getConnectTimeout();
retryAttempts = instanceConfig.getRetryAttempts();
retryInterval = instanceConfig.getRetryInterval();
pingInterval = instanceConfig.getPingInterval();
keepAlive = instanceConfig.isKeepAlive();
}
// 配置單節(jié)點(diǎn)Redis
SingleServerConfig singleServerConfig = config.useSingleServer()
.setAddress("redis://" + host + ":" + port)
.setConnectionPoolSize(poolSize)
.setConnectionMinimumIdleSize(minIdleSize)
.setIdleConnectionTimeout(idleTimeout)
.setConnectTimeout(connectTimeout)
.setRetryAttempts(retryAttempts)
.setRetryInterval(retryInterval)
.setPingConnectionInterval(pingInterval)
.setKeepAlive(keepAlive);
// 設(shè)置密碼(如果有)
if (StringUtils.isNotBlank(password)) {
singleServerConfig.setPassword(password);
}
return Redisson.create(config);
}
/**
* 根據(jù)Key獲取對(duì)應(yīng)的RedissonClient實(shí)例
*/
public RedissonClient getRedissonClient(String key) {
// 遍歷路由規(guī)則,找到匹配的規(guī)則
for (RouteRuleWrapper rule : routeRules) {
if (matchRule(key, rule)) {
String targetInstance = rule.getTargetInstance();
return redisInstances.get(targetInstance);
}
}
// 沒(méi)有匹配到規(guī)則,使用默認(rèn)實(shí)例
return redisInstances.get("default");
}
/**
* 檢查Key是否匹配路由規(guī)則
*/
private boolean matchRule(String key, RouteRuleWrapper rule) {
String type = rule.getType();
String pattern = rule.getPattern();
switch (type) {
case "prefix":
// 前綴匹配
return key.startsWith(pattern);
case "regex":
// 正則匹配
return key.matches(pattern);
case "hash":
// 哈希路由(根據(jù)Key的哈希值路由到不同實(shí)例)
// 這里簡(jiǎn)化實(shí)現(xiàn),實(shí)際可以根據(jù)哈希值和實(shí)例數(shù)量計(jì)算路由
int hash = key.hashCode();
return Math.abs(hash) % 2 == 0; // 示例:偶數(shù)哈希值匹配
default:
return false;
}
}
/**
* 路由規(guī)則包裝類(lèi)
*/
@Data
private static class RouteRuleWrapper {
private String type;
private String pattern;
private String targetInstance;
}
/**
* 注入Redis服務(wù)
*/
@Bean("redisService")
public RedisService redisService() {
return new RedisServiceImpl(this);
}
}2.4 統(tǒng)一Redis服務(wù)封裝
/**
* Redis服務(wù)接口
*/
public interface RedisService {
/**
* 設(shè)置Key-Value
*/
<T> void set(String key, T value);
/**
* 設(shè)置Key-Value,帶過(guò)期時(shí)間
*/
<T> void set(String key, T value, long expireTime, TimeUnit timeUnit);
/**
* 獲取Value
*/
<T> T get(String key, Class<T> clazz);
/**
* 刪除Key
*/
boolean delete(String key);
/**
* 設(shè)置Hash字段
*/
<T> void hset(String key, String field, T value);
/**
* 獲取Hash字段
*/
<T> T hget(String key, String field, Class<T> clazz);
// 其他Redis操作方法...
}
/**
* Redis服務(wù)實(shí)現(xiàn)類(lèi)
*/
@Service
public class RedisServiceImpl implements RedisService {
private final RedisClientConfig redisClientConfig;
public RedisServiceImpl(RedisClientConfig redisClientConfig) {
this.redisClientConfig = redisClientConfig;
}
@Override
public <T> void set(String key, T value) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RMap<String, T> map = redissonClient.getMap("cache");
map.put(key, value);
}
@Override
public <T> void set(String key, T value, long expireTime, TimeUnit timeUnit) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RBucket<T> bucket = redissonClient.getBucket(key);
bucket.set(value, expireTime, timeUnit);
}
@Override
public <T> T get(String key, Class<T> clazz) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RBucket<T> bucket = redissonClient.getBucket(key);
return bucket.get();
}
@Override
public boolean delete(String key) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RBucket<Object> bucket = redissonClient.getBucket(key);
return bucket.delete();
}
@Override
public <T> void hset(String key, String field, T value) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RMap<String, T> map = redissonClient.getMap(key);
map.put(field, value);
}
@Override
public <T> T hget(String key, String field, Class<T> clazz) {
RedissonClient redissonClient = redisClientConfig.getRedissonClient(key);
RMap<String, T> map = redissonClient.getMap(key);
return map.get(field);
}
// 其他Redis操作方法實(shí)現(xiàn)...
}2.5 使用示例
@Service
public class ActivityService {
@Autowired
private RedisService redisService;
public void cacheActivityInfo(String activityId, Activity activity) {
// 活動(dòng)相關(guān)Key,會(huì)路由到activity實(shí)例
String key = "activity_" + activityId;
redisService.set(key, activity, 1, TimeUnit.HOURS);
}
public Activity getActivityInfo(String activityId) {
String key = "activity_" + activityId;
return redisService.get(key, Activity.class);
}
public void cacheHotProduct(String productId, Product product) {
// 熱點(diǎn)Key,會(huì)路由到hot實(shí)例
String key = "hot_product_" + productId;
redisService.set(key, product, 30, TimeUnit.MINUTES);
}
public Product getHotProduct(String productId) {
String key = "hot_product_" + productId;
return redisService.get(key, Product.class);
}
public void cacheUserInfo(String userId, User user) {
// 普通Key,會(huì)路由到default實(shí)例
String key = "user_" + userId;
redisService.set(key, user, 24, TimeUnit.HOURS);
}
}3. 方案優(yōu)勢(shì)
3.1 資源隔離
- 熱點(diǎn)Key單獨(dú)存儲(chǔ)在獨(dú)立的Redis實(shí)例中,避免影響其他業(yè)務(wù)
- 不同業(yè)務(wù)線的Key可以分離到不同實(shí)例,實(shí)現(xiàn)業(yè)務(wù)隔離
3.2 靈活擴(kuò)展
- 支持動(dòng)態(tài)添加Redis實(shí)例,應(yīng)對(duì)業(yè)務(wù)增長(zhǎng)
- 支持多種路由規(guī)則,適應(yīng)不同業(yè)務(wù)場(chǎng)景
3.3 高可用性
- 單個(gè)Redis實(shí)例故障不會(huì)影響整個(gè)系統(tǒng)
- 可以為熱點(diǎn)實(shí)例配置更高的資源規(guī)格
3.4 統(tǒng)一訪問(wèn)接口
- 對(duì)外提供統(tǒng)一的Redis訪問(wèn)接口,簡(jiǎn)化開(kāi)發(fā)
- 底層實(shí)例變更對(duì)業(yè)務(wù)代碼透明
3.5 易于維護(hù)
- 集中管理Redis實(shí)例配置
- 統(tǒng)一監(jiān)控和管理所有Redis實(shí)例
4. 部署架構(gòu)
+-------------------+ +-------------------+ +-------------------+
| | | | | |
| 應(yīng)用服務(wù) | | Redis普通實(shí)例 | | Redis熱點(diǎn)實(shí)例 |
| (RedisService) |--->| (Port: 6379) | | (Port: 6380) |
| | | | | |
+-------------------+ +-------------------+ +-------------------+
|
v
+-------------------+
| |
| Redis活動(dòng)實(shí)例 |
| (Port: 6381) |
| |
+-------------------+5. 注意事項(xiàng)
- 路由規(guī)則設(shè)計(jì):路由規(guī)則應(yīng)根據(jù)業(yè)務(wù)特點(diǎn)精心設(shè)計(jì),避免規(guī)則沖突
- 數(shù)據(jù)遷移:已有數(shù)據(jù)需要考慮遷移策略,確保平滑過(guò)渡
- 監(jiān)控告警:需要為每個(gè)Redis實(shí)例配置獨(dú)立的監(jiān)控和告警
- 一致性問(wèn)題:不同實(shí)例間的數(shù)據(jù)一致性需要業(yè)務(wù)層面保證
- 連接管理:需要合理配置連接池大小,避免連接泄漏
- 性能測(cè)試:上線前需要進(jìn)行充分的性能測(cè)試,驗(yàn)證方案效果
6. 擴(kuò)展建議
- 自動(dòng)熱點(diǎn)識(shí)別:結(jié)合監(jiān)控?cái)?shù)據(jù),實(shí)現(xiàn)熱點(diǎn)Key的自動(dòng)識(shí)別和遷移
- 動(dòng)態(tài)路由調(diào)整:支持根據(jù)實(shí)例負(fù)載動(dòng)態(tài)調(diào)整路由規(guī)則
- Redis集群支持:擴(kuò)展支持Redis集群配置,提高可用性
- 多種客戶端支持:除了Redisson,支持其他Redis客戶端如Lettuce
- 緩存預(yù)熱:實(shí)現(xiàn)熱點(diǎn)數(shù)據(jù)的自動(dòng)預(yù)熱,提高系統(tǒng)響應(yīng)速度
通過(guò)以上方案,可以實(shí)現(xiàn)熱點(diǎn)Key的獨(dú)立集群部署,提高系統(tǒng)的抗風(fēng)險(xiǎn)能力和性能表現(xiàn),同時(shí)保持良好的擴(kuò)展性和維護(hù)性。
到此這篇關(guān)于Redis熱點(diǎn)Key獨(dú)立集群實(shí)現(xiàn)方案的文章就介紹到這了,更多相關(guān)redis熱點(diǎn)key內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
- Redis熱點(diǎn)Key問(wèn)題的完整解決方案:
- Redis 緩存使用的熱點(diǎn)Key問(wèn)題的解決
- Redis熱點(diǎn)Key問(wèn)題分析與解決方案
- 詳解如何發(fā)現(xiàn)并解決Redis熱點(diǎn)Key問(wèn)題
- 面試分析分布式架構(gòu)Redis熱點(diǎn)key大Value解決方案
- Redis緩存及熱點(diǎn)key問(wèn)題解決方案
- 通過(guò)Redisson監(jiān)聽(tīng)Redis集群的Key過(guò)期事件的實(shí)現(xiàn)指南
- redis集群實(shí)現(xiàn)清理前綴相同的key
- Redis集群下過(guò)期key監(jiān)聽(tīng)的實(shí)現(xiàn)代碼
相關(guān)文章
詳解Redis數(shù)據(jù)類(lèi)型實(shí)現(xiàn)原理
這篇文章主要介紹了Redis數(shù)據(jù)類(lèi)型實(shí)現(xiàn)原理,在工作中或?qū)W習(xí)中有需要的小伙伴可以參考一下這篇文章2021-08-08
使用攔截器+Redis實(shí)現(xiàn)接口冪思路詳解
這篇文章主要介紹了使用攔截器+Redis實(shí)現(xiàn)接口冪等,接口冪等有很多種實(shí)現(xiàn)方式,攔截器/AOP+Redis,攔截器/AOP+本地緩存等等,本文講解一下通過(guò)攔截器+Redis實(shí)現(xiàn)冪等的方式,需要的朋友可以參考下2023-08-08
Redis sentinel節(jié)點(diǎn)如何修改密碼
這篇文章主要介紹了Redis sentinel節(jié)點(diǎn)如何修改密碼問(wèn)題,具有很好的參考價(jià)值,希望對(duì)大家有所幫助,如有錯(cuò)誤或未考慮完全的地方,望不吝賜教2024-01-01
Redis鍵值設(shè)計(jì)的具體實(shí)現(xiàn)
本文主要介紹了Redis鍵值設(shè)計(jì)的具體實(shí)現(xiàn),文中通過(guò)示例代碼介紹的非常詳細(xì),對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值,需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)學(xué)習(xí)吧2024-06-06
spring?boot整合redis中間件與熱部署實(shí)現(xiàn)代碼
spring?boot整合redis最常用的有三個(gè)工具庫(kù)Jedis,Redisson,Lettuce,本文重點(diǎn)介紹spring?boot整合redis中間件與熱部署實(shí)現(xiàn),需要的朋友可以參考下2023-01-01
Redis結(jié)合Caffeine兩級(jí)緩存的三種實(shí)現(xiàn)方式
本文主要介紹了Redis結(jié)合Caffeine兩級(jí)緩存的實(shí)現(xiàn)示例,通過(guò)手動(dòng)、Spring注解及自定義切面三種方式實(shí)現(xiàn),具有一定的參考價(jià)值,感興趣的可以了解一下2025-08-08

