Apache ShardingSphere簡介及使用詳解
一 什么是 ShardingSphere?
1.1 背景:為什么需要分庫分表?
隨著系統(tǒng)數(shù)據(jù)量和用戶量不斷增長,傳統(tǒng)單體數(shù)據(jù)庫容易遇到瓶頸:
寫入/查詢壓力大:單表千萬級數(shù)據(jù),索引效率下降。
存儲超限:單機磁盤空間和 IOPS 不堪重負。
水平擴展困難:業(yè)務(wù)難以橫向拓展。
為了解決這些問題,開發(fā)者往往采用 分庫分表 技術(shù),將一張大表分成多個小表,分布在不同數(shù)據(jù)庫中,實現(xiàn)水平擴展和負載均衡。
1.2 什么是 Apache ShardingSphere?
Apache ShardingSphere 是一款開源的 分布式數(shù)據(jù)庫中間件框架,可以幫助開發(fā)者輕松實現(xiàn):
- 分庫分表
- 讀寫分離
- 分布式事務(wù)
- 數(shù)據(jù)脫敏
- 可觀測性 & 擴展治理能力
ShardingSphere 核心組件包括:
| 組件 | 說明 |
|---|---|
| ShardingSphere-JDBC | JDBC 封裝層,嵌入應(yīng)用中,適用于微服務(wù)架構(gòu) |
| ShardingSphere-Proxy | 獨立部署的數(shù)據(jù)庫代理,支持多語言訪問 |
| ShardingSphere-Sidecar(規(guī)劃中) | 面向云原生場景,Service Mesh 結(jié)構(gòu) |
1.3 ShardingSphere-JDBC 的核心功能
| 功能 | 簡要說明 |
|---|---|
| 分庫分表 | 按字段配置路由策略,自動將 SQL 分發(fā)到對應(yīng)的數(shù)據(jù)庫/表 |
| 讀寫分離 | 自動區(qū)分讀寫請求,路由到主/從數(shù)據(jù)庫 |
| 靈活分片策略 | 支持取模、范圍、Hint、自定義算法等多種策略 |
| 事務(wù)支持 | 支持本地事務(wù)和分布式事務(wù)(Seata/XA) |
| 透明化使用 | 對開發(fā)者來說使用方式與普通 JDBC 幾乎一致,只需配置邏輯表名 |
二、實踐部分:Java + H2 實現(xiàn)分庫分表
示例:使用 Java 原生程序 + ShardingSphere-JDBC + H2 內(nèi)存數(shù)據(jù)庫,模擬一個電商訂單系統(tǒng)按 user_id 分庫,order_id 分表。
2.1 項目結(jié)構(gòu)
- 使用 H2 數(shù)據(jù)庫模擬兩個庫
ds0,ds1 - 每個庫中創(chuàng)建兩個分表
t_order_0,t_order_1 - 邏輯表名:
t_order
2.2 Maven 依賴配置
<dependencies>
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>shardingsphere-jdbc-core</artifactId>
<version>5.4.1</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.2.220</version>
</dependency>
<dependency>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
<version>5.0.1</version>
</dependency>
</dependencies>2.3 核心代碼結(jié)構(gòu)
1. 創(chuàng)建真實表(物理分表)
public static void createTables() throws SQLException {
for (int i = 0; i < 2; i++) {
String db = "ds" + i;
try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + db + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) {
Statement stmt = conn.createStatement();
for (int j = 0; j < 2; j++) {
stmt.execute(String.format("""
CREATE TABLE IF NOT EXISTS t_order_%d (
order_id BIGINT PRIMARY KEY,
user_id INT,
status VARCHAR(50)
)
""", j));
}
}
}
}2. 配置分庫分表數(shù)據(jù)源
public static DataSource createDataSource() throws SQLException {
Map<String, DataSource> dataSourceMap = new HashMap<>();
for (int i = 0; i < 2; i++) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
dataSourceMap.put("ds" + i, ds);
}
ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration(
"t_order", "ds${0..1}.t_order_${0..1}");
orderTableRule.setDatabaseShardingStrategy(
new StandardShardingStrategyConfiguration("user_id", "dbSharding"));
orderTableRule.setTableShardingStrategy(
new StandardShardingStrategyConfiguration("order_id", "tableSharding"));
ShardingRuleConfiguration config = new ShardingRuleConfiguration();
config.getTables().add(orderTableRule);
config.getShardingAlgorithms().put("dbSharding",
new AlgorithmConfiguration("INLINE", props("algorithm-expression", "ds${user_id % 2}")));
config.getShardingAlgorithms().put("tableSharding",
new AlgorithmConfiguration("INLINE", props("algorithm-expression", "t_order_${order_id % 2}")));
return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(config), new Properties());
}3. 插入與查詢數(shù)據(jù)
public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException {
try (Connection conn = ds.getConnection()) {
PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)");
ps.setLong(1, orderId);
ps.setInt(2, userId);
ps.setString(3, status);
ps.executeUpdate();
}
}
public static void queryOrders(DataSource ds) throws SQLException {
try (Connection conn = ds.getConnection()) {
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order");
while (rs.next()) {
System.out.printf("order_id=%d, user_id=%d, status=%s%n",
rs.getLong("order_id"),
rs.getInt("user_id"),
rs.getString("status"));
}
}
}2.4 程序入口
public static void main(String[] args) throws Exception {
createTables(); // 創(chuàng)建實際分表
DataSource shardingDataSource = createDataSource();
insertOrder(shardingDataSource, 1001, 1, "INIT");
insertOrder(shardingDataSource, 1002, 2, "PAID");
insertOrder(shardingDataSource, 1003, 3, "SHIPPED");
queryOrders(shardingDataSource);
}輸出示例
order_id=1001, user_id=1, status=INIT
order_id=1002, user_id=2, status=PAID
order_id=1003, user_id=3, status=SHIPPED
ShardingSphere 已自動根據(jù)你配置的策略將數(shù)據(jù)路由到對應(yīng)的庫和表!
總結(jié)
Apache ShardingSphere 提供了強大、靈活的分庫分表能力,通過合理配置可以極大提升系統(tǒng)的性能與擴展性:
- 分片策略靈活,支持多種規(guī)則或自定義算法
- 配置簡單,無需改動業(yè)務(wù) SQL
- 兼容性強,支持 JDBC、Spring Boot、YAML 等多種使用方式
無論你是中小項目的快速原型,還是大規(guī)模高并發(fā)系統(tǒng),ShardingSphere 都是一個值得一試的解決方案。
完整代碼
package org.example;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.shardingsphere.driver.api.ShardingSphereDataSourceFactory;
import org.apache.shardingsphere.infra.config.algorithm.AlgorithmConfiguration;
import org.apache.shardingsphere.sharding.api.config.ShardingRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.rule.ShardingTableRuleConfiguration;
import org.apache.shardingsphere.sharding.api.config.strategy.sharding.StandardShardingStrategyConfiguration;
import javax.sql.DataSource;
import java.sql.*;
import java.util.*;
public class ShardingH2Example {
public static void main(String[] args) throws Exception {
createTables(); // 注意:用 DriverManager 直連底層 db 建表
DataSource shardingDataSource = createDataSource();
// 插入測試數(shù)據(jù)
insertOrder(shardingDataSource, 1001, 1, "INIT");
insertOrder(shardingDataSource, 1002, 2, "PAID");
insertOrder(shardingDataSource, 1003, 3, "SHIPPED");
// 查詢測試數(shù)據(jù)
queryOrders(shardingDataSource);
}
public static DataSource createDataSource() throws Exception {
Map<String, DataSource> dataSourceMap = new HashMap<>();
// 模擬兩個庫:ds0 和 ds1(分別用兩個內(nèi)存 H2 實例模擬)
for (int i = 0; i < 2; i++) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:ds" + i + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
dataSourceMap.put("ds" + i, ds);
}
// 分表規(guī)則配置
ShardingTableRuleConfiguration orderTableRule = new ShardingTableRuleConfiguration(
"t_order", "ds${0..1}.t_order_${0..1}");
// 分庫策略(user_id)
orderTableRule.setDatabaseShardingStrategy(new StandardShardingStrategyConfiguration(
"user_id", "dbSharding"));
// 分表策略(order_id)
orderTableRule.setTableShardingStrategy(new StandardShardingStrategyConfiguration(
"order_id", "tableSharding"));
ShardingRuleConfiguration shardingRuleConfig = new ShardingRuleConfiguration();
shardingRuleConfig.getTables().add(orderTableRule);
Properties dbProps = new Properties();
dbProps.setProperty("algorithm-expression", "ds${user_id % 2}");
shardingRuleConfig.getShardingAlgorithms().put("dbSharding",
new AlgorithmConfiguration("INLINE", dbProps));
Properties tableProps = new Properties();
tableProps.setProperty("algorithm-expression", "t_order_${order_id % 2}");
shardingRuleConfig.getShardingAlgorithms().put("tableSharding",
new AlgorithmConfiguration("INLINE", tableProps));
return ShardingSphereDataSourceFactory.createDataSource(dataSourceMap, List.of(shardingRuleConfig), new Properties());
}
public static void createTables() throws SQLException {
for (int i = 0; i < 2; i++) {
String dbName = "ds" + i;
try (Connection conn = DriverManager.getConnection("jdbc:h2:mem:" + dbName + ";DB_CLOSE_DELAY=-1;MODE=MySQL", "sa", "")) {
Statement stmt = conn.createStatement();
for (int j = 0; j < 2; j++) {
String sql = String.format("""
CREATE TABLE IF NOT EXISTS t_order_%d (
order_id BIGINT PRIMARY KEY,
user_id INT,
status VARCHAR(50)
)
""", j);
stmt.execute(sql);
}
}
}
}
// 輔助方法用于手動連接底層 H2 數(shù)據(jù)源
private static HikariDataSource getH2DataSource(String name) {
HikariDataSource ds = new HikariDataSource();
ds.setJdbcUrl("jdbc:h2:mem:" + name + ";DB_CLOSE_DELAY=-1;MODE=MySQL");
ds.setUsername("sa");
ds.setPassword("");
return ds;
}
public static void insertOrder(DataSource ds, long orderId, int userId, String status) throws SQLException {
try (Connection conn = ds.getConnection()) {
PreparedStatement ps = conn.prepareStatement("INSERT INTO t_order (order_id, user_id, status) VALUES (?, ?, ?)");
ps.setLong(1, orderId);
ps.setInt(2, userId);
ps.setString(3, status);
ps.executeUpdate();
}
}
public static void queryOrders(DataSource ds) throws SQLException {
try (Connection conn = ds.getConnection()) {
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM t_order");
System.out.println("Query Results:");
while (rs.next()) {
System.out.printf("order_id: %d, user_id: %d, status: %s%n",
rs.getLong("order_id"),
rs.getInt("user_id"),
rs.getString("status"));
}
}
}
}到此這篇關(guān)于Apache ShardingSphere 初識使用的文章就介紹到這了,更多相關(guān)Apache ShardingSphere使用內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!
- SpringBoot?整合?ShardingSphere4.1.1實現(xiàn)分庫分表功能
- springboot如何使用yml文件方式配置shardingsphere
- ShardingSphere如何進行sql重寫示例詳解
- SpringBoot3和ShardingSphere5框架實現(xiàn)數(shù)據(jù)分庫分表
- springboot整合shardingsphere和seata實現(xiàn)分布式事務(wù)的實踐
- Java基于ShardingSphere實現(xiàn)分庫分表的實例詳解
- ShardingSphere jdbc集成多數(shù)據(jù)源的實現(xiàn)步驟
- SpringBoot整合ShardingSphere的示例代碼
- Java中ShardingSphere分庫分表實戰(zhàn)
相關(guān)文章
探討如何減少Linux服務(wù)器TIME_WAIT過多的問題
本篇文章是對如何減少Linux服務(wù)器TIME_WAIT過多的問題進行了詳細的分析介紹,需要的朋友參考下2013-06-06
Linux 7.4上安裝配置Oracle 11.2.0.4圖文教程
本文通過圖文并茂的形式給大家介紹了Linux 7.4上安裝配置Oracle 11.2.0.4的方法,非常不錯,具有參考借鑒價值,需要的朋友參考下吧2017-12-12
Linux下雙網(wǎng)卡Firewalld的配置流程(推薦)
firewalld提供了一個 動態(tài)管理的防火墻,用以支持不同網(wǎng)絡(luò)區(qū)域的規(guī)則,分配對一個網(wǎng)絡(luò)及其相關(guān)鏈接和界面一定程度的信任。這篇文章給大家介紹了Linux下雙網(wǎng)卡Firewalld的配置流程,需要的朋友參考下吧2018-04-04

