史上最輕鬆入門之Spring Batch - 輕量級批處理框架實踐
阿新 • • 發佈：2019-10-30
從 MariaDB 一張表內讀 10 萬條記錄，經處理後寫到 MongoDB 。
Batch 任務模型
具體實現
1、新建 Spring Boot 應用，依賴如下：
<!-- Web 應用 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <exclusions> <exclusion> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-logging</artifactId> </exclusion> <exclusion> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-tomcat</artifactId> </exclusion> </exclusions> </dependency> <!-- Web 容器 undertow --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-undertow</artifactId> </dependency> <!-- 日誌 Log4j2 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-log4j2</artifactId> </dependency> <!-- MongoDB --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-mongodb</artifactId> </dependency> <!-- Brantch --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-batch</artifactId> </dependency> <!-- Mariadb 驅動 --> <dependency> <groupId>org.mariadb.jdbc</groupId> <artifactId>mariadb-java-client</artifactId> <version>2.0.2</version> </dependency> <!-- Lombok 程式碼簡化 --> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.16.14</version> </dependency>
2、建立一張表，並生成 10 萬條資料
DROP TABLE people IF EXISTS;
CREATE TABLE people (
id BIGINT IDENTITY NOT NULL PRIMARY KEY,
first_name VARCHAR(20),
last_name VARCHAR(20)
);3、建立 Person 類
@Data
public class Person {
private Long id;
private String lastName;
private String firstName;
}4、建立一箇中間處理器 PersonItemProcessor
import org.springframework.batch.item.ItemProcessor; @Log4j2 public class PersonItemProcessor implements ItemProcessor<Person, Person> { @Override public Person process(final Person person) throws Exception { final String firstName = person.getFirstName().toUpperCase(); final String lastName = person.getLastName().toUpperCase(); final Person transformedPerson = new Person(firstName, lastName); log.info("Converting (" + person + ") into (" + transformedPerson + ")"); return transformedPerson; } }
5、建立 PersonMapper，使用者資料庫對映
public class PersonMapper implements RowMapper {
private static final String ID_COLUMN = "id";
private static final String NICKNAME_COLUMN = "first_name";
private static final String EMAIL_COLUMN = "last_name";
@Override
public Object mapRow(ResultSet resultSet, int i) throws SQLException {
Person user = new Person();
person.setId(resultSet.getLong(ID_COLUMN));
person.setNickname(resultSet.getString(NICKNAME_COLUMN));
person.setEmail(resultSet.getString(EMAIL_COLUMN));
return person;
}
}6、建立任務完成的監聽 JobCompletionNotificationListener
@Log4j2
@Component
public class JobCompletionNotificationListener extends JobExecutionListenerSupport {
@Override
public void afterJob(JobExecution jobExecution) {
if(jobExecution.getStatus() == BatchStatus.COMPLETED) {
log.info("!!! JOB FINISHED! Time to verify the results");
}
}
}7、構建批處理任務 BatchConfiguration
@Configuration
@EnableBatchProcessing
public class BatchConfiguration {
@Autowired
public JobBuilderFactory jobBuilderFactory;
@Autowired
public StepBuilderFactory stepBuilderFactory;
@Autowired
public DataSource dataSource;
@Autowired
public MongoTemplate mongoTemplate;
@Bean
public JdbcCursorItemReader<Person> reader(){
JdbcCursorItemReader<Person> itemReader = new JdbcCursorItemReader<Person>();
itemReader.setDataSource(dataSource);
itemReader.setSql("select id, nickname, email from people");
itemReader.setRowMapper(new PersonMapper());
return itemReader;
}
@Bean
public PersonItemProcessor processor() {
return new PersonItemProcessor();
}
@Bean
MongoItemWriter<Person> writer(){
MongoItemWriter<Person> itemWriter = new MongoItemWriter<Person>();
itemWriter.setTemplate(mongoTemplate);
itemWriter.setCollection("branch");
return itemWriter;
}
@Bean
public Step step() {
return stepBuilderFactory.get("step")
.<Person, Person> chunk(10)
.reader(reader())
.processor(processor())
.writer(writer())
.build();
}
@Bean
public Job importUserJob(JobCompletionNotificationListener listener) {
return jobBuilderFactory.get("importUserJob")
.incrementer(new RunIdIncrementer())
.listener(listener)
.flow(step())
.end()
.build();
}
}任務處理結果
0出錯，耗時 2 分鐘左右，測試機 Mac
© 著作權歸作者所有,轉載或內容合作請聯絡作者
● Spring Cloud Gateway - 快速開始
● APM工具尋找了一圈，發現SkyWalking才是我的真愛
● Spring Boot 注入外部配置到應用內部的靜態變數
● 將 HTML 轉化為 PDF新姿勢
● Java 使用 UnixSocket 呼叫 Docker API
● Fastjson致命缺陷
● Service Mesh - gRPC 本地聯調遠端服務
● 使用 Thymeleaf 動態渲染 HTML
● Fastjson致命缺陷
● Spring Boot 2 整合log4j2日誌框架
● Java面試通關要點彙總集之核心篇參考答案
● Java面試通關要點彙總集之框架篇參考答案
● Spring Security 實戰乾貨：如何保護使用者密碼
● Spring Boot RabbitMQ - 優先順序佇列
原文連結： https://mp.weixin.qq.com/s?__biz=MzU0MDEwMjgwNA==&mid=2247486167&idx=1&sn=5983bd5437d14fffe48d42134da46c69&chksm=fb3f132ccc489a3aa813fb07ee64206bf3a51f57aef9b58ab7a8ab18c3d608619ed807a5828e&token=280305379&lang=zh_CN#rd
本文由部落格一文多發平臺 OpenWrite 釋出！