Spring Batch is a framework for batch processing – execution of a series of jobs. In Spring Batch, A job consists of many steps and each step consists of a READ-PROCESS-WRITE task or single operation task (tasklet).

1. For “READ-PROCESS-WRITE” process, it means “read” data from the resources (csv, xml or database), “process” it and “write” it to other resources (csv, xml and database). For example, a step may read data from a CSV file, process it and write it into the database. Spring Batch provides many made Classes to read/write CSV, XML and database.
2. For “single” operation task (tasklet), it means doing single task only, like clean up the resources after or before a step is started or completed.
3. And the steps can be chained together to run as a job.

1 Job = Many Steps.1 Step = 1 READ-PROCESS-WRITE or 1 Tasklet.Job = {
    Step 1 -> Step 2 -> Step 3} (Chained together)

Spring Batch Examples

Consider following batch jobs :

1. Step 1 – Read CSV files from folder A, process, write it to folder B. “READ-PROCESS-WRITE”
2. Step 2 – Read CSV files from folder B, process, write it to the database. “READ-PROCESS-WRITE”
3. Step 3 – Delete the CSB files from folder B. “Tasklet”
4. Step 4 – Read data from a database, process and generate statistic report in XML format, write it to folder C. “READ-PROCESS-WRITE”
5. Step 5 – Read the report and send it to manager email. “Tasklet”

In Spring Batch, we can declare like the following :

The entire jobs and steps execution are stored in database, which make the failed step is able to restart at where it was failed, no need start over the entire job.

1. Tutorial

In this Spring Batch tutorial, we will show you how to create a job, read a CSV file, process it, write the output to an XML file.

Tools and libraries used

1. Maven 3
2. Eclipse 4.2
3. JDK 1.6
4. Spring Core 3.2.2.RELEASE
5. Spring OXM 3.2.2.RELEASE
6. Spring JDBC 3.2.2.RELEASE
7. Spring Batch 2.2.0.RELEASE

2. Project Directory

Review final project directory, a standard Maven project.

3. Project Dependencies

They must have dependencies are just Spring Core, Spring Batch and JDK 1.5. Read comments for self-explanatory.

pom.xml

    	
     
      4.0.0
     	
     
      com.mkyong
     	
     
      SpringBatchExample
     	
     
      jar
     	
     
      1.0-SNAPSHOT
     	
     
      SpringBatchExample
     	
     
      http://maven.apache.org
     	
     		
      
       1.6
      		
      
       3.2.2.RELEASE
      		
      
       2.2.0.RELEASE
      		
      
       5.1.25
      		
      
       4.11
      	
     	
     		
      		
      			
       
        org.springframework
       			
       
        spring-core
       			
       
        ${spring.version}
       		
      		
      		
      			
       
        org.springframework
       			
       
        spring-jdbc
       			
       
        ${spring.version}
       		
      		
      		
      			
       
        org.springframework
       			
       
        spring-oxm
       			
       
        ${spring.version}
       		
      		
      		
      			
       
        mysql
       			
       
        mysql-connector-java
       			
       
        ${mysql.driver.version}
       		
      		
      		
      			
       
        org.springframework.batch
       			
       
        spring-batch-core
       			
       
        ${spring.batch.version}
       		
      		
      			
       
        org.springframework.batch
       			
       
        spring-batch-infrastructure
       			
       
        ${spring.batch.version}
       		
      		
      		
      			
       
        org.springframework.batch
       			
       
        spring-batch-test
       			
       
        ${spring.batch.version}
       		
      				
      		
      			
       
        junit
       			
       
        junit
       			
       
        ${junit.version}
       			
       
        test
       		
      	
     	
     		
      
       spring-batch
      		
      			
       				
        
         org.apache.maven.plugins
        				
        
         maven-eclipse-plugin
        				
        
         2.9
        				
        					
         
          true
         					
         
          false
         				
        			
       			
       				
        
         org.apache.maven.plugins
        				
        
         maven-compiler-plugin
        				
        
         2.3.2
        				
        					
         ${jdk.version}					
         
          ${jdk.version}
         				
        			
       		
      	
     
    

4. Spring Batch Jobs

A CSV file.

report.csv

1001,"213,100",980,"mkyong", 29/7/20131002,"320,200",1080,"staff 1", 30/7/20131003,"342,197",1200,"staff 2", 31/7/2013

A Spring batch job, to read above csv file with FlatFileItemReader, process the data with itemProcessor and write it to an XML file

with StaxEventItemWriter.

job-hello-world.xml

    	
     	
     	
     	
     	
     	  
      		
       			
        			
        		
       	  
      	
     	
     		
      		
      		    
       			
        				
          
           
         			
        			
        				
         				    				 
         			
        		    
       		
      	
     	
     		
      		
      		
      	
     	
     	   
      		
       			
        
         com.mkyong.model.Report
        		
       	    
      	
     
    

Map CSV value to Report object and write it to XML file (via jaxb annotations).

Report.java

package com.mkyong.model;import java.math.BigDecimal;import java.util.Date;import javax.xml.bind.annotation.XmlAttribute;import javax.xml.bind.annotation.XmlElement;import javax.xml.bind.annotation.XmlRootElement;@XmlRootElement(name = "record")public class Report {
     	private int id;	private BigDecimal sales;	private int qty;	private String staffName;	private Date date;	@XmlAttribute(name = "id")	public int getId() {
     		return id;	}	public void setId(int id) {
     		this.id = id;	}	@XmlElement(name = "sales")	public BigDecimal getSales() {
     		return sales;	}	public void setSales(BigDecimal sales) {
     		this.sales = sales;	}	@XmlElement(name = "qty")	public int getQty() {
     		return qty;	}	public void setQty(int qty) {
     		this.qty = qty;	}	@XmlElement(name = "staffName")	public String getStaffName() {
     		return staffName;	}	public void setStaffName(String staffName) {
     		this.staffName = staffName;	}	public Date getDate() {
     		return date;	}	public void setDate(Date date) {
     		this.date = date;	}	@Override	public String toString() {
     		return "Report [id=" + id + ", sales=" + sales                     + ", qty=" + qty + ", staffName=" + staffName + "]";	}}

To convert a Date, you need a custom FieldSetMapper. If no data type conversion, just use BeanWrapperFieldSetMapperto map the values by name automatically.

ReportFieldSetMapper.java

package com.mkyong;import java.text.ParseException;import java.text.SimpleDateFormat;import org.springframework.batch.item.file.mapping.FieldSetMapper;import org.springframework.batch.item.file.transform.FieldSet;import org.springframework.validation.BindException;import com.mkyong.model.Report;public class ReportFieldSetMapper implements FieldSetMapper
    
      {
     	private SimpleDateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy");		@Override	public Report mapFieldSet(FieldSet fieldSet) throws BindException {
     				Report report = new Report();		report.setId(fieldSet.readInt(0));		report.setSales(fieldSet.readBigDecimal(1));		report.setQty(fieldSet.readInt(2));		report.setStaffName(fieldSet.readString(3));				//default format yyyy-MM-dd		//fieldSet.readDate(4);		String date = fieldSet.readString(4);		try {
     			report.setDate(dateFormat.parse(date));		} catch (ParseException e) {
     			e.printStackTrace();		}				return report;			}}
    

A itemProcessor will be fired before itemWriter.

CustomItemProcessor.java

package com.mkyong;import org.springframework.batch.item.ItemProcessor;import com.mkyong.model.Report;public class CustomItemProcessor implements ItemProcessor
    
      {
     	@Override	public Report process(Report item) throws Exception {
     				System.out.println("Processing..." + item);		return item;	}}
    

Spring context and database configuration.

context.xml

database.xml

5. Run It

The most simplest way to run a batch job.

package com.mkyong;import org.springframework.batch.core.Job;import org.springframework.batch.core.JobExecution;import org.springframework.batch.core.JobParameters;import org.springframework.batch.core.launch.JobLauncher;import org.springframework.context.ApplicationContext;import org.springframework.context.support.ClassPathXmlApplicationContext;public class App {
       public static void main(String[] args) {
     	String[] springConfig  = 		{
     				"spring/batch/jobs/job-hello-world.xml" 		};			ApplicationContext context = 			new ClassPathXmlApplicationContext(springConfig);			JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");	Job job = (Job) context.getBean("helloWorldJob");	try {
     		JobExecution execution = jobLauncher.run(job, new JobParameters());		System.out.println("Exit Status : " + execution.getStatus());	} catch (Exception e) {
     		e.printStackTrace();	}	System.out.println("Done");  }}

Output

report.xml

    
    	
     		
      
       2013-07-29T00:00:00+08:00
      		
      
       980
      		
      
       213100
      		
      
       mkyong
      	
     	
     		
      
       2013-07-30T00:00:00+08:00
      		
      
       1080
      		
      
       320200
      		
      
       staff 1
      	
     	
     		
      
       2013-07-31T00:00:00+08:00
      		
      
       1200
      		
      
       342197
      		
      
       staff 2
      	
     
    

In console

Jul 30, 2013 11:52:00 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 runINFO: Job: [FlowJob: [name=helloWorldJob]] launched with the following parameters: [{
    }]Jul 30, 2013 11:52:00 PM org.springframework.batch.core.job.SimpleStepHandler handleStepINFO: Executing step: [step1]Processing...Report [id=1001, sales=213100, qty=980, staffName=mkyong]Processing...Report [id=1002, sales=320200, qty=1080, staffName=staff 1]Processing...Report [id=1003, sales=342197, qty=1200, staffName=staff 2]Jul 30, 2013 11:52:00 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 runINFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{
    }] and the following status: [COMPLETED]Exit Status : COMPLETEDDone