This tutorial will show you how we can insert a large dataset into a database at once using Spring JdbcTemplate. For this tutorial we will create a standalone maven project in Eclipse.

If you already have an idea on how to create a maven project in Eclipse will be great otherwise I will tell you here how to create a maven project in Eclipse.

Sometimes we need to insert or update large number of records in the database. It’s not a good idea to insert multiple records into database one by one in a traditional approach. It will hit the application’s performance. Spring provides batch operations with the help of JdbcTemplate, it inserts or updates records into database in one shot.

Prerequisites

The following configurations are required in order to run the application

Eclipse Kepler
JDK 1.8
Have maven installed and configured
Spring dependencies in pom.xml

Now we will see the below steps how to create a maven based spring project in Eclipse

Step 1. Create a standalone maven project in Eclipse

Go to File -> New -> Other. On popup window under Maven select Maven Project. Then click on Next. Select the workspace location – either default or browse the location. Click on Next. Now in next window select the row as highlighted from the below list of archtypes and click on Next button.

maven-arctype-quickstart

Now enter the required fields (Group Id, Artifact Id) as shown below

Group Id : com.roytuts
Artifact Id : spring-batch-insert

Step 2. Modify the pom.xml file as shown below.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.roytuts</groupId>
	<artifactId>spring-batch-insert</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>spring-batch-insert</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<jdk.version>1.8</jdk.version>
		<junit.version>4.11</junit.version>
		<mysql.version>5.1.27</mysql.version>
		<org.springframework.version>4.1.5.RELEASE</org.springframework.version>
	</properties>

	<dependencies>
		<!-- Spring framework -->
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jdbc</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<!-- mysql driver -->
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<version>${mysql.version}</version>
		</dependency>

		<!-- junit -->
		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>${junit.version}</version>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>${jdk.version}</source>
					<target>${jdk.version}</target>
				</configuration>
			</plugin>
		</plugins>
	</build>
</project>

Step 3. If you see JRE System Library[J2SE-1.4] then change the version by below process

Do right-click on the project and go to Build -> Configure build path, under Libraries tab click on JRE System Library[J2SE-1.4], click on Edit button and select the appropriate jdk 1.8 from the next window. Click on Finish then Ok.

Step 4. Create src/main/resources folder for putting the resource files.

Do right-click on the project and go New -> Source Folder. Give Folder name: as src/main/resources and click on Finish button.

Step 5. Create an XML file under src/main/resources.

Do right-click on src/main/resources in the project and go New -> file. Give File name: as spring-batch-insert-properties.xml and click on Finish button.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx"
	xmlns:task="http://www.springframework.org/schema/task"
	xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.1.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-4.1.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.1.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-4.1.xsd">

	<bean id="jdbcProperties"
		class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"
		name="configProperties">
		<property name="order" value="99999" />
		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
		<property name="ignoreUnresolvablePlaceholders" value="true" />
		<property name="properties">
			<value>
				
				JDBC.DRIVER=com.mysql.jdbc.Driver
				JDBC.URL=jdbc:mysql://localhost:3306/roytuts
				JDBC.USERNAME=root
				JDBC.PASSWORD=root

			</value>
		</property>
	</bean>

</beans>

Step 6. Create spring-batch-insert-datasource.xml file under src/main/resources with the below content.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:jee="http://www.springframework.org/schema/jee" xmlns:tx="http://www.springframework.org/schema/tx"
	xmlns:task="http://www.springframework.org/schema/task"
	xsi:schemaLocation="http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.1.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-4.1.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.1.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task-4.1.xsd">

	<context:annotation-config></context:annotation-config>
	<import	resource="spring-batch-insert-properties.xml" />

	<bean id="dataSource"
		class="org.springframework.jdbc.datasource.DriverManagerDataSource">
		<property name="driverClassName" value="${JDBC.DRIVER}" />
		<property name="url" value="${JDBC.URL}" />
		<property name="username" value="${JDBC.USERNAME}" />
		<property name="password" value="${JDBC.PASSWORD}" />
	</bean>

	<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
		<property name="dataSource" ref="dataSource" />
	</bean>
	
	<bean id="studentDao" class="com.roytuts.spring.batch.insert.dao.impl.StudentDAOImpl" />

</beans>

Step 7. Create a MySQL table

CREATE TABLE `student` (
  `student_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `student_name` varchar(30) NOT NULL,
  `student_dob` varchar(10) NOT NULL,
  `student_email` varchar(80) NOT NULL,
  `student_address` varchar(250) NOT NULL,
  PRIMARY KEY (`student_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Step 8. Create a POJO class for student.

package com.roytuts.spring.batch.insert.dto;

public class Student {

	private Integer studentId;
	private String studentName;
	private String studentDob;
	private String studentEmail;
	private String studentAddress;

	public Student() {

	}

	public Student(Integer studentId, String studentName, String studentDob,
			String studentEmail, String studentAddress) {
		this.studentId = studentId;
		this.studentName = studentName;
		this.studentDob = studentDob;
		this.studentEmail = studentEmail;
		this.studentAddress = studentAddress;
	}

	public Integer getStudentId() {
		return studentId;
	}

	public void setStudentId(Integer studentId) {
		this.studentId = studentId;
	}

	public String getStudentName() {
		return studentName;
	}

	public void setStudentName(String studentName) {
		this.studentName = studentName;
	}

	public String getStudentDob() {
		return studentDob;
	}

	public void setStudentDob(String studentDob) {
		this.studentDob = studentDob;
	}

	public String getStudentEmail() {
		return studentEmail;
	}

	public void setStudentEmail(String studentEmail) {
		this.studentEmail = studentEmail;
	}

	public String getStudentAddress() {
		return studentAddress;
	}

	public void setStudentAddress(String studentAddress) {
		this.studentAddress = studentAddress;
	}

}

Step 9. Create a DAO in com.roytuts.spring.batch.insert.dao package

package com.roytuts.spring.batch.insert.dao;

import java.util.List;

import com.roytuts.spring.batch.insert.dto.Student;

public interface StudentDAO {

	void insert(List<Student> students);

}

Note : if you get error then you can change the Compiler compliance level, Generated .class files compatibility, Source compatibility to 1.7

Step 10. Now create a DAO implementation class in com.roytuts.spring.batch.insert.dao.impl package.

package com.roytuts.spring.batch.insert.dao.impl;

import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.BatchPreparedStatementSetter;
import org.springframework.jdbc.core.JdbcTemplate;

import com.roytuts.spring.batch.insert.dao.StudentDAO;
import com.roytuts.spring.batch.insert.dto.Student;

public class StudentDAOImpl implements StudentDAO {

	@Autowired
	private JdbcTemplate jdbcTemplate;

	private final int INSERT_BATCH_SIZE = 2;
	private final String SQL_STUDENT_INSERT = "INSERT INTO student(student_name,student_dob,student_email,student_address) values(?,?,?,?)";

	@Override
	public void insert(List<Student> students) {
		for (int i = 0; i < students.size(); i += INSERT_BATCH_SIZE) {

			final List<Student> batchList = students.subList(i, i
					+ INSERT_BATCH_SIZE > students.size() ? students.size() : i
					+ INSERT_BATCH_SIZE);

			jdbcTemplate.batchUpdate(SQL_STUDENT_INSERT,
					new BatchPreparedStatementSetter() {
						@Override
						public void setValues(PreparedStatement pStmt, int j)
								throws SQLException {
							Student student = batchList.get(j);
							pStmt.setString(1, student.getStudentName());
							pStmt.setString(2, student.getStudentDob());
							pStmt.setString(3, student.getStudentEmail());
							pStmt.setString(4, student.getStudentAddress());
						}

						@Override
						public int getBatchSize() {
							return batchList.size();
						}
					});
		}
	}

}

Step 11. Create JUnit test case under src/test/java folder.

package com.roytuts.spring.batch.insert.dao.impl;

import java.util.ArrayList;
import java.util.List;

import org.junit.Before;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

import com.roytuts.spring.batch.insert.dto.Student;

public class StudentDAOImplTest {

	StudentDAOImpl studentDAOImpl;
	List<Student> students;

	@Before
	public void setUp() {
		ApplicationContext applicationContext = new ClassPathXmlApplicationContext(
				"spring-batch-insert.xml");
		studentDAOImpl = (StudentDAOImpl) applicationContext
				.getBean("studentDao");
		students = new ArrayList<Student>();
		populateStudents();
	}

	@Test
	public void insertTest() {
		studentDAOImpl.insert(students);
	}

	private void populateStudents() {
		Student student = new Student();
		student.setStudentName("Sumit");
		student.setStudentDob("01-01-1980");
		student.setStudentEmail("sumit@email.com");
		student.setStudentAddress("Garifa");

		students.add(student);

		student = new Student();
		student.setStudentName("Gourab");
		student.setStudentDob("01-01-1982");
		student.setStudentEmail("gourab@email.com");
		student.setStudentAddress("Garia");

		students.add(student);

		student = new Student();
		student.setStudentName("Debina");
		student.setStudentDob("01-01-1982");
		student.setStudentEmail("debina@email.com");
		student.setStudentAddress("Salt Lake");

		students.add(student);

		student = new Student();
		student.setStudentName("Souvik");
		student.setStudentDob("01-01-1992");
		student.setStudentEmail("souvik@email.com");
		student.setStudentAddress("Alipore");

		students.add(student);

		student = new Student();
		student.setStudentName("Liton");
		student.setStudentDob("01-01-1990");
		student.setStudentEmail("liton@email.com");
		student.setStudentAddress("Salt Lake");

		students.add(student);
	}

}

Step 12. Run the above JUnit test case you will get the below output in the console.

Apr 03, 2015 10:32:28 PM org.springframework.context.support.ClassPathXmlApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1c5e7b4: startup date [Fri Apr 03 22:32:28 IST 2015]; root of context hierarchy
Apr 03, 2015 10:32:28 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring-batch-insert.xml]
Apr 03, 2015 10:32:28 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring-batch-insert-properties.xml]
Apr 03, 2015 10:32:28 PM org.springframework.jdbc.datasource.DriverManagerDataSource setDriverClassName
INFO: Loaded JDBC driver: com.mysql.jdbc.Driver

Step 13. Now Look at the database table. You will see that all values are inserted.

Batch Insert using Spring JdbcTemplate

That’s all. Thank you for reading.

Tags:

I am a professional Web developer, Enterprise Application developer, Software Engineer and Blogger. Connect me on Roy Tutorials | TwitterFacebook Google PlusLinkedin | Reddit

Leave a Reply

Your email address will not be published. Required fields are marked *