Simple flat file to java objects converter using annotation.

With FF2J you can easily convert/transform a flat file in a collection of java objects. FF2J is an open source (Apache license) tiny (around 15KB) flat file to POJOs converter, with zero dependencies and a quick learning curve.

No XML, only Java.

• RegexEntity is an annotation that can be added on any POJO.
• RegexField is an annotation that can be added on any field of classes annotated with RegexEntity.
• Converter is an interface implemented by all converters, that convert a text (String) in a typed value.
• EntityHandler is an interface to be implemented for processing entities. For example you can write a DownloadHandler that writes all download objects in a database.

• FF2J ff2j (jar)
• FF2J Validation ff2j-validation (jar)
• FF2J Demo ff2j-demo (executable jar)

In your pom.xml you must define the dependencies to FF2J artifacts with:

<dependency>
    <groupId>ro.fortsoft.ff2j</groupId>
    <artifactId>ff2j</artifactId>
    <version>${ff2j.version}</version>
</dependency>    

where ${ff2j.version} is the last ff2j version.

You may want to check for the latest released version using Maven Search

You can convert a flat file's lines in java objects with a single line:

new FF2J()
	.map(Download.class)
	.addEntityHandler(new DownloadHandler())
	.parse(new InputStreamReader(input));

And now the story :)

Scenario: I have a DownloadServlet that for each download request writes in the container's log a line in the format

[webapp 2008/10/06 16:12:16] - 192.168.12.124, /download/next-reports-setup-1.7-jre.exe, f13dfc7fe609480297a0b15d611676b4

What I want is to see some statistics about downloads. For this reason I created a simple POJO class with the name Download having some properties.

@RegexEntity(pattern = "PATTERN")
public class Download {

	// [webapp 2008/10/06 16:12:16] - 192.168.12.124, /download/next-reports-setup-1.7-jre.exe, f13dfc7fe609480297a0b15d611676b4	
	public static final String PATTERN = "\\[webapp\\s"
		+ "(20[0-1][0-9]/\\d{2}/\\d{2}\\s\\d{2}:\\d{2}:\\d{2})" // date
		+ "\\]\\s-\\s<\\$>\\s"
		+ "(\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3})" // ip
		+ ",\\s/download/"
		+ "([^,]*)" // file 
		+ ".*";


	@RegexField(group = 1, converter = MyDateConverter.class)
	private Date date;
    
	@RegexField(group = 3)
	private String ip;

	@RegexField(group = 4)
	private String file;

	// getters and setters
	
}

I added annotation @RegexEntity(pattern = "PATTERN") to class Download. The @RegexEntity annotation may have a parameter named pattern which represents the constant name that returns the regex string. The default value is PATTERN (in Download class I added pattern value for transparency). This annotation is informing FF2J that all flat file's lines that respect the pattern will be transformed in Download objects.

The second step is to specify how FF2J will transform text fragments from flat file's lines in Download's properties. For this purpose I used @RegexField annotation. This annotation takes a mandatory parameter named group that represents the group index from the java's regex Matcher object and an optional parameter named converter that will be used by FF2J to convert the text fragment in property's value.

The third step is to create a DownloadHandler for handling Download objects.

public class DownloadHandler implements EntityHandler<Download> {

    private int count;

    @Override
    public void beforeFirstEntity() {
	    count = 0;
    }

    @Override
    public void handleEntity(Download entity) {
	    count++;
	    // only display the entity 
	    System.out.println(entity);		
    }

    @Override
    public void afterLastEntity() {
	    System.out.println("Handled " + count + " Download entities");
    }

}

The methods beforeFirstEntity() and afterLastEntity() are callback methods. FF2J will call these methods one time at the start/end of flat file parsing. For example you can start a database transaction, clear an entity table in beforeFirstEntity() and commit the database transaction in afterLastEntity().

In the example above my DownloadHandler init count variable to zero in beforeFirstEntity(), prints all download objects to System.out in handleEntity() and prints a count with download entities in afterLastEntity().

A Converter is used by FF2J to transform a text fragment into a POJO property's value.
FF2J comes with builtin converters for all primitive values ( Boolean, Byte, Short, ...).
FF2J also allows you to register new general converters (for each field's type) or you can register a converter only for a particular field.

public class MyDateConverter implements Converter<Date> {

	@Override
	public Date convert(String text) {
		try {
			return new SimpleDateFormat("yyyy/MM/dd").parse(text);
		} catch (ParseException e) {
			e.printStackTrace();
		}

		return null;
	}

}

For example MyDateConverter is used in Download class by FF2J to transform the text fragment in Date object. If you want to use MyDateConverter for all POJO fields with type Date you can do it with:

new FF2J()
    ...
    registerConverter(new MyDateConverter();

It's extremely simple to add validation support on the entity handler. For this purpose you can use ff2j-validation module.
FF2J Validation come with ValidEntityHandler class and two little dependencies: Bean Validation (JSR 303) and OVal. I choose OVal because it's lightweight (around 300K) and come with no dependencies.

In your pom.xml you must define the dependency to FF2J Validation artifact with:

<dependency>
    <groupId>ro.fortsoft.ff2j</groupId>
    <artifactId>ff2j-validation</artifactId>
    <version>${ff2j.version}</version>
</dependency>    

where ${ff2j.version} is the last ff2j version.

Validation is supported by constraints in the form of annotations placed on a field, method, or class of a POJO.

@RegexEntity(pattern = "PATTERN")
public class Download {

    ...

    @RegexField(group = 1, converter = MyDateConverter.class)
    @NotNull
    private Date date;

    @RegexField(group = 2)
    @NotNull
    private String ip;

    @RegexField(group = 3)
    @NotNull
    private String file;

    ...

}

In above snippet I want date, ip and file should be not null. For this reason I added @javax.validation.constraints.NotNull on these fields.

To validate entities you can use the ValidEntityHandler class.

public class DownloadHandler implements ValidEntityHandler<Download> {

	@Override
	public void handleValidEntity(Download entity) {
        // do something
    }

    @Override
    public void handleInvalidEntity(Download entity, List<ConstraintViolation> violations) {
        // do something
    }

}

If the entity is valid than method handleValidEntity() is called else method handleInvalidEntity() is called.

That it's all about validations :)

I have a tiny demo application that parse a log file produced by winstone (https://winstone.sourceforge.net/). The demo application is in demo package. To run the demo application use:

mvn
java -jar demo/target/demo-jar-with-dependencies.jar

Much of the conversation between developers and users is managed through [mailing list] (https://groups.google.com/group/ff2j).

Copyright 2013 Decebal Suiu

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.