Recently I read about a feature available in the .NET frametwork 2.0 or later that allows you to write transactional code blocks: that is, code blocks in which you can execute some actions or change the state of some objects and then, if something goes wrong, have the entire changes reverted to the initial state. The objects are taken to the state before the start of the transactional blocks and any other changes you performed are rolled back. The article is here.
Reading this got me thinking about how can I implement this in Java. I formulate the problem as it follows: “How transparent for the user can one implement transactional code blocks?”. Please note that I don’t say “Can you…” but “How transparent…”. I believe that allowing any amount of stress for the user, you can implement anything. The question is, how transparent and easy to use can one do it.
First, let me present some sample (pseudo) code. Let’s assume that I have two accounts, implemented with 2 atomic integers. What I want to do is transfer one amount of money from one accout to the other. The code is something like this:
public static void main( String[] args ) {
boolean someCondition = false;
AtomicInteger account1 = new AtomicInteger( 25000 );
AtomicInteger account2 = new AtomicInteger( 0 );
{ // transactional block
try {
System.out.println( "Before op: " + account1 + " | " + account2 );
account1.set( account1.get() - 5000 );
account2.set( account2.get() + 5000 );
System.out.println( "After op: " + account1 + " | " + account2 );
if ( someCondition ) {
throw new NullPointerException();
}
} catch ( Exception exception ) {
// rollback
}
} // END of transactional block
System.out.println( "Finish: " + account1 + " | " + account2 );
}
What I want is to have some helper classes so that I can mark the code block as transactional: if everything goes well, the changes performed in the block are permanent. If, however, something goes wrong (like throwing that NPE) then all the objects should be reverted to the initial state.
Therefore, I identify the following requirements to the problem:
- the state of all the objects should be reverted to the initial state, before entering the transactional block
- the commit/rollback should be as transparent as possible for the user
- the solution should work equally for objects as for primitive types
Reverting the state of an object can be accomplished in two ways. First, the class of that object can be made aware of the whole transactional process, so that it implements methods like COMMIT and ROLLBACK. This way, it is the job of the object to save it’s state and restore it later, if something went wrong. This, however, invalidates the rule no. 2, because if the user has to implement COMMIT/ROLLBACK in each of the classes involved in the operations, then this is not transparent at all.
The second way is to let the classes be implemented without the knowledge of code transactions, and provide the transactional support using some external helper classes. This is what we will see in this post. We will implement some classes that the user can involve in the process, and all the classes will be completely unaware of the transactions that are started, committed or rolled back.
Implement this second method presents one big problem. Java does not allow passing the parameters by reference. In Java you pass parameters by value. Each value that you pass is copied, and in each method you work on a copy of the value. This means that you CANNOT change, in a method, the value of a variable from outside that method. This makes the rollback of variables a hard-to-accomplish task (I think you can do it by JNI + a specially crafted DLL that allows the swapping of memory contents (perhaps using JNative or some similar library; for the time being, I will exclude the JNI approach).
With these requirements in mind, here is how I solved the problem. First, the client code using my helper classes:
public static void main( String[] args ) {
AtomicInteger account1 = new AtomicInteger( 25000 );
AtomicInteger account2 = new AtomicInteger( 0 );
Transactional trans = new Transactional();
trans.put( "account1", account1 );
trans.put( "account2", account2 );
trans.execute( new Transactional.Execution() {
protected void execute() throws Exception {
AtomicInteger account1 = get( "account1" );
AtomicInteger account2 = get( "account2" );
System.out.println( "Before op: " + account1 + " | " + account2 );
account1.set( account1.get() - 5000 );
account2.set( account2.get() + 5000 );
System.out.println( "After op: " + account1 + " | " + account2 );
throw new NullPointerException();
// rollback();
}
} );
account1 = trans.get( "account1" );
account2 = trans.get( "account2" );
System.out.println( "Finish: " + account1 + " | " + account2 );
}
The user must, first, create an instance of Transactional, place the objects that will participate in the transaction in the context, then start the transactional block. Inside, he/she must get the transactional objects and make any operations on them. Upon exiting the block, he/she must get the objects back from the context. If something goes wrong, the objects that the user gets in the end will be the original ones, with unchanged state. On the other hand, if all works well the user gets back the changed objects. The rollback is performed if some exception escapes from the block or the rollback() method is called explicitly.
If all works well, the code prints:
Before op: 25000 | 0 After op: 20000 | 5000 Finish: 20000 | 5000
If something wrong happens:
Before op: 25000 | 0 After op: 20000 | 5000 java.lang.NullPointerException ....... Finish: 25000 | 0
The helper classes are as follows:
public class Transactional {
private Map<String, Object> mappedObjects = new HashMap<String, Object>();
private Map<String, byte[]> serializedObjects = new HashMap<String, byte[]>();
private boolean rollback = false;
public void put( String name, Object object ) {
mappedObjects.put( name, object );
serializedObjects.put( name, serializeObject( object ) );
}
@SuppressWarnings( "unchecked" )
public <T> T get( String name ) {
return (T) mappedObjects.get( name );
}
public void rollback() {
rollback = true;
}
// --------------------------------------------------------------------------
public void execute( Execution exec ) {
try {
exec.transactionalParent = this;
exec.execute();
} catch ( Exception exception ) {
exception.printStackTrace();
rollback = true;
}
if ( rollback ) {
rollbackData();
}
}
private byte[] serializeObject( Object object ) {
ObjectOutputStream oos = null;
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
oos = new ObjectOutputStream( baos );
oos.writeObject( object );
return baos.toByteArray();
} catch ( Exception exception ) {
exception.printStackTrace();
return null;
} finally {
IOUtils.closeQuietly( oos );
}
}
private Object unserializeObject( byte[] content ) {
ObjectInputStream ois = null;
try {
ByteArrayInputStream bais = new ByteArrayInputStream( content );
ois = new ObjectInputStream( bais );
Object result = ois.readObject();
return result;
} catch ( Exception exception ) {
exception.printStackTrace();
return null;
} finally {
IOUtils.closeQuietly( ois );
}
}
private void rollbackData() {
Map<String, Object> originalMappedObjects = new HashMap<String, Object>();
for ( Entry<String, Object> mappedEntry : mappedObjects.entrySet() ) {
String key = mappedEntry.getKey();
Object originalObject = unserializeObject( serializedObjects.get( key ) );
originalMappedObjects.put( key, originalObject );
}
mappedObjects = originalMappedObjects;
}
// --------------------------------------------------------------------------
protected static abstract class Execution {
private Transactional transactionalParent;
protected abstract void execute() throws Exception;
protected <T> T get( String name ) {
return transactionalParent.get( name );
}
protected void rollback() {
transactionalParent.rollback();
}
}
}
As you can see, I am serializing all the objects upon entering the transactional blocks, store the initial state of them in a separate map, and let the block move on. If some exception is thrown or the user calls explicitly the rollback() method, then I restore the state of all the objects, and replace the map of objects with the restored one.
This solution is not so great, since the user must constantly get and push objects from/to the transactional context. However, the rollback is transparent to the code and also the objects are free from any transaction-oriented code.
Can you design something better? Perhaps using annotations and APT? Or somehow enhance the transactional objects and add the commit/rollback methods at runtime, on fly? Feel free to drop a comment on these issues.
Rogerio
October 13, 2010 at 8:43 pm
It is possible to get the following to work:
new Transaction(account1, account2)
{{
// transactional code here; commit or rollback issued at the end
}};
JMockit does something similar, but for the recording of expectations on mocked Java types. A ClassFileTransformer would modify the bytecode of Transaction subclasses as they are loaded, adding the necessary code to make copies of objects and to execute the final commit or rollback operation.
clojurebot
October 13, 2010 at 10:16 pm
Use Clojure! It has software transactions and it’s Java. What more could you want?
Peter Veentjer
October 13, 2010 at 11:42 pm
You need to have a look at STM.
Peter Veentjer
Multiverse: STM for Java.
http://multiverse.codehaus.org
bernhard
October 13, 2010 at 11:57 pm
Just some implementation hints:
1) Put if ( rollback ) { rollbackData(); } into a finally-block
2) add rollback=false after rollbackData(); otherwise if you once rollback you will rollback again when you invoke execute( Execution exec ) the next time
From a design perspective nice and clean code!
Greg Haines
October 14, 2010 at 12:16 am
I really liked your idea! I made some significant changes and wanted to share them back in case you or someone else finds them useful:
public class Transactional
{
private final List mappedObjects = new ArrayList();
private final List serializedObjects = new ArrayList();
private boolean rollback = false;
public Transactional(){}
@SuppressWarnings(“unchecked”)
public T get(final int pos)
{
return (T) this.mappedObjects.get(pos);
}
public boolean didRollback()
{
return this.rollback;
}
private void reset()
{
this.mappedObjects.clear();
this.serializedObjects.clear();
this.rollback = false;
}
private void finishedExecution()
{
this.serializedObjects.clear();
}
private void add(final Serializable object)
{
this.mappedObjects.add(object);
this.serializedObjects.add(serializeObject(object));
}
private void setRollback()
{
this.rollback = true;
}
private void rollbackData()
{
final List origMappedObjects =
new ArrayList(this.mappedObjects.size());
for (int i = 0; i < this.mappedObjects.size(); i++)
{
origMappedObjects.add(deserializeObject(this.serializedObjects.get(i)));
}
this.mappedObjects.clear();
this.mappedObjects.addAll(origMappedObjects);
}
public static void execute(final Transactional trans, final Execution exec,
final Serializable… transObjs)
throws Exception
{
trans.reset();
for (final Serializable transObj : transObjs)
{
trans.add(transObj);
}
try
{
exec.execute(trans.new StateImpl());
}
catch (Exception e)
{
trans.setRollback();
throw e;
}
finally
{
if (trans.didRollback())
{
trans.rollbackData();
}
trans.finishedExecution();
}
}
private static byte[] serializeObject(final Serializable object)
{
ObjectOutputStream oos = null;
try
{
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
oos = new ObjectOutputStream(baos);
oos.writeObject(object);
return baos.toByteArray();
}
catch (Exception e)
{
e.printStackTrace();
return null;
}
finally
{
closeQuietly(oos);
}
}
private static Serializable deserializeObject(final byte[] content)
{
ObjectInputStream ois = null;
try
{
ois = new ObjectInputStream(new ByteArrayInputStream(content));
return (Serializable) ois.readObject();
}
catch (Exception e)
{
e.printStackTrace();
return null;
}
finally
{
closeQuietly(ois);
}
}
private static void closeQuietly(final Closeable c)
{
try { c.close(); } catch (Exception e){}
}
public interface Execution
{
void execute(State state) throws Exception;
}
public interface State
{
T get(int pos);
void rollback();
}
private class StateImpl implements State
{
public T get(final int pos)
{
return Transactional.this.get(pos);
}
public void rollback()
{
Transactional.this.setRollback();
}
}
}
public final class TransactionalTest
{
public static void main(final String[] args)
{
final Transactional trans = new Transactional();
try
{
Transactional.execute(trans, new Execution(){
public void execute(final State state)
{
final AtomicInteger account1 = state.get(0);
final AtomicInteger account2 = state.get(1);
System.out.println(“Before op: ” + account1 + ” | ” + account2);
account1.set(account1.get() – 5000);
account2.set(account2.get() + 5000);
System.out.println(“After op: ” + account1 + ” | ” + account2);
throw new NullPointerException();
}
}, new AtomicInteger(25000), new AtomicInteger(0));
}
catch (Exception e)
{
e.printStackTrace();
}
System.out.println(“Finish: ” + trans.get(0) + ” | ” + trans.get(1));
}
private TransactionalTest(){}
}
Greg Haines
October 14, 2010 at 12:19 am
On second thought, here are pastebin links:
Transactional: http://pastebin.com/sBY6k1Mw
TransactionalTest: http://pastebin.com/iJu3UUHi
Boris
October 14, 2010 at 3:51 am
You might want to brush up on design patterns, as I recall there is something about this type of problem over there. I think it is called something like state pattern and it allows you do to precisely what you need to save a state of some object and to be able to commit or revert the actions.
And since it is one of the design patterns I believe that it is a good solution to this type of problem.
Bogdan Marian
June 26, 2011 at 1:27 pm
Actually, the design pattern needed to save object state at one moment in time is called memento (http://java.dzone.com/articles/design-patterns-memento).
Grohl
October 14, 2010 at 4:07 pm
I don’t understand why use instead of Object class
Grohl
October 14, 2010 at 4:08 pm
I don’t understand why use T ( generic type ) instead of Object class
Igor
October 14, 2010 at 4:42 pm
Try Deuce from here http://www.deucestm.org/
annotation based, no need to create Transaction object
Greg Haines
October 14, 2010 at 4:45 pm
@Boris – The state pattern isn’t really applicable here. The memento pattern describes a means of encapsulating edits and placing them on a stack which provides the means for “undo” functionality. Memento was originally designed for User Interfaces but might be able to be adapted to regular programming tasks.
However, that approach would place the onus on the coder to ensure that each edit is properly encapsulated and able to be unapplied at rollback. Bogdan started off the post asking “How transparently can we make transactional code?” meaning no major changes in the way we write it.
While I think Bogdan has done a great job in creating a simple API, it falls short, most especially in a multi-threaded environment. E.g:
Suppose that while our transaction is running, another identical transaction begins. Ours fails but the other succeeds. At the end of our transaction, our AtomicIntegers have the state that they did at the beginning and don’t reflect the changes made by the other transaction.
The only viable workaround is to serialize the transactions but that creates a giant bottleneck, which negates the whole point of multi-threading to begin with…
As Peter mentioned, STM attempts to solve this very difficult and tricky problem but, frankly, the technology is stil in its infancy. I believe that one day STM will become a very valuable tool in the software developer’s toolbox but, like any technology, is hardly a panacea for a intrinsically difficult problem.
ligerdave
October 14, 2010 at 5:03 pm
bravo! just an idea, you might wanna take a look of how jdbc transition works. it might give you some ideas of how to make this more extensible
Bogdan Mocanu
October 14, 2010 at 9:21 pm
Thank you all for your answers. They are very helpful and I learned quite a few new things.
First a small clarification: the code is very simple and indeed has a lot of weak points. I wrote it “on a napkin”
just to see how it works. Many improvements can be added. The main goal was just to see if there is any other approach to this problem.
@Rogerio: not sure how this will work. I will take a look and make some tests, but unless the user makes somethig to get the objects back from the transactional context, I really don’t see how the rollback is performed. But, as I said, I have to look at the code and usage of the Transaction class.
@clojurebot: thanks for pointing this out. I never used Clojure (though I heard and read about it). I will give it a try.
@Peter Veentjer, Igor: STM looks like the thing I need for this problem. I will try both frameworks (Multiverse and Deuce) and reply with some results. From what I saw in the examples, the impact on the code is really small (just some annotations). You just need to confgure the JVM agent for instrumentation. I have to read more in this direction.
@bernhard: thanks for the tips, indeed those fixes should be added to the code
@Greg Haines: thanks for the new version of the code. Regarding the behaviour of the code in the multithreaded env, the code is indeed problematic if you run 2 transactions on the same accounts. I think an improvement would be to deserialize the objects right after serializing them, therefore getting a clone of the original objects, and let the client work on the clones, not the real objects.
@Grohl: these are called parameterized methods. The usage of T frees you from having to do a downcast when getting the objects. If you implement the method like “Object get(String)”, then you have to do things like “Integer i = (Integer)get(“someString”)”. Using parameterized methods, Java learns from the context of the method call what is the returned type, and converts the return type of the method (at compile time). This provides a cleaner usage of the methods (see how easy is for the client to call the get method? no cast required).
@ligerdave: although similar in concepts, the JDBC transactions are pretty far from my problem. With the JDBC transactions you operate on a remote system (the DB server). On start/commit/rollback the JDBC drivers sends the appropriate SQL commands, and the server handles the transactions. However, what would be more appropriate is to study how Hibernate (or other ORM) handles transactions.
Thanks again for the tips. I hope I will find the time to update the code with the improvements and study the other directions for this problem.
Peter Veentjer
October 15, 2010 at 12:22 am
@Borgan
With the 0.5/0.6 release the impact is quite small, just a few annotations.
But I’m working on finishing up the 0.7 release (will be released in November) but instrumentation won’t be in their. Will be added in the 0.8 release. I have completely rewritten the STM so that the central clock is removed (most mvcc implementations suffer from this). And there is a lot of new stuff like isolation levels, propagation levels, pessimistic locking level. If you want to use the new 0.7 release, you need to work with managed reference (they look a lot like the AtomicReference/AtomicInteger classes from Java, with the exception that they also can participate in transactions.
E.g.
class Account{
final Ref lastUpdate = newRef();
final IntRef balance = newIntRef();
void atomicGetLastUpdate(){return lastUpdate.atomicGet();}
void atomicGetLastBalance(){return balance.atomicGet();}
static void transfer(final Account from, final Account to, final int amount){
execute(new AtomicVoidClosure(){
void execute(Transaction tx){
Date date = new Date();
if(amount<0){throw new IllegalArgumentException();}
from.balance.decrement(amount);
if(from.get()<0){throw new NotEnoughMoneyException();}
to.balance.increment(amount);
from.lastUpdate.set(date);
to.lastUpdate.set(date);
}
});
}
}
Atm I'm focussing mainly on integration in JVM based languages, and keeping the instrumentation in sync with the newest stm design, just slowed me too much down (have been working on the 0.7 release for almost 6 months since it was almost a complete rewrite).
Peter Veentjer
October 15, 2010 at 12:29 am
You can find the newest sources for the 0.7 release here.
http://git.codehaus.org/gitweb.cgi?p=multiverse.git;a=tree;h=refs/heads/multiverse-beta;hb=multiverse-beta
If you want I can send you a snapshot build.