Thursday, February 09, 2006
Extreme Java Programming
Haven't blogged about Java in a long time and figured I would share one aspect that isn't well discussed in the community that will help make your applications run faster...
Approximately two years ago, I interacted with a wonderful gentleman named Klaus Wuestefeld who enlightened me to the notion of prevalence layers which are hundreds of orders of magnitude faster than traditional databases for queries and just as fast for transaction processing.
The idea behind prevalence is adherence to the following principles:
Most folks only consider temporary storage of objects to relational databases which meets the use what you got principle but may be robbing themselves of increased performance afforded by other approaches. While I am a big fan of Hibernate, there are simply sometimes better ways of solving the same problem.
In a Prevalence system, the data doesn't really need to be "in memory" it only needs to appear as if it is in memory. Obviously, with a sufficiently wide address space (say 64 bits) and a virtual paging system to manage moving storage between disk an RAM, Prevalence will work for larger data sets. A key consideration is that as the volume of data scales, standard paging algorithms won't necessarily scale as well as a well-designed system that admits to having two types of storage - high volume slower storage (disk) and lower volume fast storage (RAM).
Anyway, I hope you will check out these benchmarks...
| | View blog reactionsApproximately two years ago, I interacted with a wonderful gentleman named Klaus Wuestefeld who enlightened me to the notion of prevalence layers which are hundreds of orders of magnitude faster than traditional databases for queries and just as fast for transaction processing.
The idea behind prevalence is adherence to the following principles:
- Keep all objects in memory all the time, and access them as in-memory objects.
- Every operation that affects the "persistent" state of an object must be implemented as a serializable command object (think command pattern). When a command is executed, its serializable form is written to a log file. When the system is restarted, the prevalence system re-executes all these serialized commands, thus restoring the states of all the in-memory objects.
- You can cause a snapshot of current object states to be stored, so that when the system is restarted, you load the snapshot and then re-execute the commands that have been logged since the snapshot was created.
- Using command pattern can sometimes restrict the architecture, so it can be also implemented transparently if you have considered approaches such as Aspect Oriented Programming (will blog on this next week).
Most folks only consider temporary storage of objects to relational databases which meets the use what you got principle but may be robbing themselves of increased performance afforded by other approaches. While I am a big fan of Hibernate, there are simply sometimes better ways of solving the same problem.
In a Prevalence system, the data doesn't really need to be "in memory" it only needs to appear as if it is in memory. Obviously, with a sufficiently wide address space (say 64 bits) and a virtual paging system to manage moving storage between disk an RAM, Prevalence will work for larger data sets. A key consideration is that as the volume of data scales, standard paging algorithms won't necessarily scale as well as a well-designed system that admits to having two types of storage - high volume slower storage (disk) and lower volume fast storage (RAM).
Anyway, I hope you will check out these benchmarks...
