🚀 go-pugleaf

TI: Give your DB a break
AU: Dion Almaer
SO: theserverside.com
    October 2003
Introduction
How many projects have you worked on where the bottleneck in testing was the
 database, for both performance and scalability? In my experience, getting t
o the data, creating new data, and updating data takes a fair amount of effo
rt.
If you are developing a small application, then you may not be worrying abou
t scalability as much. However even small applications like to run fast! Whe
n you get to larger enterprise applications, then the concern grows signific
antly. As you add more and more concurrent users your poor database gets mor
e and more bogged down.
This article discusses how caching data in front of the database can give ou
r DB a break, and allow for faster running, and more available applications.

In particular, We will look at:
    * What is clustering?
    * Using a distributed cache
    * Caching strategies
    * Read-through / Write-behind caching
    * Technologies that integrate nicely into this architecture
What is clustering?
Clustering is one of those words used in computer science to mean many thing
s. It is over used like other words in our industry such as "Sessions" :-). 
I am going to refer to clustering in the sense that "you can add more instan
ces to your cluster and the application scales in some fashion."
Clustering Issues
There are many issues with clustering. Have you ever had a manager say somet
hing like:
    "If we buy another box, our app will run twice as fast right?"
This is of course rarely the case. We can obtain an increase in performance,
 but only if we design our application to cluster for performance.
How about this gem:
    "If we buy another box, our app will be twice as reliable won��t it?"
Again, this isn��t always the case. Higher availability will only occur if y
ou design your application to cluster for availability.
Scaling Application Performance
In a distributed system, we can add more boxes running our application to th
e backend. Our application will only scale for performance, if by adding the
se boxes you enable the entire system to do more work concurrently, accommod
ate more users, etc.
It is fairly easy to scale for performance if your application lives in its 
own world and doesn��t share anything. A key to understanding the scalabilit
y of an application for performance is to characterize the application��s re
quirements to share objects. If the shared object is read-only, then it can 
make copies, which allows us to scale. We get into problems when shared obje
cts need to get updated. Depending on the policy for updates, we can shoot o
urselves in the foot here. If updates are fairly infrequent, then the work o
f making the updates allows us to scale nicely. However, if we are updating 
very frequently, then the work of pushing out updates can actually SLOW down
 the application. At this point it is better to have one point of contention
 which is managed, than to have copies that we try to keep in sync. This is 
the game that we play. How often do we need to keep our copies in sync? How 
strict do we get (do we need 100% coherence? Or can we get away with a lag).
 We need automated testing to make sure we are doing the Right Thing�6�4.
Scaling Application Availability
We have similar considerations with applications that scale for availability
. Application availability is limited to the availability of the system. We 
can get a "better box" (add more redundancies etc), but can only go so far. 
When we deal with distributed architectures, adding more boxes does not nece
ssarily increase application availability. It all depends on what services a
re shared. The worst case is that all of the machines need to be available f
or the application to run. For example, you have one web server, one applica
tion server, and one database. If one of the machines dies, you are in troub
le. In this scenario we are LESS available than if the entire application wa
s on one box.
So, just like with performance, life comes down to sharing objects. If we bu
ild nice stateless architectures where we can easily add machines to the sys
tem, then we can get extra availability.
Let��s see how a distributed cache can help.
Cache is King: Distributed Caching
TheServerSide started off its life running on a nice little Linux box. We ha
d one application server, and the database was on the same machine.
We were tasked with having TheServerSide run on a cluster of different appli
cation servers. Ok, so we get machines for each application server, one for 
a database machine, and it should run a lot faster now right? We have X mach
ines instead of one! Wrong. When we first took the app and ran it in this co
nfiguration it was deathly slow. The reason was quite simple. With the singl
e application server we were able to turn on a flag (db-is-shared in WebLogi
c) that told WebLogic that he was in charge of things. The WebLogic instance
 could assume that all access to the database would be through the applicati
on, and hence it could be cached between transactions. After the site had be
en running for awhile, the entity bean cache would basically be containing t
he data required for the system. The database would only be contacted when u
pdates/creates/deletes happened. This was nice and zippy. Unfortunately it i
s fairly rare to be in this privileged situation with applications in the re
al world.
The db-is-shared magical flag has to be turned off if you fall under any of 
the conditions below:
    * You want to run more than one instance of the application
          o E.g. using WebLogic clustering
    * Other programs access the database behind the back of the application
          o Outside processes
          o DBAs sneaking around changing things
So, it is a rare day where you can say that your application is the sole acc
ess point to the database, but take advantage of that situation if you are l
ucky enough to be in it! Use your appserver caching facilities!
Without this flag, our various application servers were always guessing abou
t the data that lay behind them. "Hmm, I don��t know if this entity has been
 changed, so I better go to the database and reload it just in case." All of
 the ejbLoad/ejbStore traffic that was taking place for no reason brought th
e application to a crawl. We now had to take the application that was NOT bu
ilt for scalability of performance, and change it.
Read-Heavy Applications
A lot of applications are read-heavy, especially web-based applications. The
re are many more concurrent users reading our news, forums, patterns, articl
es, etc, compared to those posting information. This fact allows us to desig
n our application to focus on fast reads.
We looked at different solutions, and decided on using a distributed cache t
hat would sit in front of the database. This cache would contain a subset of
 information that the database contains, and would enable the application to
 retrieve the data VERY fast.
JCache - JSR 107
One of the oldest JSRs around is the JCache API (JSR 107). Oracle originally
 started the effort (and is spec lead) and donated ideas from their implemen
tations (like BC4J). However, a long time has passed, and nothing has come o
ut of the expert group to date. A few rumors suggest that JCache is NOT dead
, and will most likely be based on wrapping around the java.util.Map interfa
ce that all Java programmers know and love.
Tangosol Coherence
The implementation of a distributed cache that we chose was Tangosol Coheren
ce. The Coherence product has been used in many large projects, and prides i
tself on the fact that it is FAST, and NEVER loses data. Coherence appears i
n the programming world as a java.util.Map that you can work with like any o
ther Map that you are used to working with. It just so happens that behind t
he scenes the caching architecture is making sure that the data is in the ri
ght place. So if you put something into cache X, you can retrieve it on anot
her machine with the same key, as though it was in memory there.
We strapped Coherence into our Entity Bean layer. Now, when we do an ejbLoad
() we first look into the cache to see if it is already loaded. If it is, we
 pass the reference back; if not, then we load it into the cache from the da
tabase. What do we put into the cache? The keys are the Primary Key classes 
for the given Entity beans, and the values are the Data Transfer Objects (DT
Os, or Value Objects) themselves. If an entity is deleted or updated, then i
t is nuked out of the cache, to be reloaded in another transaction (in ejbRe
move()/ejbStore()).
Here are some snippets from a BMP Entity bean that represents a Thread in Th
eServerSide. A couple of EJB methods are shown, with the code for interfacin
g with the cache.
 -=code here=-
  Public class ThreadBean extends ThreadDetailsBean implements EntityBean{
  //variables
  //EJB methonds
     public ThreadPK ejbFindByPrimaryKey(ThreadPK key) throws FinderExceptio
n, ObjectNotFoundException {
       //--Cache:Look in the cache for the PK
      if (CacheManager.getCache(THREAD_ENTITY_CACHE).get(key)!=null) {
            return key;
        }
        //--Go to the DB at this point as it isn't in the cache
        ... do generic DB work to SELECT the pk ...
      }
     public void ejbLoad() {
              //Query the Entity Context th get the current Primary Key, so 
we know which instance to load.
          ThreadPK ctxPK = (ThreadPK) ctx.getPrimaryKey();
          long pk = ctxPK.threadPK;
          this.threadPK=pk;
          // -- Cache: lookup the view object based on the PK ThreadDetailsB
ean threadDetails = (threadDetailsBean) CacheManager.getCache(THREAD_ENTITY_
CACHE).get(ctxPK):
          if (threadDetails!= null) {//got via cache
                     this.forunPK = threadDetails.getForumPK();
                     this.messagePK = threadDetaiks.getMessagePK();
                     this.creationDate =  threadDetails.getCreationDate();
                     ... set other attributes ...
             } else {
                   //-- not in cache, look in the db
                     ... do generic DB work to SELECT the elements ...
CacheManager.getCache(THREAD_ENTITY_CACHE).put(ctxPK,theThread);
    ... more methods ..
}
-= codes end =-
-= url:http://www.theserverside.com/resources/articles/DB_Break/CodeSample.j
pg =-
-=the maybe some mistake when I typed them =-
This work took a few days to do, and means that once again, when TheServerSi
de application is up and running, it basically contains most of the data wit
hin this distributed cache. Reading a DTO out of the cache is immediate, and
 this is how TSS was able to actually run faster in the cluster, as it scale
s for performance.
Does it scale for availability? Yes. We can startup/bring down machines with
in the cluster and no one is the wiser. As long as one application server is
 up and running, the site can work fine.
Distributed Caching Strategies
When you configure your caching architecture, there are many things to think
 about. It isn��t necessarily as easy as using the default caching that your
 product will give you. Here are some options:
Replicated Cache
Imagine having two JVMs running an application with a joint cache. If the ap
plication in JVM 1 creates a new object and places it in the cache, how will
 JVM 2 get access to that information? A replicated cache keeps a copy of th
e entire cache contents on each cache node. If memory isn��t an issue, and t
here are only a few nodes, this can be the best option. Read requests are al
ways very fast, as they are going to be similar to reading from any object i
n memory (as the data is already there in the cache). If you have a huge set
 of nodes then you may want to look to another strategy, as each put into th
e cache will cause many events kicking in, and data going to the entire set 
of nodes. Thus, the more nodes you add to the cluster, the less each will po
tentially perform.
http://www.theserverside.com/resources/articles/DB_Break/replicatedcache-dia
gram.gif
Figure 1. Shows that when an object is placed in a replicated cache, each no
de gets a copy
Partitioned Cache
To contrast the replicated cache, a partitioned cache will not have to conta
in the entire contents of a given cache. Typically there will be a way to pa
rtition the data in a cluster, so each node will share the burden. What do y
ou get with this kind of cache?
    * Load Balancing: Since the data processing is shared among the cluster 
nodes, you have some kind of automated load balancing of this work.
    * Location Transparency: Although internally data may be living on vario
us nodes in the cluster, the program has no idea about this abstraction. Dev
elopers just get objects from the cache, and the implementation works out wh
ere to get the data from.
    * Failover: Some partitioned caches will allow for some kind of failover
. This often means that a given node has a "backup" member which contains a 
copy of the data. If the primary node goes away, the backup steps up to the 
plate (and itself chooses a new backup... and the world goes on).
Since the burden is shared among the nodes, this type of cache can scale mor
e linearly than the replicated cache will be able to.
http://www.theserverside.com/resources/articles/DB_Break/distributedcache-di
agram.gif
Figure 2. Shows that when an object is placed in a distributed cache, each n
ode shares the copies
Optimistic Cache
A lot of work can go into making sure that a given cache is coherent. Some u
se cases don��t call for this guarantee. Often "most recently known" values 
are just as good as "this is definitely the right current value". For exampl
e, when we look at stock quotes online, we normally get data that is lagged 
behind the exact current value. Brokers themselves, however, need more up-to
-date stats. Optimistic caches are normally more loose with their data, with
 increased performance.
Reading through, writing behind
The caches that we have seen have the onus of the developer for putting data
 in, and extracting it out of the cache in their application code. For examp
le, our entity beans would read from the cache in ejbLoad() to see if the DT
O is in memory. It also loads up the cache from data in the database. Wouldn
��t it be nice if the cache could maybe take care of this for you? Often the
y can.
You can often configure a cache to load data from a store (db, web service, 
LDAP, legacy) on a cache miss. This is called reading through.
When an object is placed into the cache, it can be updated/created in the gi
ven backend store. This is called writing-through or writing-behind. Often y
ou can tune the interval frequency of writing the data back to the store.
We used a read-through/write-behind cache for a recent feature coming up on 
TheServerSide. We didn��t used to track the number of views that a given thr
ead received on our community. Reading a thread is a read-only activity, and
 we didn��t want to impose any delay in that use case for the user. Our solu
tion was to create a cache that contained DTOs which tracked the number of v
iews on a given thread. When a thread is viewed by a user, the cache object 
is incremented, and flow moves on. At a given frequency, the cache itself wi
ll call out to a cache hook which will update the count in the database. The
 larger the frequency the less frequent the DB access. This allows you to tw
eak the DB contention from one nice location.
An important note about this solution is that the developer that uses this c
ache is simply running gets and puts on a Map interface. The cache developer
 has the burden of writing "what do do" on reading and writing to the databa
se.
Cache Miss / DB Load
On a cache miss something like the following would be called:
-= codes here =-
 public Object load(Object key) {
    long threadPK = ((ThreadPK)key).threadPK;
    ThreadViewCountDetailsBen value = null;
    Connection conn - null;
    PreparedStatement pstmt = null;
    ResultSet rs = null;
    try {
        conn= DBUtil.getConnection()
        pstmt= conn.prepareStatement(SQL_LOAD_CELECT);
        pstmt.setLong(1,threadPK);
        rs= pstmt.executeQuery();
        if (rs.next()) {
            value = new ThreadViewCountDetailsBean(threadPK,rs.getLong("View
count")):
      }
}catch(SQLException e) {
//Handle exception
}finally {
//Realease the DB Connection, Statemnet, and ResultSet
}
 return value;
}
-= codes end =-
-=url:http://www.theserverside.com/resources/articles/DB_Break/CodeSample2.j
pg =-
The key that the cache was passed on the cache-miss gets passed down the cha
in, and this code goes to a backend database to retrieve the value. The valu
e returned from this method will then be put into the cache, and back to the
 application code that called cache.get( pk ).
Cache Write / DB Insert or Update
When a developer puts an object into the cache via cache.put( key, value), t
he following may be called at some point (depending on the write settings):
-= codes here =-
public void store(Object key, Object value) {
    long threadPk = (( ThreadPK)key).threadPK;
    TreadViewCountDetailsBean threadViewCount = (ThreadViewCountDetailBean) 
value;
    Connection conn = null;
    PreparedStatement pstmt = null;
    String sql =null;
    try {
    //CRUDE :MAY use a  DB proprietary MERGE command to get this effect
    ThreadViewCountDetailsBean loadViewCount = (ThreadViewCountDetailsBean) 
load (key);
    if ( laodViewCount == null) {
              sql = SQL_STORE_INSERT;
    } else {
              sql =SQL_STORE_UPDAE;
    }
     conn= DNUtil.getConnection();
     pstmt= conn.prepareStatement(sql);
     pstmt.setLong(1,threadViewCount.getViewCount());
     pstmt.setLong(2,threadPK);
     pstmt.executeUpdate();
    } catch)Eception e){
           //Hadle exception
    }finally {
          // Realease the DB Connection and Statement
     }
}
-= codes end =-
-=url:http://www.theserverside.com/resources/articles/DB_Break/CodeSample3.j
pg =-
This kind of setup gets you rolling in a "Cache is King" mentality. The cach
e becomes a lot smarter than just a data container, and you end up going to 
the cache itself, and don��t even think about hitting the database directly.
 This allows you to abstract the final persistence strategy completely.
Crude to elegant: Isn��t it nice when technologies mix?
We have looked at some examples of using distributed caches, different confi
gurations for different situations, and how we used distributed caching on T
heServerSide. The cache itself is really nice, but what is even more compell
ing is how it can be integrated with other technologies such as:
JDO
Java Data Objects (JDO) has a cache manager built right into the spec. When 
you combine the power of JDO (transparent persistence), and distributed cach
ing, you get a very powerful data tier. Now our applications can have transp
arent persistence which is high performing, even on a large distributed scal
e. I would definitely be compelled to choose JDO with cache for large enterp
rise projects. Some JDO vendors come with some notion of a distributed cache
, and others integrate with cache products, such as SolarMetric Kodo + Tango
sol Coherence.
JMS
Messaging is such a useful concept in computing, especially in the enterpris
e. In many situations we need to make sure that messaging is guaranteed, and
 often we turn on database storage. A nice combination can be having the mes
sage broker storing messages into the distributed cache itself (which can it
self be backed by a database).
Although not its primary use, the distributed cache itself can be used as a 
micro-messaging system. When objects (messages) are placed on the queue (in 
the cache) the consumer (the write-behind listener) can take that message an
d do whatever it needs to with it. While in most circumstances a "real" JMS 
provider would be ideal, this system can definitely work!
JNDI
You want clustered JNDI? Your application server doesn��t give it to you out
 of the box? Share the JNDI tree in a cluster!
We almost think of the cluster as a low level service that is available as a
n abstraction to the database itself. When you get using the cluster, and st
art to enjoy the speed and reliability, it is amazing how often you think ab
out using it... in many use cases.
Conclusion
On many projects the poor old database doesn��t get given a break. Managing 
the data tier isn��t an easy task, even though there are many potential solu
tions out there. If possible, I try to stay away from having to cluster the 
database machines themselves, although in some situations you find that you 
have too. In these cases I try to divide out the tasks to different database
s (e.g. a backend DB for nightly processing, a read only cluster, etc).
Many technologies give you some kind of caching out of the box, such as EJB 
CMP 2.x, and JDO. When these technologies are married to distributed cluster
s you can tweak the data tier to give you the optimal scalability for both p
erformance, and availability, as needed in your use cases.
Hopefully the JCache API specification will surprise us and get into the fas
t track, and then we will be able to write to a vendor neutral specification
 for our distributed layer. This doesn��t mean that different caching vendor
s will be "plug ��n play", but at least the code will stay pretty much the s
ame, and most of the porting time would be in our lovely XML files. :-)
About the Author
Dion Almaer is the Chief Architect of TheServerSide.com J2EE Community and f
ormerly a mentor and trainer at J2EE experts The Middleware Company. Dion is
 a columnist on Enterprise Java topics at onjava.com and theserverside.com.


--
��ʱû��ǩ��
�� ��Դ:���ɽ��ˮ bbs.neu.edu.cn��[FROM: 219.216.70.3]