Clustering is one of the most vividly ongoing topics of computer fields both in commercial and academic parts in these days. The goal of this part is providing proper basic concepts of 'Clustering' and starting points to exploit it to a novice not familiar to this strange terminology. Though I tried to deliver up-to-date and correct information of this field, there might be some important aspects that I missed and I would like to assure this.
Clustering is a form of parallel processing and therefore it will be helpful to start from the definition of parallel processing. While we are usually using a computer having only one processor to do a job, Parallel processing takes more than one processors or computers acting like a machine to process a task in a network.
To implement using more than one processor could lead us to consider several ways.
One thing is to add more processors to a computer for example, SMP or NUMA. And second method could be taking
several computers (nodes) into a group to work together. Finally, we can imagine the combination of the first and second method,
that is, combing several workstations (nodes) built in more than one processor into a group to process a work.
The second and third method could be called clustering and most of commercial clusters in these days are implementing third method.
SMP (Symmetric Multiprocessing) and NUMA (Non-uniform Memory Access) are the case of adding several
processors to a computer and using shared-memory (tightly coupled). On the contrary, clustering uses distributed-memory (loosely coupled).
SMP is faster than others but have a limitation of expansion as the processors share memory and the I/O bus or data path.
That is, as more processors are added, the shared bus or data path get overloaded and becomes a performance bottleneck.
Therefore SMP does not usually exceed 16 processors.[Web 1]
NUMA could be thought as a cluster in a box. To overcome the disadvantage of SMP, it adds an intermediate level of memory
shared among a few microprocessors so that all data accesses don't have to travel on the main bus. It contains 16 to 256 processors.
[Web 2]
But adding more processors to a computer still have the same limitation of expansion, that is, you cannot continuously
add several hundred or thousands of processors to a computer.
The easier and more efficient solution could be adding a computer when you need more computing power as we can
do in clustering. (absolute and incremental scalability)
Moreover, you don't have to shutdown all the system to add new processors to this cluster and worry about the failure
of one node as each computer is a stand-alone computer.(high availability)
One more, by using commodity building blocks, it is possible to put together a cluster with equal or greater power than
a expensive single large machine, at much lower price. (superior price/performance) [STAL99]
These are the reason why clustering is highlighted as a revolutionary solution of high computing power.
If you work with a group of people as a leader, you have to divide the main task into several sub-tasks and assign them
to each of your group member. It will be not an easy job. You have to arrange the priority of the tasks, the load of each
person and even the schedule of group members as someone of your group might leave a vacation or suddenly have
to take a day off with a cold.
The same problems should be addressed in clustering. There should be no idle computer (load balancing) and each
computer could exchange data by the order of its priority. In addition to these, the failure or addition of a computer should
not disturb the performance of others (fault tolerance).
In case of human, we can arrange these things by meetings of group members, and computers in a cluster do the same by message-passing.
MPI has focus on this state, same architecture and same vendor.
Things can become more complicated if the task of your team is also one of a sub-task of the bigger which is being
processed all over the world. You might have to communicate with other groups or persons using different languages and
overcome the problem of communication if there is no person who can speak the languages of both parts.
Again, the same problem could happen in clustering, different architectures (interoperability) and PVM takes focus on this.
To solve these problems, most of the clusters in these days implement PVM (Parallel Virtual Machine) and MPI
(Message Passing Interface). Though PVM and MPI both make it possible for different processors to communicate with
each other acting as if they are on one machine, they have a lot of different features at the beginning of their approach.
While PVM was sprang from several researchers focusing the interconnection between heterogenous computers,
MPI was specified by a committee of about 40 high performance computing experts from research and industry in a series
of meetings in 1993-1994 (MPI-1), 1997 (MPI-2), to set up a standard message-passing specification focusing the performance
of their machine.
PVM have strong features in interoperability and fault tolerance at the price of degradation of performance and have a weak message-passing structure. On the contrary, MPI have strong features in message-passing to improve performance without the consideration of interoperability as they were not considered the interoperability with other vendor's computer. [PVMvsMPI]
PVM (Parallel Virtual Machine)In these days, most of commercial cluster vendors follow MPI and are improving their own MPI implementations.
Now clustering is almost possible on all the types of architectures and many commercial programs are available. But most of them are
still hardware dependent and actually not affordable for a person. However, there is a way you can set up a cluster relatively easily and at the
minimum cost for the purpose of studying. These kinds of clusters are possible to be built on Linux machines with several free-wares.
Beowulf provides useful informations of clustering and have news groups which might willingly give you valualbe
advices in the process of building and configuring a cluster.
Miscellany
We can consider different implementations of clustering according to the property of its application, computation intensive or communication intensive. In case of communication intensive, we have to take a focus on message-passing.
The most impressive property of Java language is code portability and in my opinion, it will accelerate the continuous innovation of clustering.
Where can we use this clustering (supercomputer?) in our actual life? Everyone dose not need complicated scientific computation or 3D films.
In the middle of this research, I found an interesting news in clustering of the recent.
'new technology derives new demands.'
The image of right side was derived from Window NT clusters-links. It shows one of the applicable example of clustering. These kinds of model can be used in where higher computing powers are needed, for example, in a database server environment which performs very complicated queries and therefore needs more computing powers than others.