Cluster Group Modules

Home Project Closed Site Index News Recent Old Download Documents User Docs Development Introduction to ANet High-Level Design Run-Time Wrapper Client Connection Modules Data Transformation Modules Client SDK Cluster Group Modules Cluster Filter Modules Core Modules Handshaking Protocol Modules Packet Protocol Modules Connection Protocol Modules Bandwidth Manager Module Document Type Definition (DTD) Low-Level Design Protocol Specifications Task List Development Roadmap Other Docs Development Mailing Lists Contacting Us

PROJECT CLOSED

by Benad

The Cluster Group Modules are used to duplicate, when needed, data between different clusters[1].

Table of Contents

• Design
  • Role of the Cluster Group Modules
  • Input/Output for the Cluster Group Modules
  • The ANet Internal Packets (AIP)
• Implementation Notes
• DTD

Design

Role of the Cluster Group Modules

Each module instance represents by itself a cluster group. When clients want to send data to the network, they have to send the data to one specific cluster group. Thus, one of the things that a Cluster Group Module has to do is that, given the packets coming from the clients, it has to send the incoming packets to the proper clusters.

Also, it is up to the Cluster Group Module to re-distribute the incoming packets from one cluster to another. This allows different clusters to exchange information.

Input/Output for the Cluster Group Modules

As input, the Cluster Groups need two things each time their "main" function is called. First, they need a list of cluster IDs. Those are unique numbers that identify all the clusters you have to receive or send data to. You can get additional information about the clusters from the parsed configuration file. The IDs are not part of the configuration file[1], but are generated as the file is parsed, and they are attributes as if they were written in the configuration file itself(1). Also, they need an "input buffer", which is a list of ANet Internal Packets (AIP). Each packet is identified with the cluster ID it came from.

As output, they produce a list of AIPs, each one identified with the cluster ID it has to go to.

The ANet Internal Packets (AIP)

Now it's a good time to introduce the most important data structure in the ANet deamon: the ANet Internal Packets, AIP in short.

An ANet Packet is very similar to packets in IP[2]: they represent an independent block of information that cannot be broken down to be useful. You can split a large block of data into several ANet packets, as long as they contain enough information to be able to merge them back later in your client.

The ANet Internal Packet is a representation of an actual External Packet (AEP), but for use within the deamon. The goal of this structure is to make it easy to transform and process. Thus, it follows byte-ordering and memory alignment of the compiled deamon. Also, the structure is much more flexible than the External one(2).

The AIPs are broken into two: the header and the data. The size of both the header and data are not fixed, but may have a limitation in size. By allowing the header to be of any size, we allow future versions of the ANet deamon to store additional information (at the end of the header) that can be safely ignored by previous versions of the modules.

So here is a list of what the header will, at minimum, contain:

• The total size of the packet.
• The total header size, which includes the space allocated for the value of the total size of the packet.
• The packet type, which can be query, static, packet for a TWDTC, and so on.
• The service number, which identifies what kind of data is contained in the packet (ftp, chat...).
• The origin or destination of the packet. It's meaning depends on the context. Here, with Cluster Group Modules, they represent cluster IDs.
• Specific information related to the packet type. For queries, you have a checksum. For static data, you have both the primary and secondary keys.

Functions in the run-time wrapper will be given to all the modules to simplify access to the data contained in the AIPs.

Implementation Notes

• Even though it is expected that most Cluster Group Modules will work a lot with the Bandwidth Manager, I don't see the point of trying to add special access functions to the Bandwidth Manager. I'll make sure that the Inter-Module Communication[1] is simple enough to work with.
• Re-creating a new output buffer each time to fit the size of the buffer you want for the current function call might be slow. You should try to reuse as much as possible the same Memory Tag[1], and resize it only when you find that it is not big enough.
• The run-time wrapper will assume that the data is not modified when it is duplicated. Otherwise, the modified data is considered a new data, which might cause a slight speed degradation in the ANet Core.

DTD

See the complete DTD for more information.

<!-- Cluster Group. Contains references to Clusters that compose
that Cluster Group. -->
<!ELEMENT ClusterGroup (ClusterRef)+>
<!ATTLIST ClusterGroup %name; %moduleName; %args; %security;>

Notes

(1) So, the IDs must not be saved back to disk. They are re-generated each time the deamon is launched, except when it is "resumed".

(2) What I mean is that, most of the time, doing a simple typecasting is enough to retrieve the data you want.

References

About the references...

Last update for this document: August 24, 2001, at 1:23:37 PST