Core Modules

Home/Documents/Development/High-Level Design/Core Modules
 Project Closed
 Site Index
 User Docs
 Introduction to ANet
 High-Level Design
 Run-Time Wrapper
 Client Connection Modules
 Data Transformation Modules
 Client SDK
 Cluster Group Modules
 Cluster Filter Modules
 Core Modules
 Handshaking Protocol Modules
 Packet Protocol Modules
 Connection Protocol Modules
 Bandwidth Manager Module
 Document Type Definition (DTD)
 Low-Level Design
 Protocol Specifications
 Task List
 Development Roadmap
 Other Docs
 Mailing Lists
 Contacting Us


by Benad

The Core Modules are doing the actual data distribution within a network.

Table of Contents


Why Core Modules?

We need to distribute data somewhere, so we need the Core Modules, right? Actually, if data distribution is so important, how come the most important part of the ANet Protocol resides in external modules?

There is a simple answer: the rules of distribution themselves can be changed, and new distribution methods could be added. For example, let's say that we just found a way to optimize the distribution of Static Data. Then, the only thing we need to install is a newer version of the Static Data Module. No painful version upgrade is needed.

Input/Output of the Core Modules

The "main" function of each module is called with both a list of packets to distribute and a list of commands that were called by the clients. The list of packets is destroyed between each function call. Commands will also be deleted, but only if you mark them as "was used". All "used" commands will be deleted, while the other ones will be sent in a similar manner to the other Core Modules. Unused commands will be finally deleted if they were unused by all the Core Modules.(1)

The input packets are AIPs. There is one buffer for all the cluster, thus it is up to the Core Module to figure out to which connections the packets should be re-transmitted.

As output, the Core Module will have to produce one buffer per connection ID. The output packets must be ANet External Packets (AEP), which are totally platform independant. The deamon will provide some functions to transform an AIP to an AEP, though the modules might have to set some values by themselves(2).

Core Modules

Here is a high-level description of the Core Modules that will be implemented for the first versions of ANet.(3)

Query Module

This module will implement queries[1] for ANet. A query packet is sent to a connection only if the same packet was not sent recently and if the "Time-to-Live" value of the packet is not 0. Also, the packet must not be re-sent to the connection that originally sent the packet.

To do that, the Query Module will keep a list of the packets recently sent, in the form of a simple checksum(4) and the origin of each packet.

Static Data Module

This module will implement Static Data[2] in ANet. It will mostly use the hard disk to store the data, though it might need a larger cache than the Query Module to keep its indexes and recently used static packets.

TWDT Module

This module will implement Anonymous Two-Way Data Transfers (TWDT[3]) in ANet. Its input come only from the client commands[4] made for the TWDT module.

This module will need to use the Query Module through Inter-Module Communication[5], even though the caching of the queries it needs to produce is handled by the TWDT Module itself.

Managing Connections

As input, the modules will always have a list of the connections, and their state. That list is kept in the parsed configuration file[5]. Here are the possible states.

The connection is open, thus it will accept input data and produce output data.
On Hold
The connection cannot do any input or output, but will soon become either Open again or be Closed.
The connection is forever closed. The connection will soon be removed from the connection list.

The Core Modules cannot, by themselves, change the state of a connection. Only Cluster Filter Modules[6] and by Handshaking Protocol Modules.

Implementation Notes


See the complete DTD for more information.

<!-- Core Modules. Simply contains a list of CoreModule elements. -->
<!ELEMENT CoreModules (CoreModule)+>
<!ATTLIST CoreModules %security;>

<!-- Core Module -->
<!ATTLIST CoreModule %moduleName; %args; %security;>


(1) The Run-Time wrapper[5] will allow you to create as many new commands as you want. Actually, commands that are unrecognized by the wrapper will be sent to the Core Modules.

(2) For example, the "Time-to-Live" exists only in the AEPs (for query AEPs), so it is up to the Query Module to fill the value. Note that the format or the AEPs will be very similar to the format of the AIP, though without memory alignment and with strict rules for byte ordering.

(3) The modules were already described in the development introduction. They don't really have a high-level design, and the low-level design will be covered during development. That's because it's too simple (Query Module) or too complex (Static Data and TWDT Module) to be worth the time investment. Don't worry, the modules will be thoroughly documented once a stable implementation will be done.

(4) By simple, I mean easy to compute. For example, MD5[7] checksums would take too much CPU, and isn't very good for small data anyways.


About the references...

[1] Benad: "Queries". Local link.
[2] Benad: "Static Data". Local link.
[3] Benad: "Anonymous Two-Way Data Transfers". Local link.
[4] Benad: "Client Connection Modules". Local link.
[5] Benad: "Run-Time Wrapper". Local link.
[6] Benad: "Cluster Filter Modules". Local link.
[7] Network Working Group: "The MD5 Message-Digest Algorithm". External link.

Last update for this document: August 24, 2001, at 1:22:47 PST