Data Transformation Modules

Home/Documents/Development/High-Level Design/Data Transformation Modules
 Project Closed
 Site Index
 User Docs
 Introduction to ANet
 High-Level Design
 Run-Time Wrapper
 Client Connection Modules
 Data Transformation Modules
 Client SDK
 Cluster Group Modules
 Cluster Filter Modules
 Core Modules
 Handshaking Protocol Modules
 Packet Protocol Modules
 Connection Protocol Modules
 Bandwidth Manager Module
 Document Type Definition (DTD)
 Low-Level Design
 Protocol Specifications
 Task List
 Development Roadmap
 Other Docs
 Mailing Lists
 Contacting Us


by Benad

The data transformation modules are use to change data in various ways, such as encryption and compression.

Table of Contents


The goal of the Data Transformation modules is to change some data into another form. Their most common use is compression and decompression, but they can also be used for encryption, or for any other function that has an inverse function(1).

The data can be identified by a "format" tag. This 4-bytes value identifies the kind of data once it has been transformed. The "NULL" value identifies data that has not been transformed yet. The transformation function for each format use the input data as if it was untransformed data. Thus, the format tag has to be stored with the data before it is transformed to another kind of data(2).

One instance is created for each Data Transform Module. Its initialization function should return a list of what format tags it supports, with both the actual, 4-byte tag and a Human-readable string, to make it easier to make the configuration file. It is assumed that for each supported format tag, both "transform to" and "transform from" functions are supported. Only one function is needed for each module, as the function will be passed the format tag, the "transformation direction" and a memory tag[1] that points to the data.

Since some transformation functions might need to have additionnal "contextual" information, such as compression level or encryption settings, they can accept a fourth argument, which is a memory tag that refers to the additional information.

The run-time wrapper will allow all modules to call any data transformation function. If you need any kind of security system, it has to be implemented within the data transformation module.

Implementation Notes


See the complete DTD for more information.

<!-- Data Transformation Module. Data Transformation Modules will be used
in ascending order of "order" for output (to the network), in descending
order for input (from the network). -->
<!ELEMENT DataTransform EMPTY>
<!ATTLIST DataTransform %moduleName; %args; %security; %security;
          order CDATA "-1">


(1) Actually, it could be used for thigs like tar[2], MIME[3], binhex[4], or anything you can think of. Also, the input data for a transformation function could point to something outside the deamon, for example a database or the hard disk. So, it could be used to generate, if this is what you need.

(2) That is, you should store the previous format tag in a place that you can retrieve it once the data will be decoded back. Obviously, that's not needed if you assume that you're working with untransformed data.

(3) Please try to make your module "fowards-compatible". If you always use the same format tag accross versions, then it will be impossible to install two different versions of your module at the same time, which might be needed if your module is not backwards-compatible or if the latest version introduces new bug (don't worry, it always happen).


About the references...

[1] Benad: "Run-Time Wrapper". Local link.
[2] Free Software Foundation: "tar". External link.
[3] Network Working Group: "Multipurpose Internet Mail Extensions". External links: Part 1, Part 2, Part 3, Part 4, Part 5.
[4] Network Working Group: "MIME Content Type for BinHex Encoded Files". External link.

Last update for this document: August 24, 2001, at 23:52:32 PST