Data Transformation Modules
Home/Documents/Development/High-Level Design/Data Transformation Modules
by Benad
The data transformation modules are use to change data in various ways, such as encryption and compression.
Table of Contents
The goal of the Data Transformation modules is to change some data into another form. Their most common use is compression and decompression, but they can also be used for encryption, or for any other function that has an inverse function(1).
The data can be identified by a "format" tag. This 4-bytes value identifies the kind of data once it has been transformed. The "NULL" value identifies data that has not been transformed yet. The transformation function for each format use the input data as if it was untransformed data. Thus, the format tag has to be stored with the data before it is transformed to another kind of data(2).
One instance is created for each Data Transform Module. Its initialization function should return a list of what format tags it supports, with both the actual, 4-byte tag and a Human-readable string, to make it easier to make the configuration file. It is assumed that for each supported format tag, both "transform to" and "transform from" functions are supported. Only one function is needed for each module, as the function will be passed the format tag, the "transformation direction" and a memory tag[1] that points to the data.
Since some transformation functions might need to have additionnal "contextual" information, such as compression level or encryption settings, they can accept a fourth argument, which is a memory tag that refers to the additional information.
The run-time wrapper will allow all modules to call any data transformation function. If you need any kind of security system, it has to be implemented within the data transformation module.
- I think that relying on Inter-Module Communication for calling Data Transformation functions would be too complex. The run-time wrapper could have a single "TrasformData" function, to which you give all the needed data, without having to care to which module it relates to.
- If two different modules can support the same format tag, than the one that will be used is undefined. So, if the data format could be implemented into various versions, it is recommended to store the version in the format tag itself(3). Note that both the tag itself and its corresponding Human-readable string should be unique.
See the complete DTD for more information.
<!-- Data Transformation Module. Data Transformation Modules will be used
in ascending order of "order" for output (to the network), in descending
order for input (from the network). -->
<!ELEMENT DataTransform EMPTY>
<!ATTLIST DataTransform %moduleName; %args; %security; %security;
order CDATA "-1">
|
Notes
(1) Actually, it could be used for thigs like tar[2], MIME[3], binhex[4], or anything you can think of. Also, the input data for a transformation function could point to something outside the deamon, for example a database or the hard disk. So, it could be used to generate, if this is what you need.
(2) That is, you should store the previous format tag in a place that you can retrieve it once the data will be decoded back. Obviously, that's not needed if you assume that you're working with untransformed data.
(3) Please try to make your module "fowards-compatible". If you always use the same format tag accross versions, then it will be impossible to install two different versions of your module at the same time, which might be needed if your module is not backwards-compatible or if the latest version introduces new bug (don't worry, it always happen).
References
About the references...
[1] Benad: "Run-Time Wrapper". Local link.
[2] Free Software Foundation: "tar". External link.
[3] Network Working Group: "Multipurpose Internet Mail Extensions". External links: Part 1, Part 2, Part 3, Part 4, Part 5.
[4] Network Working Group: "MIME Content Type for BinHex Encoded Files". External link.
Last update for this document: August 24, 2001, at 23:52:32 PST
Home