Overview

From ESM Wiki
Jump to: navigation, search

Contents

ESM is a capability of Mailspect's MPP software that allows to define complex email processing logic on email server's side. ESM integrates email server with other software systems for the sake of integration of email communication with business process management systems (such as CRM, ERP, BPM, etc.). ESM is a conjuction of a simple yet powerful higher-level configuration language with a number of precompiled components each implementing useful lower-level functionality.

Connection with MPP core logic

MPP is a software that processes requests. Processing logic is defined by the vendor and to some extent can be tuned by a customer to his/her particular needs with the help of configuration tools. This processing logic and its configuration is documented on Mailspect documentation site and is referenced as "MPP core logic" in further explanations. In some predefined extension points of MPP core logic processing control is passed to a ESM pipeline which is a root abstraction for ESM logic. After a pipeline finishes control is returned back to MPP core logic. For example WBL pipeline is executed when MPP checks for WBL (white/black/access lists). Pipeline processing logic, its configuration and connection with MPP core logic is documented on this site.

Processing logic

Pipeline structure is illustrated on the following diagram:

Overview Pipeline.jpg

Pipeline consists of a sequence of engines that are executed in turns one after another and a postprocessor. Each engine checks for a precondition and if precondition is met engine performs action and calculates result. Engine may use results from preceeding engines. Postprocessor comes into play after all engines are done. The job of a postprocessor is to perform additional pipeline-specific processing that is normally not allowed for engines.

Specific action that an engine performs depends on an engine type. Engine can provide a bridge to a Web Service or a database software (MySQL database, LDAP directory, etc.), access to a useful functionality of a third-party library (regular expressions matching, MIME generation etc.), access to a system resource (saving files, etc.) or something else. The point is that each engine type performs specialized atomic action and by combining engines into a pipeline administrator can configure complex processing logic that consists of multiple dependent actions. Available engine types make a toolbox for creating a pipeline. As number of available engine types will grow the toolbox will grow. Thus the number of functionality that administrator can combine together will grow.

Configuration

Pipeline configuration is specified with the help of XML plus special template and condition expressions for relevant XML element values. Pipeline XML extends the structure of MPP configuration file (known as mppd.conf.xml). Most of a pipeline configuration is engines configuration that is concentrated under /mppd/engines XML element. Engine sequence and postprocessor are specified as MPP policy group options. For example:

<mppd>
    <groups>
        <group id="some_MPP_policy_group">
            <wbl_engines>mysql.check_sender mysql.check_recipient</wbl_engines>
        </group>
    </groups>

    <engines>
        <mysql>
            <query id="check_sender">
                ...
            </query>

            <query id="check_recipient">
                ...
            </query>
        </mysql>
    </engines>
</mppd>

In this example <wbl_engines> element defines engine sequence for WBL pipeline for some_MPP_policy_group Postprocessor is not defined. One <mysql> engine is defined under <engines> with two <query> methods. Internal structure of <query> elements is omitted (three dots stands for "something goes here") for simplicity as well as other MPP configuration options that might be present in normal config.

Template and condition expressions are the means to parametrize configuration options with request-specific (runtime) values such as:

  • message sender, recipient, body part, etc.;
  • results from previous engines;
  • results from embeded functions.

For example:

<mppd>
    <engines>
        <http>
            <query>
                <precondition>$body_part.malformed $EQ no $AND $body_part.multipart $EQ no</precondition>

                <headers>
                    Content-Type: $body_part.mime_type
                    Content-Size: ${size $body_part}
                </headers>
                
                <entity>$body_part</entity>

                ...
            </query>
        </http>
    </engines>
</mppd>

In this example there are one condition element <precondition> and two template elements <headers> and <entity>. Dolar character "$" starts macro substitution that will be expanded during request processing. In the condition dolar character also starts boolean operations $EQ (equal) and $AND (logical AND).

Multiplicity of runtime properties

Some template macros that refer to runtime properties such as recipient or body part may actually stand not for a single value but for a set of values. In ESM such macros are said to have one or more dimension. In programing languages this is modeled with arrays and handled with looping operators. ESM doesn't introduce arrays and looping operators. Instead it makes use of "implicit iteration" concept. In the root of this concept is the idea that if an expression contains a dimensional macro then an operation that depends on expression must be repeated for each value of a macro. For example:

<mppd>
    <engines>
        <mysql>
            <query>
                <template>SELECT COUNT(*) FROM recipient_acl WHERE email='${escape $recipient}'</template>
                ...
            </query>
        </mysql>
    </engines>
</mppd>

In this example dimensional macro $recipient is used for MySQL query template thus query will be repeated for each recipient.

Communication protocol processing stage

Some macros, engines and pipelines may be not applicable at some stages. This should be taken into account when designing processing logic. For example strip body pipeline will not be run at policy-rcpt stage.

Personal tools