From OpenRate Community Wiki
In order to understand the OpenRate processing architecture, you will need to be familiar with certain key concepts about the way that OpenRate works. The Concepts documents introduce you to the most important elements of the OpenRate system, and try to explain them in a clear and concise way.
We recommend that you read the concepts documents fully before going on to the reference section. This will aid you in getting started in the right direction with OpenRate.
The key concepts that you should understand before getting to work are:
- Framework - How the Framework supports the operation of OpenRate, how it loads and what it does
- Pipelines - How groups of processing modules are grouped together to provide processing
- Module Stack - How modules are constructed in a highly ordered way to make sure that the architecture always remains aligned
- Records - What records are and how they work in OpenRate
- Resources - The concept of having supporting modules hosted by the Framework at the service of the real processing modules
- Transactions - How operations are made safe by implementing a "voting" system for each transaction
The OpenRate reference documents give you in depth information about the Modules and Resources that form part of the OpenRate architecture, and you should use these when you are configuring or designing an OpenRate application.
The modules in this section are divided by functional area, so that you can quickly find a module that does the job you need, or so that you can find the closest one to dervice a custom class from. Only the main implementation modules are listed here, please see the individual file specification to view the module stack definition.
To have a running framework there is a minimum of configuration you should do on the framework itself, apart from the configuration of the framework resource modules.
- Framework Configuration - Describes the the configuration that must be performed on the framework itself
To configure a working pipeline, you will have to do some configuration.
- Pipeline Configuration - Describes the configuration on the pipelines in the framework that you should do
Modules are the main processing elements of the OpenRate architecture. They are responsible for handling Records, which are the basic data representation element. Modules are organised into Pipelines, composed of Input Modules, Processing Modules and Output Modules.
Input Modules are used to read external streams of information and turn them into OpenRate records. Currently defined Input Modules are:
- Flat File Input Adapter
- Binary File Input Adapter
- DB Input Adapter
- TCP Input Adapter
- ASN.1 Input Adapter
- TAP Input Adapter
- Buffer Input Adapter
Other Input Modules are easily definable by sub-classing elements of the Module Stack.
Processing Modules are used to transform, read, delete, modify or create records. Processing modules can also reference Resources in order to perform lookups, store intermediate results or access other sources of information.
OpenRate provides a rich mix of processing modules, and this linked with the highly flexible and extendable architecture means that the number of possible processing modules is virtually limitless. We therefore present a list of the most important modules here.
Lookup for static values
- Best Match - A configurable best match lookup algorithm to perform zone matching in a mobile environment, using only the B-Number
- RUM Best Match - A configurable best match lookup algorithm to perform zone matching in a mobile environment for all Charge Packets of a Rating Record
- Best Match Fixed Line - A configurable best match lookup algorithm to perform zone matching in a fixed line (two dimensional) A-B environment, using both the A-Number and the B-Number
- Regex Match - A configurable regular expression matching algorithm of records files or intermediate results against a sorted list of regular expression patterns
- Indexed Lookup Match - A module that can index an array of values, configurably indexing multiple columns in order to be able to return the whole list of cross reference values
- Validity Segment Match - A module that can locate a key, respecting non-overlapping periods of validity. This is useful for situations where a resource can be re-allocated between different accounts at different times. An example of this could be an MSISDN that is re-used, or a port number on a dial-up device that gets re-allocated many times.
Lookup customer data
- Customer Match - Locate customer information based on a very simple file update interface, suitable for minimal installations with primitive provisioning systems.
- Customer Match Audited - Locate customer information based on a fully featured temporal database structure that can be filled using the OpenRate Customer Interface. This provides support for complex rating scenarios.
- Customer Match Infranet - A module that links directly to a Portal Infranet (Oracle BRM) Database to extract customer information. This can be used to run OpenRate in an Oracle BRM environment.
- Customer Match JBilling - A module that links directly to a JBilling Database to extract customer information. This can be used to run OpenRate in a JBilling integrated environment.
- Rate Calculation - A configurable rating module that calculates the cost of a record based on definable record attributes. It uses the Rate Cache.
- RUM Rate Calculation - A configurable rating module that calculates the cost of a record based on definable record attributes. It uses the RUM Rate Cache.
- Discount Calculation - A configurable discounting module that calculates the discount applicable on a record according to period volume thresholds (e.g. Discount on ever call after the first 100 minutes), or granted resources to be consumed (e.g. 100 free megabytes a month).
- Time Match - Used to determine if the call was made during peak or off-peak hours. It uses the Time Model Cache.
- RUM Time Match - Used to determine if the call was made during peak or off-peak hours for all Charge Packets of a Rating Record. It uses the Time Model Cache.
- RUM Holiday Match - Used to determine if the call was made on a holiday day or not. If it was, it marks the call with the defined holiday time result. It uses the RUM Holiday Cache.
- Gather RUM Impacts - Used to summarise the impacts that have been made to different resources after rating. In effect this rolls up all of the impacts from the Charge Packets, creating Balance Impacts from them.
- Minimum Fee Lookup - Sometimes it is necessary to calculate the minimum price for a record. This module reads the configured minimum fee from the Min Fee Cache and applies it to the record.
- Balance Handler PlugIn - allows access to the functions of the Balance Cache. Used for calculating discounts, bundles, pre-paid usage allotments etc.
Lookup of dynamic values
- Duplicate Check - used to detect if a record has been previously processed, and to allow it to be rejected if so
- Dump - A diagnostic output producer used in on the fly debugging and problem resolution.
- DumpNT - A diagnostic output producer used in on the fly debugging and problem resolution in Non-Transactional settings.
- Processing Stub - a very simple primitive module that forms the starting point for your own implementations of processing modules.
Other Processing Modules are easily definable by sub-classing elements of the Module Stack.
Output Modules are used to write processed records to external streams of information. Currently defined Output Modules are:
- Flat File Output Adapter
- Flat File Multi Stream Output Adapter
- JDBC Output Adapter
- JDBC Batch Output Adapter
- TCP Output Adapter (In preparation)
- XML File Output Adapter
- Null Output Adapter
- Null NT Output Adapter
- TAP Output Adapter
Other Output Modules are easily definable by sub-classing elements of the Module Stack.
Service Resources provide services to the Framework and Modules. Normally service resources do not store information but instead provide access to other elements of the system. The most important Service Resources are:
- Log - Provides trouble free and fast access to the log devices to log information.
- Event Handler - Provides dispatching and coordination services to allow control messages to be passed to system components, and to allow monitoring. Communicates with the Configuration Manager
- Transaction Manager - Used by Transactional modules as a coordinator of the status of processing, to ensure that only fully completed work is committed to storage
- Data Source Manager - Provides easy and trouble free access to database resources
- Data Cache Controller - manages the initialisation of the data cache objects during Framework start up.
Creating Service Resource modules is usually out of the scope of normal configuration, and requires in depth knowledge of the system architecture. Should you require other Service Resources not listed here, get in contact with us and we can put you on the right path.
Data resources provide persistent or temporary data storage facilities that can be accessed by Modules. For example a cache of information may be loaded on Framework startup and can be referenced by modules to enrich each record as it passes through the OpenRate system.
Most data resources may be reloaded without stopping the processing framework, and in this case, the pipelines are halted until a Sync Point is reached, and then the loading is performed while no transaction is active. This ensures that reference data is consistent during processing.
The most important Information Resources are:
Lookup static data
- Best Match Cache - Provides the configuration database for the Best Match processing module, based on a mobile (one dimensional "B Number") zoning scheme
- Best Match Cache Fixed Line - Provides the configuration database for the Best Match Fixed Line processing module, based on a fixed line (two dimensional "A Number-B Number") zoning scheme
- Regex Match Cache - Provides the configuration database for the Regex Match processing module
- Validity Segment Cache - Manages the lookup of shared resources allocated to differing users at different times (for example, shared modem racks that allocate ports to users on a as-required basis)
- Indexed Lookup Cache - Reads lookup information into cache memory, stores the information as an object in the cache memory, and creates any number of user definable hash table indexes to provide fast object retrieval
- Persistent Indexed Object - A general purpose persistent hash table. Persistent in this context means that the values are saved periodically or on system shutdown, and reloaded on system startup
- Customer Cache - Provides caching for the Customer Match module
- Customer Cache Audited - Provides caching for the Customer Match Audited module, using a historicised mode of operation.
- Customer Cache Infranet - Provides caching for the Customer Match Infranet module. To use this module you need to have a configured Portal Infranet database.
- Customer Cache JBilling - Provides caching for the Customer Match JBilling module. To use this module you need to have a configured JBilling database.
- Rate Cache - Provides the configuration database for the Rate Calculation processing module. This is a simplified rating module for cases where you do not need to perform the complex rating offered by RUM Rate Cache, which is part of the RUM Rating Model.
- RUM Rate Cache - Provides the configuration database for the RUM Rate Calculation processing module as part of the RUM Rating Model.
- Time Model Cache - Provides the configuration database for the Time Match processing module. This is a simplified time module for cases where you do not need to perform the complex rating offered by RUM Time Model Cache, which is part of the RUM Rating Model.
- RUM Time Model Cache - Provides the configuration database for the RUM Time Match processing module as part of the RUM Rating Model.
- RUM Holiday Cache - Provides the configuration database for the RUM Holiday Match processing module as part of the RUM Rating Model.
- Balance Cache - Provides the persistent usage accumulation counters for use in the Discount Calculation processing module
- Shared Memory - A general purpose shared memory cache for holding intermediate results or running totals
- Persistent Shared Memory - A general purpose persistent shared memory cache for holding intermediate results or running totals. Persistent in this context means that the values are saved on system shutdown or periodically, and reloaded on system startup
- Aggregation Cache - Standardised module for creating aggregations (counting, summing, averaging, finding the minimum or maximum) on a stream of records. Fully configurable to filter (in conjunction with Regex Match Cache) and aggregate on any collection of fields.
- Duplicate Check Cache - Manages the detection of records that have already been processed, storing the in-memory cache to persistent storage (file or DB) on demand or at framework shutdown.
- AbstractSyncLoaderCache - Used to provide the basis for caches that are reloadable, and which enforce a Sync Point before reloading.
- AbstractStubPlugin - Used to provide the basis for processing modules which do not need to manage Header and Trailer records.
OpenRate is a convergent platform, meaning that it can handle batch and real time processing with the same modules.
This interface allows the creation of customer related information to allow rating to be performed using OpenRate standard modules. The structure of the interface is described in the section Customer Interface. The interface creates Audited Customer Information. Examples of the code needed to implement these calls is presented in the developer guide.
Automatic or Manual Transaction Handling
The CustomerInterface can be used with automatic or manual transaction handling. Manual transaction handling models business events more closely, by allowing you to define the transaction boundaries over multiple methods, and commit (or rollback) the entire group of methods if any of the component methods fails. If you do not wish to manage transactions manually, you simply ignore this parameter, and each method will be atomic and auto-committing. For a discussion of this see Modeling Business Events.
Note that if you use Manual Transaction Handling, your underlying database must be able to support transactions. This means that for MySQL, you have to use tables that are managed by the "InnoDB" engine, not the default "myISAM" engine"
The methods available in the customer interface are:
- CreateCustomer - Creates a new customer
- CreateProduct - Creates a new Product
- CreateAlias - Creates a new Alias for a customer
- CreateERA - Creates a new Extended Rating Attribute for a customer
- ModifyCustomer - Allows the end date of a customer relationship to be changed
- ModifyProduct - Allows the end date of a product instance to be changed
- ModifyAlias - Changes the end date of the Alias
- ModifyERA - Changes the value of the ERA
- DeleteERA - Deletes an ERA
- MigrateProduct - Change the Product identifier
- GetCustomer - Returns whether a customer is known in the database or not, and if it is, returns the internally used account identifier, start and end date
- GetProduct - Retrieve the products associated with an account, and the services, subscription IDs start and end dates
- GetAlias - Retrieve the Alias associated with an account, and the services, subscription IDs start and end dates
- GetERA - Retrieve the ERAs associated with an account
- GetProducts - Retrieve the internal product list and services
The OpenRate package also contains several example configurations that are designed to run out of the box and to allow you to have a working platform to explore. You can read more about these example platforms in the section on OpenRate Example Applications.