Python query extensions

Query API

To create a python query extension, it is required to implement a QueryPlugin class:

class QueryPlugin:
   def __init__(self, system, user, password):
     def GetParameters(self):
     def GetMetadata(self, arguments):
   def ExecuteQuery(self, arguments, columnSchema):

Constructor

def __init__(self, system, user, password):

The constructor is provided with the system, user and password. The user and password are credentials mapped in the OmniFi Administration application.

GetParameters

def GetParameters(self):

Extensions that require some form of user input can define parameters. The return value is a list (Parameter).

A parameter is created as follows:

Parameter(Name, Description, Type, DefaultValue, Domain, Mandatory, Comment, MultiSelect, Operators)

Option	Type	Description
Name	str	Context unique identifier.
Type	DataType (enum)	Data type [string	integer	double	date	long	uint	bool
DefaultValue	Value	A typed Value.
Mandatory	Boolean	Defines if the parameter is mandatory or optional.
Comment	str	Longer comment visible to the user.
MultiSelect	Boolean	Optional, enables multi-select for the parameter.
Operators	str	Optional, defines operators for the parameter.

Default value

You can provide a default value in the form of a typed Value. The Value class ensures type safety in communication between OmniFi and the python extension by including data type information.

Value(DataType.string, "Wednesday")

Parameter domains

Domains allow parameter input to be restricted to a certain subset of discrete values. For example, a yield curve gap set could be described as a set of discrete values [O/N | SPOT | 1W | 1M | 3M | 1Y …]. To provide the user with a convenient drop-down you can use the parameter domain feature.

def GetParameters(self):
        parameters = list()
        gapDomain = list()
        gaps = list(["O/N", "SPOT", "1W", "1M", "3M", "6M", "1Y", "10Y"])
        for g in gaps:
            gapDomain.append(DomainItem(Value(DataType.string, g), g))
        gap = Parameter("gap", "Period", DataType.string, Value(DataType.string, "SPOT"), gapDomain, True, "")
        parameters.append(gap)
        return (parameters);

A domain consists of a set of DomainItem. A DomainItem has a descriptive string, and a typed Value.

Multi-Select parameters

Multi-select can be enabled for string domain parameters, and allows the user to select multiple items from the domain.

Multi-select values are sent to the extension as a CSV encoded string, where each element is one selected item, e.g. OPTION_A,OPTION_B.

Items that themselves contain quotation characters or list separators are CSV encoded, e.g. "value,1","value""2".

Parameter operators

A parameter can be associated with a set of operators. OmniFi doesn't have support for any particular operators, instead you define a set of string options that the user can choose from, e.g. `["=", ">", "<"]. The extension is responsible for implementing the operator logic, allowing you to define any operators that make sense in your context.

def GetParameters(self, configArgs):
    
    parameters = list()

    # Add a date parameter for the period
    parameters.append(Parameter("period_start", "Period Start", DataType.date, None, None, True, "The period start date"))

    # Add a period length parameter. This parameter is expressed in # of days, 
    # with an operator + or - indicating forward or backward from period_start
    parameters.append(Parameter("period_length", "Period Length", DataType.integer, None, None, True, "The period length +/- n days", False, ["+", "-"]))

The selected operator is provided to GetMetadata() and ExecuteQuery() with the parameter argument.

def ExecuteQuery(self, arguments, columnSchema):
    
    # Create a reader for the arguments.
    # This includes also Configuration Parameters.
    argReader = ArgumentReader(arguments)
    
    rate_date = argReader.GetArgument("period_start").Value
    period_length = argReader.GetArgument("period_length").Value
    period_operator = argReader.GetArgument("period_length").Operator

GetMetadata

def GetMetadata(self, arguments):

The query output is defined by the MetadataCollection returned by GetMetadata.

To define an output column, simply add it so a MetadataCollection, as we have seen in the hello world example:

def GetMetadata(self, arguments):
        metadata = MetadataCollection()
        metadata.add(Metadata("hello_world", "Hello World", DataType.string))
        return (metadata);

protected override Plugin_MetadataCollection GetMetadataImpl(List<Plugin_Argument> Parameters)
{
    Plugin_MetadataCollection metadata = new Plugin_MetadataCollection();
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.INTEGER,
            Name = "int",
            Description = "Integer"
        });
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.STRING,
            Name = "str",
            Description = "String"
        });
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.INTEGER,
            Name = "int",
            Description = "Integer"
        }, "SubEntity");
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.STRING,
            Name = "str",
            Description = "String"
        }, "SubEntity");
    return (metadata);
}

This adds the column to the default output table. Ensure that the column name (“hello_world” in this example) is unique in the table!

The above method simply adds the column to the default table “main”, but you can define any number of tables by adding metadata for them:

def GetMetadata(self, arguments):
        metadata = MetadataCollection()
        metadata.add(Metadata("hello_world", "Hello World", DataType.string))
        metadata.add(Metadata("other_column", "Other Column", DataType.string), "ANOTHER_TABLE")
        return (metadata);

The arguments passed to GetMetadata are the parameter values provided by the user. Note that while sometimes necessary, it is recommended to not allow user input to affect output metadata, as this makes using the extension in reports, interfaces etc. so much more difficult.

ExecuteQuery

def ExecuteQuery(self, arguments, columnSchema):

Execute query is the main worker method of the extension.

Arguments

Arguments are provided as a list of Argument. For simplicity this structure can be read using an ArgumentReader.

def ExecuteQuery(self, arguments, columnSchema):
        argReader = ArgumentReader(arguments)
        paramValue = argReader.TryGetArgumentValue("paramName")
        if (paramValue != None):
            pass # Do something

var paramLookup = Parameters.ToDictionary((x) => x.ParameterName);

Schema

The columnSchema argument defines the output the user has chosen to include in the query. The data type is ColumnSchema.
Utilizing the provided schema is optional. As long as your data matches your metadata (as returned by GetMetadata) precisely, OmniFi will arrange the query output to match the user configuration. However, for large volume scenarios is recommended you make use of the provided schema to minimize the data and improve performance.

Output

The output of ExecuteQuery is a QueryResult structure, containing a dictionary of column-based Datasets. As shown in the hello world example, you can use a QueryResultWriter to populate the result structure row-by-row. While convenient, this method does come with a performance penalty. If optimal performance is your priority, you can create the column based structure directly.

Return value

The OmniFi data structure is column based. A table consists of a set of columns of data points as opposed to rows of data points. Plugin_QueryResult class consists of one or more entity/sub-entity sets of columns.

def ExecuteQuery(self, arguments, columnSchema):
        values = list()
        for x in range(0, 100):
            values.append("I amd here!")
        
        schema = self.GetMetadata(None).Datasets["main"].metadata;
        mainSet = Dataset()
        mainSet.AddColumn(DataColumn(schema["hello_world"], values))
        result = QueryResult()
        result.AddDataset(mainSet)
        return (result)

Note that to create subentity tables, just create a named Dataset:

subSet = Dataset("SubEntityName")
# Add columns to subentity table …
result = QueryResult()
result.AddDataset(subSet)