Python query extensions

Query API

To create a python query extension, it is required to implement a QueryPlugin class:

class QueryPlugin:
   def __init__(self, system, user, password):
     def GetParameters(self):
     def GetMetadata(self, arguments):
   def ExecuteQuery(self, arguments, columnSchema):

Constructor

def __init__(self, system, user, password):
public class MyDotNetQueryExtensions : ACustomQueryExtension
{
    public MyDotNetQueryExtensions(string System, string UserId, string Password)
            : base(System, UserId, Password)
    {
    }
}

The constructor is provided with the system, user and password. The user and password are credentials mapped in the OmniFi Administration application.

GetParameters

def GetParameters(self):

Extensions that require some form of user input can define parameters. The return value is a list (Parameter).

A parameter is created as follows:

Parameter(Name, Description, Type, DefaultValue, Domain, Mandatory, Comment)
protected override List<Plugin_Parameter> GetParametersImpl()
{
    return (new List<Plugin_Parameter>
    {
        new Plugin_Parameter
        {
            Type = Plugin_DataType.INTEGER,
            Name = "count",
            Description = "Count",
            Mandatory = true,
            DefaultValue = new Plugin_Value(11),
            Comment = "Number of records to generate"
        }
    });            
}
OptionTypeDescription
NamestrContext unique identifier.
TypeDataType (enum)Data type [string | integer | double | date | long | uint | bool]
DefaultValueValueA typed Value.
MandatoryBooleanDefines if the parameter is mandatory or optional.
CommentstrLonger comment visible to the user.

Default value

You can provide a default value in the form of a typed Value. The Value class ensures type safety in communication between OmniFi and the python extension by including data type information.

Value(DataType.string, "Wednesday")

Parameter domains

Domains allow parameter input to be restricted to a certain subset of discrete values. For example, a yield curve gap set could be described as a set of discrete values [O/N | SPOT | 1W | 1M | 3M | 1Y …]. To provide the user with a convenient drop-down you can use the parameter domain feature.

def GetParameters(self):
        parameters = list()
        gapDomain = list()
        gaps = list(["O/N", "SPOT", "1W", "1M", "3M", "6M", "1Y", "10Y"])
        for g in gaps:
            gapDomain.append(DomainItem(Value(DataType.string, g), g))
        gap = Parameter("gap", "Period", DataType.string, Value(DataType.string, "SPOT"), gapDomain, True, "")
        parameters.append(gap)
        return (parameters);
protected override List<Plugin_Parameter> GetParametersImpl()
{
    return (new List<Plugin_Parameter>
    {
        new Plugin_Parameter
        {
            Type = Plugin_DataType.INTEGER,
            Name = "count",
            Description = "Count",
            Mandatory = true,
            DefaultValue = new Plugin_Value(11),
            Comment = "Number of records to generate",
            Domain = new List<Plugin_DomainItem>
            {
                new Plugin_DomainItem
                {
                    DisplayMember = "First",
                    ValueMember = new Plugin_Value(1)
                },
                new Plugin_DomainItem
                {
                    DisplayMember = "Second",
                    ValueMember = new Plugin_Value(2)
                },
            }
        }
    });            
}

A domain consists of a set of DomainItem. A DomainItem has a descriptive string, and a typed Value.

GetMetadata

def GetMetadata(self, arguments):

The query output is defined by the MetadataCollection returned by GetMetadata.

To define an output column, simply add it so a MetadataCollection, as we have seen in the hello world example:

def GetMetadata(self, arguments):
        metadata = MetadataCollection()
        metadata.add(Metadata("hello_world", "Hello World", DataType.string))
        return (metadata);
protected override Plugin_MetadataCollection GetMetadataImpl(List<Plugin_Argument> Parameters)
{
    Plugin_MetadataCollection metadata = new Plugin_MetadataCollection();
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.INTEGER,
            Name = "int",
            Description = "Integer"
        });
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.STRING,
            Name = "str",
            Description = "String"
        });
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.INTEGER,
            Name = "int",
            Description = "Integer"
        }, "SubEntity");
    metadata.Add(
        new Plugin_Metadata
        {
            Type = Plugin_DataType.STRING,
            Name = "str",
            Description = "String"
        }, "SubEntity");
    return (metadata);
}

This adds the column to the default output table. Ensure that the column name (“hello_world” in this example) is unique in the table!

The above method simply adds the column to the default table “main”, but you can define any number of tables by adding metadata for them:

def GetMetadata(self, arguments):
        metadata = MetadataCollection()
        metadata.add(Metadata("hello_world", "Hello World", DataType.string))
        metadata.add(Metadata("other_column", "Other Column", DataType.string), "ANOTHER_TABLE")
        return (metadata);

The arguments passed to GetMetadata are the parameter values provided by the user. Note that while sometimes necessary, it is recommended to not allow user input to affect output metadata, as this makes using the extension in reports, interfaces etc. so much more difficult.

ExecuteQuery

def ExecuteQuery(self, arguments, columnSchema):
protected override Plugin_QueryResult ExecuteQueryImpl(List<Plugin_Argument> Parameters, Plugin_ColumnSchema ColumnSchema)
        {
            var paramLookup = Parameters.ToDictionary((x) => x.ParameterName);
            Plugin_Argument countValue;
            if (!paramLookup.TryGetValue("count", out countValue))
                throw new ApplicationException("Count parameter not provided");
 
            int count = countValue.IntegerValue ?? 10;
            Plugin_QueryResult result = new Plugin_QueryResult(
                new Plugin_Dataset(
                    new Plugin_DataColumn("int", "Integer", Enumerable.Range(0, count)
                        .Select((x) => (int?)x).ToArray()),
                    new Plugin_DataColumn("str", "String", Enumerable.Range(0, count)
                        .Select((x) => $"number {x}").ToArray())
                ),
                new Plugin_Dataset("SubEntity",
                    new Plugin_DataColumn("int", "Integer", Enumerable.Range(0, count)
                        .Select((x) => (int?)x).ToArray()),
                    new Plugin_DataColumn("str", "String", Enumerable.Range(0, count)
                        .Select((x) => $"number {x}").ToArray())
                )
            );
            
            return (result);
        }

Execute query is the main worker method of the extension.

Arguments

Arguments are provided as a list of Argument. For simplicity this structure can be read using an ArgumentReader.

def ExecuteQuery(self, arguments, columnSchema):
        argReader = ArgumentReader(arguments)
        paramValue = argReader.TryGetArgumentValue("paramName")
        if (paramValue != None):
            pass # Do something
var paramLookup = Parameters.ToDictionary((x) => x.ParameterName);

Schema

The columnSchema argument defines the output the user has chosen to include in the query. The data type is ColumnSchema.
Utilizing the provided schema is optional. As long as your data matches your metadata (as returned by GetMetadata) precisely, OmniFi will arrange the query output to match the user configuration. However, for large volume scenarios is recommended you make use of the provided schema to minimize the data and improve performance.

Output

The output of ExecuteQuery is a QueryResult structure, containing a dictionary of column-based Datasets. As shown in the hello world example, you can use a QueryResultWriter to populate the result structure row-by-row. While convenient, this method does come with a performance penalty. If optimal performance is your priority, you can create the column based structure directly.

Return value

The OmniFi data structure is column based. A table consists of a set of columns of data points as opposed to rows of data points. Plugin_QueryResult class consists of one or more entity/sub-entity sets of columns.

def ExecuteQuery(self, arguments, columnSchema):
        values = list()
        for x in range(0, 100):
            values.append("I amd here!")
        
        schema = self.GetMetadata(None).Datasets["main"].metadata;
        mainSet = Dataset()
        mainSet.AddColumn(DataColumn(schema["hello_world"], values))
        result = QueryResult()
        result.AddDataset(mainSet)
        return (result)

Note that to create subentity tables, just create a named Dataset:

subSet = Dataset("SubEntityName")
# Add columns to subentity table …
result = QueryResult()
result.AddDataset(subSet)