SymetryML Documentation
Data drift

Estimate performance drift between two projects

Scores a validation project and a production project against the same target and metric, so their performance can be compared — e.g. to check whether the production model's performance has drifted relative to the validation baseline. The request body is an inline object (not a named schema) with required `prodDf` and `valDf` fields, both of type `DataFrame`. The response's `KSVDMap` field carries the metric scores; its keys depend on `metric`: - **`accuracy`, `precision`, `recall`, `f1`, `specificity`**: a single key matching the metric name. - **`roc_curve`**: `auc`, `youden_threshold`, and one `FPR:<threshold>`/`TPR:<threshold>` pair per distinct score value in the production data. - **`precrec_curve`**: `ap`, `f1_threshold`, and one `precision:<threshold>`/`recall:<threshold>` pair per distinct score value.

POST
/{user}/projects/performanceEstimator

Scores a validation project and a production project against the same target and metric, so their performance can be compared — e.g. to check whether the production model's performance has drifted relative to the validation baseline.

The request body is an inline object (not a named schema) with required prodDf and valDf fields, both of type DataFrame.

The response's KSVDMap field carries the metric scores; its keys depend on metric:

  • accuracy, precision, recall, f1, specificity: a single key matching the metric name.
  • roc_curve: auc, youden_threshold, and one FPR:<threshold>/TPR:<threshold> pair per distinct score value in the production data.
  • precrec_curve: ap, f1_threshold, and one precision:<threshold>/recall:<threshold> pair per distinct score value.

Authorization

SymetryMLAuth
AuthorizationBearer <token>

HMAC-SHA256 signature-based authentication: requests are signed with your secret key and sent with the Customer-ID, Sym-date, Authorization, Content-MD5 and sym-version headers. See SymetryML REST API Security for the signature algorithm and a complete example.

In: header

Path Parameters

user*string

User/Customer identifier

Query Parameters

targetName*string
validationProject*string
productionProject*string
metric*string

Value in

  • "roc_curve"
  • "precrec_curve"
  • "accuracy"
  • "precision"
  • "recall"
  • "f1"
  • "specificity"
threshold*number

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

application/json

application/json

curl -X POST "https://example.com/c1/projects/performanceEstimator?targetName=string&validationProject=string&productionProject=string&metric=roc_curve&threshold=0" \  -H "Content-Type: application/json" \  -d '{    "valDf": {      "attributeNames": [        "Cover_Type",        "score_validation"      ],      "attributeTypes": [        "B",        "C"      ],      "data": [        [          "1",          "0.82"        ],        [          "0",          "0.13"        ]      ]    },    "prodDf": {      "attributeNames": [        "score_production"      ],      "attributeTypes": [        "C"      ],      "data": [        [          "0.77"        ],        [          "0.09"        ]      ]    }  }'

{  "statusCode": 200,  "statusString": "OK",  "values": {    "KSVDMap": {      "values": [        {          "accuracy": 0.638028809892168        }      ]    }  }}

{  "statusCode": 500,  "statusString": "Generic Exception [java.lang.NullPointerException: Cannot invoke \"com.rtlm.core.projects.SymetryProject.getAttributesNameAndId()\" because \"this.mValidationProject\" is null]",  "values": {}}

Analyze data drift between two projects POST

The request body is an `MLContext` whose `extraParameters` map carries: - **`drift_pref_project_name`** (required): the reference/baseline project. - **`drift_analysis_project_name`** (required): the project being analyzed for drift. May also be the literal value `rollingWindowEmbeddedProject`, to use the reference project's embedded rolling-window project. - **`drift_metric`** (optional, default `mahalanobis`): one of `mahalanobis`, `marginal_stat`, `marginal_non_stat`, or `pca_reconstruction_error`. - **`drift_confidence`** (optional, default `0.95`): applies to the `mahalanobis` and `marginal_stat` metrics. - Advanced `mahalanobis`-only tuning parameters, all optional: `drift_regularize`, `drift_force_spd`, `drift_filter_for_rank`, `drift_do_pca`, `drift_pca_explained_var`, `drift_sensitivity`, and `drift_do_frobenius`. `inputAttributeNames` is required (only the first name is used) when `drift_metric` is `marginal_stat` or `marginal_non_stat`. It is optional for `mahalanobis` and `pca_reconstruction_error`, where all attributes are used when omitted. The response's `KSVDMap` keys are metric-specific: each `drift_metric` returns a different set of score names, as shown in the examples below.

Create new data source POST

Create a data source. The request body's `type` field selects the DSInfo validation/reader path.