Project Data Source Logs

This functionality allows to get report metrics on the various data sources that were processed by a SymetryML project. Data Sources logs can be requested as well as the logs about the SymetryML streaming API Learn and Forget.

Data Source Log Information

The data source log contains information about each data source that was processed (learn / forget) by a SymetryML project. It also contains other meta information about each invocation:

Data Source Log Definitions

Real Streaming, AKA Learn / Forget Log Information

The Learn / Forget Stream log is a bit different. As these REST endpoints are designed to continuously stream data into a project, they can easily grow large if we keep track of meta-data for each invocation that pushes new data into a SymetryML project. For more information on the SymetryML streaming API consult the following sections:

• Real Time Streaming Data
  • Incremental Learning
  • Decremental Forget

The learn / forget logging REST endpoint aggregate data based on the type of data that was pushed into the learn / forget streaming API for a given project. More particularly the following fields - or primary keys - are used to aggregate logging information:

• client_id
• client_request_type
• client_request_reponse_code
• client_request_df_attributes_names
• client_request_df_attributes_types

Learn / Forget Log Definitions

The following table lists what information is returned for the learn / forget logging rest API based on the previous list. The primary keys are in listed in bold.

Data Source Logs Rest API

Data Source Get Log

Get a given project data source log. This log contains information about the data sources that were processed by a given project.

URL

GET /symetry/rest/{cid}/projects/{pid}/getDsLog

HTTP Responses

HTTP Response Entity

Sample Request/Response Fed Create

Example response with project with two data sources successfully learned:

GET url="http://charm:8080/symetry/rest/c1/projects/p1/getDsLog"

Response:
{"statusCode":"OK","statusString":"OK","values":
        {"stringList":{
                "values":
                ["{"ds_request_datetime":"1605629571980","ds_request_response_code":"200","ds_name":"Iris_category_csv","ds_request_type":"learn","ds_client_id":"web-api","ds_request_process_time":"266","ds_client_ip":"172_16_104_1","ds_request_df_numrows":"150"}",
                "{"ds_request_datetime":"1605629597958","ds_request_response_code":"200","ds_name":"Iris_rtlm2_csv","ds_request_type":"learn","ds_client_id":"web-api","ds_request_process_time":"257","ds_client_ip":"172_16_104_1","ds_request_df_numrows":"150"}"]
                }
        }
}

SymetryML Project Learn / Forget Logs

The REST endpoint provides logging information about data that was pushed to a given project via the various stream APIs.

URL

GET /symetry/rest/{cid}/projects/{pid}/getLFStats

HTTP Responses

HTTP Response Entity

Sample Request/Response

GET url="http://charm:8080/symetry/rest/c1/projects/p1/getLFStats"

Response:
{"statusCode":"OK","statusString":"OK","values":
        {"stringList":{
                "values":
                ["{"client_request_df_attributes_numrows_total":"506","client_request_type":"learn","client_request_df_attributes_names":"CRIM,ZN,INDUS,CHAS,NOX,RM,AGE,DIS,RAD,TAX,PTRATIO,B,LSTAT,MEDV","client_request_reponse_code":"200","client_request_count":"1","client_request_df_attributes_types":"C,C,C,B,C,C,C,C,C,C,C,C,C,C","client_id":"default-rest-client","client_request_process_time":"301"}"]
                }
        }
}