Data Source API

Using a Data Source

In SymetryML, a data source is an abstraction of a CSV file that resides somewhere and that can be used by:

• SymetryML projects to learn new data
• Models to make predictions and assessments
• Encoder to update their internal encoding table

SymetryML supports various types of data sources:

• Secure File Transfer Protocol (SFTP)
• HTTP/HTTPS URL
• Amazon Simple Storage Service (S3)
• Microsoft Azure Blob Storage
• Google Cloud Storage
• Oracle OCI Object Storage
• Amazon RedShift
• Snowflake
• Spark Processing: Amazon S3, Google cloud storage, Oracle OCI object storage and Microsoft Azure blob storage data sources can be processed in parallel leveraging a Spark Cluster.
• SymetryML data source plugins
• JDBC
• Local Data Source, that is allows to browse the local file system of the jetty web server with same privileges as the user running the Jetty web server.

To use a data source, create a JSON data structure described in DSInfo that contains the fields in table below.

Required DSInfo Fields

Field Validation

Data source fields are validated on create, update and sample preview. A missing or empty required field is rejected with 400 BAD REQUEST and a message naming what is missing:

In 6.3 these conditions surfaced as generic 500 errors.

The Required column in the tables below reflects exactly what this validation enforces. A field marked Optional is not rejected at create time, which is not the same as saying the data source will function without it.

Note: this applies to built-in types only. Data source plugins declare their own fields through uiInputFields in the plugin configuration and are not checked here.

Additional Information Stored in Data Source

The info field of a data source contains specific information based on the type of data source. The following tables describes what they are for each different type of data source

HTTP/HTTPS Data Source

| field | Required Description | | ------ | -------- ------------------------ | | path | Required http:// or https:// URL. |

Note: http and https data sources follow redirects. A non-2xx response fails with HTTP GET <url> failed with status N. When the server sends no Content-Length header the reported size is -1, so progress cannot be reported for chunked responses. HTTPS accepts self-signed certificates. Connection and read timeouts come from the rtlm.option.vfs.http.connection.timeout and rtlm.option.vfs.http.so.timeout server settings — the vfs. prefix is historical and the names are unchanged.

Secure FTP (SFTP)

| field | Required Description | | ------------ | -------- ------------------------------------------------- | | path | Required path to the file on the server. | | sftpuser | Required user name used to connect to the SFTP server. | | sftppasswd | Required user password used to connect to the SFTP server. | | sftphost | Required host to which you want to connect. |

Amazon S3

| field | Required Description | | ------------- | -------- ---------------------------------------------------------- | | path | Required path to file on the server, excluding the Amazon S3 bucket | | s3accessKey | Required Amazon S3 access key to use to connect to S3. | | s3secretKey | Required Amazon S3 secret key to use to connect to S3. | | s3bucket | Required Amazon S3 bucket to use. |

Oracle OCI Object Storage with S3 Compatibility

Please consult https://docs.oracle.com/en-us/iaas/Content/Object/Tasks/s3compatibleapi.htm for details on how to configure your Oracle OCI account so that the Amazon S3 Compatibility API can be used.

| field | Required Description | | -------------- | -------- ------------------------------------------------------------------------------------------- | | path | Required path to file on the server, excluding the Oracle OCI Object Storage bucket | | s3accessKey | Required Oracle Oracle OCI Object Storage access key to use to connect to Oracle OCI Object Storage. | | s3secretKey | Required Oracle Oracle OCI Object Storage secret key to use to connect to Oracle OCI Object Storage. | | s3bucket | Optional Oracle Oracle OCI Object Storage bucket to use. | | ocinamespace | Required Oracle Oracle OCI Object Storage namespace | | ociregion | Required Oracle Oracle OCI Object Storage region |

Google Cloud Storage

| field | Required Description | | -------------- | -------- ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | path | Required Path to file | | gcsaccessKey | Required GCS HMAC Access Key | | gcssecretKey | Required GCS HMAC Secret Key | | gcsbucket | Required GCS Bucket | | gcsproject | Optional GCS Project | | gcsmarker | Optional Optional marker parameter indicating where in the GCS bucket to begin listing. The list will only include keys that occur lexicographically after the marker. | | gcsdelimiter | Optional GCS File/Folder delimiter. / is used by default |

Microsoft Azure Blob Storage

| field | Required Description | | --------------------------------------------- | -------- ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | path | Required Path to file | | azure.credentials.connection.string | Optional Connection string that specifies credentials to authorize access to Azure Blob Storage. Use one of this, account key or SAS token. | | azure.account.name | Optional Name of the Azure account to use | | azure.credentials.sharedkey.account.key | Optional Account key that specifies credentials to authorize access to Azure Blob Storage. Use one of this, connection string or SAS token. | | azure.credentials.sharedkey.sas.token | Optional SAS token (account or service) that specifies credentials to authorize access to Azure Blob Storage. Use one of this, connection string or account key. | | azure.blob.container.name | Required Name of the Azure Blob Storage container that contains the blob | | azure.blob.inputstream.chunk.size.max.bytes | Optional Maximum size in bytes of each chunk of data when reading the blob contents chunk by chunk. Default: 4194304 | | azure.blob.path.delimiter | Optional String that separates elements of the path to the blob file. Default: / | | azure.blob.list.marker | Optional

Marker that specifies the beginning of the next page of a list of
Azure Blob Storage items to fetch from Azure. This property is used internally.

Note: only azure.blob.container.name is required. Credentials are optional — anonymous public read works when the container's access level is set to Container.

Amazon Redshift

| field | Required Description | | ---------- | -------- ------------------------- | | path | Required name of the table to use. | | rsuser | Required Redshift database user | | rspasswd | Required Redshift user password | | rsurl | Required Redshift connection url |

Spark Map Reduce

| field | Required Description | | ----------------------------------------------------------------------------------------- | -------- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | sparkmaster | Required address of the spark’s cluster master | | spark.job.process.jvm.heap.size.min | Optional Mininum JVM size used for the spark Driver process launched by the Jetty Rest Server. Default : 1024m | | spark.job.process.jvm.heap.size.max | Optional Maximum JVM size used for the spark Driver process launched by the Jetty Rest Server. Default: 2048m | | Any Spark parameters be used also like: spark.executor.memory or spark.executor.cores | Optional To pass such parameters, prefix them with ‘sml.sparkenv.’ as in the following examples: - sml.sparkenv.spark.executor.cores - sml.sparkenv.spark.cores.max | | spark.automl.sample.random.seed | Optional If AutoML is used, one can set the randomizer seed that will be used to select a random sample of tuple from the data source to be used to bootstrap the AutoML environement. |

Spark Map Reduce Data Source Type

JDBC

PostgreSQL and Amazon Redshift over JDBC

PostgreSQL and Amazon Redshift are reached through the jdbc data source type rather than a type of their own. Set jdbcprefix to jdbc:postgresql or jdbc:redshift to select this behaviour.

Browsing with no database selected lists databases as directories. A PostgreSQL connection URL with no database named defaults to the connecting user's own name, which usually does not exist, so the browser connects through a maintenance database instead — trying postgres first, then template1. The connecting user needs CONNECT permission on one of those two, or browsing fails.

Browsing with a database selected lists tables as files, and only the public schema is listed. Views are included, matching the behaviour of the MySQL listing.

Snowflake

SymetryML supports connecting to Snowflake data warehouses using Snowflake's Snowpark Container Services (CP3O) for distributed processing.

Modes. Setting snowflake.cp3o.compute.pool.instance.family puts the data source in CP3O mode, which distributes the work across a compute pool; leaving it unset selects Reader mode, which reads through a warehouse. The Required column above marks which fields each mode adds.

Locating the data. Supply either path in the exact form DB/SCHEMA/TABLE, or all three of snowflake.data.database, snowflake.data.schema and snowflake.data.table. A path with anything other than three non-empty segments — including a trailing / — is rejected with DS type 'snow': path must be in the format [DB/SCHEMA/TABLE]. Supplying neither is rejected with DS type 'snow': either 'path' or all of 'snowflake.data.database', 'snowflake.data.schema', 'snowflake.data.table' must be provided.

Note: snowflake.job.role is not a data source field. It is a server setting — see SymetryML REST Configuration.

Amazon EMR

Google Dataproc

Runs SymetryML processing on a Google Cloud Dataproc cluster, reading the data from Google Cloud Storage. Set type to dataproc.

Authentication resolves in three steps, stopping at the first that produces credentials:

1. dataproc.service.account.key.json on the data source.
2. The dataproc.gcp.credentials.json.path server setting.
3. Google Application Default Credentials — workload identity, GOOGLE_APPLICATION_CREDENTIALS, or the GCE metadata server.

This is why the service-account key is not a required field: when SymetryML runs on GCP it can be left blank and step 3 supplies the credentials. A failure at step 3 is reported as Dataproc ADC credentials unavailable: <message>.

Multi-file paths. A path that ends in /, or that contains * or ?, is treated as multi-file. For preview the reader resolves it to the first matching object — a directory-prefix preview shows one file, not the union of all of them.

The server-side settings that support Dataproc — including dataproc.listener.host, which must be changed from its default — are listed in SymetryML REST Configuration.

Additional CSV Options

You can specify additional parameters that describe the ‘type of csv files’. You can also add the following parameters to a data source to change how SymetryML parses your data:

Note: these options apply to file-based data sources only — the CSV, delimited and archive files read from SFTP, HTTP/HTTPS, S3, GCS, Azure Blob, OCI and local disk. They have no effect on database sources such as JDBC, Redshift and Snowflake, which read typed values directly from the result set and never pass them through CSV parsing.

Missing Values

When a value in a continuous column cannot be read as a number, SymetryML checks it against a list of tokens that mean "missing". A match is stored as NaN rather than failing the row. Matching is case-insensitive.

Additional tokens can be declared server-side with the rtlm.option.sml.nan.values setting, which takes a comma-separated list and defaults to -. Tokens added this way are also matched case-insensitively.

A number wrapped in parentheses is read as negative, so (1.5) is -1.5.

Additional Information on Spark S3 Data Source

SymetryML can leverage a spark cluster to speed up processing of large amounts of data significantly. Currently, your data must reside on Amazon S3. Depending on the size of your data, it may take more or less time for the job to start, as the Spark Cluster must compute the partitions of your data before starting the job. Consequently, if your data is very large, this may take a few minutes.

Please be sure to consult the Spark Installation Guide for additional Spark information.

Best practices for Spark S3 Data Source:

• Performance may vary depending on Amazon resource utilization when you run your job.
• Be sure all executor nodes in your cluster reside in the same Amazon EC2 placement group.

About Data Source Plugins (DSPlugins)

The SymetryML data source API allows you to create a new data source by the form of Java library (jar) that can be added the server. Instead of transforming data into CSV files, for example, you can write a DS plugin that reads the data natively.

Data Source Encryption

Data sources might contain sensitive information that should never be passed in the clear. To avoid having to use HTTPS for these services, the SymetryML REST API forces you to pass such information in encrypted form. This can be done easily, as each SymetryML secret key is also a 128-bit Advanced Encryption Standard (AES) secret key.

1. Create your DSInfo data structure and enter the appropriate information so that the SymetryML server can access it.
2. Extract the JSON string from that data structure.
3. Encrypt the JSON string representation using your SymetryML secret key:
   • Initialization vector in Base 64: LzM5QUtXZXWHm7HJ4wAePg==
   • Block cipher algorithm: AES/CBC/PKCS5Padding
4. Send the encrypted string as part of the body to any REST service that requests a DSInfo as the body.
5. The server decrypts the string using the client secret key and reconstructs the DSInfo.

Data Source Create

This API function creates a new data source.

URL

POST /symetry/rest/{cid}/dss/create [Body=DSInfo (encrypted)]

HTTP Responses

HTTP Response Entity

None.

Sample Request/Response

Please note that in the following example the BODY is encrypted. Refer to section Data Source Encryption for encryption details.

Request:
POST url="http://charm:8080/symetry/rest/c1/dss/create"

Body:
emUmJ7LXKaICPww/dKRoMR/Go6+B3ATTn+GwgE1/vcK9pN/mLoqGyKiMtGiTYMct4Gth1ElniKLXtmXfHfs7Rfn+QhJHZ+s00w2PBdbvYZIoF3My04H5XCboY21Fh4SkBhsxo+DhYuardN7R+uGphN/DSbiHRLIXe51HijSpfuq0fJuQYW0ccN4FM/B74LJccuDnbb+IouS9u/9rybKu/wsVbDVRTK/zZpWXyn4qtds=

Response:
{"statusCode":"CREATED","statusString":"DS created with name [ds1] for customer [c1]","values":{}}

Data Source Update

This API function update an existing data source.

URL

PUT /symetry/rest/{cid}/dss/{dsname} [Body=DSInfo (encrypted)]

HTTP Responses

HTTP Response Entity

None.

Sample Request/Response

Please note that in the following example the BODY is encrypted. Refer to section Data Source Encryption for encryption details.

Request:
PUT url="http://charm:8080/symetry/rest/c1/dss/aDataSourceName"

Body:
emUmJ7LXKaICPww/dKRoMR/Go6+B3ATTn+GwgE1/vcK9pN/mLoqGyKiMtGiTYMct4Gth1ElniKLXtmXfHfs7Rfn+QhJHZ+s00w2PBdbvYZIoF3My04H5XCboY21Fh4SkBhsxo+DhYuardN7R+uGphN/DSbiHRLIXe51HijSpfuq0fJuQYW0ccN4FM/B74LJccuDnbb+IouS9u/9rybKu/wsVbDVRTK/zZpWXyn4qtds=

Response:
{"statusCode":"OK","statusString":"DS updated with name [ds1] for customer [c1]","values":{}}

List Customer Data Sources

This API function returns all the data sources that belong to a user.

URL

GET /symetry/rest/{cid}/dss/

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
GET url="http://charm:8080/symetry/rest/c1/dss"

Response:=
{"statusCode":"OK","statusString":"OK","values":{"stringList":{"values":["Iris_rtlm-out-neil.csv:s3","Iris_rtlm.csv:s3","BigData11g_num.csv:s3","Smaato_Bids_20130812_CTR.csv:s3"]}}}

Delete Data Source

This API function deletes a data source.

URL

DELETE /symetry/rest/{cid}/dss/{dsname}

HTTP Responses

HTTP Response Entity

None

Data Source Information

This API function returns information about a data source. Because this information can be sensitive, the server encrypts it to ensure that the information will be returned safely, even when using HTTP. Use your SymetryML secret key to decrypt the information. For information about how to decrypt the response, see the section Data Source Encryption.

URL

GET /symetry/rest/{cid}/dss/{dsname}

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
GET url="http://charm:8080/symetry/rest/c1/dss/BigData11g_num.csv"

Response:
{"statusCode":"OK","statusString":"OK","values":{"dsinfo":"emUmJ7LXKaICPww/dKRoMRNJEMepL37Eq9CgZfhZPWj93mo3A+C8ucfIOGaPwwn2dip/JEuLFjUT/fjHjy18XKFnzFz5Ujp0WmS0uA4ssvAJwNPL6BvnsY6+a/lKa+c/q9/5tz5lr13N13I7OGAhuYhXYV+xb8oFZqsn+bH5spBXRb5u+oyEMXNKLCaNt3pzc/xCyW47KCwIi9V5iSA+fcJAWfetm9ZsIHNbI6utkxKrqrU5OfLmgriGAP++yQtutlGR7r/bKV1bRc8UDsgsXQg1HgoxHcKXCgsAFDFzqsJmZ/5/uQDc0ytc5Fk85GUx"}}

Data Source Browsing

This API function lists content about a remote data source directory.

URL

POST /symetry/rest/{cid}/projects/dsbrowse/ [Body=DSListingRequest (encrypted)]

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
POST url="http://charm:8080/symetry/rest/c1/projects/dsbrowse"

Body:
drhSjndw6G15pgevCsDqaSfjX9x3hMo+dNqd/MV943Dsd2rl2guhvq2qUhjEORcfKAEjHaoRZMKmSbQB6bcca2YT6HmUyRxuOG0wiKgGy0MOEq7+iIncbX4orpGr4rhro1Frw909Uy8qcWskaInQHJT4EGRPcwxvwInFlea39hsMkycFK4pKlTpanOUYgcv7

Response:
{"statusCode":"OK","statusString":"OK","values":{"dsdirectoryListing":{"ok":true,"dirs":["data2/","folder-a/","folder-datasets/","folder-demo/","folder-dev/","folder-dev-pub-http/","folder-dev-pub-https/","folder-docs/","folder-perf-all-reports/","folder-source/"]}}}

Fetching Sample Data 1

This API function fetches a data source sample by specifying the data source information as part of the request body. The response returns up to 128 lines. This REST call needs a request body that contains an encrypted DSInfo data structure (see the section Data Source Encryption).

URL

POST /symetry/rest/{cid}/dss/sample/preview [body=DSInfo encrypted]

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
POST url="http://charm:8080/symetry/rest/c1/dss/sample/preview

Body:
WTqUHBoXHbl+cMMjdc0zgjBP8e44G1os15V+I4GZgDOr1dX9uOfvY5uK9ZgC9yral9XC1ohD1W+UvkPlKR4dQT00EgCdS2UPgZz2NwwooHOM+KY1Ysf5qZlkFKiOkxwoWH/mr3mvvgdTUpZS8zrDJk3gwsavFT5fe0J2lTR33F1OH7FwxP4qs5nzRbVz546l

Response:
{"statusCode":"OK","statusString":"OK","values":{"dataframe":{"attributeNames":["adexchange","imp_width","imp_height","imp_btype","preference","pub","domain","site_base_url","category","device_ip","device_country","device_dma","device_state","device_city","zip","device_carrier","language","device_os","device_make","device_model","device_osv","lat","lon","restriction_bcat","restriction_badv","position","gender","user_keyword","user_yob","user_age","view_count","campaign","creative","creative_type","winner","bid_price","win_price","clicked","rtlm_ctr_score","rtdm","req_hour","req_day","category_count","bcat_count","badv_count"],"data":[["Smaato","320","50","","APP","New IT Solutions Ltd","","","Technology \u0026 Computing","157.55.32.83","US","","WASHINGTON","REDMOND","98052","","","Unknown","","","","47.67398834","-122.1215134","IAB7-28|IAB19-30|IAB22-1|IAB19-3|IAB17-18|IAB26|IAB25|IAB24|IAB9-9|IAB7-

(...)

,"attributeTypes":["S","C","C","L","S","L","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","L","S","S","S","S","S","S","B","S","S","S","B","C","C","B","S","B","C","C","B","C","B"]}}}

Fetching Sample Data 2

URL

GET /symetry/rest/{cid}/dss/{dsname}/sample

Query Parameters

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
GET url="http://charm:8080/symetry/rest/c1/dss/Smaato_Bids_20130812_CTR.csv/sample"

Response:
{"statusCode":"OK","statusString":"OK","values":{"dataframe":{"attributeNames":["adexchange","imp_width","imp_height","imp_btype","preference","pub","domain","site_base_url","category","device_ip","device_country","device_dma","device_state","device_city","zip","device_carrier","language","device_os","device_make","device_model","device_osv","lat","lon","restriction_bcat","restriction_badv","position","gender","user_keyword","user_yob","user_age","view_count","campaign","creative","creative_type","winner","bid_price","win_price","clicked","rtlm_ctr_score","rtdm","req_hour","req_day","category_count","bcat_count","badv_count"],"data":[["Smaato","320","50","","APP","New IT Solutions Ltd","","","Technology \u0026 Computing","157.55.32.83","US","","WASHINGTON","REDMOND","98052","","","Unknown","","","","47.67398834","-122.1215134","IAB7-28|IAB19-30|IAB22-1|IAB19-3|IAB17-18|IAB26|IAB25|IAB24|IAB9-9|IAB7-

(...)

,"attributeTypes":["S","C","C","L","S","L","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","S","L","S","S","S","S","S","S","B","S","S","S","B","C","C","B","S","B","C","C","B","C","B"]}}}

SymetryML Project Data Source API

Add Data Source to a SymetryML project

This API function lets you add a data source to a project.

URL

GET /symetry/rest/{cid}/projects/{pid}/addds/{dsname}

HTTP Responses

HTTP Response Entity

None

Remove Data Source from a SymetryML project

This API function lets you remove a data source from a project.

URL

GET /symetry/rest/{cid}/projects/{pid}/detachds/{dsname}

HTTP Responses

HTTP Response Entity

None

Learning Data from a Data Source

This API function lets you learn from a previously created data source.

URL

GET /symetry/rest/{cid}/projects/{pid}/dss/{dsname}/learn

HTTP Responses

HTTP Response Entity

None

Sample Request/Response

Request 1 create ds:
POST url=http://charm:8080/symetry/rest/c1/dss/create

Body 1:
{ENCRYPTED}

Request 2 learn ds:
POST url=http://charm:8080/symetry/rest/c1/projects/r1/dss/irisds/learn

BODY 2
{"errorHandling":1,"attributeNames":["sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"data":[],"attributeTypes":["C","C","C","C","B","B","B","B","B","B","B","B","B","B","B"]}

Response 2:
{"statusCode":"ACCEPTED","statusString":"Job Created","values":{}}

Response 2 Header:
Location: http://charm:8080/symetry/rest/c1/jobs/4

Job 3 Request:
GET url="http://charm:8080/symetry/rest/c1/jobs/4

Job 3 Response:
{"statusCode":"OK","statusString":"Job is finished","values":{"smlInfo":{"pid":"r1","isDirty":true,"modelsList":[],"modelTypeList":[],"attributeNames":["sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"attributeIndexes":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14],"attributeTypes":["C","C","C","C","B","B","B","B","B","B","B","B","B","B","B"],"modelAssessments":{},"modelPredictions":{},"hash":-1,"categorySeparator":"$","type":"cpu","creationDate":1488220762857,"lastModificationDate":1488226030143,"loaded":true,"persisted":true}}}

Forgetting Data from a Data Source

This API function lets you forget data from a previously created data source.

URL

GET /symetry/rest/{cid}/projects/{pid}/dss/{dsname}/forget

HTTP Responses

HTTP Response Entity

None

Sample Request/Response

See the section previous Learn endpoint and replace learn with forget in the url.

Prediction Based on a Data Source

After a model is built, you can use this API function to make predictions using a data source. This action can be performed on very large files if they reside on Amazon S3. A prediction file is created that contains the original rows, along with additional prediction information based on the type of model used.

URL

Request:
POST /symetry/rest/{cid}/projects/{pid}/dss/predict/{modelid}?indsname={indsname}&outdsname={outdsname}&impute=false
Request Body:
{
  "attributeNames":[{input attributes names}],
  "attributeTypes":[{input attribute types}]
}

Query Parameters

HTTP Responses

HTTP Response Entity

None

Sample Request/Response

Request:
POST url="http://charm:8080/symetry/rest/c1/projects/irisTest/dss/predict/testLDA?indsname=dsin&outdsname=dsout&impute=false
Request Body:
{
  "attributeNames":["sepal_length","sepal_width","petal_length","petal_width"],
  "attributeTypes":["C","C","C","C"]
}

Response Header:
Location: http://charm:8080/symetry/rest/c1/jobs/2

Response:
{"statusCode":"ACCEPTED","statusString":"Job Created","values":{}}

Job Request:
GET url="http://charm:8080/symetry/rest/c1/jobs/2

Job Response:
{"statusCode":"OK","statusString":"Job is finished","values":{"dataframe":{"attributeNames":["resZ_Result","res_Result","normZ_Result","sepal_length","sepal_width","petal_length","petal_width","sepal_lengt_b1","sepal_lengt_b2","sepal_width_b1","sepal_width_b2","petal_length_b1","petal_length_b2","petal_width_b1","petal_width_b2","Iris_setosa","Iris_versicolor","Iris_virginica"],"data":[["-0.4559922893809558","1","0.2929126883636288","4.3","3","1.1","0.1","1","0","0","1","1","0","1","0","1","0","0"],["-0.5225734165408404","1","0.2965158290613048","

(…)

["-2.6526076600477295","1","0.4232667751179995","6.9","3.1","5.4","2.1","0","1","0","1","0","1","0","1","0","0","1"]]}}}

Encoder Data Source API

Updating an Encoder with a Data Source

This API function updates an Encoder with data from a data source.

URL

GET /symetry/rest/{cid}/encoders/{encodername}/learnds [body = DSInfo (encrypted)]

HTTP Responses

HTTP Response Entity

None

Sample Request/Response

Request:
POST url="http://charm:8080/symetry/rest/c1/encoders/enctest/learnds

Request Body:
emUmJ7LXKaICPww/dKRoMZo4y++JZksKFU3xx80mZ0E83ED6eCXhZ7PR1jzizIQg/uH19jVkSUWnDxul9+NIqZe7a9Yz5a8gW1kpRHJ7SyOYoo7oV90atrkMcZ73Jj0FRn53P81t8Q+7fCTwMYRfD52hjCbwtvdwFCbPbhPo1c9CAk9QLhtAFLDWVqblBeBXTPR8/0zfKJWtwj30Yr0gwqeTI3+BMOMEvH28WFWXo+wBdgkMBpXJsQH/zbPvVdCp9P+BJWp/E1Ju2hUzPO5c2k3/Dmqv3xWNmsQzEJWNMFTJnBG33hTUyTr/+j87NsM2e1luWvf1KaNzaatjS1ZZ9AQCp7gv48vMVrzEHs4ePiOFq5t0UElAu1kzFerhtNxEFmC9A90Gjt3FSLrEGgx5emZ0uogJa6m4nufbQnKUrUZL1sILmLfReZOqnPKp59HqjiiczpkPQ2vPfhmFpaM2RYaEHbqSpMGCnBO9Axij9ExPjR4X9aqHOzKw2yBba4Em

Response Header:
Location: http://charm:8080/symetry/rest/c1/jobs/4

Response:
{"statusCode":"ACCEPTED","statusString":"Job Created","values":{}}

Job Request:
GET url="http://charm:8080/symetry/rest/c1/jobs/4

Job Response:
{"statusCode":"OK","statusString":"Job is finished","values":{}}

Listing a Data Source Used by an Encoder

This API function lists the data source(s) that were used to update an Encoder.

URL

GET /symetry/rest/{cid}/encoders/{encodername}/dss

HTTP Responses

HTTP Response Entity

Sample Request/Response

Request:
GET url="http://charm:8080/symetry/rest/c1/encoders/enctest/dss

Response:
{"statusCode":"OK","statusString":"OK","values":{"stringList":{"values":["s12"]}}}

Data Source Job Status

When invoking a JobStatus for a job that was initiated for learning, forgetting, or making predictions based on a data source, the response might contain an entity. The following sections describe these cases.

Learning a Data Source

HTTP Responses

Forgetting a Data Source

HTTP Responses

Prediction Based on Data Source

HTTP Responses