Export profiles to databases

This type of pipeline exports profiles from the workspace's data warehouse to a database table. It is available only for destination database connections.

Create pipeline

Create a destination pipeline that exports profiles to a database.

Request

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • connection

    int Required

    The ID of the connection to which the profiles will be written. It must be a destination database.

  • target

    string Required

    The entity on which the pipeline operates, which must be "User" in order to create a pipeline that exports profiles.

    Possible values: "User".

  • enabled

    boolean

    Indicates if the pipeline is enabled once created.

  • filter

    nullable object

    The filter applied to the workspace's profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • tableName

    string Required

    The name of the table where the profiles will be exported.

    Must be a non-empty string with a maximum of 1024 characters.

  • tableKey

    string Required

    The name of the table column that contains a value identifying a profile within the table. This column must be included in the output schema.

    Typically, this is the column used as the table's primary key. However, it can also be a column with a unique constraint, or one that is guaranteed to contain only unique values.

    If a row with the same value in this column already exists, it will be updated; otherwise, a new row will be created for the exported profile.

    The type of this column must match one of the following Krenalis types: string, int, or uuid.

  • transformation

    object Required

    The mapping or function responsible for transforming profiles into database users.

    One of either a mapping or a function must be provided, but not both. The one that is not provided can be either missing or set to null.

    • transformation.mapping

      nullable object with string values Conditionally Required

      The transformation mapping. A key represents the name of a column in the database table, and its corresponding value is an expression. This expression can reference property paths from the profile schema.

    • transformation.function

      nullable object Conditionally Required

      A JavaScript or Python function that, given a profile, returns a user record to be written to the database table.

      • transformation.function.source

        string Required

        The source code of the JavaScript or Python function.

        Must be a non-empty string with a maximum of 50000 characters.

      • transformation.function.language

        string

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean Required

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string Required

        The paths of the properties that will be passed to the function. At least one path must be present.

      • transformation.function.outPaths

        array of string Required

        The paths of the properties that may be returned by the function. At least one path must be present.

  • inSchema

    schema Required

    The schema of the properties used in the filter and the input properties within the transformation.

    It should be a subset of the profile schema.

  • outSchema

    schema Required

    The schema for the table key property and the output properties within the transformation.

    In addition to the table key property, it must contain at least one other output property.

    When exporting profiles to a database, this should be a subset of the database table schema.

Response

  • id

    int

    The ID of the pipeline.
POST /v1/pipelines
curl https://example.com/v1/pipelines \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Export users on database",
"connection": 230527183,
"target": "User",
"enabled": true,
"filter": {
"logical": "and",
"conditions": [
{
"property": "address.country",
"operator": "is",
"values": [
"US"
]
}
]
},
"tableName": "customers",
"tableKey": "email",
"transformation": {
"mapping": {
"firstname": "first_name",
"lastname": "last_name",
"email": "email"
}
},
"inSchema": {
"kind": "object",
"properties": [
{
"name": "first_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "First name"
},
{
"name": "last_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "Last name"
},
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 254
},
"readOptional": true,
"description": "Email"
},
{
"name": "address",
"type": {
"kind": "object",
"properties": [
{
"name": "country",
"type": {
"kind": "string",
"maxLength": 2
},
"readOptional": true,
"description": "Country"
}
]
},
"readOptional": true,
"description": "Address"
}
]
},
"outSchema": {
"kind": "object",
"properties": [
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 120
},
"createRequired": true
},
{
"name": "firstname",
"type": {
"kind": "string",
"maxLength": 100
},
"nullable": true
},
{
"name": "lastname",
"type": {
"kind": "string",
"maxLength": 100
},
"nullable": true
}
]
}
}'
Response
{
  "id": 705981339
}
Errors
404
workspace does not exist
422
connection does not exist
422
connector does not exist
422
transformation language is not supported

Update pipeline

Update a destination pipeline that exports profiles to a database.

Request

  • :id

    int Required

    The ID of the destination database pipeline that exports profiles.

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • enabled

    boolean

    Indicates if the pipeline is enabled. Use the Set status endpoint to change only the pipeline's status.

  • filter

    nullable object

    The filter applied to the workspace's profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • tableName

    string Required

    The name of the table where the profiles will be exported.

    Must be a non-empty string with a maximum of 1024 characters.

  • tableKey

    string Required

    The name of the table column that contains a value identifying a profile within the table. This column must be included in the output schema.

    Typically, this is the column used as the table's primary key. However, it can also be a column with a unique constraint, or one that is guaranteed to contain only unique values.

    If a row with the same value in this column already exists, it will be updated; otherwise, a new row will be created for the exported profile.

    The type of this column must match one of the following Krenalis types: string, int, or uuid.

  • transformation

    object Required

    The mapping or function responsible for transforming profiles into database users.

    One of either a mapping or a function must be provided, but not both. The one that is not provided can be either missing or set to null.

    • transformation.mapping

      nullable object with string values Conditionally Required

      The transformation mapping. A key represents the name of a column in the database table, and its corresponding value is an expression. This expression can reference property paths from the profile schema.

    • transformation.function

      nullable object Conditionally Required

      A JavaScript or Python function that, given a profile, returns a user record to be written to the database table.

      • transformation.function.source

        string Required

        The source code of the JavaScript or Python function.

        Must be a non-empty string with a maximum of 50000 characters.

      • transformation.function.language

        string

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean Required

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string Required

        The paths of the properties that will be passed to the function. At least one path must be present.

      • transformation.function.outPaths

        array of string Required

        The paths of the properties that may be returned by the function. At least one path must be present.

  • inSchema

    schema Required

    The schema of the properties used in the filter and the input properties within the transformation.

    It should be a subset of the profile schema.

  • outSchema

    schema Required

    The schema for the table key property and the output properties within the transformation.

    In addition to the table key property, it must contain at least one other output property.

    When exporting profiles to a database, this should be a subset of the database table schema.

Response

No response.
PUT /v1/pipelines/:id
curl -X PUT https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Export users on database",
"enabled": true,
"filter": {
"logical": "and",
"conditions": [
{
"property": "address.country",
"operator": "is",
"values": [
"US"
]
}
]
},
"tableName": "customers",
"tableKey": "email",
"transformation": {
"mapping": {
"firstname": "first_name",
"lastname": "last_name",
"email": "email"
}
},
"inSchema": {
"kind": "object",
"properties": [
{
"name": "first_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "First name"
},
{
"name": "last_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "Last name"
},
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 254
},
"readOptional": true,
"description": "Email"
},
{
"name": "address",
"type": {
"kind": "object",
"properties": [
{
"name": "country",
"type": {
"kind": "string",
"maxLength": 2
},
"readOptional": true,
"description": "Country"
}
]
},
"readOptional": true,
"description": "Address"
}
]
},
"outSchema": {
"kind": "object",
"properties": [
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 120
},
"createRequired": true
},
{
"name": "firstname",
"type": {
"kind": "string",
"maxLength": 100
},
"nullable": true
},
{
"name": "lastname",
"type": {
"kind": "string",
"maxLength": 100
},
"nullable": true
}
]
}
}'
Errors
404
workspace does not exist
404
pipeline does not exist
422
transformation language is not supported

Get pipeline

Get a destination pipeline that exports profiles to a database.

Request

  • :id

    int Required

    The ID of the destination database pipeline that exports profiles.

Response

  • id

    int

    The ID of the destination database pipeline that exports profiles.

  • name

    string

    The pipeline's name.

    It is not longer than 60 characters.

  • connector

    string

    The code of the connection's connector.

  • connectorType

    string

    The type of the connection's connector. It is always "Database" when the pipeline exports profiles to a database.

    Possible values: "Application", "Database", "FileStorage", "MessageBroker", "SDK" or "Webhook".

  • connection

    int

    The ID of the connection to which the profiles will be exported. It is a destination database.

  • connectionRole

    string

    The role of the pipeline's connection. It is always "Destination" when the pipeline exports profiles to a database.

    Possible values: "Source" or "Destination".

  • target

    string

    The entity on which the pipeline operates. It is always "User" when the pipeline exports profiles to a database.

    Possible values: "User" or "Event".

  • enabled

    boolean

    Indicates if the pipeline is enabled.

  • filter

    nullable object

    The filter applied to the workspace's profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Possible values: "and" or "or".

    • filter.conditions

      array of object

      A filter's condition.

      • property

        string

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • tableName

    string

    The name of the table where the profiles are exported.

    It is not longer than 1024 characters.

  • tableKey

    string

    The name of the table column that contains a value identifying a user within the table.

    Typically, this is the column used as the table's primary key. However, it can also be a column with a unique constraint, or one that is guaranteed to contain only unique values.

    If a row with the same value in this column already exists, it will be updated; otherwise, a new row will be created for the exported profile.

    The type of this column must match one of the following Krenalis types: string, int, or uuid.

  • transformation

    object

    The mapping or function responsible for transforming profiles into database users.

    One of either a mapping or a function is present, but not both. The one that is not present is null.

    • transformation.mapping

      nullable object with string values

      The transformation mapping. A key represents the name of a column in the database table, and its corresponding value is an expression. This expression can reference property paths from the profile schema.

    • transformation.function

      nullable object

      A JavaScript or Python function that, given a profile, returns a user record to be written to the database table.

      • transformation.function.source

        string

        The source code of the JavaScript or Python function.

        It is not longer than 50000 characters.

      • transformation.function.language

        string

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string

        The paths of the properties that will be passed to the function. It contains at least one property path.

      • transformation.function.outPaths

        array of string

        The paths of the properties that may be returned by the function. It contains at least one property path.

  • inSchema

    schema

    The schema of the properties used in the filter and the input properties within the transformation.

  • outSchema

    schema

    The schema of the table key property and the output properties within the transformation.

  • running

    boolean

    Indicates if the pipeline is running.

  • scheduleStart

    nullable int

    The start time of the schedule in minutes, counting from 00:00. It specifies the minute when the first scheduled run of the day begins. Subsequent runs occur based on the interval defined by the scheduler period. If the scheduler is disabled, this value is null.

  • schedulePeriod

    nullable string

    The schedule period, which determines how often the export runs automatically. If it is null, the scheduler is disabled, and no automatic runs will occur.

    To change the schedule period, use the Set schedule period endpoint.

    Possible values: "5m", "15m", "30m", "1h", "2h", "3h", "6h", "8h", "12h" or "24h".
GET /v1/pipelines/:id
curl https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx"
Response
{
  "id": 705981339,
  "name": "Export users on database",
  "connector": "postgresql",
  "connectorType": "Database",
  "connection": 1371036433,
  "connectionRole": "Destination",
  "target": "User",
  "enabled": true,
  "filter": {
    "logical": "and",
    "conditions": [
      {
        "property": "address.country",
        "operator": "is",
        "values": [
          "US"
        ]
      }
    ]
  },
  "tableName": "customers",
  "tableKey": "email",
  "transformation": {
    "mapping": {
      "firstName": "first_name",
      "lastName": "last_name",
      "email": "email"
    }
  },
  "inSchema": {
    "kind": "object",
    "properties": [
      {
        "name": "first_name",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "readOptional": true,
        "description": "First name"
      },
      {
        "name": "last_name",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "readOptional": true,
        "description": "Last name"
      },
      {
        "name": "email",
        "type": {
          "kind": "string",
          "maxLength": 254
        },
        "readOptional": true,
        "description": "Email"
      },
      {
        "name": "address",
        "type": {
          "kind": "object",
          "properties": [
            {
              "name": "country",
              "type": {
                "kind": "string",
                "maxLength": 2
              },
              "readOptional": true,
              "description": "Country"
            }
          ]
        },
        "readOptional": true,
        "description": "Address"
      }
    ]
  },
  "outSchema": {
    "kind": "object",
    "properties": [
      {
        "name": "email",
        "type": {
          "kind": "string",
          "maxLength": 120
        }
      },
      {
        "name": "firstname",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "nullable": true
      },
      {
        "name": "lastname",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "nullable": true
      }
    ]
  },
  "running": false,
  "scheduleStart": 15,
  "schedulePeriod": "1h"
}
Errors
404
workspace does not exist
404
pipeline does not exist