Export profiles to files

This type of pipeline exports profiles from the workspace's data warehouse to a newly created file. It is available only for destination file storage connections.

Create pipeline

Create a destination pipeline that exports profiles to a file.

Request

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • connection

    int Required

    The ID of the connection to which the file will be written. It must be a destination file storage connection.

  • target

    string Required

    The entity on which the pipeline operates, which must be "User" in order to create a pipeline that exports profiles.

    Possible values: "User".

  • enabled

    boolean

    Indicates if the pipeline is enabled once created.

  • format

    string Required

    The file format. It corresponds to the code of a file connector.

    Possible values: "csv", "excel", "parquet" or "json".

  • path

    string Required

    The file path relative to the root path defined in the file storage. Refer to the file storage documentation for details on the specific format.

    Must be a non-empty string with a maximum of 1024 characters.

  • sheet

    string Conditionally Required

    The sheet name. It can only be used with the Excel format, where it is required.

    When provided, it must have a length between 1 and 31 characters, not start or end with a single quote ', and cannot contain any of the following characters: *, /, :, ?, [, \, and ].

  • compression

    string

    The format used to compress the file. If not provided or empty, the file will remain uncompressed.

    Possible values: "", "Zip", "Gzip" or "Snappy".

  • orderBy

    string Required

    The property path by which profiles will be ordered in the exported file.

    Must be a non-empty string with a maximum of 1024 characters.

  • formatSettings

    nullable json

    The specific settings of the pipeline, which vary based on the file connector specified in the format field.

    Please refer to the page that documents the settings for each connector type.

  • filter

    nullable object

    The filter applied to the workspace profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • inSchema

    schema Required

    The schema of the properties in the filter, along with the other properties from the profile schema, to be exported into the file.

Response

  • id

    int

    The ID of the pipeline.
POST /v1/pipelines
curl https://example.com/v1/pipelines \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Newsletter Subscribers",
"connection": 230527183,
"target": "User",
"enabled": true,
"format": "excel",
"path": "subscribers.xlsx",
"sheet": "Sheet1",
"orderBy": "address.country",
"filter": {
"logical": "and",
"conditions": [
{
"property": "address.country",
"operator": "is",
"values": [
"US"
]
}
]
},
"inSchema": {
"kind": "object",
"properties": [
{
"name": "address",
"type": {
"kind": "object",
"properties": [
{
"name": "country",
"type": {
"kind": "string",
"maxLength": 2
},
"readOptional": true,
"description": "Country"
}
]
},
"readOptional": true,
"description": "Address"
}
]
}
}'
Response
{
  "id": 285017124
}
Errors
404
workspace does not exist
422
connection does not exist
422
format does not exist
422
format settings are not valid

Update pipeline

Update a destination pipeline that exports profiles to a file.

Request

  • :id

    int Required

    The ID of the destination file pipeline.

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • enabled

    boolean

    Indicates if the pipeline is enabled. Use the Set status endpoint to change only the pipeline's status.

  • path

    string Required

    The file path relative to the root path defined in the file storage. Refer to the file storage documentation for details on the specific format.

    Must be a non-empty string with a maximum of 1024 characters.

  • sheet

    string Conditionally Required

    The sheet name. It can only be used with the Excel format, where it is required.

    When provided, it must have a length between 1 and 31 characters, not start or end with a single quote ', and cannot contain any of the following characters: *, /, :, ?, [, \, and ].

  • compression

    string

    The format used to compress the file. If not provided or empty, the file will remain uncompressed.

    Possible values: "", "Zip", "Gzip" or "Snappy".

  • orderBy

    string Required

    The property path by which profiles will be ordered in the exported file.

    Must be a non-empty string with a maximum of 1024 characters.

  • format

    string Required

    The file format. It corresponds to the code of a file connector.

    Possible values: "csv", "excel", "parquet" or "json".

  • filter

    nullable object

    The filter applied to the workspace profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • inSchema

    schema Required

    The schema of the properties in the filter, along with the other properties from the profile schema, to be exported into the file.

Response

No response.
PUT /v1/pipelines/:id
curl -X PUT https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Newsletter Subscribers",
"enabled": true,
"path": "subscribers.xlsx",
"sheet": "Sheet1",
"orderBy": "address.country",
"format": "excel",
"filter": {
"logical": "and",
"conditions": [
{
"property": "address.country",
"operator": "is",
"values": [
"US"
]
}
]
},
"inSchema": {
"kind": "object",
"properties": [
{
"name": "address",
"type": {
"kind": "object",
"properties": [
{
"name": "country",
"type": {
"kind": "string",
"maxLength": 2
},
"readOptional": true,
"description": "Country"
}
]
},
"readOptional": true,
"description": "Address"
}
]
}
}'
Errors
404
workspace does not exist
404
pipeline does not exist
422
format does not exist
422
format settings are not valid

Get pipeline

Get a destination pipeline that exports profiles to a file.

Request

  • :id

    int Required

    The ID of the destination file pipeline.

Response

  • id

    int

    The ID of the destination file pipeline.

  • name

    string

    The pipeline's name.

    It is not longer than 60 characters.

  • connector

    string

    The code of the connection's connector.

  • connectorType

    string

    The type of the connection's connector. It is always "FileStorage" when the pipeline exports profiles to a file.

    Possible values: "Application", "Database", "FileStorage", "MessageBroker", "SDK" or "Webhook".

  • connection

    int

    The ID of the connection to which the file is written. It is a destination file storage.

  • connectionRole

    string

    The role of the pipeline's connection. It is always "Destination" when the pipeline exports profiles to a file.

    Possible values: "Source" or "Destination".

  • target

    string

    The entity on which the pipeline operates. It is always "User" for a pipeline that exports profiles.

    Possible values: "User" or "Event".

  • enabled

    boolean

    Indicates if the pipeline is enabled.

  • format

    string

    The file format. It corresponds to the code of a file connector.

    Possible values: "csv", "excel", "parquet" or "json".

  • path

    string

    The file path relative to the root path defined in the file storage. Refer to the file storage documentation for details on the specific format.

    It is not longer than 1024 characters.

  • sheet

    nullable string

    The name of the sheet. It is null if the format is not "excel".

  • compression

    string

    The format used to compress the file. If empty, the file will remain uncompressed.

    Possible values: "", "Zip", "Gzip" or "Snappy".

  • orderBy

    string

    The property path by which profiles will be ordered in the exported file.

    It is not longer than 1024 characters.

  • filter

    nullable object

    The filter applied to the workspace profiles. If it's not null, only the profiles that match the filter will be included.

    See the filters documentation for more details.

    • filter.logical

      string Possible values: "and" or "or".

    • filter.conditions

      array of object

      A filter's condition.

      • property

        string

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • inSchema

    schema

    The input schema.

  • running

    boolean

    Indicates if the pipeline is running.

  • scheduleStart

    nullable int

    The start time of the schedule in minutes, counting from 00:00. It specifies the minute when the first scheduled run of the day begins. Subsequent runs occur based on the interval defined by the scheduler period. If the scheduler is disabled, this value is null.

  • schedulePeriod

    nullable string

    The schedule period, which determines how often the export runs automatically. If it is null, the scheduler is disabled, and no automatic runs will occur.

    To change the schedule period, use the Set schedule period endpoint.

    Possible values: "5m", "15m", "30m", "1h", "2h", "3h", "6h", "8h", "12h" or "24h".
GET /v1/pipelines/:id
curl https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx"
Response
{
  "id": 705981339,
  "name": "Newsletter Subscribers",
  "connector": "s3",
  "connectorType": "FileStorage",
  "connection": 1371036433,
  "connectionRole": "Destination",
  "target": "User",
  "enabled": true,
  "format": "json",
  "path": "subscribers.json",
  "sheet": null,
  "compression": "Gzip",
  "orderBy": "customer_id",
  "filter": {
    "logical": "and",
    "conditions": [
      {
        "property": "address.country",
        "operator": "is",
        "values": [
          "US"
        ]
      }
    ]
  },
  "inSchema": {
    "kind": "object",
    "properties": [
      {
        "name": "address",
        "type": {
          "kind": "object",
          "properties": [
            {
              "name": "country",
              "type": {
                "kind": "string",
                "maxLength": 2
              },
              "readOptional": true,
              "description": "Country"
            }
          ]
        },
        "readOptional": true,
        "description": "Address"
      }
    ]
  },
  "running": false,
  "scheduleStart": 15,
  "schedulePeriod": "1h"
}
Errors
404
workspace does not exist
404
pipeline does not exist