Import users from events

This type of pipeline imports users into the workspace's data warehouse from events received from websites, mobile apps, backend systems, and connected devices. It is available only for source SDK and webhook connections.

Create pipeline

Create a source pipeline that imports users from events.

Request

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • connection

    int Required

    The ID of the connection from which the events are received. It must be a source SDK or webhook connection.

  • target

    string Required

    The entity on which the pipeline operates, which must be "User" in order to create a pipeline that imports users.

    Possible values: "User".

  • enabled

    boolean

    Indicates if the pipeline is enabled once created.

  • filter

    nullable object

    The filter applied to the events. If it's not null, only the users of events that match the filter will be imported.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • transformation

    nullable object

    The mapping or function responsible for transforming incoming events into identities linked to the pipeline. Once the identity resolution process is complete, the identities associated with all pipelines are merged into profiles.

    • If no transformation is provided (or if it's set to null), the imported identity will not have any properties.
    • If a mapping or function is provided, only one of them should be specified. The other must either be absent or set to null.

    In any case, the imported identity will always include an Anonymous ID and, if available, a User ID.

    • transformation.mapping

      nullable object with string values Conditionally Required

      The transformation mapping. A key represents a property path in the profile schema, and its corresponding value is an expression. This expression can reference property paths from the event schema.

    • transformation.function

      nullable object Conditionally Required

      The transformation function. A JavaScript or Python function that given an event, returns an identity.

      • transformation.function.source

        string Required

        The source code of the JavaScript or Python function.

        Must be a non-empty string with a maximum of 50000 characters.

      • transformation.function.language

        string Required

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string Required

        The paths of the properties that will be passed to the function. At least one path must be present.

      • transformation.function.outPaths

        array of string Required

        The paths of the properties that may be returned by the function. At least one path must be present.

  • outSchema

    nullable schema Conditionally Required

    The schema for the output properties of the transformation. It is required if a transformation is present.

    When importing users from events, this should be a subset of the profile schema.

Response

  • id

    int

    The ID of the pipeline.
POST /v1/pipelines
curl https://example.com/v1/pipelines \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Ingest users",
"connection": 230527183,
"target": "User",
"enabled": true,
"filter": {
"logical": "and",
"conditions": [
{
"property": "type",
"operator": "is",
"values": [
"identify"
]
}
]
},
"transformation": {
"mapping": {
"email": "traits.email",
"first_name": "traits.firstName",
"last_name": "traits.lastName"
}
},
"outSchema": {
"kind": "object",
"properties": [
{
"name": "first_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "First name"
},
{
"name": "last_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "Last name"
},
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 254
},
"readOptional": true,
"description": "Email"
}
]
}
}'
Response
{
  "id": 705981339
}
Errors
404
workspace does not exist
422
connection does not exist
422
connector does not exist

Update pipeline

Update a source pipeline that imports users from events.

Request

  • :id

    int Required

    The ID of the source SDK or webhook pipeline.

  • name

    string Required

    The pipeline's name.

    Must be a non-empty string with a maximum of 60 characters.

  • enabled

    boolean

    Indicates if the pipeline is enabled. Use the Set status endpoint to change only the pipeline's status.

  • filter

    nullable object

    The filter applied to the events. If it's not null, only the users of events that match the filter will be imported.

    See the filters documentation for more details.

    • filter.logical

      string Required Possible values: "and" or "or".

    • filter.conditions

      array of object Required

      A filter's condition.

      • property

        string Required

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string Required

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • transformation

    nullable object

    The mapping or function responsible for transforming incoming events into identities linked to the pipeline. Once the identity resolution process is complete, the identities associated with all pipelines are merged into profiles.

    • If no transformation is provided (or if it's set to null), the imported identity will not have any properties.
    • If a mapping or function is provided, only one of them should be specified. The other must either be absent or set to null.

    In any case, the imported identity will always include an Anonymous ID and, if available, a User ID.

    • transformation.mapping

      nullable object with string values Conditionally Required

      The transformation mapping. A key represents a property path in the profile schema, and its corresponding value is an expression. This expression can reference property paths from the event schema.

    • transformation.function

      nullable object Conditionally Required

      The transformation function. A JavaScript or Python function that given an event, returns an identity.

      • transformation.function.source

        string Required

        The source code of the JavaScript or Python function.

        Must be a non-empty string with a maximum of 50000 characters.

      • transformation.function.language

        string Required

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string Required

        The paths of the properties that will be passed to the function. At least one path must be present.

      • transformation.function.outPaths

        array of string Required

        The paths of the properties that may be returned by the function. At least one path must be present.

  • outSchema

    nullable schema Conditionally Required

    The schema for the output properties of the transformation. It is required if a transformation is present.

    When importing users from events, this should be a subset of the profile schema.

Response

No response.
PUT /v1/pipelines/:id
curl -X PUT https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx" \
--json '{
"name": "Ingest users",
"enabled": true,
"filter": {
"logical": "and",
"conditions": [
{
"property": "type",
"operator": "is",
"values": [
"identify"
]
}
]
},
"transformation": {
"mapping": {
"email": "traits.email",
"first_name": "traits.firstName",
"last_name": "traits.lastName"
}
},
"outSchema": {
"kind": "object",
"properties": [
{
"name": "first_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "First name"
},
{
"name": "last_name",
"type": {
"kind": "string",
"maxLength": 100
},
"readOptional": true,
"description": "Last name"
},
{
"name": "email",
"type": {
"kind": "string",
"maxLength": 254
},
"readOptional": true,
"description": "Email"
}
]
}
}'
Errors
404
workspace does not exist
404
pipeline does not exist

Get pipeline

Get a source pipeline that imports users from events.

Request

  • :id

    int Required

    The ID of the source SDK or webhook pipeline.

Response

  • id

    int

    The ID of the source SDK or webhook pipeline.

  • name

    string

    The pipeline's name.

    It is not longer than 60 characters.

  • connector

    string

    The code of the connection's connector.

  • connectorType

    string

    The type of the connection's connector. It is always "SDK" or "Webhook" when the pipeline imports users from events.

    Possible values: "Application", "Database", "FileStorage", "MessageBroker", "SDK" or "Webhook".

  • connection

    int

    The ID of the connection from which the events are received. It is a source SDK or webhook connection.

  • connectionRole

    string

    The role of the pipeline's connection. It is always "Source" when the pipeline imports users from events.

    Possible values: "Source" or "Destination".

  • target

    string

    The entity on which the pipeline operates. It is always "User" when the pipeline imports users from events.

    Possible values: "User" or "Event".

  • enabled

    boolean

    Indicates if the pipeline is enabled.

  • filter

    nullable object

    The filter applied to the events. If it's not null, only the users of events that match the filter will be imported.

    See the filters documentation for more details.

    • filter.logical

      string Possible values: "and" or "or".

    • filter.conditions

      array of object

      A filter's condition.

      • property

        string

        The name or path of the property. If the property has a json type, it can include a json path.

      • operator

        string

        The condition's operator. The allowed values depend on the property's type.

        Possible values: "is", "is not", "is less than", "is less than or equal to", "is greater than", "is greater than or equal to", "is between", "is not between", "contains", "does not contain", "is one of", "is not one of", "starts with", "ends with", "is before", "is on or before", "is after", "is on or after", "is true", "is false", "is empty", "is not empty", "is null", "is not null", "exists" or "does not exist".

      • values

        array of string

        The values the operator applies to, if any. These depend on both the operator and the property's type, including whether they're present and how many there are.

  • transformation

    nullable object

    The mapping or function responsible for transforming incoming events into user identities linked to the pipeline. Once the identity resolution process is complete, the user identities associated with all pipelines are merged into unified users.

    • If no transformation is present, it is null.
    • If a transformation is present, either a mapping or a function will be present, but not both. The one that is not present it is null.

    • transformation.mapping

      nullable object with string values

      The transformation mapping. A key represents a property path in the profile schema, and its corresponding value is an expression. This expression can reference property paths from the event schema.

    • transformation.function

      nullable object

      The transformation function. A JavaScript or Python function that given an event, returns an identity.

      • transformation.function.source

        string

        The source code of the JavaScript or Python function.

        It is not longer than 50000 characters.

      • transformation.function.language

        string

        The language of the function.

        Possible values: "JavaScript" or "Python".

      • transformation.function.preserveJSON

        boolean

        Specifies whether JSON values are passed to and returned from the function as strings, keeping their original format without any encoding or decoding.

      • transformation.function.inPaths

        array of string

        The paths of the properties that will be passed to the function. It contains at least one property path.

      • transformation.function.outPaths

        array of string

        The paths of the properties that may be returned by the function. It contains at least one property path.

  • inSchema

    schema

    The schema for the properties used in the filter and any input properties for the transformation.

  • outSchema

    nullable schema

    The schema for the output properties of the transformation. If no transformation is present, it is null.
GET /v1/pipelines/:id
curl https://example.com/v1/pipelines/705981339 \
-H "Authorization: Bearer api_xxxxxxx"
Response
{
  "id": 705981339,
  "name": "Ingest users",
  "connector": "java",
  "connectorType": "SDK",
  "connection": 1371036433,
  "connectionRole": "Source",
  "target": "User",
  "enabled": true,
  "filter": {
    "logical": "and",
    "conditions": [
      {
        "property": "type",
        "operator": "is",
        "values": [
          "identify"
        ]
      }
    ]
  },
  "transformation": {
    "function": {
      "source": "const transform = (event) => { ... }",
      "language": "JavaScript",
      "preserveJSON": false,
      "inPaths": [
        "traits.firstName",
        "traits.lastName"
      ],
      "outPaths": [
        "first_name",
        "last_name"
      ]
    }
  },
  "inSchema": {
    "kind": "object",
    "properties": [
      {
        "name": "connectionId",
        "type": {
          "kind": "int",
          "bitSize": 32
        },
        "description": "Connection ID"
      },
      {
        "name": "anonymousId",
        "type": {
          "kind": "string"
        },
        "description": "Anonymous ID"
      },
      {
        "name": "channel",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Channel"
      },
      {
        "name": "category",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Category"
      },
      {
        "name": "context",
        "type": {
          "kind": "object",
          "properties": [
            {
              "name": "app",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "name",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "version",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Version"
                  },
                  {
                    "name": "build",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Build"
                  },
                  {
                    "name": "namespace",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Namespace"
                  }
                ]
              },
              "readOptional": true,
              "description": "App"
            },
            {
              "name": "browser",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "name",
                    "type": {
                      "kind": "string",
                      "values": [
                        "Chrome",
                        "Safari",
                        "Edge",
                        "Firefox",
                        "Samsung Internet",
                        "Opera",
                        "Other"
                      ]
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "other",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Other"
                  },
                  {
                    "name": "version",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Version"
                  }
                ]
              },
              "readOptional": true,
              "description": "Browser"
            },
            {
              "name": "campaign",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "name",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "source",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Source"
                  },
                  {
                    "name": "medium",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Medium"
                  },
                  {
                    "name": "term",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Term"
                  },
                  {
                    "name": "content",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Content"
                  }
                ]
              },
              "readOptional": true,
              "description": "Campaign"
            },
            {
              "name": "device",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "id",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Id"
                  },
                  {
                    "name": "advertisingId",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Advertising ID"
                  },
                  {
                    "name": "adTrackingEnabled",
                    "type": {
                      "kind": "boolean"
                    },
                    "readOptional": true,
                    "description": "Ad tracking enabled"
                  },
                  {
                    "name": "manufacturer",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Manufacturer"
                  },
                  {
                    "name": "model",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Model"
                  },
                  {
                    "name": "name",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "type",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Type"
                  },
                  {
                    "name": "token",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Token"
                  }
                ]
              },
              "readOptional": true,
              "description": "Device"
            },
            {
              "name": "ip",
              "type": {
                "kind": "ip"
              },
              "readOptional": true,
              "description": "IP"
            },
            {
              "name": "library",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "name",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "version",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Version"
                  }
                ]
              },
              "readOptional": true,
              "description": "Library"
            },
            {
              "name": "locale",
              "type": {
                "kind": "string"
              },
              "readOptional": true,
              "description": "Locale"
            },
            {
              "name": "location",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "city",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "City"
                  },
                  {
                    "name": "country",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Country"
                  },
                  {
                    "name": "latitude",
                    "type": {
                      "kind": "float",
                      "bitSize": 64
                    },
                    "readOptional": true,
                    "description": "Latitude"
                  },
                  {
                    "name": "longitude",
                    "type": {
                      "kind": "float",
                      "bitSize": 64
                    },
                    "readOptional": true,
                    "description": "Longitude"
                  },
                  {
                    "name": "speed",
                    "type": {
                      "kind": "float",
                      "bitSize": 64
                    },
                    "readOptional": true,
                    "description": "Speed"
                  }
                ]
              },
              "readOptional": true,
              "description": "Location"
            },
            {
              "name": "network",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "bluetooth",
                    "type": {
                      "kind": "boolean"
                    },
                    "readOptional": true,
                    "description": "Bluetooth"
                  },
                  {
                    "name": "carrier",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Carrier"
                  },
                  {
                    "name": "cellular",
                    "type": {
                      "kind": "boolean"
                    },
                    "readOptional": true,
                    "description": "Cellular"
                  },
                  {
                    "name": "wifi",
                    "type": {
                      "kind": "boolean"
                    },
                    "readOptional": true,
                    "description": "Wi-Fi"
                  }
                ]
              },
              "readOptional": true,
              "description": "Network"
            },
            {
              "name": "os",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "name",
                    "type": {
                      "kind": "string",
                      "values": [
                        "Android",
                        "Windows",
                        "iOS",
                        "macOS",
                        "Linux",
                        "Chrome OS",
                        "Other"
                      ]
                    },
                    "readOptional": true,
                    "description": "Name"
                  },
                  {
                    "name": "other",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Other"
                  },
                  {
                    "name": "version",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Version"
                  }
                ]
              },
              "readOptional": true,
              "description": "OS"
            },
            {
              "name": "page",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "path",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Path"
                  },
                  {
                    "name": "referrer",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Referrer"
                  },
                  {
                    "name": "search",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Search"
                  },
                  {
                    "name": "title",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Title"
                  },
                  {
                    "name": "url",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "URL"
                  }
                ]
              },
              "readOptional": true,
              "description": "Page"
            },
            {
              "name": "referrer",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "id",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "ID"
                  },
                  {
                    "name": "type",
                    "type": {
                      "kind": "string"
                    },
                    "readOptional": true,
                    "description": "Type"
                  }
                ]
              },
              "readOptional": true,
              "description": "Referrer"
            },
            {
              "name": "screen",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "width",
                    "type": {
                      "kind": "int",
                      "bitSize": 16
                    },
                    "readOptional": true,
                    "description": "Width"
                  },
                  {
                    "name": "height",
                    "type": {
                      "kind": "int",
                      "bitSize": 16
                    },
                    "readOptional": true,
                    "description": "Height"
                  },
                  {
                    "name": "density",
                    "type": {
                      "kind": "decimal",
                      "precision": 3,
                      "scale": 2
                    },
                    "readOptional": true,
                    "description": "Density"
                  }
                ]
              },
              "readOptional": true,
              "description": "Screen"
            },
            {
              "name": "session",
              "type": {
                "kind": "object",
                "properties": [
                  {
                    "name": "id",
                    "type": {
                      "kind": "int",
                      "bitSize": 64
                    },
                    "readOptional": true,
                    "description": "ID"
                  },
                  {
                    "name": "start",
                    "type": {
                      "kind": "boolean"
                    },
                    "readOptional": true,
                    "description": "Start"
                  }
                ]
              },
              "readOptional": true,
              "description": "Session"
            },
            {
              "name": "timezone",
              "type": {
                "kind": "string"
              },
              "readOptional": true,
              "description": "Timezone"
            },
            {
              "name": "userAgent",
              "type": {
                "kind": "string"
              },
              "readOptional": true,
              "description": "User agent"
            }
          ]
        },
        "readOptional": true,
        "description": "Context"
      },
      {
        "name": "event",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Event"
      },
      {
        "name": "groupId",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Group ID"
      },
      {
        "name": "messageId",
        "type": {
          "kind": "string"
        },
        "description": "Message ID"
      },
      {
        "name": "name",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Name"
      },
      {
        "name": "properties",
        "type": {
          "kind": "json"
        },
        "readOptional": true,
        "description": "Properties"
      },
      {
        "name": "receivedAt",
        "type": {
          "kind": "datetime"
        },
        "description": "Received at"
      },
      {
        "name": "sentAt",
        "type": {
          "kind": "datetime"
        },
        "description": "Sent at"
      },
      {
        "name": "originalTimestamp",
        "type": {
          "kind": "datetime"
        },
        "description": "Original timestamp"
      },
      {
        "name": "timestamp",
        "type": {
          "kind": "datetime"
        },
        "description": "Timestamp"
      },
      {
        "name": "traits",
        "type": {
          "kind": "json"
        },
        "description": "Traits"
      },
      {
        "name": "type",
        "type": {
          "kind": "string",
          "values": [
            "alias",
            "identify",
            "group",
            "page",
            "screen",
            "track"
          ]
        },
        "description": "Type"
      },
      {
        "name": "previousId",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "Previous ID"
      },
      {
        "name": "userId",
        "type": {
          "kind": "string"
        },
        "readOptional": true,
        "description": "User ID"
      }
    ]
  },
  "outSchema": {
    "kind": "object",
    "properties": [
      {
        "name": "first_name",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "readOptional": true,
        "description": "First name"
      },
      {
        "name": "last_name",
        "type": {
          "kind": "string",
          "maxLength": 100
        },
        "readOptional": true,
        "description": "Last name"
      }
    ]
  }
}
Errors
404
workspace does not exist
404
pipeline does not exist