Data types

View as Markdown

When defining data types and schemas in a connector, use the functions from the types package:

"github.com/krenalis/krenalis/tools/types"

This package provides functions to construct data types to use in connectors. For example, types.Boolean() returns a Type value representing the boolean type, and types.Array(types.Int(32)) returns a Type value representing an array of 32-bit signed integers.

In connectors, data types need to be defined for:

How to construct data types

Below are the data types and how to construct them using the types package functions.

string

Represents UTF-8 encoded string.

types.String()

String can be limited by allowed values, a pattern, or maximum lengths characters and bytes, for example:

// String limited to specific values.
types.String().WithValues("Hearts", "Diamonds", "Clubs", "Spades")

// String matching a pattern.
types.String().WithPattern(regexp.MustCompile(`\d+`))

// String with a maximum length of 2 characters.
types.String().WithMaxLegth(2)

// String with a maximum length of 1000 bytes.
types.String().WithMaxByteLength(1000)

You can combine maximum character and byte lengths:

// String with a maximum of 20 characters and 25 bytes.
types.String().WithMaxLength(20).WithMaxByteLength(25)

boolean

Represents a boolean value.

types.Boolean()

int(n)

An integer with n bytes, where n can be 8, 16, 24, 32, or 64.

types.Int(n)

You can also declare the integer as unsigned, meaning it can only hold non-negative values, for example:

// 32-bit unsigned integer.
types.Int(32).Unsigned()

You can limit integers to a minimum and maximum range, for example:

// 8-bit integer with range [-20, 20].
types.Int(8).WithIntRange(-20, 20)

// 32-bit unsigned integer with range [100, 1000].
types.Int(32).Unsigned().WithUnsignedRange(100, 1000)

float(n)

A floating-point number with n bytes, n can be 32, or 64. Includes +Inf, -Inf, and NaN values.

types.Float(n)

Floats can have a minimum and maximum value and can be restricted to real numbers only, for example:

// 32-bit float with range [-20.5, 8].
types.Float(32).WithFloatRange(-20.5, 8)

// 64-bit float with range [0, 56.481782].
types.Float(64).WithFloatRange(0, 56.481782)

// 64-bit float, excluding +Inf, -Inf, and NaN.
types.Float(64).Real()

decimal(p,s)

A decimal number with precision p and scale s, where p ranges from 1 to 76, s from 0 to 37, and s is less than or equal to p.

types.Decimal(p, s)

Decimals can also have a minimum and maximum value range, for example:

import "github.com/krenalis/krenalis/core/decimal"

...

min := decimal.MustInt(-10.5)
max := decimal.MustInt(8.25)

// Decimal with precision 5 and scale 2, range [-10.5, 8.25].
types.Decimal(5, 2).WithDecimalRange(min, max)

datetime

Represents a date and time within the range [1, 9999] with nanosecond precision and no timezone.

types.DateTime()

date

Represents a date within the range [1, 9999].

types.Date()

time

Represents a time of day with nanosecond precision and no timezone.

types.Time()

year

Represents a year within the range [1, 9999].

types.Year()

uuid

Represents a UUID.

types.UUID()

json

Represents JSON data.

types.JSON()

ip

Represents an IP4 or IP6 address.

types.IP()

array(T)

Represents an array of elements of type T.

types.Array(T)

For example:

// Array of strings.
types.Array(types.String())

// Array of 16-bit signed integers.
types.Array(types.Int(16))

// Array of UUID arrays.
types.Array(types.Array(types.UUID()))

Arrays can be limited in the minimum and maximum number of elements:

// Array of 32-bit unsigned integers with at least 1 element.
types.Array(types.Int(32)).WithMinElements(1)

// Array with a maximum of 10 decimal values.
types.Array(types.Decimal(10, 2)).WithMaxElements(10)

// Array with 5 to 15 string values.
types.Array(types.String()).WithMinElements(5).WithMaxElements(15)

Arrays can also be constrained to have unique values for their elements, except for arrays of json, array, map, and object:

// Array of 64-bit unsigned integers with unique values.
types.Array(types.Int64().Unsigned()).WithUnique()

// Array of UUIDs with unique values.
types.Array(types.UUID()).WithUnique()

object

Represents an object with specified properties.

types.Object([]types.Property{...})

For example:

types.Object([]types.Property{
    {Name: "first_name", Type: types.String().WithMaxLength(30)},
    {Name: "last_name", Type: types.String().WithMaxLength(30)},
    {Name: "birth_date", Type: types.Year()},
})

You can also use the types.ObjectOf function to construct an object. Unlike types.Object, it does not panic if a property is invalid but returns an error instead:

typ, err := types.ObjectOf([]types.Property{...})
if err != nil {
    ...
}

Properties

An object property is defined as follows:

type Property struct {
    Name           string
    Prefilled      string
    Type           Type
    CreateRequired bool
    UpdateRequired bool
    ReadOptional   bool
    Nullable       bool
    Description    string
}
  • Name: The name of the property. It must start with a letter [A-Za-z_] and can only contain alphanumeric characters and underscores [A-Za-z0-9_] after that. To check if a name is valid, use the types.IsValidPropertyName function.
  • Prefilled: A prefilled value to use in transformation mappings for events sent to applications. It pre-fills the input with the expression that evaluates to the property's value.
  • Type: The type of the property, which can be any data type.
  • CreateRequired: Indicates whether the property is required for creation.
  • UpdateRequired: Indicates whether the property is required for the update.
  • ReadOptional: Indicates whether the property may not be present when reading.
  • Nullable: Indicates whether the property can be null. In Go, this means it can be nil. In JavaScript, it can be null, and in Python, it can be None.
  • Description: A description providing additional information about the property's usage.

map(T)

Represents a map with string keys and values of type T.

types.Map(T)

For example:

// Map with 64-bit signed integers as values.
types.Map(types.Int(64))

// Map with arrays of strings as values.
types.Map(types.Array(types.String()))