Data Transformation

In this section, we'll guide you on how to perform data transformations.

Use Cases

When the data you've extracted from a webpage does not meet your needs, you can transform it by adding transformers. For example, you can trim whitespace, use regular expressions to replace or extract data, format dates, or convert data types.

example

You can add multiple transformers and they will be executed in sequence.

Next, we'll go over the usage of each transformer.

Trim

Trims whitespace from the start or end of a string.

trim

Arguments

  • Value: String, the input value
  • Position: Enum, which position to trim the whitespace

Returns

  • String, the trimmed string
  • Any, if Value is not a string, it will return the original value

Examples

Given the string:

"  abc  "

Example 1: If Position = Start, the output will be:

"abc  "

Example 2: If Position = End, the output will be:

"  abc"

Example 3: If Position = Both, the output will be:

"abc"

Replace

Replaces the matched string with a specified replacement.

replace

Arguments

  • Value: String, the input value
  • Pattern: RegExp, a regular expression
  • Replacement: String, the replacement string. If not provided, it will replace with an empty string.

Returns

  • String, the resulting string after replacement
  • Any, if Value is not a string or if Pattern is incorrect, it will return the original value

Examples

Given the string:

"Hello, Anna"

Example 1: If Pattern = /Hello/g and Replacement = "Hi", it will replace "Hello" with "Hi", resulting in:

"Hi, Anna"

Example 2: If Pattern = /Hello/g and Replacement = "", it will replace "Hello" with an empty string, resulting in:

", Anna"

Example 3: If Pattern = /abc/g and Replacement = "", there will be no match, and the result will be:

"Hello, Anna"

Match

Extracts matched parts of a string based on a pattern.

match

Arguments

Returns

  • Array, returns an array of matched results
  • Null, returns null if no match is found
  • Any, if Value is not a string or Pattern is incorrect, it will return the original value

Examples

Given the string:

"Hello, Anna! Hello, Nick!"

Example 1: If Pattern = /Hello/g, the output will be:

[ "Hello", "Hello" ]

Example 2: If Pattern = /abc/g, the output will be:

null

Extract

Extract is a variant of Match. The difference is that Extract converts the array returned by Match into a string, separating each value with a comma ,.

extract

Arguments

Returns

  • String, returns a string of matched results
  • Null, returns null if no match is found
  • Any, if Value is not a string or Pattern is incorrect, it will return the original value

Examples

Given the string:

"Hello, Anna! Hello, Nick!"

Example 1: If Pattern = /Hello/g, the output will be:

"Hello,Hello"

Example 2: If Pattern = /abc/g, the output will be:

null

Modify

Adds a prefix and suffix to a string.

modify

Arguments

  • Value: String, the input value
  • Prefix: String, the prefix to add
  • Suffix: String, the suffix to add

Returns

  • String, returns the modified string
  • Any, if Value is not a string, it will return the original value

Examples

Given the string:

"Anna"

Example 1: If Prefix = "xxx-", Suffix = "-yyy", the output will be:

"xxx-Anna-yyy"

Split

Splits a string into an array.

split

Arguments

  • Value: String, the input value
  • Separator: String, the separator used to split the string

Returns

  • Array, returns the array of split values
  • Any, if Value is not a string, it will return the original value

Examples

Given the string:

"Anna,Nick"

Example 1: If Separator = ",", the output will be:

[ "Anna", "Nick" ]

Size

Returns the length of a string or the number of elements in an array.

size

Arguments

  • Value: Array | String, the input value

Returns

  • Number, returns the length of the string or the number of elements in the array
  • Any, if Value is not an array or a string, it will return the original value

Examples

Example 1: If Value = [ "Anna", "Nick" ], the output will be:

2

Example 2: If Value = "Anna", the output will be:

4

Slice

Extracts a portion of an array or substring from a string.

slice

Arguments

  • Value: Array | String, the input value
  • Start: Number, the starting index (starts from 0)
  • End: Number, the ending index (starts from 0)

Returns

  • Array, returns the portion of the array
  • String, returns the substring
  • Any, if Value is not an array or string, it will return the original value

Examples

Example 1: If Value = [ "A", "B", "C", "D", "E" ], the output will be:

[ "A", "B", "C" ]

Example 2: If Value = "ABCDE", the output will be:

"ABC"

At

Retrieve an element from an array or a character from a string.

at

Arguments

  • Value: Array | String, the input value
  • Index: Number, the index of the element (starting from 0)

Returns

  • Any, the specified element of an array
  • String, the specified character of a string
  • Any, if Value is neither an array nor a string, it will be returned as-is

Examples

Example 1: When Value = [ "A", "B", "C", "D", "E" ], the output will be:

"A"

Example 2: When Value = "ABCDE", the output will be:

"A"

Join

Concatenate an array into a string.

join

Arguments

  • Value: Array, the input value
  • Separator: String, the delimiter used to separate elements

Returns

  • String, the concatenated string
  • Any, if Value is not an array, it will be returned as-is

Examples

Suppose we have the following array:

[ "A", "B", "C" ]

Example 1: When Separator = ", ", the output will be:

"A, B, C"

Example 2: When Separator = " | ", the output will be:

"A | B | C"

Sort

Sort the elements of an array.

sort

Arguments

  • Value: Array, the input value

Returns

  • Array, the sorted array
  • Any, if Value is not an array, it will be returned as-is

Examples

Example 1: When Value = [ "C", "A", "B" ], the output will be:

[ "A", "B", "C" ]

Unique

Make the elements of an array unique.

unique

Arguments

  • Value: Array, the input value

Returns

  • Array, the array with unique elements
  • Any, if Value is not an array, it will be returned as-is

Examples

Example 1: When Value = [ "A", "A", "B", "B" ], the output will be:

[ "A", "B" ]

Compact

Remove falsy values from an array, such as: "", 0, false, undefined, null, NaN.

compact

Arguments

  • Value: Array, the input value

Returns

  • Array, the array with falsy values removed
  • Any, if Value is not an array, it will be returned as-is

Examples

Example 1: When Value = [ "A", "", 0, "B" ], the output will be:

[ "A", "B" ]

Format

Format a date. For more information, refer to Day.js.

format

Arguments

  • Value: Number | String | Date, the input value

Returns

  • String, the formatted date string
  • Any, if Value is not a valid date, it will be returned as-is

Examples

Example 1: When Value = 1728718675568, the output will be:

"2024-10-12 15:37:55"

Example 2: When Value = "2024-10-12T07:37:55.568Z", the output will be:

"2024-10-12 15:37:55"

toString

Convert a value to a string.

toString

Arguments

  • Value: Any, the input value

Returns

  • String, the converted string

Examples

Example 1: When Value = 123, the output will be:

"123"

Example 2: When Value = true, the output will be:

"true"

Example 3: When Value = [ "A", "B" ], the output will be:

"[ 'A', 'B' ]"

toNumber

Convert a value to a number.

toNumber

Arguments

  • Value: Any, the input value

Returns

  • Number, the converted number

Examples

Example 1: When Value = "123", the output will be:

123

toBoolean

Convert a value to a boolean. Values such as "", 0, false, undefined, null, NaN will be converted to false, while other values will be converted to true.

toBoolean

Arguments

  • Value: Any, the input value

Returns

  • Boolean, the converted boolean value

Examples

Example 1: When Value = 1, the output will be:

true

Example 2: When Value = 0, the output will be:

false

toArray

Convert a value to an array.

toArray

Arguments

  • Value: String, the input value

Returns

  • Array, the converted array. If the conversion fails, it returns an empty array [].

Examples

Example 1: When Value = '[ "A", "B" ]', the output will be:

[ "A", "B" ]

toJSON

Convert a value to JSON.

toJSON

Arguments

  • Value: String, the input value

Returns

  • Object, the converted JSON object. If the conversion fails, it returns an empty JSON object {}.

Examples

Example 1: When Value = '{ "foo": "bar" }', the output will be:

{ foo: "bar" }

On This Page