File Format Triggers

File format triggers let you export, convert, and compress your scraped data into various file formats. These triggers run after your automation completes and process the extracted data into the format you need.

RTILA X provides six file format triggers:

Trigger	Purpose
Export to File	Export data to CSV, JSON, XLSX, or Parquet
Convert Format	Convert data from one format to another
Compress File	Compress a file into ZIP or GZIP format
Parquet Export	Export data to Apache Parquet format
XML Processing	Export data to XML or parse XML files
JSONL Export	Export data to JSON Lines format

Export to File

Exports your scraped data to a local file in your chosen format. This is the most commonly used file trigger.

Description

When your automation completes, the Export to File trigger takes all the extracted data and writes it to a file on your computer. You can choose the format and whether to overwrite or append to an existing file.

Parameters

Parameter	Type	Required	Default	Description
format	string	✅ Yes	"csv"	The output format. Options: csv, json, xlsx, parquet
output_path	string	✅ Yes	—	The file path where the data will be saved. Supports ${variable} substitution. Example: "output/products.csv"
mode	string	No	"overwrite"	How to handle an existing file. Options: overwrite (replace the file) or append (add to the end). Only applies to CSV format.

Format Details

Format	Extension	Best For	Opens In
csv	.csv	Spreadsheets, data analysis	Excel, Google Sheets, LibreOffice
json	.json	Developers, APIs, data pipelines	Any text editor, programming tools
xlsx	.xlsx	Excel spreadsheets with formatting	Microsoft Excel
parquet	.parquet	Large datasets, data analytics	Apache Spark, Pandas, data tools

Examples

Export to CSV (overwrite):

{
  "format": "csv",
  "output_path": "output/products_${date}.csv",
  "mode": "overwrite"
}

Export to JSON:

{
  "format": "json",
  "output_path": "output/products_${date}.json",
  "mode": "overwrite"
}

Append to existing CSV:

{
  "format": "csv",
  "output_path": "output/all_products.csv",
  "mode": "append"
}

Behavior Notes

• If mode is "overwrite", the existing file is completely replaced.
• If mode is "append", new data is added to the end of the file. Only works with CSV format.
• If the file doesn’t exist, it is created regardless of the mode.
• The output_path is relative to RTILA X’s output directory.
• The ${date} variable is automatically available and formatted as YYYY-MM-DD.

---

Convert Format

Converts data from one format to another. This is useful when you need to take data in one format and produce a different format for a downstream tool.

Description

The Convert Format trigger reads data from an input file or from an in-memory variable, and writes it to a new file in a different format. This is useful when you’ve already exported data in one format but need it in another.

Parameters

Parameter	Type	Required	Default	Description
input_path	string	No	—	The file path to read data from. Use this if your data is in a file.
input_data	list	No	—	The data to convert, passed as a list of objects. Use this if your data is in memory.
input_format	string	No	—	The format of the input file. Required if using input_path. Options: csv, json, parquet, xlsx
output_format	string	✅ Yes	—	The desired output format. Options: csv, json, parquet, xlsx
output_path	string	✅ Yes	—	The file path where the converted file will be saved.

Examples

Convert CSV to JSON:

{
  "input_path": "output/products.csv",
  "input_format": "csv",
  "output_format": "json",
  "output_path": "output/products.json"
}

Convert JSON to Excel:

{
  "input_path": "output/data.json",
  "input_format": "json",
  "output_format": "xlsx",
  "output_path": "output/data.xlsx"
}

Convert in-memory data to Parquet:

{
  "input_data": "${dataset}",
  "output_format": "parquet",
  "output_path": "output/data.parquet"
}

Behavior Notes

• Use either input_path (for file-based data) or input_data (for in-memory data), not both.
• If using input_path, you must specify input_format so RTILA knows how to read the file.
• If using input_data, you can reference the ${dataset} variable to use all extracted data.
• The conversion preserves all data fields. Column names become JSON keys or CSV headers.

---

Compress File

Compresses a file into a ZIP or GZIP archive. This is useful for reducing file size before uploading to cloud storage or sending via email.

Description

The Compress File trigger takes a file and compresses it into an archive format. This reduces file size, making it easier to store, transfer, or share.

Parameters

Parameter	Type	Required	Default	Description
source_path	string	✅ Yes	—	The path to the file you want to compress.
output_path	string	✅ Yes	—	The path where the compressed archive will be saved.
format	string	✅ Yes	—	The compression format. Options: zip or gzip

Format Comparison

Format	Extension	Best For	Compression Ratio
zip	.zip	General use, Windows/Mac compatibility	Good
gzip	.gz	Linux servers, web servers, data pipelines	Better than ZIP

Examples

Compress a CSV file to ZIP:

{
  "source_path": "output/products.csv",
  "output_path": "output/products_${date}.zip",
  "format": "zip"
}

Compress a JSON file to GZIP:

{
  "source_path": "output/data.json",
  "output_path": "output/data.json.gz",
  "format": "gzip"
}

Behavior Notes

• The source file must exist before this trigger runs. Typically, you’d use this after an Export to File trigger.
• The original file is NOT deleted. You get both the original and the compressed version.
• GZIP can only compress a single file. ZIP can also compress a single file (or a directory).
• Compression ratio depends on the data. Text files (CSV, JSON) compress very well. Already-compressed files (images, videos) won’t compress much.

---

Parquet Export

Exports data to Apache Parquet format. Parquet is a columnar storage format optimized for large datasets and analytics workloads.

Description

The Parquet Export trigger converts your scraped data into Apache Parquet format. Parquet is the preferred format for big data tools like Apache Spark, Pandas, and data warehouses. It stores data in columns rather than rows, making it extremely efficient for analytics.

Parameters

Parameter	Type	Required	Default	Description
output_path	string	✅ Yes	—	The file path where the Parquet file will be saved.
data	list	✅ Yes	—	The data to export. Use ${dataset} to export all extracted data.
compression	string	No	"snappy"	The compression codec. Options: snappy, gzip, brotli, none
row_group_size	integer	No	10000	The number of rows per row group. Larger values improve compression but use more memory.

Compression Comparison

Codec	Speed	Compression	Best For
snappy	Very fast	Good	General use, real-time processing
gzip	Moderate	Better	Storage efficiency
brotli	Slow	Best	Maximum compression, archival
none	Fastest	None	Fastest processing, largest files

Examples

Basic Parquet export:

{
  "output_path": "output/products.parquet",
  "data": "${dataset}",
  "compression": "snappy",
  "row_group_size": 10000
}

High-compression export for archival:

{
  "output_path": "output/archive/products_${date}.parquet",
  "data": "${dataset}",
  "compression": "brotli",
  "row_group_size": 50000
}

Behavior Notes

• Parquet is a binary format. You cannot open it in Excel or a text editor. Use tools like Apache Spark, Pandas, or Parquet viewers.
• The row_group_size controls how many rows are grouped together. Larger groups compress better but use more memory during writing.
• Parquet preserves data types. Numbers stay as numbers, strings stay as strings.
• For very large datasets (millions of rows), Parquet is significantly more efficient than CSV.

---

XML Processing

Exports data to XML format or parses an existing XML file. XML is still widely used in enterprise systems, data feeds, and legacy integrations.

Description

The XML Processing trigger can either export your scraped data to an XML file or parse an existing XML file and extract data from it. This makes it a two-way tool: you can both produce and consume XML data.

Parameters

Parameter	Type	Required	Default	Description
operation	string	✅ Yes	—	The operation to perform. Options: export_to_xml or parse_xml
data	list	No	—	The data to export to XML. Required for export_to_xml. Use ${dataset} for all extracted data.
output_path	string	No	—	The file path for the XML output. Required for export_to_xml.
input_path	string	No	—	The path to the XML file to parse. Required for parse_xml.
root_element	string	No	"records"	The name of the root XML element. Default: records
record_element	string	No	"record"	The name of each record element. Default: record
encoding	string	No	"utf-8"	The character encoding for the XML file. Default: utf-8

XML Structure

When you export data to XML, the structure looks like this:

<?xml version="1.0" encoding="utf-8"?>
<records>
  <record>
    <name>Product A</name>
    <price>29.99</price>
    <url>https://example.com/product-a</url>
  </record>
  <record>
    <name>Product B</name>
    <price>49.99</price>
    <url>https://example.com/product-b</url>
  </record>
</records>

The root_element controls the outer tag (records), and the record_element controls each item tag (record).

Examples

Export data to XML:

{
  "operation": "export_to_xml",
  "data": "${dataset}",
  "output_path": "output/products.xml",
  "root_element": "products",
  "record_element": "product",
  "encoding": "utf-8"
}

This produces:

<?xml version="1.0" encoding="utf-8"?>
<products>
  <product>
    <name>Product A</name>
    <price>29.99</price>
  </product>
</products>

Parse an XML file:

{
  "operation": "parse_xml",
  "input_path": "input/feed.xml",
  "output_variable": "parsed_data"
}

Behavior Notes

• For export_to_xml, the data and output_path parameters are required.
• For parse_xml, the input_path parameter is required.
• The root_element and record_element only apply to export_to_xml.
• XML special characters (<, >, &, ", ') are automatically escaped in exported data.
• The encoding parameter controls the character set. Use utf-8 for international text.

---

JSONL Export

Exports data to JSON Lines format (JSONL). In this format, each line of the file is a separate JSON object, one per line.

Description

The JSONL Export trigger writes your data as JSON Lines. Each record becomes one line of JSON. This format is popular in data pipelines, machine learning workflows, and log processing because each line can be processed independently.

Parameters

Parameter	Type	Required	Default	Description
output_path	string	✅ Yes	—	The file path where the JSONL file will be saved.
data	list	✅ Yes	—	The data to export. Use ${dataset} to export all extracted data.
encoding	string	No	"utf-8"	The character encoding for the file. Default: utf-8

JSONL Format Example

A JSONL file looks like this (each line is a complete JSON object):

{"name": "Product A", "price": 29.99, "url": "https://example.com/a"}
{"name": "Product B", "price": 49.99, "url": "https://example.com/b"}
{"name": "Product C", "price": 19.99, "url": "https://example.com/c"}

Notice that each line is a complete, valid JSON object. There is no wrapping array or commas between lines.

Examples

Export all data to JSONL:

{
  "output_path": "output/products.jsonl",
  "data": "${dataset}",
  "encoding": "utf-8"
}

Export with a dated filename:

{
  "output_path": "output/products_${date}.jsonl",
  "data": "${dataset}",
  "encoding": "utf-8"
}

JSONL vs JSON

Feature	JSON	JSONL
Structure	One large array [{...}, {...}]	One object per line
Streaming	Must read entire file	Can process line by line
Memory usage	Loads entire file into memory	Only one line at a time
Best for	Small datasets, APIs	Large datasets, data pipelines
Appending	Must rewrite entire file	Can append one line at a time

Behavior Notes

• Each line in the output file is a complete JSON object.
• There is no wrapping array. Each line is independent.
• JSONL is ideal for large datasets because you can process it line by line without loading the entire file.
• Many data tools (Apache Spark, Pandas, BigQuery) can read JSONL natively.

---

Choosing the Right File Format

Scenario	Recommended Format	Why
Opening in Excel	xlsx or csv	Native Excel support
Sharing with non-technical users	csv	Universal compatibility
Feeding data to an API	json	Standard API format
Large datasets (100K+ rows)	parquet	Columnar storage, very efficient
Data pipelines / ETL	jsonl or parquet	Line-by-line processing
Enterprise / legacy systems	xml	Still required by many enterprise systems
Archival / backup	parquet with brotli	Best compression ratio
Quick sharing / email	csv + zip	Small file size, universal format

Chaining File Triggers

You can chain multiple file triggers together. For example:

1. Export to File — Export data to CSV.
2. Compress File — Compress the CSV into a ZIP.
3. SFTP — Upload the ZIP to a server.

Or:

1. Export to File — Export data to JSON.
2. Convert Format — Convert JSON to Parquet.
3. Cloud Storage — Upload the Parquet file to S3.