Dataset Basics

A Dataset defines what data to extract from a web page. Every scraping project needs at least one dataset. Without a dataset, your automation can navigate pages and click buttons, but it won’t know what information to collect and save.

Think of a dataset as a template: you tell RTILA X where to look on the page (the item selector), what information to grab from each item (the properties), and how to process that information (transformations). RTILA X then applies this template to every matching element on the page, producing a clean table of data.

Core Concepts

Concept	Description	Example
Dataset	A named collection of properties to extract from a page	“products”, “reviews”, “metadata”
Item Selector	The CSS/XPath selector for the repeating parent container	css=.product-card
Property	A single field to extract from each item	title, price, image_url
Property Type	How the data is extracted	text, attribute, html, count
Record	One row of extracted data (one per matched item)	One product = one record
Transformation	A processing step applied to extracted data	trim, replace, cast
Required Fields	Fields that must have a value for a record to be valid	title, price
Parent Dataset	A parent dataset that child datasets inherit from	Parent: products → Child: reviews

How Extraction Works

RTILA X follows a precise, step-by-step process every time it extracts data:

1. Navigate to the page — RTILA X loads the target URL in the browser.
2. Wait for content — RTILA X waits for the page to finish loading (respects your wait_for_load_state settings).
3. Find items — RTILA X searches the page for all elements matching the Item Selector. Each matched element becomes one record.
4. For each item found:
   • RTILA X looks inside the item for each Property’s selector.
   • It extracts the value based on the property type (text, attribute, html, etc.).
   • It applies any Transformations in order (trim, replace, cast, etc.).
   • It checks Required Fields and logs warnings for missing values.
5. Deduplicate — RTILA X compares each extracted item against previously extracted items in the current run. Duplicates are skipped automatically.
6. Save the data — Results are saved to the configured output destination (file, database, cloud storage, etc.).

Dataset Structure

Every dataset has the following structure:

Field	Type	Required	Description
name	String	✅ Yes	A unique name for the dataset. Used in commands like extract_data and when viewing results.
item_selector	String	✅ Yes	The CSS/XPath selector that identifies the repeating container. Each match becomes one record. Format: css=..., xpath=..., text=..., or variable=...
properties	Array	✅ Yes	A list of property definitions. Each property extracts one field from each item.
required_fields	Array	No	A list of property names that must have values. If a required field is empty, a warning is logged.
parent	String	No	The name of a parent dataset. Child datasets can inherit properties from the parent.

Properties

Properties define the individual columns of data extracted from each item. Each property has the following fields:

Field	Type	Required	Description
name	String	✅ Yes	The column name in your output data (e.g., “title”, “price”, “image_url”).
selector	String	✅ Yes*	The selector to find this specific piece of data within each item. Relative to the item selector. *Not required for page_url and index types.
type	String	✅ Yes	What kind of data to extract. See Property Types below.
attribute	String	If attribute type	Which HTML attribute to read (e.g., href, src, data-id).
property	String	If property type	Which DOM property to read (e.g., value, checked).
download	Boolean	No	If checked and the value is a URL, the file is downloaded automatically.
transformations	Array	No	A chain of transformations applied to the value after extraction.

Property Types

RTILA X supports 9 property types:

Type	What It Extracts	Example Use Case
text	Visible text content of an element	Product names, descriptions, prices
html	Inner HTML including tags	Rich text content, formatted descriptions
attribute	A specific HTML attribute value	Links (href), image sources (src), data attributes
property	A JavaScript DOM property	Input values, checkbox states, scroll positions
count	Number of matching elements	Number of reviews, ratings count
page_url	The current page URL	Tracking which page data came from
selector_path	The CSS path of the matched element	Debugging selector issues
list	A nested list of sub-items	Reviews within a product, tags within an article
index	The row number (1-based)	Adding sequential row numbers to output

See Property Types for detailed documentation of each type with examples.

Item Selector

The Item Selector is the most important part of a dataset. It tells RTILA X which element on the page repeats to form your data rows.

How It Works

Imagine a product listing page with 20 product cards. Each card has the same HTML structure:

<div class="product-card">
  <h2>Product Name</h2>
  <span class="price">$29.99</span>
  <img src="/images/product.jpg" />
</div>

The Item Selector would be: css=.product-card

RTILA X finds all 20 elements matching .product-card and creates 20 records.

Selector Format

Selectors use the format type=value:

Format	Example	Description
css=.classname	css=.product-card	Find elements with a specific class
css=#idname	css=#main-content	Find element with a specific ID
xpath=//expression	xpath=//div[@class='item']	Find elements using XPath
text=Exact Text	text=Add to Cart	Find elements with exact text
variable=var_name	variable=my_selector	Use a runtime variable as the selector

You can also combine multiple selectors separated by commas. RTILA X tries each one in order:

css=.product-card,xpath=//div[@class='item'],text="Product Card"

See Selectors for complete selector documentation.

Dataset Settings Modal

The Dataset Settings modal lets you configure advanced dataset options. Access it by clicking the gear icon next to a dataset in the Data Extraction tab.

Setting	Description
Selector for Each Item’s Container	The item selector (see above). Use the element picker to select visually.
Parent Dataset	Link this dataset to a parent dataset for data inheritance.
Required Fields	Comma-separated list of property names that must have values.

Adding a Dataset

In the Visual Builder

1. Open the project in the Visual Builder (click Builder in the ribbon).
2. Switch to the Data Extraction tab.
3. Click the + button to add a new dataset.
4. A default dataset named “Default” is created with one property.
5. Click the + button in the properties area to add more properties.
6. Use the element picker (pointer icon) to select elements on the page.
7. The generated selector is inserted into the selector field automatically.

Via AI Assistant

Simply describe what you want to extract:

“Extract all product names, prices, and image URLs from this page.”

The AI will analyze the page, identify the data structure, and create the dataset automatically.

Via AI Visual Extraction (Wand Icon)

1. In the Data Extraction tab, click the wand icon (✨).
2. RTILA X takes a full-page screenshot and sends it to the AI.
3. The AI identifies data structures and creates datasets automatically.
4. The generated datasets appear in the Data Extraction tab.

See AI Vision Analysis for details.

Multiple Datasets

A project can have multiple datasets. Each dataset is extracted independently and stored separately.

Common multi-dataset setups:

Dataset Name	What It Extracts
products	Product listing data (name, price, image)
reviews	Review data within each product
metadata	Page-level metadata (page title, total count)
categories	Category navigation links
pagination	Pagination info (page numbers, next/prev URLs)

When you run a project with multiple datasets, each extract_data command specifies which dataset to extract. You can extract different datasets at different points in your automation.

Dataset Previews

The Data Extraction tab shows a live preview of the data. This updates in real time as you modify selectors.

Preview Element	Description
Column headers	Match your property names exactly
Cells	Show the extracted value (truncated to 50 characters)
Empty required cells	Show “(Empty Required)” in red if the property is marked as required
Image columns	Show a small thumbnail preview with a download button
Scripted values	Show “(Scripted value)” in italics if a script transformation is applied
Variable values	Show “(Variable value)” in italics if using a variable selector

Refreshing the Preview

Click the refresh icon (🔄) floating button in the bottom-right corner to re-run the extraction on the current page state. Use this after:

• Navigating to a different page
• Modifying selectors
• Adding or removing properties

Missing Required Fields Warning

If your dataset has required fields configured but the corresponding properties are missing, a red warning banner appears:

⚠️ Warning: The following required fields are missing from properties: title, price

This warning helps you catch configuration errors before running your automation.

Required Fields

Required fields ensure data quality. When you mark a property as required, RTILA X checks that the field has a value for every extracted record.

Setting Required Fields

1. Open the Dataset Settings modal (gear icon).
2. In the Required Fields field, enter property names separated by commas.
3. Example: title,price,image_url
4. Click Save.

Behavior

• If a required field is empty for a record, a warning is logged.
• The record is still included in the results (required fields generate warnings, not errors).
• Use the filter_condition parameter in extract_data to exclude incomplete records.

Download Option

For properties that extract URLs (like image sources or file links), you can enable the Download option:

1. Select a property in the Data Extraction tab.
2. Check the Download file if value is a URL checkbox.
3. Save your project.

When the automation runs:

1. The property extracts the URL value (e.g., https://example.com/images/product.jpg).
2. RTILA X detects that the value is a URL.
3. RTILA X downloads the file to the local output directory.
4. The extracted value becomes the local file path instead of the URL.

See Exporting Data for details on downloaded files.

Transformations

Transformations process extracted values after extraction. You can chain multiple transformations together, and they run in order from top to bottom.

RTILA X provides 11 transformation types: trim, prefix, suffix, replace, regex, extract_regex, join, cast, json_parse, json_path, and script.

See Transformations for complete documentation with examples.

Complete Example

Scenario: Scrape a product listing page.

Page HTML:

<div class="product-card">
  <h2>Wireless Headphones</h2>
  <span class="price">$49.99</span>
  <a href="/products/headphones" class="product-link">View Product</a>
  <img src="/images/headphones.jpg" alt="Headphones" />
</div>

Dataset Configuration:

Property	Selector	Type	Attribute
title	css=h2	text	—
price	css=.price	text	—
link	css=a.product-link	attribute	href
image	css=img	attribute	src

Item Selector: css=.product-card

Extracted Data:

title	price	link	image
Wireless Headphones	$49.99	/products/headphones	/images/headphones.jpg
Bluetooth Speaker	$29.99	/products/speaker	/images/speaker.jpg
USB-C Cable	$9.99	/products/cable	/images/cable.jpg

Tips and Best Practices

Tip	Why It Matters
Use specific item selectors	css=.product-card is better than css=div
Use the element picker	More reliable than typing selectors manually
Test with the live preview	Verify selectors work before running the full project
Use required fields for quality	Flag incomplete records early
Use transformations for cleanup	Trim whitespace, remove symbols, convert types
Use the Download option for images	Saves images locally alongside your data
Test on multiple pages	Verify selectors work across different pages of the same site
Use multiple datasets for complex pages	Separate products, reviews, and metadata into different datasets
Use parent datasets for nested data	Link child datasets to parent datasets for inheritance
Name datasets descriptively	Use names like “products”, “reviews”, not “dataset1”

##