Description

The SFTP upload route lets you programmatically update the Single Fan View. You periodically push data to your SFTP server, and Cortex checks that server for new files. When Cortex finds a new file matching one of our modules, it triggers an update to SFV with that file's data.

Pre-Requisites

DWH

You set up a process ingesting source data from your different systems, modelling it to fit each module you wish to populate in the SFV.
eg: Ticketing, Retail, etc
SFTP Server

You set up your SFTP server, with an automated process that periodically populates it with new files containing each SFV modules' data. You decide the batch periodicity, which is also configured on Cortex.
Files can be incremental or full refresh - see below.

File Naming

File names follow a fixed convention so we can map each file to the right module, and so the naming itself can trigger different types of refresh:

Incremental loads: <module_name>_module_<load_description>.<file_format>
Full refresh: full_<module_name>_module_<load_description>.<file_format>

⚠️ Important: Each file is processed only once, so <load_description> must make the name unique - for example, a timestamp, as in user_module_20260101.csv for a daily User load.

Supported Modules

We currently support SFTP uploads for the following modules on your Fan Data Platform:

Module Name	File name Prefix	Incremental Example	Full refresh Example
User	user_module	user_module_20250101.csv	full_user_module_20250101.csv
Contact Preferences	contact_preference_module	contact_preference_module_abc.csv	full_contact_preference_module_abc.csv
Custom Data Fields	user_data_field_module	user_data_field_module_incremental_1.csv	full_user_data_field_module_refresh_all_1.csv
Form	form_module	form_module_123.csv	full_form_module_123.csv
Digital Usage	digital_usage_module	digital_usage_module_20250101.csv	full_digital_usage_module_20250101.csv
Email	email_module	email_module_20250101.csv	full_email_module_20250101.csv
Retail	retail_module	retail_module_load.csv	full_retail_module_load.csv
Structured Event	structured_event_module	structured_event_module_xyz.csv	full_structured_event_module_xyz.csv
Ticket	ticket_module	ticket_module_zzz.csv	full_ticket_module_zzz.csv
Streaming	streaming_module	streaming_module_1234.parquet	full_streaming_module_1234.parquet
Push	push_module	push_module_3456.parquet	full_push__module__3456.parquet
Opportunity	opportunity_module	opportunity_module_abc.csv	full_opportunity__module_abc.csv
Entitlement	entitlement_module	entitlement_module_xyz.csv	full_entitlement__module_xyz.csv
Hospitality	hospitality_module	hospitality_module_999.parquet	full_hospitality_module_999.parquet

For more details on each module, see the "Modules Structure" section.

Refresh Type

Incremental

Naming convention: no file prefix.

Smaller batches added on top of what's already on the Fan Data Platform. Always additive, and typically depends on an upstream process for the delta calculation.

Example: the daily load of the previous day's Transactions.
Full Refresh

Naming convention: file name starts with full_.

Larger batches that entirely replace the existing data for that module - old data is erased and replaced with the new entries.

Example: a monthly complete refresh of the Users database, used to clear unwanted data and keep existing records consistent.

File Naming Examples

Incremental load for the User module: user_module_20250101.csv

Full refresh for the Contact Preferences module: full_contact_preferences_module_20250101.csv

Supported formats

CSV
Parquet