Sync Scheduling

Sync Scheduling

Querri’s sync scheduling keeps your connected data sources up to date automatically. Set up recurring syncs on a schedule — hourly, daily, weekly, or custom cron — and Querri pulls the latest data from your connectors without manual intervention.

Why Schedule Syncs?

Without scheduling, connector data is a snapshot from the moment you first connected. Scheduling ensures your dashboards, reports, and AI analyses always reflect the current state of your source systems.

Common patterns:

• Daily morning sync — fresh data ready when your team starts work
• Hourly sync — near-real-time visibility into fast-moving data (sales pipelines, support tickets)
• Weekly full refresh — clean baseline that catches schema changes and deleted records

Sync Types

Querri supports two sync types, and the recommended approach is to use both together.

Full Sync

A full sync re-downloads every record from the source. It replaces the existing dataset with a complete, current snapshot.

When to use full sync:

• You need to catch deleted records (incremental sync can’t detect deletions)
• The source schema has changed (new columns, renamed fields)
• Data needs a clean baseline periodically
• The data source is small enough that a full re-download is fast

Trade-offs:

• Guarantees a complete, accurate dataset
• Slower for large data sources
• Uses more API calls and bandwidth
• Overwrites the existing dataset entirely

Incremental Sync

Incremental sync fetches only records that have changed since the last sync. It uses a cursor-based high-water mark — a timestamp or sequential value that advances after each successful sync.

How it works:

1. First sync — pulls all records (identical to a full sync) and records the highest cursor value (e.g., the most recent LastModifiedDate)
2. Subsequent syncs — adds a filter (e.g., WHERE LastModifiedDate > '2026-03-07T15:30:00Z') so only new/modified records are fetched
3. Append — new records are appended to the existing dataset, preserving full history

When to use incremental sync:

• Large data sources where full re-download is slow or expensive
• Data that grows continuously (transactions, events, logs)
• API rate limits make full syncs impractical
• You need frequent updates without the overhead of full re-reads

Trade-offs:

• Much faster after the first sync
• Uses fewer API calls
• Cannot detect deleted records
• Requires a reliable cursor column (timestamp or sequential ID)

Comparison

Aspect	Full Sync	Incremental Sync
What it fetches	Every record, every time	Only new/changed records
Speed	Slower for large datasets	Much faster after first sync
API usage	Higher	Lower
Handles deletes	Yes	No
Handles schema changes	Yes	No
Best for	Small datasets, weekly baselines	Large datasets, frequent updates

The Recommended Pattern: Both Together

The best approach is to combine both sync types:

Schedule	Type	Example Cron	Purpose
Frequent	Incremental	0 */1 * * * (hourly)	Catch new and updated records quickly
Periodic	Full	0 4 * * 0 (Sunday 4 AM)	Clean slate — catches deletes, schema changes, missed records

This gives you near-real-time freshness during the week with a guaranteed-correct baseline on a regular cadence.

Setting Up a Schedule

Creating a Schedule

1. Go to Settings → Connectors
2. Select your connector
3. Open the Schedule tab
4. Click Add Schedule
5. Choose the Sync Type — Full or Incremental
6. Enter a Cron Expression (e.g., 0 8 * * * for daily at 8 AM)
7. Select your Timezone
8. Click Save

Cron Expression Reference

Expression	Meaning
0 8 * * *	Daily at 8:00 AM
0 */2 * * *	Every 2 hours
0 */4 * * *	Every 4 hours
30 6 * * 1-5	Weekdays at 6:30 AM
0 3 * * 0	Sundays at 3:00 AM
0 0 1 * *	First day of each month at midnight

All times are in the timezone you select. Querri uses 5-field cron syntax (minute, hour, day-of-month, month, day-of-week).

Dual Schedules

A connector can have up to two schedules — one full and one incremental. To set up dual schedules:

1. Create an incremental schedule (e.g., hourly)
2. Create a full schedule (e.g., weekly)
3. Optionally enable Skip sources with active incremental sync on the full schedule

Skip Sources with Active Incremental Sync

When this option is enabled on a full-sync schedule, sources that are already covered by an incremental schedule will be skipped during the full sync. This is useful when you have a mix of sources — some with reliable cursor columns (use incremental) and some without (need full sync only).

Incremental Sync by Connector

Business Application Connectors

These connectors have built-in cursor fields — no additional configuration required:

Connector	Cursor Field	What Supports Incremental
QuickBooks	MetaData.LastUpdatedTime	Query operations only (Invoices, Customers, Payments, etc.)
HubSpot	hs_lastmodifieddate	All CRM objects (Contacts, Companies, Deals, Tickets, custom objects)
Salesforce	LastModifiedDate	All standard and custom objects

When you create an incremental schedule, these connectors automatically filter API requests using their built-in cursor field.

QuickBooks: Queries vs. Reports

QuickBooks has two types of data sources:

• Query operations (Invoices, Customers, Payments, Items, etc.) — support incremental sync via MetaData.LastUpdatedTime
• Report operations (Profit & Loss, Balance Sheet, etc.) — do not support incremental sync because reports are computed snapshots without individual record timestamps

During an incremental sync, report sources are automatically skipped. They will sync only during full syncs.

Recommended QuickBooks setup:

• Incremental schedule every 2–4 hours (keeps transactional data fresh)
• Full schedule weekly or daily (refreshes reports and catches deletes)

HubSpot

All HubSpot CRM objects support incremental sync. The connector uses HubSpot’s Search API with lastmodifieddate filtering to fetch only changed records. Custom objects follow the same pattern.

Recommended HubSpot setup:

• Incremental schedule every 1–2 hours
• Full schedule weekly

Salesforce

All Salesforce standard and custom objects support incremental sync via SOQL WHERE LastModifiedDate > ... filtering. Incremental syncs automatically use the REST API instead of Bulk 2.0 for faster small-delta fetches.

Recommended Salesforce setup:

• Incremental schedule every 1–4 hours depending on data volume
• Full schedule weekly

Database Connectors

Database connectors (PostgreSQL, MySQL, SQL Server, Redshift, BigQuery) require you to specify which column to use as the cursor.

Choosing a Cursor Column

Pick a column that reliably updates whenever a row is inserted or modified:

Column Type	Good For	Example
updated_at / modified_at	Tables with update tracking	Best choice — catches both inserts and updates
created_at	Append-only tables	Only catches new rows, not updates to existing rows
Auto-incrementing id	Append-only tables	Only catches new rows; sequential guarantee

Not All Tables Support Incremental Sync

Tables without a reliable timestamp or sequential column cannot use incremental sync:

• Lookup/reference tables — rarely change, no timestamp column
• Tables with soft deletes — may lack an updated_at column
• ETL-populated tables — often truncate and reload, making cursors meaningless

For these tables, use full sync.

Configuring the Cursor

1. In the schedule configuration, expand Cursor Configuration
2. Enter the column name to use as the cursor (e.g., updated_at)
3. Querri validates that the column exists on the selected tables before saving

Recommended database setup:

• Incremental schedule every 1–4 hours on tables with good updated_at columns
• Full schedule daily or weekly on all tables (catches schema changes and tables without cursors)
• Enable “Skip sources with active incremental sync” on the full schedule so tables with cursors aren’t double-synced

Monitoring Sync Runs

Run History

Every sync run is logged in the connector’s Run History tab. Each entry shows:

• Status — completed, failed, or skipped
• Sync type — full or incremental
• Per-source detail — which sources were synced, skipped, or errored
• Duration — how long each source took
• Rows fetched — number of records retrieved

Skipped Sources

When a source is skipped, the run history shows the reason:

• “Source does not support incremental sync” — the source type doesn’t have a cursor field (e.g., QuickBooks reports)
• “No cursor field configured” — database table without a cursor column specified
• “Skipped by full-sync schedule” — the source has an active incremental schedule and was excluded from the full sync

Manual Sync

You can trigger a sync outside the schedule at any time:

1. Go to Settings → Connectors
2. Select your connector
3. Click Sync Now

Manual syncs always run a full sync by default.

Troubleshooting

Schedule Not Running

Problem: The schedule was saved but syncs aren’t happening.

Check:

• The schedule is enabled (not paused)
• The cron expression is valid
• The connector is still connected (check connection status)
• The timezone is correct (a schedule set to 8 AM Pacific won’t run at 8 AM Eastern)

Incremental Sync Missing Records

Problem: Some records appear in the source but not in Querri after an incremental sync.

Possible causes:

• The record was modified before the cursor window
• The cursor column doesn’t update on every type of change
• Clock skew between the source system and Querri

Solution: Run a manual full sync. If this happens regularly, shorten the incremental interval or switch to full sync for that source.

Duplicates After Incremental Sync

Problem: The same records appear multiple times.

Cause: This is expected behavior for incremental append operations, particularly when cursor values have low precision (date-only instead of datetime).

Solution: Querri deduplicates at query time. Schedule periodic full syncs to reset the dataset if exact deduplication is important.

Cursor Not Advancing

Problem: The sync keeps re-fetching the same records.

Check:

• The run history for errors — a failed sync doesn’t advance the cursor
• The cursor column for NULL values or non-monotonic values
• Whether the source system’s clock is synchronized

”Source does not support incremental sync”

Problem: A source you expected to sync incrementally was skipped.

Solution: Check the per-connector details above to confirm whether that source type supports cursors. QuickBooks reports and database tables without timestamp columns cannot use incremental sync — they will sync on the full schedule instead.

Best Practices

1. Start with full sync, then add incremental — get a complete baseline before switching to deltas
2. Always keep a full-sync schedule — catches deletes, schema changes, and edge cases the cursor misses
3. Choose timestamp over ID for database cursors — timestamps catch both inserts and updates; IDs only catch inserts
4. Don’t over-sync — hourly incremental is usually sufficient; every-minute syncing creates unnecessary load
5. Monitor run history — watch for increasing sync durations or growing numbers of skipped sources
6. Use “skip incremental sources” on full schedules — avoids redundant work on sources that are already syncing incrementally

Next Steps

• Connectors Overview — all available connectors
• Managing Connections — edit, test, and troubleshoot connectors
• QuickBooks — QuickBooks-specific setup
• HubSpot — HubSpot connector setup
• Database Best Practices — optimize database connections