Steps API

The Steps API allows you to create and manage individual execution steps within projects. Steps represent discrete data operations like SQL queries, transformations, exports, or visualizations.

Base URL

https://api.querri.com/api

Authentication

All Steps API endpoints require JWT authentication. See Authentication for details.

Endpoints

Create Step

Add a new step to a project.

POST /api/projects/{uuid}/steps

Headers:

Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json

Path Parameters:

Parameter	Type	Description
uuid	string	Project UUID

Request Body:

{
  "name": "Extract customer data",
  "type": "sql_query",
  "order": 1,
  "config": {
    "connector_uuid": "conn_01ABCDEF",
    "query": "SELECT customer_id, name, total_revenue FROM customers WHERE created_at >= '2024-01-01' ORDER BY total_revenue DESC LIMIT 100",
    "parameters": {
      "min_date": "2024-01-01"
    }
  },
  "dependencies": []
}

Response: 201 Created

{
  "step_id": "step_01ABCDEF",
  "project_uuid": "proj_01ABCDEF",
  "name": "Extract customer data",
  "type": "sql_query",
  "order": 1,
  "status": "pending",
  "config": {
    "connector_uuid": "conn_01ABCDEF",
    "query": "SELECT customer_id, name, total_revenue FROM customers...",
    "parameters": {
      "min_date": "2024-01-01"
    }
  },
  "dependencies": [],
  "created_at": "2025-01-15T10:00:00Z",
  "created_by": "user_01ABCDEF"
}

Permission Required: project:edit on the parent project.

---

List Steps

Retrieve all steps for a project.

GET /api/projects/{uuid}/steps

Query Parameters:

Parameter	Type	Default	Description
status	string	-	Filter by status: pending, running, completed, failed
type	string	-	Filter by step type

Example Request:

curl "https://api.querri.com/api/projects/proj_01ABCDEF/steps?status=completed" \
  -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."

Response: 200 OK

{
  "data": [
    {
      "step_id": "step_01ABCDEF",
      "name": "Extract customer data",
      "type": "sql_query",
      "order": 1,
      "status": "completed",
      "last_executed_at": "2025-01-15T14:30:00Z",
      "execution_duration_seconds": 12,
      "result_summary": {
        "rows": 100,
        "columns": 3,
        "size_bytes": 8192
      }
    },
    {
      "step_id": "step_02GHIJKL",
      "name": "Calculate revenue metrics",
      "type": "transform",
      "order": 2,
      "status": "completed",
      "last_executed_at": "2025-01-15T14:30:15Z",
      "execution_duration_seconds": 3,
      "dependencies": ["step_01ABCDEF"]
    }
  ],
  "total": 2
}

Permission Required: project:read on the parent project.

---

Execute Step

Execute a single step independently or as part of project execution.

POST /api/projects/{uuid}/steps/{step_id}/execute

Headers:

Authorization: Bearer {JWT_TOKEN}
Content-Type: application/json

Request Body:

{
  "parameters": {
    "min_date": "2024-10-01",
    "max_date": "2024-12-31"
  },
  "mode": "async",
  "cache": {
    "enabled": true,
    "ttl_seconds": 3600
  }
}

Parameters:

Field	Type	Required	Description
parameters	object	No	Runtime parameters to override step config
mode	string	No	async (default) or sync
cache.enabled	boolean	No	Use cached results if available (default: true)
cache.ttl_seconds	integer	No	Cache validity duration (default: 3600)

Response: 202 Accepted (async)

{
  "execution_id": "exec_01ABCDEF",
  "step_id": "step_01ABCDEF",
  "status": "running",
  "started_at": "2025-01-15T15:00:00Z",
  "estimated_duration_seconds": 10
}

Response: 200 OK (sync)

{
  "execution_id": "exec_01ABCDEF",
  "step_id": "step_01ABCDEF",
  "status": "completed",
  "started_at": "2025-01-15T15:00:00Z",
  "completed_at": "2025-01-15T15:00:12Z",
  "duration_seconds": 12,
  "result": {
    "rows": 100,
    "columns": 3,
    "size_bytes": 8192,
    "schema": [
      {"name": "customer_id", "type": "integer"},
      {"name": "name", "type": "string"},
      {"name": "total_revenue", "type": "float"}
    ],
    "data_url": "/api/projects/proj_01ABCDEF/steps/step_01ABCDEF/data"
  }
}

Permission Required: project:execute on the parent project.

Error Responses:

// 409 Conflict - Step already running
{
  "error": "conflict",
  "message": "Step is already executing",
  "execution_id": "exec_01EXISTING"
}

// 424 Failed Dependency - Dependency step not completed
{
  "error": "dependency_failed",
  "message": "Dependent step 'step_02GHIJKL' has not completed successfully",
  "dependency_step_id": "step_02GHIJKL"
}

---

Get Step Data

Retrieve paginated result data from a completed step execution.

GET /api/projects/{uuid}/steps/{step_id}/data

Query Parameters:

Parameter	Type	Default	Description
page	integer	1	Page number
limit	integer	100	Rows per page (max 1000)
format	string	json	Response format: json, csv, parquet
execution_id	string	latest	Specific execution to retrieve data from

Example Request:

curl "https://api.querri.com/api/projects/proj_01ABCDEF/steps/step_01ABCDEF/data?page=1&limit=50&format=json" \
  -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."

Response: 200 OK (JSON format)

{
  "step_id": "step_01ABCDEF",
  "execution_id": "exec_01ABCDEF",
  "schema": [
    {"name": "customer_id", "type": "integer"},
    {"name": "name", "type": "string"},
    {"name": "total_revenue", "type": "float"}
  ],
  "data": [
    {"customer_id": 1001, "name": "Acme Corp", "total_revenue": 125000.00},
    {"customer_id": 1002, "name": "TechStart Inc", "total_revenue": 98500.50},
    {"customer_id": 1003, "name": "Global Solutions", "total_revenue": 87200.25}
  ],
  "pagination": {
    "page": 1,
    "limit": 50,
    "total_rows": 100,
    "total_pages": 2,
    "has_next": true,
    "has_prev": false
  },
  "metadata": {
    "executed_at": "2025-01-15T15:00:12Z",
    "duration_seconds": 12,
    "size_bytes": 8192
  }
}

Response: 200 OK (CSV format)

customer_id,name,total_revenue
1001,Acme Corp,125000.00
1002,TechStart Inc,98500.50
1003,Global Solutions,87200.25

Response: 200 OK (Parquet format)

Binary parquet file download with Content-Type: application/octet-stream

Permission Required: project:read on the parent project.

---

Schedule Step Execution

Schedule recurring execution of a step.

POST /api/projects/{uuid}/steps/{step_id}/schedule

Request Body:

{
  "enabled": true,
  "cron": "0 9 * * 1-5",
  "timezone": "America/New_York",
  "parameters": {
    "min_date": "{{today - 7d}}",
    "max_date": "{{today}}"
  },
  "notify_on_failure": true
}

Parameters:

Field	Type	Required	Description
enabled	boolean	Yes	Enable/disable schedule
cron	string	Yes	Cron expression for schedule
timezone	string	No	Timezone for cron (default: UTC)
parameters	object	No	Dynamic parameters using template syntax
notify_on_failure	boolean	No	Send notification on failure (default: false)

Response: 201 Created

{
  "schedule_id": "sched_01ABCDEF",
  "step_id": "step_01ABCDEF",
  "enabled": true,
  "cron": "0 9 * * 1-5",
  "timezone": "America/New_York",
  "next_execution": "2025-01-16T09:00:00-05:00",
  "created_at": "2025-01-15T15:30:00Z"
}

Permission Required: project:edit on the parent project.

---

Step Types and Schemas

SQL Query Step

Execute a SQL query against a connector.

{
  "type": "sql_query",
  "config": {
    "connector_uuid": "conn_01ABCDEF",
    "query": "SELECT * FROM customers WHERE created_at >= {{min_date}}",
    "parameters": {
      "min_date": "2024-01-01"
    },
    "timeout_seconds": 300
  }
}

Configuration Schema:

Field	Type	Required	Description
connector_uuid	string	Yes	Database connector to use
query	string	Yes	SQL query with optional parameter placeholders
parameters	object	No	Query parameters
timeout_seconds	integer	No	Query timeout (default: 300)

---

Transform Step

Apply data transformations using Python or SQL.

{
  "type": "transform",
  "config": {
    "input_step_id": "step_01ABCDEF",
    "language": "python",
    "code": "df['revenue_category'] = df['total_revenue'].apply(lambda x: 'high' if x > 100000 else 'medium' if x > 50000 else 'low')",
    "output_columns": ["customer_id", "name", "total_revenue", "revenue_category"]
  }
}

Configuration Schema:

Field	Type	Required	Description
input_step_id	string	Yes	Step ID providing input data
language	string	Yes	python or sql
code	string	Yes	Transformation code
output_columns	array	No	Columns to include in output

---

Export Step

Export data to external destinations.

{
  "type": "export",
  "config": {
    "input_step_id": "step_01ABCDEF",
    "destination": {
      "type": "s3",
      "bucket": "querri-exports",
      "path": "customers/{{date}}/export.parquet",
      "format": "parquet"
    },
    "compression": "snappy"
  }
}

Configuration Schema:

Field	Type	Required	Description
input_step_id	string	Yes	Step ID providing data to export
destination.type	string	Yes	s3, gcs, local, sftp
destination.path	string	Yes	Export file path with template support
destination.format	string	Yes	parquet, csv, json, excel
compression	string	No	none, gzip, snappy, zstd

---

Visualization Step

Generate charts and visualizations.

{
  "type": "visualization",
  "config": {
    "input_step_id": "step_01ABCDEF",
    "chart_type": "bar",
    "x_axis": "name",
    "y_axis": "total_revenue",
    "title": "Top Customers by Revenue",
    "options": {
      "sort": "desc",
      "limit": 10,
      "color_scheme": "blues"
    }
  }
}

Configuration Schema:

Field	Type	Required	Description
input_step_id	string	Yes	Step ID providing data
chart_type	string	Yes	bar, line, pie, scatter, area
x_axis	string	Yes	Column for X axis
y_axis	string	Yes	Column for Y axis
title	string	No	Chart title
options	object	No	Chart-specific options

---

API Request Step

Make HTTP requests to external APIs.

{
  "type": "api_request",
  "config": {
    "method": "POST",
    "url": "https://api.external.com/data",
    "headers": {
      "Authorization": "Bearer {{api_key}}",
      "Content-Type": "application/json"
    },
    "body": {
      "query": "latest data"
    },
    "response_mapping": {
      "data": "$.results[*]"
    }
  }
}

Configuration Schema:

Field	Type	Required	Description
method	string	Yes	HTTP method: GET, POST, PUT, DELETE
url	string	Yes	API endpoint URL
headers	object	No	HTTP headers
body	object	No	Request body (for POST/PUT)
response_mapping	object	No	JSONPath mapping for response data

---

Step Dependencies

Steps can depend on other steps, creating execution graphs.

Defining Dependencies

{
  "name": "Calculate metrics",
  "type": "transform",
  "dependencies": ["step_01ABCDEF", "step_02GHIJKL"],
  "config": {
    "input_step_ids": ["step_01ABCDEF", "step_02GHIJKL"],
    "code": "merged = pd.merge(step_01_data, step_02_data, on='customer_id')"
  }
}

Dependency Resolution

• Steps execute in topological order
• A step waits until all dependencies complete
• If any dependency fails, dependent steps are skipped
• Circular dependencies are rejected

---

Parameter Templating

Steps support dynamic parameters using template syntax.

Template Variables

Variable	Description	Example
{{today}}	Current date	2025-01-15
{{now}}	Current timestamp	2025-01-15T10:00:00Z
{{today - Nd}}	N days ago	{{today - 7d}} → 2025-01-08
{{user.email}}	Current user email	user@example.com
{{user.org_id}}	Current organization	org_01ABCDEF
{{env.VAR}}	Environment variable	{{env.API_KEY}}

Example Usage

{
  "query": "SELECT * FROM events WHERE date >= '{{today - 7d}}' AND date < '{{today}}'",
  "parameters": {
    "user_email": "{{user.email}}",
    "export_path": "exports/{{today}}/data.parquet"
  }
}

---

Execution Status Lifecycle

Steps progress through these statuses:

1. pending - Created but not executed
2. queued - Waiting for dependencies or resources
3. running - Currently executing
4. completed - Successfully finished
5. failed - Execution failed
6. cancelled - User cancelled execution
7. skipped - Skipped due to dependency failure

---

Caching

Step results are automatically cached to improve performance.

Cache Behavior

• Results cached by step configuration hash
• Cache key includes: query, parameters, connector
• Default TTL: 1 hour
• Manual cache invalidation available

Cache Control

# Execute with cache disabled
curl -X POST "https://api.querri.com/api/projects/proj_01ABCDEF/steps/step_01ABCDEF/execute" \
  -H "Authorization: Bearer ${JWT_TOKEN}" \
  -d '{"cache": {"enabled": false}}'

# Invalidate cache
curl -X DELETE "https://api.querri.com/api/projects/proj_01ABCDEF/steps/step_01ABCDEF/cache" \
  -H "Authorization: Bearer ${JWT_TOKEN}"

---

Error Responses

Common Errors

// Invalid step configuration
{
  "error": "validation_error",
  "message": "Invalid SQL query syntax",
  "field": "config.query",
  "details": "Unexpected token 'FORM' at line 1"
}

// Dependency not met
{
  "error": "dependency_failed",
  "message": "Cannot execute step: dependency failed",
  "failed_dependency": "step_02GHIJKL",
  "dependency_error": "SQL execution error"
}

// Timeout
{
  "error": "execution_timeout",
  "message": "Step execution exceeded timeout of 300 seconds",
  "duration_seconds": 300
}

---

HTTP Status Codes

Code	Description
200	Success (GET, sync execution)
201	Created (POST step)
202	Accepted (async execution)
400	Bad Request (validation error)
401	Unauthorized
403	Forbidden (insufficient permissions)
404	Not Found (step doesn’t exist)
409	Conflict (already executing)
424	Failed Dependency
429	Too Many Requests
500	Internal Server Error