Pipelines

Not everything that matters arrives through an Edge Server. Power quality meters email event files, utilities export CSVs, contractors upload test reports. A pipeline is a small JavaScript program you write inside the platform that receives those pushes over HTTPS, runs your parsing logic, and turns the result into stored files and events, with no server of your own to host.

Overview

Pipelines is reached from Settings: open Settings, find the Pipelines card ("Transform and route incoming data with custom processing pipelines.") and click Manage, which lands on the list at /administration/pipelines.

Each pipeline is a named endpoint plus the code behind it. An external system POSTs a JSON payload or a file to the pipeline's URL; the platform runs your handler in an isolated serverless function and executes the actions it returns. What comes out the other end is ordinary platform data: files on the Files page and events in Alarm History.

How a Pipeline Runs

The flow is push, run, act. A sender POSTs to the pipeline's endpoint with an API key. The platform invokes your handler with the request payload: JSON fields arrive as fields, and an uploaded file arrives with its filename, content and type. Your code does whatever parsing it needs and returns a list of actions; the platform then performs them, storing files and writing events. Every invocation, success or failure, increments the pipeline's execution count, and a run counts as failed when the handler throws or returns something that is not a valid action list.

Runs are stateless and short-lived: each request gets a fresh execution with a 30 second limit, so pipelines suit event files and periodic exports rather than continuous streams. Continuous device data belongs to the Edge Server path in the Connect and Collect stages.

The Pipelines List

The list shows each pipeline's name, its Execution Count, and Last Modified. The search box (Search pipelines…) filters by name, the column and density toggles adjust the table, and the export button writes the rows out. Clicking a row, or Open in its actions, opens the pipeline's configuration; there is no delete, so a retired pipeline is switched off with its enable toggle instead.

Creating a Pipeline

Click New Pipeline to open the Create Pipeline dialog. Give the pipeline a name, and note the code editor arrives pre-filled with a complete working example handler, which is the fastest way to learn the contract: it parses an uploaded file's name, stores the file, and writes an event bound to it.

Create deploys the pipeline and opens its configuration page. New pipelines start disabled; flip Enable Pipeline when the code is ready to receive real traffic. Saving from the configuration page redeploys the code, and the change is live on the next request.

Writing Pipeline Code

The editor is a real multi-file JavaScript environment. The Files panel on the left lists the module files; index.mjs is the entry point, always present, and cannot be renamed or removed. The New file button adds helper modules (names get .mjs appended automatically), so a parser, a report generator and a lookup table can live in their own files and be imported from the entry file with standard ES module imports.

The contract is a single exported function:

export const handler = async (event) => {
    const body = event.body ?? {};
    const file = body.file; // { filename, value (base64), mimetype } when a file was uploaded

    // ...parse, transform, decide...

    return [
        {
            action: 'STORE_FILE',
            base64file: file.value,
            filename: file.filename,
            mimetype: file.mimetype,
            directory: 'power-quality/meter-12',
        },
        {
            action: 'WRITE_EVENT',
            message: 'Power quality event recorded',
            isFileBoundToEvent: true,
        },
    ];
};

The return value must be an array of actions, and it is the only way a pipeline affects the platform: no actions, no side effects. Code runs on Node.js 22 with 256 MB of memory and the 30 second limit above.

Actions

Store File (STORE_FILE) saves a file into the Organization's storage. It takes the content (base64file), a filename, a mimetype and an optional directory; the directory becomes the folder shown on the Files page, so the folder tree there mirrors what your pipelines write.

Write Event (WRITE_EVENT) records a manual event, the same kind that appears in Alarm History. It takes a message and optional dateTime, can attach the event to an Asset with assetId, can bind the file stored by the same run with isFileBoundToEvent, and can notify an alarm group by email with notifyAlarmGroupId, optionally attaching the file with attachFileToEmail. That last combination is how "meter 12 recorded a sag, report attached" reaches the on-call inbox with no one watching a screen.

No Action (NA) is the explicit way to do nothing, for payloads your code decides to ignore.

A single run can return several actions, and typically does: store the evidence, then write the event that points at it.

Layers

Pipeline code cannot install its own npm packages; shared dependency bundles called layers provide them instead. The Layers selector on the pipeline form attaches any of the published bundles: a base utility layer (HTTP requests, lodash, date handling with Luxon), a file layer (CSV parsing with Papa Parse), and a PDF layer (PDF generation and charting). Attach only what the code imports, and expect a selected layer to display its version; an outdated version is flagged on the form after the platform publishes a newer one.

Sending Data to a Pipeline

Each pipeline listens at its own execute URL:

POST /p/v1/organizations/{organizationId}/pipelines/{endpointId}/execute?apiKeyId={id}&apiKeySecret={secret}

The endpointId is a short identifier generated for the pipeline when it is created. Requests authenticate with an Organization API key, created under Settings → API keys (see Settings), passed as the apiKeyId and apiKeySecret query parameters.

The body is either JSON (application/json) or a file upload (multipart/form-data, one file per request, up to 50 MB). Form fields arrive in your handler alongside the file, which is how a sender passes context like which meter or site the upload belongs to. A disabled pipeline rejects requests until it is enabled again.

Permissions

Related Workflow Stages

Pipelines are the Collect stage's second door: the Edge Server pulls readings from equipment on site, while a pipeline lets outside systems push what they have. Its output lands in the Store stage, as documents on Files and events beside your Alarms, and the API keys that secure it belong to the Integrate stage covered in Settings and the API reference. For continuous device data, start instead with the Edge Server in Device Administration.