Polling Triggers

Polling Triggers check whether an App has new data on a regular interval. For example, a "New Email" Trigger in Gmail might fetch a list of emails in the user's inbox to determine if there are any new emails since the last poll.

Defining a Polling Trigger#

Polling Triggers are defined using the definePollingTrigger function.

The logic for checking for new data that should result in Trigger Events is defined in the Trigger's poll function. The poll function will return an object with two properties: newState and events. The newState is any data that should be persisted between Polling requests in order to determine what has changed since the last poll. The state stored as of the last poll is passed to the poll function as prevState. The events property is an array of Trigger Event payloads, which should adhere to the Trigger's payload schema.

What to Store as State#

A Polling Trigger's state should contain any data that the Trigger's poll function will need to determine if there is new data since the last poll.

Some values that might potentially be stored in a Polling Trigger's state:

• The date of the newest item seen when fetching data during the current poll. This can be used to filter items to only newer (and therefore not yet seen) data on subsequent polls. See the example Polling Trigger below.
• A set containing ids of the items returned in the first page of results. This can be useful for APIs that support sorting by the date an item was created, but doesn't support filtering.
• A set containing compound keys like <itemId>_<itemUpdatedDate>. This type of state could be useful for a Trigger that should fire when an existing item has been updated rather than when a new one is created.

Example Polling Trigger#

import { definePollingTrigger } from "@rollout/framework";

import { MyAppCredential } from "../../auth";
import { inputParamsSchema } from "./input";
import { payloadSchema } from "./payload";

type State = Date;

export const trigger = definePollingTrigger<MyAppCredential, State>()({
  name: "My Polling Trigger",
  inputParamsSchema,
  payloadSchema,
  async poll({ inputParams, prevState, credential }) {

    // prevState is null on first poll. Since Rollout ignores Trigger events on the first poll, we return early
    if (prevState == null) {
      return { newState: new Date(), events: [] };
    }

    // Fetch items created after the most recently created item that we've seen in previous polls
    const itemsFilterParams = new UrlSearchParams( { createdAtGte: prevState} );
    const itemsRes = await fetch(`https://api.example.com/items?${itemsFilterParams}`, { 
      headers: {
        "Content-Type": "application/json",
        "Authorization": `Bearer ${credential.data.accessToken}`,
      }
    });

    const { data: rawItems } = await itemsRes.json();

    const items = rawItems.map((ri) => ({
      ...ri,
      createdAt: new Date(ri.createdAt),
      updatedAt: new Date(ri.createdAt),
    }));

    // Find the most recently created item, so that we can use its createdAt date as our state
    const sortedItems = items
      .map((i) => i.createdAt)
      .sort((i1, i2) => i1.getTime() - i2.getTime())
    const newestItem = sortedItems[0];

    const newState = newestItem ? newestItem.createdAt: prevState;

    return { newState, events: items };
  },
});

Polling Interval#

By default, Polling Triggers will check for new data every 10 minutes, but this can be configured for each Rollout Project Environment via an environment variable.

For example, to set the interval to 2 minutes, set ROLLOUT_TRIGGERS_MIN_TIME_BETWEEN_POLLING_SECS to 120.

Polling Jobs Timeout#

By default, the polling jobs (execution of the poll function) time out after 120 seconds, but this can be configured for each Rollout Project Environment via an environment variable.

For example, to increase the timeout to 5 minutes, set ROLLOUT_POLLING_TRIGGERS_JOB_TIMEOUT_SECS to 300.

Polling Jobs Concurrency#

By default, the polling jobs are executed serially, that is one at a time, but the concurrency can be configured for each Rollout Project Environment via an environment variable.

For example, setting ROLLOUT_POLLING_TRIGGERS_CONCURRENCY to 2 will increase the concurrency to 2 jobs.