Automate Your Export

Automate recurring Pulsedive Feed exports and integrate them into your existing security tooling.

​

How Often to Pull

For CSV exports, pull at most every 8 hours. Meaningful changes will still be captured within a reasonable timeframe for detection and alerting if you pull as little as once a day. Pulling more frequently than that adds unnecessary load without a meaningful benefit, since CSV data may be cached for up to an hour on Pulsedive’s end. For STIX/TAXII exports, use the added_after filter or the next pagination token to pull only new or updated objects since your last successful request. To learn more, visit Pagination.

​

Understand How Filtering Affects Your Dataset

This is the most important thing to get right when automating. When you filter by risk level or retirement status, your export only contains indicators that match your filters at the time of the pull. If an indicator’s risk score or retirement status changes between pulls, it may appear in one export and not the next. For example:

• An indicator filtered in at high risk may drop out of your next export if Pulsedive reassesses it to medium.
• An indicator that was active may disappear from your export once it’s retired, if you’re pulling active-only.

This means a simple diff between two exports may not reflect the true state of the dataset for your use case. Full reload vs. diff: If your use case requires an accurate, up-to-date picture of the entire filtered dataset, wipe your local storage and reload from the full export on each pull. If you’re only interested in what’s new, a diff approach works, but understand that deletions from your filtered view won’t surface as explicit records.

​

Custom Pipelines for CSV

These practices apply to any custom integration, regardless of your tooling or platform:

• Store your Direct URL as a secret: It contains your API key. Use environment variables or a secrets manager, not hardcoded values.
• Handle empty fields: Any optional CSV field may be empty for a given indicator. Your parser should handle this without errors.
• Parse multi-value fields carefully: The threats, feeds, and riskfactors fields use commas to separate multiple values within a single column. Make sure your parser treats these as inner delimiters, not column separators.
• Handle the :retired suffix: If you include retired indicators, the risk field may contain values like high:retired. Split on : before comparing risk values in your logic.

To learn more about CSV field formats, visit CSV Export.