Dedup Plugin

Run webhooks, queue messages, and user actions once per window across the fleet.

Overview

Webhook providers redeliver. Queues are at-least-once. Users double-click. @nestjs-redisx/dedup gives you an atomic "have I seen this key in the last N seconds?" - one SET NX PX per check - plus a distributed trailing debounce for collapsing bursts into a single execution.

Dedup vs idempotency: this plugin is a lightweight first-wins marker - nothing about the execution is stored or replayed. If the caller needs the original response back (HTTP retries of a payment request), use @nestjs-redisx/idempotency, which fingerprints requests and replays results.

Key Features

• First-wins markers - exactly one winner per key per window, fleet-wide, in a single atomic operation
• Failures don't burn the window - a thrown first execution releases its own marker (CAS on the owner token), so retries aren't locked out until the TTL
• Distributed trailing debounce - a burst of calls collapses into one execution after the quiet window, on the instance that made the LAST call
• @Dedupe - proxy-based decorator for any @Injectable method with {n} key templates
• errorPolicy - gates fail open (default: run, duplicates possible during an outage) or closed (skip + throw); manual APIs are always strict
• Testable without Redis - runs on the in-memory testing driver

Installation

ioredis

npm install @nestjs-redisx/core @nestjs-redisx/dedup ioredis

node-redis

npm install @nestjs-redisx/core @nestjs-redisx/dedup redis

Quick Start

import { Module } from '@nestjs/common';
import { RedisModule } from '@nestjs-redisx/core';
import { DedupPlugin } from '@nestjs-redisx/dedup';

@Module({
  imports: [
    RedisModule.forRoot({
      clients: {
        host: 'localhost',
        port: 6379,
      },
      plugins: [
        new DedupPlugin({
          defaultWindow: 60_000, // repeats within a minute are duplicates
          errorPolicy: 'fail-open', // store down -> run anyway (at-least-once)
        }),
      ],
    }),
  ],
})
export class AppModule {}

Deduplicate a webhook:

import { Injectable } from '@nestjs/common';
import { Dedupe } from '@nestjs-redisx/dedup';
import { StripeEvent } from './types';

@Injectable()
export class WebhookService {
  // Providers redeliver: each event id is handled once per 5 minutes,
  // fleet-wide. Duplicate deliveries resolve to undefined without running
  // the body. A THROWN first attempt releases the marker, so the provider's
  // retry is not locked out for the rest of the window.
  @Dedupe({ key: 'stripe:{0}', window: 300_000 })
  async handleStripeEvent(eventId: string, event: StripeEvent): Promise<void> {
    // ...apply the event exactly once per window...
  }
}

How It Works

• isFirst / dedupe set a marker key (SET NX PX) holding this caller's token; only the caller that created the key is "first".
• When a dedupe execution throws, the marker is released via a CAS Lua script keyed on the token - a newer marker set by someone else is never erased.
• debounce stamps the caller as the LAST one for the key, waits out the quiet window in-process (unref'd timer), then atomically claims the slot - only the still-last caller executes.

Next Steps

• Configuration - windows, errorPolicy, prefixes
• Service API - isFirst / dedupe / debounce / release
• @Dedupe - decorator options
• Recipes - webhooks, queue consumers, double-click guards, burst collapsing
• Testing - no-Redis test setups
• Troubleshooting