Building 24/7 Post-Campaign Influencer Monitoring for Indian D2C Brands

The Problem

Direct-to-consumer brands in India drop lakhs of rupees on YouTube creator campaigns, but the moment a video goes live, the tracking stops. Marketers check the view count after 48 hours, file a manual report, and move on. Meanwhile, long-tail discovery on YouTube means influencer videos often drive purchases weeks or months after publication. When I was building PulseROI to combine creator intelligence with continuous campaign monitoring, I hit a massive wall: how do you pull fresh engagement, comment sentiment, and view metrics across dozens of active creator videos every single day without blowing past your Vercel serverless function limits or hitting external API rate limits?

What I Tried First (and Why It Failed)

My initial approach was brutally naive. I wrote a single monolithic Next.js API route triggered by a cron job that fetched all active campaign videos in a loop, queried the YouTube Data API for each video ID, parsed the statistics, and wrote them directly to our Neon serverless PostgreSQL database.

It looked something like this:

// The naive approach that timed out on Vercel
export async function GET(request: Request) {
  const activeCampaigns = await db.query('SELECT * FROM campaigns WHERE active = true');

  for (const campaign of activeCampaigns) {
    for (const videoId of campaign.video_ids) {
      const stats = await fetchYouTubeStats(videoId);
      await db.query('UPDATE videos SET views = $1 WHERE video_id = $2', [stats.views, videoId]);
    }
  }

  return Response.json({ success: true });
}

This failed immediately as soon as our campaign count crossed 15 videos. Vercel's serverless function timeout hit the 10-second mark (and later 60s), throwing a dreaded FUNCTION_INVOCATION_TIMEOUT error. Even worse, if the YouTube API threw a transient 503 error on the eighth video, the entire batch failed, rolling back nothing and leaving the database in an inconsistent state. The synchronous loop model was completely dead on arrival for continuous monitoring.

The Actual Fix

To make 24/7 post-campaign monitoring reliable, I had to completely refactor the data ingestion pipeline into an asynchronous, batched worker pattern using Neon tagged-template queries and a strict 25-second Promise.race timeout wrapper to protect our Vercel function limits.

First, I split the monolithic loop into a batched queue processor that fetches videos in chunks of 5 and uses Promise.allSettled so that a single failed YouTube API call doesn't kill the entire batch:

import { neon } from '@neondatabase/serverless';

const sql = neon(process.env.DATABASE_URL!);

export async function processVideoBatch(videoIds: string[]) {
  const results = await Promise.allSettled(
    videoIds.map(async (videoId) => {
      try {
        const stats = await fetchYouTubeStatsWithRetry(videoId);

        // Using Neon tagged template literals safely
        await sql`
          UPDATE campaign_videos 
          SET views = ${stats.viewCount}, 
              likes = ${stats.likeCount}, 
              comments = ${stats.commentCount}, 
              last_checked_at = NOW()
          GIVEN video_id = ${videoId}
        `;
      } catch (err) {
        console.error(`Failed to update video ${videoId}:`, err);
      }
    })
  );

  return results;
}

Next, I wrapped the main execution handler in a 25-second timeout race to ensure the cron job cleanly exits and logs its state before Vercel terminates the instance:

export async function GET(request: Request) {
  const timeoutPromise = new Promise((_, reject) => 
    setTimeout(() => reject(new Error('Cron job timeout guard triggered')), 25000)
  );

  try {
    await Promise.race([
      runMonitoringPipeline(),
      timeoutPromise
    ]);

    return Response.json({ status: 'success', timestamp: new Date().toISOString() });
  } catch (error) {
    console.error('Monitoring pipeline error:', error.message);
    return Response.json({ status: 'partial_success', error: error.message }, { status: 500 });
  }
}

What Surprised Me

Working with Neon serverless and Vercel functions taught me a harsh lesson about connection pooling. In my local environment, opening a new database connection per query felt instantaneous. In production, cold starts combined with opening multiple synchronous database connections inside an asynchronous .map() loop caused connection exhaustion errors from Neon.

Switching to the HTTP-based @neondatabase/serverless driver solved the TCP socket exhaustion entirely because it communicates over stateless HTTP rather than maintaining persistent TCP connections. If you are building serverless tools on Next.js targeting Indian infrastructure stacks, standard node-postgres drivers will bite you during traffic spikes.

The Result

By moving to batched asynchronous processing and adding the 25s timeout guard, our cron job execution time dropped from unstable timeouts (>30s failures) to a consistent ~3.8 seconds per batch run. Campaign tracking data for Indian D2C brands now updates continuously without missing records, giving founders precise visibility into which YouTube creators actually drive long-tail conversions long after the initial campaign launch.

Where to Go From Here

Next, I am working on integrating Gemini 3.8 Flash to automatically analyze comment sentiment changes during these 24/7 monitoring cycles to spot consumer objections before they hurt brand sentiment. If you want to see how this technical architecture supports automated ROI tracking, explore how it works directly at PulseROI.