Designing Explainable Opportunity Detection in SaaS Analytics

Analytics products often start with a simple request: “Show users where they should take action.”

The tempting implementation is a score.

const score =
  impressions * 0.2 +
  clicks * 0.4 +
  position * 0.3 +
  ctr * 0.1;

It looks useful. It is also difficult to explain.

What does a score of 74 actually mean? Why did one item outrank another? And does the same threshold make sense for every customer?

A better architecture keeps observations, context, hypotheses, and actions separate.

Treat dimensions as pairs

Metrics rarely make sense without their associated resource.

Instead of analyzing a query independently, model the query and destination together:

type OpportunityCandidate = {
  query: string;
  resourceId: string;
  period: DateRange;
  comparisonPeriod: DateRange;
  impressions: number;
  clicks: number;
  ctr: number;
  averagePosition: number;
};

This prevents a common analytics mistake: detecting an interesting metric while ignoring where the traffic actually landed.

Preserve context before classification

Do not immediately label a candidate as “good” or “bad.”

First store the observed state:

type Observation = {
  candidateId: string;
  filters: Record<string, string>;
  trend: "up" | "down" | "stable";
  relevance: "high" | "medium" | "low";
  resourceMatch: "strong" | "weak";
};

Keeping filters and comparison periods attached to the observation is important. Otherwise, two values that look comparable may come from different devices, countries, date ranges, or traffic segments.

Avoid magic thresholds

Rules such as:

if (position < 15 && ctr < 2) {
  return "opportunity";
}

are convenient but brittle.

Thresholds hide assumptions and often become invalid as customer size, traffic volume, and market conditions change.

Instead, let the system surface a small candidate set and explain why each item was selected.

{
  reason: "High visibility with weak interaction",
  evidence: {
    impressions: 12400,
    ctr: 0.8,
    trend: "stable"
  }
}

Users can now understand the recommendation.

Store hypotheses separately from facts

An observation is not a diagnosis.

Model the next step explicitly:

type Experiment = {
  observationId: string;
  hypothesis: string;
  change: string;
  reviewAt: Date;
  status: "planned" | "running" | "reviewed";
};

This makes analytics actionable without pretending the system knows causality.

It also encourages one controlled change at a time, making later comparisons far more useful.

Good analytics software should not merely generate more numbers. It should help users move from evidence to a test while preserving enough context to understand whether that test actually worked.