column, newspaper, dispute, opinion, author, a book, news, medium, media, article, mood, text, writing, write, publication, journalist, journalism, publish, column, opinion, article, journalist, publish, publish, publish, publish, publish. Fraud screening produces numbers: these examples turn them into written decisions
Photo by viarami on Pixabay

Industry

Part of 6 points on choosing fraud detection tools

Fraud screening produces numbers: these examples turn them into written decisions

Fraud detection tools produce numbers; these four written artifacts turn them into decisions. A screening note, a threshold policy, a dispute paragraph, a log.

Detection output is only as good as the writing around it. Whether the number comes from a fraud detection product such as HypeAuditor or Modash or from a manual review, four short documents do most of the work in agencies that handle this well, and none of them requires a particular product. The examples below give each one's wording. They are reproduced here in outline, with the reasoning, so they can be adapted rather than copied. Every value in them is left blank on purpose: a threshold that suits one agency's client mix is wrong for another's, and inventing one here would be worse than useless.

What to take away

  • A screening note that records what was checked, and what was not, is what makes a decision defensible later.
  • Thresholds belong in a policy written before the campaign, not chosen while looking at a result.
  • The most valuable artifact is the log of cases you decided were fine, because that is what calibrates the next decision.

The screening note

One short block per creator, written at shortlist stage, kept with the shortlist rather than in a tool.

Screening note: six lines

  • Checked onthe date
  • What was checkedspecific things
  • What was not checkedblind spots
  • Observationwhat you saw
  • Judgmentproceed, ask, decline
  • Who decideda name

Written out, the six lines read like this:

  1. Creator and date.The account checked, the platform, and the day the check was run.
  2. What was checked.The tool or report used, and the signals read: follower growth, engagement pattern, audience location, a sample of comments.
  3. What was not checked, and why.For example: "Audience location not reviewed; the creator has not shared platform analytics."
  4. What was seen.One sentence, without a verdict word: "Follower growth arrived in two steps in March; comments read as people."
  5. Decision.Clear, look further, or decline, and which zone of the threshold policy the case fell in.
  6. Who decided.The name of the person, not the team.

The third row is the one people leave out and the one that matters most. A note that says what was outside the check is honest. A note that omits it reads, six months later, as a claim that everything was fine. What a check can establish in the first place, and what it cannot, is the subject of the guide to choosing detection tools.

The threshold policy

The policy is two sentences with blanks the agency fills from its own record:

Look threshold. Any creator whose [measure] in [tool] is beyond [value] gets a manual review, written up as a screening note, before the shortlist goes to the client.

Act threshold. Any creator beyond [value] is not booked unless [named role] records in the decision log why the campaign should proceed.

Two thresholds, three zones

Where does the number sit?

Yes

Below lower threshold -> no action

No

Between thresholds -> human decides and records why

Writing them apart prevents the most common failure in this work, which is treating an unusual number as a finding. Unusual is common. In a large enough population, most people are unusual on some measure, and a rule that flags outliers will flag a great many creators who have simply had one post travel further than the rest.

The policy should also say what the thresholds are not for. They are not a quality judgment, not a reason to renegotiate a fee, and not something to share with a client as a headline figure. A client who sees a score will start asking for a score, and then the score becomes the deliverable. What does belong in a client report is covered in the guide to measurement and reporting.

The dispute paragraph

This one goes in the agreement, not in a report. Its job is to make a disagreement about facts rather than about method. Four elements.

  1. The evidence that counts.The platform's own analytics for a stated period, supplied by the creator, outrank any third-party score.
  2. The time to answer.The creator has [number] working days from a written request to supply that evidence.
  3. Who decides.One named person on each side reviews the evidence, and the agreement says who has the final word.
  4. What follows.What happens to the fee, the content and the usage rights if the concern stands, and if it does not.

In the agreement it can read: "If the Agency raises a written concern about audience authenticity, the Creator will supply screenshots of the platform's native analytics covering [period] within [number] working days. The Agency's [role] will review them with the Creator's representative, and if the concern is not resolved, [consequence]."

Four elements of the dispute paragraph

  • Evidence each side will accept, named in advance
  • Who produces it, and within what period
  • What happens while the dispute is unresolved
  • The remedy, expressed concretely

Agencies that have this paragraph resolve findings in days. Agencies that do not spend the same days arguing about whether a third party's number counts. The way this sits with disclosure obligations, which are a separate matter with their own rules, is set out in the Federal Trade Commission guidance on endorsements and reviews.

The decision log

The last artifact is the one nobody builds and the one that improves the work most. One line per case, including the cases you cleared.

One line per case

  • Date
  • What triggered the look
  • What was seen
  • What was decided
  • What happened afterwards

Date, what triggered the look, what was seen, what was decided, and what happened afterwards if anything did. A cleared case reads like this: "[date] | engagement jump crossed the look threshold | one post spread after a repost, comments read as people | cleared | campaign delivered as briefed."

Kept for a year, this becomes the only dataset you have that is actually about your creators, your categories and your clients. It tells you whether your thresholds are set sensibly, which is otherwise unknowable, and it lets a new member of staff learn in an afternoon what took the last person two years.

Kept properly, it is also the first thing a new account manager reads, in the same way a written record of what a tool was bought for serves the guide to campaign workflow software.

It also protects you from a subtle problem. If you only record the cases you rejected, your picture of a bad case is built entirely from cases you already thought were bad, and it will confirm whatever you believed when you started. Logging the clears breaks that loop.

What these four deliberately leave out

No scores. No percentages. No thresholds with numbers in them. This is not caution for its own sake: a number published on a page like this becomes a default, gets copied into a policy by someone who has not tested it against their own base rate, and then makes decisions about real people's work.

What the four artifacts leave out

Left out

Scores
Becomes a default
Percentages
Not interpretable alone
Numbered thresholds
Copied untested
Creator's side
Fifth document, unwritten

Why

Scores
Percentages
Numbered thresholds
Creator's side

A score without a base rate is not interpretable, and the same reasoning applies to any figure printed here.

They also leave out the creator's side of the conversation, which is a fifth document worth having: what you tell someone when a check goes against them, written once, calmly, in advance. Nobody writes that one well while a campaign is on fire. Where the counterparty is a manager rather than the creator, the conversation runs differently again, and that difference is set out in the guide to working through representation.

Common questions

Can we keep these inside our campaign tool instead of as documents?

Yes, if the tool lets you export them in a form a person can read. The point is that the reasoning survives the software, and most products model the score but not the judgment.

How long should we keep a decision log?

Long enough to be useful and no longer. It contains observations about identifiable people, so it needs a retention period and a reason, not an indefinite drawer.

Is a screening note not just extra admin?

It is admin, and it takes a couple of minutes. The alternative is that in eight months nobody can say what was checked, which turns a routine question from a client into a problem.

What if the tool and the reviewer disagree?

Record both, and record which one the decision followed. Over a year that record tells you whether the tool is earning its price, which no vendor material can tell you.

More in Industry

Latest from Field Desk