Week 4 · Day 18 · Thursday 1 October

Measurement

Sivaguru Ayyadurai · Founder, LotsTech13 min readTask: 60 minSubmit your work

You now have work in eight channels and no way of knowing whether any of it did anything. Without today, everything you produce gets judged on how it felt — and how it felt is reliably wrong.

1. A number with no decision attached is decoration.

The test before you measure anything: what will someone do differently depending on the answer?

If nobody will act on it either way, do not collect it. This single question removes most dashboards.

It also explains why "impressions went up" is not a result. Nobody has ever changed a plan because impressions rose. It is a number that exists to be reported, which is a different job from being useful.

2. Instrumentation comes first, or nothing else is possible.

Before any analysis: do the events even exist? A small set of stable, named events through the core flow, ending at the value moment, each firing once and in order, with a written definition of what each means.

If they are not there, that is the finding. Name the events needed and where they belong. Do not approximate.

An analysis built on events that fire twice is worse than no analysis, because someone will believe it.

This is the least glamorous work in marketing and it gates everything else. A team that skips it spends a year making confident decisions from numbers that were wrong in a consistent direction.

3. The small set that actually matters.

Resist the dashboard. Before scale, these are the numbers:

ActivationShare of new accounts reaching the value moment. Not signup — the moment from Monday's lesson.
RetentionMeasured against the product's real rhythm. A weekly tool and a daily tool need different windows.
ConversionVisit → signup → activation → payment. Every step, not just the ends.
Channel performanceCustomers produced. Section 4.
Churn and its reasonsOnce there are customers to lose.

On retention, one trap worth naming: using the wrong window manufactures good news. Measure a weekly tool on a monthly window and retention looks excellent, because almost everyone returns within a month. The number is true and it is telling you nothing.

4. Channels are judged by customers, not traffic.

This is the one that changes behaviour.

Traffic on its own does not tell you a channel is working. A channel sending 10,000 visitors and no customers is not obviously better than one sending 40 and producing three, and it is more dangerous, because it consumes attention, feels like success, and keeps being fed. How to judge a channel that is genuinely too young for customer numbers is the next section.

Traffic is the easiest thing to move and the easiest to mistake for progress. Every marketer who has looked busy for a year while nothing happened was watching traffic.

5. The ladder — and why week one cannot use the top rung.

Section 4 is right and it cannot be applied immediately. With five weeks of execution and a product with no customers, no channel will produce enough customers to judge on that alone.

So you need rungs, and the discipline to judge a young channel on the right one:

  1. Business outcome — customers, revenue. The real answer, and slow.
  2. Leading indicator — signups, replies, demo requests. Moves earlier, correlates decently.
  3. Activity — published, sent, posted. Only tells you the work happened.
  4. Quality signal — a reply that says something real, a thread that goes well, someone quoting you.

A three-week-old channel gets judged on rungs two and four. Abandoning it because rung one is empty is a mistake — and so is declaring victory from rung three, which is the more common one.

The rule: decide which rung a channel is being judged on before you start it, and write it down. Otherwise you will pick whichever rung looks best when the review arrives.

6. Always report the window and the denominator.

A number alone is unusable and frequently misleading.

Unusable: "Activation is 34%."

Usable: "34% of the 61 accounts created in August reached first publish within 7 days."

The second tells you the population, the window, the definition of success and the sample size — so you can tell that 34% of 61 is twenty-one people, and that moving it by one person changes the percentage by nearly two points.

Which leads to the discipline that stops most bad decisions: small numbers move a lot on their own. Do not editorialise normal fluctuation into a trend. If a change is inside the noise, say so.

7. The weekly review, and the decision to do nothing.

Same slot, same numbers, every week. The rhythm matters more than the dashboard, because a fixed cadence is what makes an unusual week visible.

Each review produces exactly three things:

1. What changed — one line, with what you think caused it. And say when you do not know rather than inventing a cause. "Signups doubled and I cannot tell why" is a legitimate and useful entry.

2. One decision. "Do nothing this week" is a valid decision — when it is written down as one. An unrecorded decision is not a decision, it is drift.

3. The history kept, so trends stay visible across months rather than being re-litigated weekly.

The second point is the one that distinguishes a review from a report. A report tells you what happened. A review forces someone to choose.

8. Gauge — and honesty about what the data cannot tell you.

Gauge describes itself the right way: you are the feedback loop. Without you the rest of the team ships work and judges it on how it felt.

It starts every request by asking what decision the number serves, and if nobody will act on it either way, it says so and does not spend the query.

The part worth taking most seriously is what it is told about gaps:

Say what the data cannot tell you. Attribution is partial. Sessions are approximate. Small samples move on their own.

Never fill a gap with an estimate presented as a measurement. If a number is unavailable, the answer is that it is unavailable.

That is Day 7's source tiers, arriving in the place where the temptation is strongest. Nobody wants to answer "how did the blog do" with "we cannot tell yet" — but a fabricated attribution number is worse than an honest gap, because someone will plan against it.

9. What this did not cover, and how to go deeper.

Not covered:

  • Analytics tooling — which product to use and how to configure it.
  • Attribution models — first touch, last touch, multi-touch, and why all of them are partly wrong.
  • Cohort analysis — comparing groups over time properly.
  • Statistical significance — when a difference is real. Most small-site tests never reach it.
  • Unit economics — acquisition cost against lifetime value.
  • Privacy and consent — what you may track, which is narrower than it was.

Where to go. Pick one number for something you care about, write down what you expect it to be, then look. The gap between your expectation and reality is the fastest way to learn how bad all of us are at estimating this.

Today’s task — a measurement plan for your two channels (60 min)

Objective: know in advance how you would tell whether your two channels worked.
Evidence: your Day 9 channel choice, everything you have built since.
Deliverable: a measurement table, a baseline, and instrumentation gaps.
Review: Editor and your checklist.
Signal: could someone else run your weekly review from this table alone?

1. Name the value moment again (5 min)

From Monday. Everything on the table below has to point at it, so it goes at the top of the page.

2. Build the table (25 min)

One row per channel, for the two you chose on Day 9:

Channel · Objective · Leading indicator · Lagging indicator · Baseline today · Which rung it is judged on · Review period · Stop or change rule

Then, for every row, the question that stops metric-collecting: what decision will this number help me make?

If a row has no answer, delete the row.

And one more column, because a number can be honest and still mislead you: what would make this number misleading? Too small a sample, a conversion that arrives weeks later, one person counted twice, a seasonal week, tracking that was broken on Tuesday. Write the specific risk for that row, not the general worry.

3. Record today's baseline honestly (15 min)

For each number: what is it right now? For most of them the answer is zero, or unavailable. Both are real answers.

You already have one real baseline — Day 13's AI visibility record. Use it, with its date.

Where a number is unavailable, write unavailable. Do not estimate.

4. Name the instrumentation gaps (10 min)

What would need to exist for you to measure what you just listed? Name the specific events and where they belong — signup, account connected, first post published, and so on.

This list is genuinely useful to me. It is the difference between measuring next month and guessing next month.

5. Write one stop rule you would honour (5 min)

For your first channel: what would have to be true, by when, for you to stop and do something else?

Make it something you would actually act on. A stop rule you would explain away is not one.

Then post one line

In #daily-log: your stop rule — the condition and the date.

Tomorrow you build the plan these numbers judge.

Tomorrow — Day 19: The first 30 days. The last lesson. Everything you have made becomes one dated plan — and the best one becomes what we actually do.