How Daynostic decides an insight is real

Telling a real pattern apart from a coincidence is a statistics problem, and not a simple one. Two variables that drift together on a chart, or a handful of days that happen to line up, are not enough to call something an insight. Daynostic does not work by plotting two series, finding where they cross, and declaring a discovery. Behind every insight is an analysis pipeline assembled from established statistical methods, the kind used in scientific research and taught in graduate statistics courses. This article describes what that pipeline does to a set of daily check-ins before anything reaches the screen.

Rank correlations

Daynostic compares pairs of tracked variables, such as sleep quality and next-day mood, using a Spearman rank correlation.¹ A Spearman correlation operates on the order of the values, not their raw magnitudes: it measures whether the days that rank highest on one variable tend to rank highest on the other.

The reason for working with ranks is that daily life data is noisy and poorly behaved. A single extreme day, food poisoning or a 2 a.m. fire alarm, can drag an ordinary average well off course while having almost no effect on a ranking. Rank-based methods also avoid treating a 1 to 5 mood scale as a precise measurement instrument. It is not one, and analysing ordinal ratings as if they were exact quantities is a common source of error in this kind of work.

Permutation testing

A correlation coefficient on its own says little. A coefficient of 0.4 across ten days carries almost no weight; the same value across several hundred days is a different matter. The textbook approach derives a p-value from a formula, but those formulas assume the data follow a particular distribution, which life data rarely does.

Daynostic uses a permutation test instead.² It takes the actual data, shuffles the day labels thousands of times, and measures how often a correlation at least as strong as the observed one appears by chance in the reshuffled history. When random shuffles routinely produce a correlation as strong as the real one, the pattern cannot be separated from chance, and it is discarded. The more data behind a correlation, the harder it becomes for a shuffled version to match it by accident.

Correcting for multiple comparisons

The number of variable pairs grows quickly as more things are tracked. Testing many pairs raises the probability that at least one will appear significant by accident, a well understood issue in any field that runs large numbers of statistical tests at once.

Daynostic applies the Benjamini-Hochberg false-discovery-rate correction³, the same procedure used in genomics and clinical research when thousands of hypotheses are evaluated at the same time. The effect is that the more comparisons are made against the data, the higher the threshold each individual result has to clear.

The weekend problem

Saturdays differ from Tuesdays. People tend to sleep longer, exercise more and report better mood on weekends. Left uncorrected, this produces apparent correlations between almost everything and almost everything else, when the common cause is the structure of the week. Daynostic controls for day-of-week effects before judging a correlation, so a weekly rhythm is not mistaken for a discovery about sleep or exercise.

Confidence tiers

Each association that passes the pipeline is labelled with one of three confidence tiers:

• Strong: passed every test with a clear margin, on a large number of days.
• Moderate: passed the tests, but on fewer days or with a weaker effect.
• Preliminary: an early result close to the threshold, which further data will either confirm or drop.

When no association passes, none is shown. A result appears once the statistics support it.

Putting it together

Each of these methods is standard in statistics. The work lies in combining them into a pipeline that runs automatically on a single person's data and holds to the same standards a researcher would apply by hand. An insight on the screen represents an association that has been ranked, tested against its own shuffled history, corrected for the number of comparisons made, and adjusted for the structure of the week. When Daynostic reports that sleep quality is associated with next-day mood, that statement has passed the same kind of scrutiny used to evaluate findings in published research.