Our newsletter platform picks the winning subject line by open rate. In a recent test, one version had noticeably more opens, but both versions produced almost the same number of clicks and purchases.
We have subscribers using Apple Mail, and I’m trying to understand how Apple Mail Privacy Protection affects the result. I don’t want to call a subject line successful just because tracking images were loaded without someone reading the email.
For email subject line testing, would you still use opens as a directional signal, exclude opens the platform flags as privacy-generated, or choose a different primary metric? Clicks seem more useful, but our list is small enough that a single campaign produces relatively few of them.
How would you design a test that gives a small newsletter a meaningful result without overstating the evidence?
