The Sample publisher report (7 Mar)

Last week I discussed a new experiment we were planning: after someone rates or subscribes to a newsletter, instead of sending them to a list of their previous newsletter recommendations, just pick a newsletter at random and send them to that one. That way you never have to pick what to read; you just have to say if you liked it or not.

We started the test on Tuesday morning and in less than a day we were seeing a statistically significant improvement. Specifically, there was a 27% lift in the number of 1-click subscribes, which is kind of ridiculous. So we switched everyone over to that. Looks like we’re going all-in on “Tinder for newsletters.”

As part of that, I made it so if you’re signed in and you go to https://thesample.ai/, we’ll redirect you to the “show a random newsletter” page. (You get signed in automatically whenever you rate or subscribe to a newsletter from one of the emails, or when you click “your settings” near the bottom.) So feel free to bookmark that link. I added it as an icon to my home screen—like this, except I’m using Android + Firefox, not iOS + Safari.

We’re now in the middle of enabling “on-demand recommendations.” One of the nice things about structuring The Sample as a newsletter is that it was easy to code. We have a daily pipeline thing that runs every evening (pacific time). It takes a few hours, and at the end it decides what newsletters to send people the following morning. The upside is that it’s OK if our code is slow, and it’s operationally pretty simple—”run this thing once a day.” However that means that if you go through all the newsletters we’ve already sent you, you’ll have to wait till the next day to get another one.

So with the Tinder-for-newsletters thing, we really need to be able to always serve up another newsletter. Last week we did some architectural planning for how to do that, and I spent a couple days starting on the implementation. I’m hoping to finish that up within a few more days.

We’ve also made some plans for re-recommending newsletters. If we’ve previously sent you a newsletter, and (1) you didn’t subscribe to it, and (2) you didn’t click “show less like this,” and (3) there is a new issue from that newsletter, then we’ll give the algorithm the option of recommending it again. There will be a handful of guard rails to make sure nothing obviously annoying happens; e.g. if we’ve recommended the same newsletter to you 3 or 4 times and you still haven’t subscribed, we’ll probably stop recommending it.

Oh, and one last thing: to make on-demand recommendations work, we’ve had to redo the way that cross-promotion forwards are assigned. Ideally there shouldn’t be any noticeable change. I’m keeping an eye on it, and I’ll make further adjustments if it looks like anyone isn’t getting as many forwards as they should.

—Jacob