Jacob O'Bryant
Home · Archive · Twitter · GitHub · LinkedIn
Findka survey results
20 November 2020

Last week I set up an automated survey for Findka users, based on the Superhuman survey. It gets sent to anyone who has used Findka on at least four different days and has been active in the past week.

So far it’s been sent to 83 people, and I’ve received 22 responses. The first question’s breakdown was:

If at least 40% of the responses are “Very disappointed,” it’s considered a strong indicator of product-market fit.

There were also six people who said some variation on “thanks for making this/I’ve enjoyed using this,” which made me happy.

Small features

I got a number of suggestions, a few of which I have implemented in the past couple days:

There are a few more features which I haven’t added yet but plan to soonish:

Algorithm

I also made some fairly significant algorithm improvements. Here’s a snippet from my daily usage stats email:

         Click  Like  Dislike  Ignore  n
Exploit  38.6%  7.2%  0%       54.2%   83
Explore  20.0%  0%    0%       80.0%   15

I made an algorithm change last night, so this only includes data from one day so far, and it’s not enough to be statistically significant. But hopefully it continues in this direction. Each link Findka sends has a 20% chance of being picked completely at random (“Explore”), and the rest of the time, the link is chosen by the recommendation algorithm (“Exploit”). This has the nice side effect of providing a control for my experiments.

(“Ignore” only includes links if at least one other link in the same email was clicked. So if a user doesn’t open an email, that doesn’t affect the statistics above.)

My next todo item is another significant algorithm change: I’m going to start doing some content-based filtering instead of just collaborative filtering. (Update next day: this is already done. One of the rare times where something actually takes less time than you expect it to…)

Bigger features

Multiple people suggested each of these features. I think they’re good ideas, but I’m not confident that now is the right time to implement them:

The first idea was particularly appealing to me initially. Every content platform has a small percentage of hyper-consuming users. Serving recommendations via the website would encourage those users to rate more articles, which means Findka’s algorithm would accumulate training data faster (!). However, on further reflection, I’m concerned about potential unintended side effects. A regular email encourages forming a habit which is good for retention. It seems plausible that if Findka provided recommendations via the website, some users might use that exclusively and ignore the email, which could reduce retention. (I have no clue if this would actually be a problem or not, but that’s what makes me hesitate: I’d rather be sure the feature is a good idea first, or at least have enough users to run an A/B test). I also like the focus that comes from serving recommendations via email exclusively.

I’ll be thinking about these features as Findka grows.

Target audience

A few people people mentioned that they liked Findka’s minimalism/lack of distraction. For example: “I seem to read a lot about how people want to give up their reliance on social media… Findka gives you a lot of the good stuff with almost none of the bad stuff.” I definitely want to avoid the suck-you-in/engagement-maximizing-at-the-expense-of-users aspect that many apps have. Maybe that would be a good angle to focus more on. (“Discover good content without going insane”).

Speaking of social media

At some point (not soon), I want to try letting users submit links to online communities they’re part of (Slack workspaces, forums, Facebook groups, whatever). The website could then have a “Discover communities” section where you can get recommendations for communities based on your interests (i.e. the articles you like). I think something like this would be a much better/less toxic model for social media: “content discovery” and “people talking to each other” should be kept separate, and the latter should mainly happen in small, focused communities.

The thing that really interests me about this is not aiding discovery of existing communities per se, but how it would facilitate the creation of new communities. You could set up a Discord server with just yourself, and you could post your favorited articles from Findka. Maybe a handful of somewhat likeminded people would join and you could have high quality discussions with random strangers. I’ve always preferred a small number of close connections rather than a large number of shallow connections.