Findka survey results

Last week I set up an automated survey for Findka users, based on the Superhuman survey. It gets sent to anyone who has used Findka on at least four different days and has been active in the past week.

So far it’s been sent to 83 people, and I’ve received 22 responses. The first question’s breakdown was:

• Not disappointed: 6 (27%)

• Somewhat disappointed: 12 (55%)

• Very disappointed: 2 (9%)

• No answer: 2 (9%)

If at least 40% of the responses are “Very disappointed,” it’s considered a strong indicator of product-market fit.

There were also six people who said some variation on “thanks for making this/I’ve enjoyed using this,” which made me happy.

Small features

I got a number of suggestions, a few of which I have implemented in the past couple days:

• You can now view all your favorited and submitted articles.

• I’ve added metadata (title and description) for all links which were missing it. I set up an admin console so I could do this manually (there are only ~280 articles so far). At some point I’ll improve my scraping methods so Findka can fetch the title and description even if they aren’t included in the appropriate HTML meta tags.

• I made a bookmarklet for submitting essays, similar to the one Hacker News has.

• I updated some links to use https:// instead of http:// (where it didn’t break the link). Again, I’ll continue to do this manually for now.

There are a few more features which I haven’t added yet but plan to soonish:

• Choose email delivery time.

• Include reading time estimate for links.

Algorithm

I also made some fairly significant algorithm improvements. Here’s a snippet from my daily usage stats email:

         Click  Like  Dislike  Ignore  n
Exploit  38.6%  7.2%  0%       54.2%   83
Explore  20.0%  0%    0%       80.0%   15

I made an algorithm change last night, so this only includes data from one day so far, and it’s not enough to be statistically significant. But hopefully it continues in this direction. Each link Findka sends has a 20% chance of being picked completely at random (“Explore”), and the rest of the time, the link is chosen by the recommendation algorithm (“Exploit”). This has the nice side effect of providing a control for my experiments.

(“Ignore” only includes links if at least one other link in the same email was clicked. So if a user doesn’t open an email, that doesn’t affect the statistics above.)

My next todo item is another significant algorithm change: I’m going to start doing some content-based filtering instead of just collaborative filtering. (Update next day: this is already done. One of the rare times where something actually takes less time than you expect it to…)

Bigger features

Multiple people suggested each of these features. I think they’re good ideas, but I’m not confident that now is the right time to implement them:

• Get recommendations via the website instead of just email. Perhaps give these one at a time instead of in a batch.

• A browsable/searchable directory of submitted articles. Perhaps include non-personalized recommendations (i.e. “people who liked this article also liked these other articles”, without necessarily taking into account the current user’s preferences).

The first idea was particularly appealing to me initially. Every content platform has a small percentage of hyper-consuming users. Serving recommendations via the website would encourage those users to rate more articles, which means Findka’s algorithm would accumulate training data faster (!). However, on further reflection, I’m concerned about potential unintended side effects. A regular email encourages forming a habit which is good for retention. It seems plausible that if Findka provided recommendations via the website, some users might use that exclusively and ignore the email, which could reduce retention. (I have no clue if this would actually be a problem or not, but that’s what makes me hesitate: I’d rather be sure the feature is a good idea first, or at least have enough users to run an A/B test). I also like the focus that comes from serving recommendations via email exclusively.

I’ll be thinking about these features as Findka grows.

Target audience

A few people people mentioned that they liked Findka’s minimalism/lack of distraction. For example: “I seem to read a lot about how people want to give up their reliance on social media… Findka gives you a lot of the good stuff with almost none of the bad stuff.” I definitely want to avoid the suck-you-in/engagement-maximizing-at-the-expense-of-users aspect that many apps have. Maybe that would be a good angle to focus more on. (“Discover good content without going insane”).

Speaking of social media

At some point (not soon), I want to try letting users submit links to online communities they’re part of (Slack workspaces, forums, Facebook groups, whatever). The website could then have a “Discover communities” section where you can get recommendations for communities based on your interests (i.e. the articles you like). I think something like this would be a much better/less toxic model for social media: “content discovery” and “people talking to each other” should be kept separate, and the latter should mainly happen in small, focused communities.

The thing that really interests me about this is not aiding discovery of existing communities per se, but how it would facilitate the creation of new communities. You could set up a Discord server with just yourself, and you could post your favorited articles from Findka. Maybe a handful of somewhat likeminded people would join and you could have high quality discussions with random strangers. I’ve always preferred a small number of close connections rather than a large number of shallow connections.