26 June 2021
I paused Findka Essays, an essay recommender system, because it was struggling to deal with the amount of data that had accumulated. Relatively speaking there isn't all that much data—about 600k documents—but I wrote the code quick-and-dirty, and it was time for a refactoring.
Until recently, Findka Essays and The Sample have each had their own machine learning code running on their own web servers (a Python script ran as a subprocess). I have now set up a separate recommendation service that can store preferences for any type of item (essays, newsletters, or anything else) and provides recommendations via an API. I've rewritten Findka Essays to use this API, and my next step is to have The Sample use it as well.
Along the way, I've made a few simplifications. Most significantly, I decided to remove the feed subscription thing that mixed RSS feeds in with your other recommendations. I've done this because my overall strategy for Findka has shifted significantly in the past several months: instead of making a single application that does lots of things, I plan to make many small apps that work together seamlessly. I would rather keep Findka Essays focused on its regular essay recommendations; feed muxing can be handled by another app.
This new recommendation service will be at the heart of all these small apps. For example, I'd like to make a bookmarking service (like a free version of Pinboard); so you can bookmark things and then have those influence your recommendations from Findka Essays and The Sample. Eventually I'd like to release an API so that anyone can write apps (like a feed muxer!) and integrations that plug into Findka's preference database. I'll expose an endpoint for recommendations, too.
But for now, my top priority is to grow The Sample.