The recommender system you've always wanted

I mentioned last week that I’m planning to start freelancing. I’ve been about two weeks away from doing that for the past six months. On one hand, it’s not necessarily a huge change—I’ll still be working on The Sample. But shifting away from full-time startup mode has some interesting psychological effects.

I’ve been focused primarily on marketing and growth for a while now. Since The Sample has passed the “quantum of utility” threshold, I figured better to get that growing week-over-week before I invest more in product development. But… ultimately my core strength is in software invention, and I kept worrying about if focusing 100% on growth was actually not the highest leverage use of my time.

So the nice thing about freelancing-soon-I-guess and not thinking about The Sample as a “startup” anymore is that I feel much more free to work on whatever feels worthwhile, instead of just what my head says is worthwhile. And… perhaps that’s the best startup strategy anyway.

I’ve thought before that perhaps it’s better in the early stages not to think of yourself as doing a startup; and I attempted that strategy at least once. The problem is I’m not good at tricking myself. I can cover my ears and say “I’m not doing a startup, lah lah lah,” but unless I actually have some sort of income… then I’m still doing something full-time that I’m really hoping starts to make money at some point. The psychological pressure is still there.

Since I decided to start freelancing, I felt sad for a day, then after that I could feel the psychological pressure start to lift. Today I jokingly thought “with all this morale, I could forget about freelancing and just keep working on the startup!” (However I’d prefer to have an ongoing stream of morale, rather than a quick hit).

The timing is pretty good since Findka Essays finally toppled over the other day, so on all fronts I have the perfect excuse to do something I’ve been thinking about for a couple weeks: set up a separate API server for The Sample’s and Findka Essay’s algorithm. Right now, for both products, the algorithm code runs on the web server. Also, when I wrote it, I was more focused on getting things done quickly than making it very scalable in general. So the result is that Findka Essay has finally accumulated enough data that the poor web server can’t take it any more.

Since I hate rushing myself, I just told everyone that it’ll be down for the next few weeks. In the mean time, we’re going to set up a single new server just for the algorithm—and we’ll design the data model so it can handle multiple types of content. That way the new server can generate recommendations for both Findka Essays and The Sample. Besides making maintenance easier, there are some other benefits:

• We can easily create new recommendation services, like a book recommender. All we have to do is set up a simple website to ingest users’ book preferences somehow and deliver the recommendations from the API server.

• We can share preference data between the services easily. So if you subscribe to Findka Essays and The Sample, we can recommend newsletters based on essays you like (and vice-versa). And if we make a book recommender, it can take advantage of both your newsletter and your essay preferences.

• When we make improvements to the algorithm, it will benefit all the services at once.

So basically, we can focus on making a great recommendation algorithm and start releasing a proliferation of front ends for different kinds of content. (Side note, I announced a huge update to Biff today! That’ll make it fast for us to build all those front ends). We could even make the front ends open-source.

Anyway. The proliferation thing is just a possibility; all we know for now is we gotta at least get that API server up. And I need to actually get at least one client. (If you have any leads, let me know). I’m also going to apply to Emergent Ventures with a complementary idea, and perhaps also Clojurists Together for Biff. And of course, I’ll keep trying to grow The Sample. With freelancing income, I’ll probably focus more on paid acquisition. I’ve had newsletter ads that got subscribers for under $2 a pop, so honestly if I can figure out how to do that consistently, that might be good enough.