2024: Getting Back to Blogging, and Building in Public
In which I lay out the projects I plan to half-finish this year, and set the intention to start blogging regularly again.
I’m in serious danger of breaking a very achievable resolution I set for 2024: to blog once a month. Since I’ve already broken my other resolutions (avoid New Jersey, stop calling ChatGPT “homie”), it’s important to me to follow-through on this one.
I’ve been wanting to get back into blogging for years. Putting my thoughts down on paper and hitting publish is a deeply clarifying process for me - something about publishing to an audience, even an imagined one, forces me to structure my thoughts.
The main barrier is my fear of writing poorly - in high school, I could confidently publish drivel week after week (see 16 year old me’s “Harry Potter Spells Are Like Websites”, or the smug Machine Learning blog I wrote while getting a C in Precalculus). Unfortunately, today, a crust of minimal self-awareness now provides friction to being a dumbass-at-large on Wordpress.
My goal is to once again shed that crust - if I want to make my writing less drivelsome, I’ll only get there through forced practice and repetition.
In addition to writing practice, and clarity of thought, another benefit of blogging is the sense of accountability when I hit “Publish”. Is it rational to feel accountable to a crowd of crawler bots and the occasional salesperson trolling through LinkedIn? Maybe not, but the Baidu web spider has visited every one of my projects, and I’d hate to disappoint it.
The Building in Public movement has resonated with me for years on these grounds; I have a huge graveyard of 20% finished side projects, and if the parasociality of blogging about them can get me to actually finish one, it’d be a successful experiment. At this point, I’d even take getting to the 40% mark!
So, all of that being said, what do I want to work on this year?
Ursula
Ursula (named for Ursula Le Guin) will be a a stack-ranking app for books, a la https://beliapp.com/. I had this idea last year, when I started using Beli more. A stack rank, as opposed to stars, forces you to decide which is better between two options, and, at least on Beli, provides a much stronger signal than a star ranking.
A huge caveat here is that Beli is young, and hasn’t yet had to contend with the poisonous forces of monetization, which is probably a larger factor in driving down the rating quality of Yelp and Google than the star form factor. I think it’s cool, though, and I’d love to try it out with books.
I’d also like to integrate AI book recommendations with LLMs. Hopefully more on this soon!
AI Engineering Musings and Weatherwax
I have big plans to be another asshole talking about OpenAI on the internet. But like, really.
Working on Waverly for the past few months has thrown me in the deep end with AI engineering. I don’t want to oversell my experience here - it’s just been about 4 months of working with LLMs full time. But ~500 hours in, while stochastic programming still feels just plain strange, I’m starting to form opinions and intuition about it.
I don’t have any particuraly ground-breaking insights here (evals are really important, and I agree with pretty much everything this blog post: https://arvid.xyz/posts/prompt-design/). I still want to talk about it, though!
I’ve also come to dislike LangChain, and most other libraries that provide abstractions over the actual process of prompting the model yourself. It’s undeniable that you want some level of abstraction over any individual provider’s API, though - it’s crucial that any AI project be able to hot-swap models.
Everybody has their own solutions to this (and about half the class of any given YC batch is trying to build a new one), but after wrestling with it a few times in a few different projects, my preferred answer is an LLM proxy server that just abstracts the OpenAI API spec (which is becoming the de-facto standard among all providers) into a websocket server that you can use as a service.
I’d like to build a Rust implementation of this proxy server and publish it with clients in a number of languages. I’d also like to call it Weatherwax, after the character from Discworld.
I’ll write more extensively about motivations for this when I get around to building it, but a large one is that I’ve also come to believe that LLM integration should happen as close to the client as possible - I consider separating LLM calls into a service with a different API in your apps backend to be an antipattern.
Revamp of This Website
(update 2/20/24: this is done! you're lookin' at it :) )
I’ve been meaning to do this for years. I hacked together the current version of this website on a roadtrip freshman year of college, when I realized I was out of money and needed to start freelancing. It still uses jQuery (🙁), and I’m afraid to touch the server it runs on for fear I’ll never get it back up.
I started remaking it in the summer of 2022 but two things went wrong:
- I got instantly distracted by every other possible task.
- My goals for it got impossibly large - not only was it going to have a custom built portal for my consulting, but I’d also build the CMS myself, and I’d make a little link shortener too. And ooh! Maybe I can also copy that cool file sharing tool that BCG has internally. And it has to look sick.
This time I’ll aim to keep myself to somewhat more modest goals. I want a flexible foundation that I can eventually add all these things to, but first I must make the foundation. And I should build it in something profoundly boring and reliable.
Playground for AI experiments
I have a number of small one-off ideas I want to publish with LLMs.
For example, I think most chat-with-your-documentation implementations have fallen short for me because they try to chat with the full site. What I really want is a little chatbox under each page where I can ask follow-up questions. “How do I use this with middleware?”, “This isn’t working, what CORS settings can I mess with?”.
Other small things I might put here are a john henry (https://en.wikipedia.org/wiki/John_Henry_(folklore)) style game where you have to try and build a working web app faster than an LLM, some fun interactive creative writing exercises, and maybe some home grown evals (perhaps in the form of a snarky “are we agentic yet”?)
This should be pretty easy to set up - I just need to revamp my site first!
Do a Triathlon
This isn’t code related, but maybe writing it down will force me to actually find a good pool to train in in Brooklyn! The Gowanus Canal is starting to look mighty appealing with what pool access costs in this city.
In Conclusion
Time will tell whether I follow-through on any of these things. If you’re a human who’s made it to end though, and any of these interest you, please get in touch about them! I’d love a collaborator, user, or just an interesting conversation on any of them. My email is will@willbeddow.com.