Personal Problems into Productive Side Projects

I really enjoy building side projects. I love coding and I love solving problems, no matter how trivial or large they might seem. After creating dozens of side projects over the years, I thought it might be worth sharing the approach I take from concept to launch.

Scratch An Itch

All my projects start off with a personal problem I want to solve. When I built the COVID19API I wanted to understand the likelihood or being trapped in Switzerland, when I built Fight A Bear I needed an easy way of having randomised HIIT workouts created, and recently GitPilotAI was built to automate away the mundane.

The scope of these problems is narrow enough for me to be productive and get to work, without spending loads of time trying to understand technical aspects that will have a large impact down the line. The key is to get coding as soon as possible, and in order to do that you need to understand a narrow problem deeply enough.

Some problems I want to tackle are larger, like building an automated trading algorithm system, but I still take the same approach. I understand roughly the system architecture and scope of work and find some logical grouping for pieces of functionality. These are hardly ever perfectly scoped, but you work to minimise any potential damage from bad decisions.

Code Early And Rough

When I get down to coding, it’s normally hacking at first. I want to test one single thing first and I focus on that. I write in Go, so this normally means putting that functionality in the main function and just running the program.

For example, if I want to have an automated note categorisation system, one key element is to be able to parse notes in a given format. I start there, building out a parsing function using a hardcoded file path on disk. Once it works, I break the functionality out into a separate function and abstract out what I can.

When you’re doing this part of the process, don’t think too deeply about what this might mean for the broader system. Just focus on a very narrow piece, hack until it works, and then work your way up. All the tidying and good coding practices can be built from the solid foundation of functioning code.

It’s normally during this time that I’ll start adding in some unit tests. Once it’s working, write a test confirming it’s working (both positive and negative testing, with a few use cases for each).

Release And Iterate

If I want to learn new patterns in Go, or a new way to develop (TDD or DDD) I might pick a problem and use it to explore these skills. I also might just want to solve a problem in the best way I know how to.

Some projects are used as a vehicle to develop your skills and others are used to implement your existing skills. This will impact what you do when you are ready for release. Being ready for release can mean be being satisfied that you learnt some skill and leaving the repo static, using the tool as is and never touching it again, or using it as a starting point for feedback from yourself (and maybe others) using this product.

If improving this product is something you want, then you’ll need to iterate from feedback. The advice here is the same: focus narrowly. Take note of all feedback, but choose what to implement based on your area of focus in the project. Only once you are satisfied that narrow part is working as it should then you should move on.

A typical case of this is as follows. You’ve built your little project, maybe it gives spending insights of government based on public data. The problem you wanted to solve was “how does the government spend collected tax money, and how much return do they get on that spending?”. The first implementation could result in a webpage with no CSS at all, a table and some summary text. The biggest piece of initial feedback you are likely to get is “this needs to be way better looking, I want to see the data in graphs and query the data”. You need to decide if this aligns with the narrow scope, and it will likely not. Is the data being returned accurate? Does it give you the base level of functionality to answer your question? If not, you’re not ready to move on.

Expand Or Be Satisfied

Depending on your initial goal, you now decide where you want to head with the project. It might be time to move on, or there might be some desire left to move it forward. If you move forward, you’d repeat the process above.

Using this iterative approach lets you tackle a few things. You learn how to break a project down into what actually matters, work on that until it is solved, and through doing so get to learn a new skill and way of working. Side projects are my main vehicle for self-improvement and refreshing my skills, and when the projects are interesting I find myself spending almost all my free time on them. The upshot is all that time is spent growing my skill level and capacity.