Yes, it’s always time to learn something new, but this is just what I’m focusing on now. Java was my first programming language, that I’ve been using since I learned in college. I’ve picked up some other languages, search as just enough C/C++ to get through class, Progress (which appears to have been renamed over the years), Python, a very brief experimentation with Scala, Javascript, and just enough Ruby to be dangerous to your Chef scripts. However, almost everything I’ve done professionally has been Java. And while I’ve been working as a software developer for my whole professional career, during that time the idea that developers own their code in production really took off, which means operations is becoming an increasing part of the software developer’s job description. So far, that’s been mitigating by using managed services on public clouds like AWS, which in turn has led to an explosion of more managed services, which handily are also big cash cows for these companies. But the emphasis on running services in addition to writing them means I need to be good not just at writing code, but also able optimize how I build it to answer questions about what’s going on when there’s user complaints, or things seem off, as well as be able to tell the business what the code I’m writing is doing for them.
Operating software is not running software, and infrastructure is not architecture
1 of the things I want to practice is operating software and dealing with infrastructure outside of public cloud vendor’s managed services. I’m still going to run my code on the cloud (because I’m not ready to go full-on home lab, although they do apparently make racks sized for Raspberry Pis and other similarly sized computers). But I don’t want to spin up cloud provider-specific resources (I may try to avoid even a managed Kubernetes cluster, but we’ll see how I feel about that the closer I get to putting this live – as best as I can tell, Kubernetes and Docker Swarm are the closest I’ll get to “deploying” locally before pushing to a cloud). However my software runs, I want to be able to deploy the code on my machine and have it be as close as possible to what it would be like on the public Internet.
Here’s the thing, it’s 1 thing to say “Oh, I’m just going to run this as a Docker container,” but you don’t just SSH into a server and type docker run... into the command line. We have container orchestration tools that launch containers and keep them running (creating new containers if 1 crashes, doing rolling updates, the whole 9 yards). I’m not planning to use Kubernetes for this – I’m tempted because it’s the biggest container orchestrator out there – but I’d be running that on a managed cluster on account of I’m not ready to go full on Kubernetes the Hard Way just yet (although that may come at some point). I want to force myself to think through things that some of these managed services handle for me on platforms like AWS. Ideally, I’ll gain a few things out of focusing on operating this code both locally and on a live server:
- I want to focus on properly instrumenting an application for observability. Not just spitting out logs to a centralized service like Splunk, or letting something else see container stats, but full-on canonical log lines, queryable metrics, the whole thing. It’s something I’ve read about, and I want to use this as an opportunity to try to put it into practice, so I can get a feel of the best way to instrument the code itself for observability, capturing traces, and including enough context for both debugging and having answers to any “business” (it’s a family app, but it’ll be business when I bring my learnings to my day job) people have.
- I want to have a greater appreciation for some of things we rely on managed services to do. These services are useful to a lot of people for a reason, but I want to learn a little more about some of the things they do for us and see a) is there a way to have an equivalent running locally so my dev setup better mirrors production and b) is there a way we can bring some of this stuff we’re relying on managed services to do for us something we manage in a way that saves us on AWS bills without dramatically increasing our workload.
This will hopefully force me to start thinking about my software much more holistically than as a thing that gets run on AWS. If you’ve ever set up an application to run on AWS, there’s always a lot of other stuff you end up setting up to support that application. So not only do I need to structure and organize my code in a way that makes sense for the application, but I also want to think more about setting up the infrastructure and other resources that my application relies on and get better at working with and around those.
It’s not just the infrastructure
Trying to build an application in a way that makes me think about properly operating software is nice, but I also want to learn a new language, ideally something that’s back-end focused (I prefer servers to browsers, although I can put together a functional enough web UI if I need to), but different from Java so I a) learn how to think about solving problems differently and b) pick up a new language that’s in-demand so I’m keeping my skills sharp and current. So I’m going to try learning Go.
The problem
So what am I going to build for myself? Why the best possible application for mastering everything there is to master about a language – a to-do list! Just kidding, kind of. I’m not building a to-do list, but I am going to try to build a gift registry. If you’ve gotten married and had a first child, you probably have a pretty good idea of the big picture here – you can build a list of things you’d like to get as gifts (the main use case I’m thinking of is birthdays and Christmas within the family, although there are potentially other occasions this could be made to work, such as having a self-hosted, independent baby registry of your own), people can commit to getting them for you, and update the registry with how things stand (e.g. someone’s said they’re going to get it, ordered, shipped, etc.). It’s a fairly straightforward problem domain that covers a lot of basic CRUD-y operations that give a good introduction to a new programming language that you can start exploring doing something actually interesting when you’re done (I already have ideas for that).
Here’s some context on the problem itself: things like birthdays and Christmas can get a little frustrating if you have multiple family members asking what you want, then trying to keep organized on who’s getting what. For Christmas there’s an added wrinkle of keeping track of what all you’re getting for everyone else. For the first part (coordinating who wants what and who’s getting what off which list), we used multiple email chains to communicate what we want and then other reply chains to discuss who’s ordering what. This year, that was replaced by a set of Google sheets for the lists of who wants want and to let people commit to different gifts. That works pretty well, but you can still pull up the tab where you entered your list and see who’s getting you want (we’re supposed to be on the honor system to not look, but while I’m building a gift registry I’ll just add some logic to limit that visibility). The other big issue is that for some gifts, multiple people get together to order a joint gift for someone, and the sheets setup doesn’t really allow that, so I’ll be adding that.
The other half of that (at Christmas as least) is keeping track of what all you’re buying for whom. At my house, that’s a shared checklist in a notes app for my wife and me. That should be a new tab or view in a web page that grabs everything I’m committed to getting so I can shop from that. This can feed back into the list where people view what people want so family members can see what gifts are claimed and where things stand with them (useful if you need a spouse to make sure they’re on the lookout for a package with someone’s gift), as well as keeping everyone banding together on a gift in the loop. I can also use this to give people a record of who got what for whom by letting the list show who participated in what after the gift-giving.
There’s a little bit of an interesting wrinkle in that individuals ask for gifts, but generally families give them (e.g. my sister may ask for something, but my wife and I are the ones who give it to her, so my wife saying we’ll order that item should show up as I did too). There’s also the logic around filtering what the gift requests show based on who’s looking at it For example: I should see that my parents volunteered to order a gift for my sister, as well as the exact status (whether my parents just called dibs on getting it, it’s been ordered, shipped, etc.). My sister, on the other hand, should just see that someone claimed dibs on the item, (maybe even ordered, shipped and delivered), but not who is getting it until after we exchanged gifts.
Other things to practice
There’s a few other goals that I want to pursue while I’m working on this. FIrst and foremost is user account management – I want to enable signing in via username and password (both for the practice and because not everyone in my family necessarily likes having to have Google and/or Facebook as a prerequisite for everything). So that’s ensuring TLS, hashing, plus salting and peppering the account passwords, which means secret management.
After that – web sockets for automatically updating as items are changed and added. This isn’t a “must-have” feature (I doubt an online gift registry gets widely used even for the limited number of people in my family), but it makes sense and I’ve never built anything with web sockets before, so why not?
Next up, there seems to be a growing interest (or renewal of interest) in coding in the negative space (Tiger Style being the most common example). The idea is that you actively and aggressively guard against an invalid state (TIger Style and it’s derivatives use asserts heavily) in order to catch invalid state early and abort immediately upon detection. Probably a better way to think about it is discussed in Chelsea Troy’s article on edge-free programming. That’s something I’m going to try to do more of everywhere, but this seems like a low-risk place to go big on it (I doubt my co-workers would appreciate random assert statements showing up out of nowhere in the codebase).
Fourth on the list is feature flags. I know what they are and how they can be used, but I’ve never actually built with them. I doubt trying to shoehorn them into this project is adding any value (short of deploying something I want to test on a remote server and then release to other users), but I do actually see some benefit to implementing them in my day job, so if I can find an excuse to practice here that would be great.
And so much more!
OK, there’s more I can (and 1 day want) to do with this (no, I’m not making the next great SaaS product that will make me a multi-millionaire from people paying $5+ a month for a glorified online checklist). This actually doesn’t make a lot of sense as a centralized web application at all (this whole online family gift registry idea sounds really stupid when you remember you have in-laws, who are married and thus have their own in-laws, who may be married and…you get the idea). That means this should really be a distributed application, but first, I need to learn how to build it (remember, I’m writing this in a language I don’t already know), and properly operate this as deployed application. Then I can start working on packaging this up so other people can run it themselves, all while building a way to sync data from all our individual lists back and forth. That’s a whole new class of problems to explore and stuff to learn about.
It’s easy to get lost in the day-to-day tickets at work, and sometimes it’s hard to feel motivated to do something on your own, especially something that’s going to see limited use. But the key is to remember you don’t need a project that turns into a unicorn company. Build something that offers an improvement to your life (even if it’s a first world problem like keeping track of who’s getting what for whom for Christmas and birthdays), ideally lets you do it in a way that you aren’t doing in your day job. The worst-case scenario is you now have something you can show people to prove you can do things that doesn’t appear in job history on your resume, you’ve learned new stuff, and may have even gotten better at the stuff you’re already doing. So with that in mind, I’m going to see if I can’t build and run a system locally, and then translate that to a live server with minimal differences between my laptop and the public hosting. I want to be see if I can not only have “live” versions of the application locally, that mirror production in as many ways as possible (just with fewer resources), or at the very least see if I can build it in such a way that I can move it, as-is between cloud hosts with minimal work on my part. I’m not going to be revolutionizing anything, but I am going be better than I was when I sat down to write this post.