I mentioned in a previous blog post that we had used DynamoDB on an internal project at Bronto (my employer, but I don’t speak for them – they have people for that). That project is the Bronto Cantina – which launched to the whole Durham office a couple of weeks ago. It was a pretty neat little application, most of which was written over the course of a couple of days, followed by bits and pieces of cleanup afterwards to get a test setup up and running. Now we’re live, and I wanted to say a few words about it.
The motivation
The apps team at Bronto did about 90% of the work on this app during Bronto’s “hack days”, a couple of days where engineers can work on something interesting so long as it relates back to Bronto. At the time, I was on “kitchen duty“, a week-long responsibility for helping take care of 1 of the 2 kitchens in our Durham office. When our team was spitballing ideas, I mentioned that having something that would automatically email me when the dishwasher was ready to be unloaded would be great – at the time my partner in the engineering kitchen and I would have to go back to our desks, email each other that the dishwasher was running and the estimate of when it would be finished. In fact, that process was basically how we kept in the loop on what had or hadn’t been done in terms of kitchen duty responsibilities. Having that automated would have been a huge help on knowing what, if anything, needed to be done in the kitchen.
Initial features
Since Cantina started as a hack days project, our initial version was largely a proof-of-concept. We could track who did what work in the kitchen, have kitchen dutiers (we really don’t have a better name for people on kitchen duty) “check in” and let their other kitchen duty workers know what work they had done, and (mostly) the ability to send an email notification whenever the kitchen duty crew had goodies (called “Suite Treats” since Bronto is owned by Netsuite, who I also don’t speak for and also has people for that sort of thing) in 1 of the kitchens (because Bronto is not above bribery when picking which kitchen duty crew gets to jump to the front of the next month’s massive lunch and learn line). I say “mostly” on that last point because our notification email was running afoul of corporate spam filtering rules. People not on kitchen duty could leave a happy vote if they were happy with the way things were in the kitchen, or a shame vote listing why you’re not satisfied (e.g. there’s no coffee in the coffee machine, no snacks set out, no clean dishes available, or the kitchen’s dirty just to list a few). Since we’re pretty sure mischievous Brontos are going to want to mess with the kitchen duty crew, we even added a “just trolling” option in the shame vote list.
Whenever someone casts a shame vote, the current kitchen duty crew for that kitchen gets notified via email so they can fix the issue as soon as possible. We track the kitchen “state” so people on kitchen duty don’t get spammed about the same issue repeatedly. As an example, if the coffee machine is empty first thing 1 morning, a lot of people are probably going to complain to the kitchen duty crew, but only the first complaint will trigger an email. Once the coffee machine is refilled, we clear that “state” from the kitchen, so future complaints about the lack of coffee will trigger emails to the kitchen duty team. This way, the kitchen duty crew gets word whenever something’s wrong in the kitchen, but they’re not getting spammed. Trolling doesn’t spam the kitchen dutiers, but it is included in the stats because that’s the whole point of having the option in the first place.
Lastly, kitchen dutiers have the ability to list the work they did in the kitchen If the work done fixed a complaint someone had about the kitchen, we track that too so if the issue comes up again the kitchen duty crew can be notified. This lets our administrative assistant know who all did what work during their week on duty. The workload stats for the current week are also available in the company-facing app so it’s easy to see who’s not pulling their weight, and their fellow Brontos can shame them (offline) appropriately.
Updates made since the hack days presentation
Bronto Cantina is a simple HTML page with some JavaScript to handle processing. Since we’re not anticipating a lot of changes to these files now that we’re live, we added some logic to explicitly load the page using what’s cached in the browser whenever possible. The actual files are pretty small and hosted on S3, so it’s not like page reloads would be breaking the bank or taxing our network resources, but there’s still no point pulling from from Amazon when we don’t need to.
The big addition we made after the hack days presentation was the creation of an admin console that runs as an app within Bronto. This lets our administrative assistant, who’s in charge of all things kitchen duty, see a full breakdown of stats of the various kitchen dutiers. This includes who all did what work (set when they checked in with the work they did), the number of times people complained along with breakdowns on what the complaints where, and the treats kitchen dutiers brought in to bribe their way into kitchen duty glory.
Non-coding issues
Probably the biggest set of issues we had wasn’t code- or infrastructure-related at all, but rather with our own spam filtering setup. Just having Bronto email the Durham office whenever someone left goodies in the kitchen caused the emails to be blocked. After getting things set up so we could email the office on the email server so we could email a mailing list instead of every individual Bronto, we immediately found all our emails running afoul of another email security service. That took some whitelisting from Netsuite corporate to get up and running. The real important lesson from all of this is that sometimes the biggest issue with your application isn’t your code, but aspects about your users’ environment that never crossed your mind during the development process, nor that you can control or simulate easily.
Our internal playground
1 of the benefits of doing this application was that we were able to use it as an internal beta test for a lot of stuff we were thinking about doing in production, but hadn’t really tested yet. Since Cantina is just for Bronto, this was a great opportunity to try out not only using DynamoDB (spoiler alert, I was not a fan), but also integrating a few other internal services we had built to help make our lives with our other apps easier. This gave us an internal beta of these services before using them for customer-facing applications. It was also a good test of our documentation for these apps, since I was hooking Cantina into these services but hadn’t ever worked with any of them. We’re not quite done using Cantina as our personal testbed. There’s a few other services we’d like to play around with, and a Bronto-only app that’s actively being used is a great resource for trying these things out.
Future work
Now that Cantina’s been live in both our office kitchens for a few weeks, we’ve observed some user behavior that’s going to drive some changes to the application in the near future. First and foremost, we need some type of “Note” field in the shame votes for non-obvious issues, like a lack of creamer in the coffee machine. Just putting in a status of “coffee” is likely to have the kitchen duty crew check the beans and think that everything’s OK. Another feature is a “reverse shame” button, that will allow kitchen duty to call out Brontos that aren’t holding up to their end of taking care of the kitchen, such as putting dirty dishes in the dishwasher.
The lack of those 2 features has resulted in many Brontos abusing the “Suite Treats” feature instead – spamming the Durham office instead of sending a single email to the kitchen duty crew (for things like lack of creamer) or emailing just the building in question (to tell people to bring their dishes back to the kitchen and put them in the dishwasher).
After a morning where the engineering coffee machine broke down, we realized that we should probably add a “report broken equipment” feature. There’s no sense in trying to “shame” the current kitchen duty crew for something that’s not their fault, but it would have been nice for whoever first realized it to go ahead and notify facilities right there on the spot. This would also give us the ability to ensure that facilities is only emailed about broken equipment once instead of spammed by every groggy engineer in the building.
Part of the benefit of having a platform to work with is that you can create applications for them that the people who built the platform never intended at the time. Granted, this is much smaller scale than the types of things that Bronto was built to support, but it still works. In fact, the only thing that had to be done to support this on the Bronto end was for us to create an unlisted application and set up a Cantina account in Bronto. If we were a real customer, doing real productive coding and things, we could have built automated email integration around any part of our product without any special support from Bronto (I’m not counting getting an app registered or listed with Bronto as “special” since Bronto does this for customers already).
Cantina was a pretty informative project for us since we got to try DynamoDB out and field-test a couple of internal services that we had written). It solved a problem for some of our fellow employees, and most of the effort outside of the coding itself was around corporate email filtering settings. Now we have Cantina up and running in both our kitchens at the Durham office, it’s on to our next applications.