Dev Debrief #1: An Interview With the Developer of Bookstack
Interview with Dan Brown of Bookstack - a customizable, self-hosted platform for organizing and sharing knowledge
Hello! I’m Dan, a full-stack web developer based in the Hampshire countryside of the UK. Before I got into coding I was a temperature sensor sales engineer. I started off learning Java so I could build Android apps and Minecraft mods. While I never built any Minecraft mods, I did publish some apps on the play store but along the way I started a JavaScript tutorial, thinking it was the same as Java (Spoiler: turns out they’re quite different things), and that set me on a path of learning how to develop for the web.
From then I built various sites until I changed careers to be a developer at a Marketing Automation company, which I then left after 7 years in late 2021 to work on BookStack full time. I still don’t have any formal development education (I don’t think my engineering college BTEC helps too much) but the unofficial school of YouTube and online tutorial sites has taken me quite far.
In addition to BookStack, I often build & share various other open source projects and tools like RSS, Mailbag and Wirestat.
Following, is an interview with Dan Brown consisting of a handful of questions I came up with. This is the first interview and I already have others lined up so expect them to improve over time! So sorry to Dan for being the guinea pig on this but he was more than honored to be the first. 🙂
What inspired you to create BookStack?
At my last company we had, what I believe is, a common documentation problem. Content was disorganized, distributed across multiple platforms & formats, making it a pain to recall and share knowledge. I wanted to get everything into a single centralized go-to location, in an easily searchable format.
I had a look around for solutions to suit my needs and the closest was Atlassian Confluence, but that had a user-count based cost which I thought was a massive shame since I never would want the finance department to be a factor in freely sharing knowledge. I could envision future discussions about if it was “worth” adding an extra department if it takes us to the next payment tier. I did look to open source solutions too. DokuWiki seemed like a potential option but we’d need to depend on a range of plugins to get it suitable for the intended audience and I wasn’t sure of the maintenance dept of that.
While searching I saw a gap in the open source space for a multi-user documentation platform that focuses on ease-of-use and user-experience out of the box. I thought it’d just be a case of “Bodging together a WYSIWYG editor with a database” so I started building my own and now here we are almost 8 years later, still bodging together that database-backed WYSIWYG editor.
How did you go about designing and developing BookStack?
I started off working to an MVP for usage within my workplace, using the “Laravel” PHP framework that I had recently got to grips with. The platform built-up and changed quite rapidly over the first 3 months. The variation and evolution of the design can be seen in my old blog-post here.
Before it was “BookStack”, it was called “Oxbow” and had just books and pages, where pages could be infinitely nested within each-other for unlimited depth. While this sounds great from a technical point-of-view, I found it troublesome from a user-experience perspective as content could easily get lost within the depths of the nesting, and you’d have to limit visibility as paths & breadcrumbs go beyond what you can reasonably show on screen. I needed something simple to use as my audience wasn’t just technical people but instead composed of different teams with mixed-technical skill.
Eventually I landed on the fixed Book > Chapter > Page structure, which provided some real-world familiarity & abstraction, so I refactored what I had built around that idea and then came the “BookStack” name.
BookStackAppCan you describe a particular challenge you faced during development and how you overcame it?
My biggest challenge with developing BookStack has been the social side of things and the sheer amount of energy it requires to maintain a community, at least at the level of support & communication I want to sustain. Quite quickly I found working on code, and the platform itself, became a lesser part of the time consumed by the project, with handling issues and supporting the community taking up much of the time. You can spend hours or even days on a single GitHub issue, just to have 5 more opened right after. The sheer weight, and often the strange social dynamics involved, of the requests & comments within the GitHub issues list has put me close to giving up on the project a few times for my own sanity.
I’ve got better with this over time as I gained confidence in my actions & responses, bolstered by gaining a historical path to look back upon. Having much more time, after leaving my job to work on BookStack full-time, has also helped with this although I’m not yet covering my living costs to make this sustainable (although getting closer!).
If you were after a technical example, then the recent permissions changes to Bookstack have been quite a challenge. In the system we have role permissions, and users can have multiple roles, and these roles permission can be overridden at a per item-basis, and those may cascade to child items, which may have their own permission overrides. The sheer amount of combinations and scenarios has become super complex, but this is also a system which needs to be super performant since permissions are queried for every action and every piece of content you see in the interface. Many scenarios weren’t even handle consistently between different parts of BookStack itself since the exact logic could be very open to interpretation. To address this, I spent some fair time formalising the exact target logic along with a large set of scenario tests, for every eventuality I could imagine & discover, and backed each up with automated testing for consistent ongoing enforcement. Got there in the end with the v23.01 release but it was brain-melting stuff which caused the rollback of some additional permission features I had already written with intent to implement, therefore loosing a few weeks’ worth of effort.
How do you ensure that BookStack is secure and stable for users?
Testing is so important for something continually evolving like BookStack. I try to cover as much as possible with tests which are automatically run upon codebase change, helping to ensure everything remains working as expected and that we’re not breaking existing features.
A continuous and steady release cycle also helps, I’m quite proud in that we’ve provided a continuous upgrade path since the initial public release with very little breaking changes outside of system requirement changes. Keeping our requirements modern helps us stick to the most up-to-date version of our dependencies which helps with security. I also keep an eye on the community-driven BookStack hosting efforts (Like the docker container projects) to help out with user reports there, and warn of any upcoming impacting changes in the interest of stable releases.
How do you balance user requests and feedback with your own vision for BookStack?
This has been really tricky and somewhat stressful, but something I learnt to better deal with over time. What I’ve found is that actual BookStack established user base is naturally aligned in vision, since they’re using the platform that I’ve built to said vision. Conversely, a lot of the requests I get from new users are along the lines of “We want to use BookStack but we’d need feature x”, and a lot of the time those requests are bolstered with reasoning like “This would open BookStack to a wider audience”. While adding new features would be better to attract new users, I’ve found that focusing on these kinds of things makes the project wider, meaning there’s more to maintain & support while you’re also risking loss of focus on the core use-case of the platform. I don’t want to make the platform something to suit all needs, that’s a dangerous path to exhaustion and there’s room for multiple options and alternatives in this space to cover different requirements. Therefore I very strongly focus on the existing user-base, improving and building upon what we already have, choosing to take on requests that strengthen the existing image of BookStack rather than widening it.
I’ve also rejected paid feature build/addition quite a few times now, often due to the ideas not making sense for the platform or due to concerns of those features impacting long-term maintainability. While I’m open to accept funding to help cover the work going into a feature, I’d never want the idea or implementation of a BookStack feature to be dictated/influenced based on money over the actual needs of the community.
Can you share a success story or memorable experience with a user of BookStack?
Something that I see as a success story is that my largest sponsor to date has just simply been a happy BookStack user. They didn’t take me up on the advertising the sponsorship provides, nor do they specifically gain anything, they just wanted to give back and support the project.
Upon that, reaching the top of Hacker News earlier last year was quite memorable. I’m still fairly afraid to share the project about, so was quite nervous to see the post get traction but I ended up getting a lot of activity and positive feedback from that. Additionally I’m often seeing positive comments pop up on Reddit which means a lot and definitely helps recharge my motivation for the project.
What advice would you give to someone who wants to create a self-hosted application?
Focus on building to your own needs and desires, at least to start with. That focus will help with long-term maintenance by keeping the scope narrow and motivation alive. If your application does a good job to suit your specific use-case, it’s likely it’ll suit others as-well.
Also, if you release your application as open-source, be sure to fully understand the license you’re using and think about how that interplays with your plans & desires for the project. I often see folks slap on a popular license, or copy the licensing of a popular project, without actually understanding the rights and requirements they’re applying to their users.
What do you see as the future of self-hosting and or open-source?
I’m looking forward to seeing ActivityPub develop further to become more widespread and further connect up applications in a decentralized manner. Seeing support from groups like Medium, Tumblr & Wordpress indicates it’s increasingly becoming an established standard.
In regards to self-hosted hardware, I think we’ll see more of a focus on building smaller, energy efficient setups. This has already become a much more significant factor over the last year in Europe with energy prices becoming excessive and an ever increasing focus on being green. Hopefully we’ll also see hardware diversify further with ARM and RISC-V systems becoming more ubiquitous.
Within the open source self-hosted space, there currently seems to be a lot of VC investment going on which can be good or bad. It could be a good way to fund/push new open source alternatives but on the other hand, these heavily-funded projects can often overshadow community projects with their marketing budget & focus, and they’re often set-up as “open core” with features locked-off behind a non-free & paid license. In many cases it can seem like they’re just using “open source” as a marketing tool and not really aligned with the “spirit” of open source. We’ll have to see how things pan out over time.
How do you keep up with the latest technology and trends in the industry? Any fav blogs or websites?
Reddit and HackerNews mostly, in particular the /r/selfhosted and /r/opensource subreddits, then I use RSS to follow a bunch of blogs and news sites. I also watch a massive amount of YouTube, with some self-hosted relevant favourites being Jeff Geerling, Level1Techs, Veronica Explains and ServeTheHome.
What are you favorite self-hosted apps (other than your own)
My favorites that I use on a daily basis are:
- Proxmox - Love using LXC containers to self-host my services which Proxmox makes easy. In addition, being able to simply manage test containers & VMs for BookStack dev work has been super helpful.
- Pihole - Been running this for quite a while as a way to block tracking domains and catch naughty devices on the network.
- Plausible - Used this to self-host website analytics and found it ideal in regards to providing exactly what I want to know without being overly complex like Google Analytics.
- Uptime Kuma - Does exactly what I need to monitor my sites. I also use this as a cheap & bodged-together Linux server monitor system by using it alongside wirestat, a little HTTP-based metric monitor system I wrote in Go.
And as a bonus, here’s a fun question. What’s your favorite programming joke or meme?
Does the Ballmer Peak XKCD count? This is my favorite as I’m convinced, from experience, it’s a real thing.
How can we follow you?
You can find me at danb.me which includes my blog, projects and other random bits. I Just wrote a bit on my Homelab setup. Otherwise you can find me on the Fediverse at @[email protected], and on GitHub at @ssddanbrown and finally of course the Bookstack community Discord. 😊
Thank you for sharing your inspiring journey as a full-stack web developer with us Dan!
Your transition from a temperature sensor sales engineer to a self-taught developer is truly impressive, and your dedication to learning through online resources like YouTube and online tutorials is a testament to your passion for the field. It's great to see you making contributions to the open-source community by building and sharing various tools and projects like BookStack, RSS, Mailbag, and Wirestat. We appreciate your time and insight, and wish you all the best in your future endeavors!
