This is indeed the crux of the matter - and thank you @Warren for bringing this discussion point to the table . There are two definitive schools of thought when it comes to forum threading - outlined in the link that @neo posted. Since Discourse implemented a “flat-threaded” model, discussions that wander off topic are not very useful as references nor as live discussions. There are a few features that the Discourse devs have introduced to facilitate the viewing of threads, but there can really be no true marriage between the flat-threaded model and the hierarchical model.
This is the reason why some of the moderation may seem heavy-handed at times - unfortunately it is a necessary trade-off.
For my part, I believe that the Discourse model is the correct model for this site - at least among the implementations that exist today. Many singularly-focused forums (ours is “The SAFE Network”) do benefit from this type of model, as it leads to both easily index-able threads as well as constructive, focused discussions. This allows not just the “best” reply at each step in the conversation to be followed, but all of the responses that could have legitimate points to be shown.
A hierarchical structure would facilitate off-topic discussions that happen as a natural offshoot of the original topic. This structure is better suited for sites where any topic about any subject is welcome, and off-topic discussion is the norm. This is not the case with this forum. One reason for this is that it facilitates vying for the spot of the “best” reply at each step in the conversation, which has the potential to obfuscate subsequent posts.
While I believe that off-topic subjects that naturally evolve out of existing discussions are absolutely beneficial to the health of the community in general, moderators are already in in possession of the tool to facilitate these discussions. The ability to split topics to other threads is a great way to keep the original threads clean while maintaining valuable discussions that otherwise may appear off-topic under the original thread. As well, Discourse implemented the collapsible “reply” feature and the “reply as linked topic” to lead users’ thought process towards this way of thinking while navigating forums such as these.
So, while the ability to view a thread in a hierarchical fashion - and the benefits to off-topic conversation branches that such a setup would provide - is not available in Discourse, it does provide moderation tools as well as tools for the users in order to facilitate the clean, organized forum structure that is beneficial to the discussions that take place here.