WHAT IS SO DIFFICULT ABOUT STAYING ON SUBJECT OF A THREAD?
That's the nature of the beast. You have to weigh the length of the thread (and thus the time you'd have to spend "mining" it) against the possible value - which you can't know ahead of time.
So, for me, when I don't know a subject well it's worth going into a forum like this, reading through the biggest issues to find answers to my questions and learn the questions I should be asking but didn't know about, then present my questions as new threads.
The reality, though, is that this is a DIY forum, and like most similar forums you aren't going to get a straight answer to any question. That's the reason threads meander - people have a little to contribute, or the conversation lit up one of the areas of their expertise they do know about and might be relevant, if only tangentially, so they contribute what little they can.
If a thread doesn't go off topic, it's because no one is paying any attention to it - at all.
This is one of the things Discourse forum software was meant to help solve - calving off a portion of a thread into its own topic is trivial for moderators and long-term "trusted" users. So thread management is built in as a primary goal and feature. Forums like this have features to support this sort of use, but not very good, and they require a lot of tending from, usually few, dedicated moderators to perform the same work. Not that I'm suggesting we change forum software, simply pointing out that this is a problem endemic to forum style social networks, and you ultimately need to come to terms with it, and determine your own techniques for dealing with it.
Of course, the reality is that if you're looking for answers and not conversations, then forums will forever be frustrating for you. Maybe quora,
https://diy.stackexchange.com/ , or
https://sustainability.stackexchange.com/ would better meet your needs - where going off topic is a capital offense, and quickly squashed.
