Things that might need to be considered (not complete) in this pursuits might be:
- Success of the Theory
The first and most obvious factor is time. Directly feeding into the exploration-exploitation-dilemma, so more or less the question whether you should spend your active hours with research (exploration) or application (exploitation). Where research has the potential to provide improvements and optimization of processes or to be a waste of time, while running applications provides results, but at the risk of doing it very inefficiently (also wasting time and resources).
So the first question is probably, do you have the time for that? Or maybe is your current system lacking so much that you need to take the time for that because otherwise you'd be failing anyway? Or in general do you think it's time worth spending? Whether the answer is yes or no depends on the specifics and often comes down to the decision of the person being asked...
So for example the inventor of homeopathy cited as reason for his research that the medicine at the time was severely lacking, in that extensive use of bloodletting and use of laxatives were making the patients weaker and more ill rather than helping them. And in that regard homeopathy might have ironically worked, ... in the sense that it doesn't work and hence doesn't make things worse than they already had been.
So there might have been good reason to do research (which always goes outside of the mainstream and into the unknown), given that the state of medicine wasn't great and that there was potential for low hanging fruits of improvement. Now it still ended up mostly being a waste of time and at best accidentally useful. So apparently he improved "clinical trials" as a way of testing his substances (though due to being that fringe that was of little significance to what we now use as clinical trials) or his drug testing revealed stuff that actually has an effect... that is before he gave it the homeopathic treatment of diluting it... but still that is an inherent risk of any research (that it ends up being a waste of time).
Now closely linked to the problem of limited time is the potential to be useful. So is this either a low hanging fruit that can improve without much effort or is it something that has a groundbreaking effect that is worth the time investment. Of course especially the overuse of claims of the latter usually taints this risk-reward calculation so much often enough the claim of a groundbreaking/revolutionary/all-purpose whatever is an immediate red flag for bullshit.
That being said it could still happen that this is not an overused PR claim, but what you yourself might get as impression from your own work.
Next up would be compatibility&applicability. That is the problem that the "revolutionary" usually comes with the problem of being incompatible with most other things. So for example if homeopathy were to be correct, we'd have a problem with the entire rest of science, because as of right now, according to those sciences it shouldn't work. Like the substances are diluted below the level where chemistry could have any measurable effect. Seriously they dilute it to the point where there might only be a statistical chance that there is even 1 atom of the "active" ingredient present, in the otherwise inert "solution". And the claim that the water has a memory of what it interacted with is in some conflict with physics. Not to mention that the higher potency by diluting would have "interesting" consequences with respect to idk pissing in the ocean...
So if that were to be true we might need to discard most of what we think we know and start from the ground up.
That's not to say that thing like this don't happen and there is the concept of the scientific revolution where you have stages of building upon previous knowledge within a paradigm and then phases of a paradigm shift where people suddenly cite different papers and build upon those. Though usually the evidence largely remain unchanged it's rather the explanation that changes, so even in that regard it's very unlikely that homeopathy is true and all of science is wrong, given that this would mean that we should at least see an effect and get an explanation for why we don't see one now.
So the problem is that the further you walk off the mainstream, the more you are alone and have to supply yourself with new structures and frameworks that the existing institutions already have. So idk if millions of people do or have checked the theories for internal consistency it becomes much easier to build on the work of other people than if you reinvent the wheel and have nobody around who knows what you're doing. In the latter case there is a lot more stuff to do which has nothing to do with the actual research, for example.
So the more niche the subject and the more fringe the assertion the less compatible it is with other research and the more it's harder or unlikely to find application for that.
Which blends into the next point "success of the theory". So basically whether it works. Like if it's not just a nice idea, but if there is an actual proof of concept, if there might already be a prototype and it shows promising results for a relevant problem, then even if their idea is bullshit that topic might nonetheless be interesting.
And usually the aggregate "potential" of a theory correlates with "popularity". Now popularity is not an indicator that something is correct, far from it, however in the end humans are social animals so even if the huge mass is chasing bullshit and might still be worth paying attention as that is likely going to shape the environment, blend into language, determine which problems are considered relevant and which get funding aso.
So at first you might check whether you have time for that and if that is well spend on the issue. Whether the topic in general is worth it and then whether the paper in particular is worth it. Which includes the evidence cited, the internal and external consistency with facts. The thing is the more easy errors there are in the paper the more likely there either are better sources or the more likely you are better off to tackle the problem yourself if you feel up for that kind of challenge.
Which brings us to the last point on my non-conclusive list: trust. Now that probably sounds counter-intuitive as it's literally a fallacy (appeal to authority) to trust in institutions, experts and whatnot. And no it's not appeal to "false-authority" it's appeal to authority, the fact that an expert says it does NOT mean that they are correct. As you can see very plainly if you look at the start, namely that research is not inevitably leading to a positive outcome but could also be a waste of time, so failure is not even necessarily a flaw of the researcher, might just be bad luck.
Though when push comes to shove you only have 2 options and realistically only 1. Either you trust experts or you do it yourself (and your time is limited so even if you do parts of it yourself you'd end up with a pretty limited dataset). So practically you'd need to trust at least some people.
And that includes trust in the ability of data collection (because if they fuck up that part the entire rest is completely worthless), in the ability to analyze the data, in their integrity to not obfuscate or tamper with the data, to not exaggerate their findings. Now the more you are into a topic the more you have a more realistic part on the expectations and the analysis part, but if you can't trust the data itself that's not going to help you either.
So ironically the currency within the scientific community is trust. The more institutionalized experts, have proven under pressure that they are theoretically capable of performing the job that they want to attempt (got their degrees and reputation), their work has been tested by their peers and ultimately their reputation and career relies on their integrity. Because the thing is if they fuck up, and I don't mean being wrong, but being sloppy or even cheating, then their career is over. If you can't trust the data that people present then their entire body of work is worthless. Like you'd need to redo every experiment, every analysis and so on. And not only is their work worthless, they might have tainted the work of all the people that build upon them and that might have wasted their time doing so.
So often enough the default trust for established experts is higher as they a) proven some level of expertise, b) have a reputation among other experts and c) would lose a lot if they worked sloppy.
Conversely the mechanisms of the scientific community with regards to quality management are usually "peer-review" and "impact factor". So whether a subset of your peers approve of your methodology and your work standards and how relevant the article and the journal is with regards to how well received it is in the scientific community. Which is measured by citations. So if other people cite your work as their theoretical foundation those papers of yours increase in value and if journals publish lots of those high ranking papers their impact factor increases and thus their interest in curating their publications and getting rid of the bullshit.
Though at the same time peer review is only as useful as the peers that review. So the more fringe and novel the subject matter (and that's not limited to the "cranks", but also includes idk the AI hype or less flashy fundamental research with not a long list of researchers) the less peers and the less knowledge about what you're actually doing and in turn the less competent reviews you can expect. Like usually the process of reviewing might be blind, but often enough is the subject matter is fringe enough you can already tell by writing style, the complaints or just the low number who your reviewers are...
Same with impact factors, apparently it has breed a culture of publish or perish where your popularity and being around so that people know you, know your stuff and read and in consequence cite your papers and lots of them creates a feedback loop where socializing and networking is rewarded rather than proper scientific research.
So there are attempts to hold a certain standard for research and to provide a value in terms of highlighting what's worth reading. But in the end a lot of that is based on at least some good faith and trust in people's integrity and it's far from being fail safe.
So in the end "science" is usually what you end up with years or decades after the initial publications. After the hype cycle has moved on, after people tried to build upon the findings (and accidentally checked them) and either failed or succeeded, after things have been canonized.
Though that's usually when you're no longer reading papers but text books or only look at the initial paper as a historic document but no longer for it's content. While papers are usually either read because they are fresh and interesting or because they are specific to a particular experiment and that always comes with some risk.
Sure it can be the next scientific revolution or you could have stumbled upon a Ramanujan and even people like Niels Bor (?) said something along the lines of "you first need to publish a lot of normal stuff before you've established yourself enough for people to take your more daring approaches serious" and despite the revolutionary papers it might be more like Planck said "Progress is made one funeral at a time", so essentially the acceptance of a new theory comes due to the fact that proponents of the old theory die out and the students want to learn another.
So in the end you probably need to develop an "intuition" of your own with regards to whether it's worthwhile wasting your time on that. Which isn't meant to be "emotional", but in the end you yourself also are some sort of expert and you probably have a professional opinion as to whether an idea is interesting, promising, whether it's feasible, whether your peers talk about it, whether the person is a master of their craft, a novice or a career jumper. And if it's riddled with errors, red flags, isn't interesting and no one is paying attention, then that's probably for a reason.
So a scientist should be professionally open minded given that most of what is being done is model building where it's never a certainty that those are actually correct, but there are only so many thing that you can be an expert in and if you already feel that something is being suspicious you should probably let people develop it further on their own and see where it is going before you jump a band waggon.