underdetermination in the engineering world

Question

Newbie here- please go easy on me.

Now that I am retired from the world of physics and engineering, I am struggling to come to terms with my career failings. The most spectacular of these occurred when a high-level manager dismissed the validity of 6 months' worth of my work in identifying the root cause of a catastrophic problem in our factories. The "reasoning" behind that dismissal was as follows:

"The existence of your putative model describing the physical mechanism behind the quality collapse occurring in production lines (a, b, c, d...) cannot in and of itself disprove the existence of a completely different model that assigns root cause to a portion of the factory under the control of someone other than my people.

"Since it then is possible that the real cause of this problem is actually to be found in a part of the factory that you did not search, you have furnished us with no basis upon which to believe that your model is correct, and until you do, we will take no action on it.

Her viewpoint carried the day, my model was dismissed, and no action was taken. The problem came and went on its own over the next ten years (!!!) of production, until the affected product line was declared obsolescent and shut down.

In retrospect, it appears that she pulled the underdetermination trick on me, and to defeat it via exhaustion would have required more years than remained in my career. So, she won, and I lost. Here then is my question for this community:

The excuse-by-underdetermination, or inexhaustion, or unconceived alternatives, appears to be so powerful a technique as to dismiss upon its invocation the validity of any model at all, and thereby justify inaction in any possible set of circumstances. How is it possible to refute it, and thereby make progress towards the solution of problems in the real world (restricted here to mean the factory floor of a globally-distributed manufacturing enterprise where one minute of productive downtime costs tens of thousands of dollars)?

Answer 1

Let's distinguish between the belief that a model is true and accepting it as effective for certain practical purposes.

It's a safe guess that your model was incomplete — it left out some factors that you assumed to be less important and/or difficult to model. Your model also might have involved some simplifications — maybe using linear approximations, or continuous models of fluids, or representing three-dimensional processes in one or two dimensions.

For these reasons — even setting aside the possibility of some other unknown causal factors playing a role in the factory — your model was false, and indeed known to be false. And so that means we shouldn't believe that it's true.

But the purpose of your model wasn't to be a complete and perfect representation of the factory. The purpose of your model was to identify some of the major factors contributing to the problem, so that the problem could be fixed. And it might have been more than adequate for this purpose.

That is, even though your model wasn't true, and we shouldn't believe it, it still might have been very effective for the practical purposes at hand, and in that case we should have accepted it.

The manager, positing the possible existence of other, unknown causal factors, is giving a reason to believe the model is false. But this isn't a reason to believe that the model wouldn't be effective.

However, once we shift from belief to acceptance, we need to recognize that standards of effectiveness can differ between people. Let's shift the characters in the scenario a bit. Suppose the interventions suggested by your model might have led to unsafe working conditions. Specifically, suppose that, from your perspective as a systems engineer, your primary goal was for the apparatus to work efficiently; while the union steward put more emphasis on worker safety. Then your model might have been perfectly acceptable, according to your view of the practical purposes at hand. But, simultaneously, it might have been unacceptable according to the union steward's view.

Shifting from belief to acceptance lets us avoid certain kinds of sweeping skepticism about science and engineering. It also clarifies how science gets entangled in value-based disagreements — or, in other words, how science becomes political.

Answer 2

Thanks for sharing.

"The existence of your putative model describing the physical mechanism behind the quality collapse occurring in production lines (a, b, c, d...) cannot in and of itself disprove the existence of a completely different model

That's correct, but Occam's razor deals with this

that assigns root cause to a portion of the factory under the control of someone other than my people.

That is stupid, the problem should be fixed regardless of where it is

"Since it then is possible that the real cause of this problem is actually to be found in a part of the factory that you did not search, you have furnished us with no basis upon which to believe that your model is correct, and until you do, we will take no action on it.

@John Forkosh in the comments pointed out cost-benefits analysis. It seems the changes required as a result of your analysis were too costly compared to how unconvinced she was of the root cause.

In that case, to build confidence, your hypothesis should have been tested to confirm that the simulated root issue would indeed trigger the observed quality defects, I know this can be difficult.

Answer 3

The excuse-by-underdetermination, or inexhaustion, or unconceived alternatives, appears to be so powerful a technique as to dismiss upon its invocation the validity of any model at all, and thereby justify inaction in any possible set of circumstances. How is it possible to refute it, and thereby make progress towards the solution of problems in the real world (restricted here to mean the factory floor of a globally-distributed manufacturing enterprise where one minute of productive downtime costs tens of thousands of dollars)?

Since the tactic you describe can be used against absolutely any model, it is completely useless for the purpose of deciding between models or refuting any particular model.

The way to deal with this tactic is to think in terms of solving problems instead of justification. which is a good idea in general since the whole idea of justification is a dead end:

Do all epistemologies suffer from the "regress of justifications" problem?

The person who objected to your model didn't have any suggestions for a solution to the factory's problems. Nor did she have any argument that could refute your model. You could have pointed this out and asked if she was aware of any specific reason why your suggestions can't be implemented.

Other people have suggested that you should have tested your model. I think the only relevant test in this case would be to make the suggested changes and see if improvements happened. Since no alternative model was offered of what was happening in the factory, you couldn't compare your ideas to some other specific idea in terms of their implications. You might like to read "The Goal" by Goldratt, which is a novel about understanding what is happening in a factory and improving its productivity.