Is there a term that defines the distinction between a claim based on measurements versus claim based on subjective "guess"?

Specifically, in s/w development I can estimate ("make a claim") the cost of a project by counting the nouns in the requirements document. I enter the count into a formula and get estimated hours. In contrast another person simply reads the document and says, "Based on my experience, I estimate cost as x hours."

Is the 'guess' any more, or less, valid than the noun count?

  • I'm not really seeing the question about philosophy here. Can you make it clearer? – virmaior Nov 21 '16 at 3:28
  • Thanks for taking a look. Maybe this is trivial. I want to give a legitimate reason to reject a practice at my work: People estimating project costs by just guessing--they call it "engineering judgement." I want estimation to be based on measurable characteristics. – user1704475 Nov 21 '16 at 6:40
  • Thanks for taking a look. I think there's an epistemological term that distinguishes knowledge ("claims"?) based on quantifiable characteristics versus claims based on "intuition," i.e., "I know X because I measured it" vs. "I know X because, well I just know. Don't me how." People at my work do a lot of guessing--they call it "engineering judgement." I'm telling them they're making a fundamental error, but I'm ignorant of some basic terms in how-do-we-know-what-we-know-ology. – user1704475 Nov 21 '16 at 6:49

As I read your question, the two methods (counting text features, such as nouns; and expert judgment) differ in their intersubjectivity, or the extent to which two different people using the method in the same circumstances will get the same result. Intersubjectivity doesn't require quantification. For example, pretty much everyone who meets my cat agrees that he's black, but not because we've quantified the radiation he reflects. On the other hand, quantification through reliable measurement can improve intersubjectivity. You don't spend part of every supermarket trip negotiates how many apples you're buying because a (state-certified, reliable) scale simply measures the weight of the apples.

Now, the substance of your question is whether intersubjective methods are better than expert judgment. What does "better" mean here? I think here "better" means something like "tends to give a more accurate estimate of the billable hours required to complete projects." By definition, intersubjective methods will be more precise than non-intersubjective methods; but they are not necessarily accurate or true estimators. For example, suppose your method was to count the number of "e"s in the proposal, then multiply by 700 hours. That method would be extremely precise, but probably wildly inaccurate (unless your company does really complex engineering that takes a lot of time.)

(A counting method might have other advantages over expert judgment, even if counting is less accurate. The counting method might be easier to adjust — when it turns out that multiplying by 700 hours gives numbers that are way too high, you can modify the formula to multiply by 200 hours instead, and then test whether that works better. The counting method can be implemented in a software tool, and then anyone can use it, so junior engineers would be just as good as senior engineers at estimating the time required for a project. And so on. I'm going to assume all you care about is accuracy, though.)

So you can't say a priori (without any empirical evidence) which method is more accurate. But you can do a simple experiment to test and compare the methods. Over the next six months or so, for every project that comes through your company, keep track of the expert estimates, the estimates produced by your counting method, and the actual billable hours needed to complete the project. (Don't tweak your method to improve it while the experiment is running!) This experiment will produce intersubjective evidence that you can use to determine which method is more accurate.

  • Fantastic, thanks Dan (Dan Hicks as in "and is Hot Licks"?). I omitted details to make question shorter. We use "noun-counting" just as you mentioned: plug into a formula, along with a few other parameters, and get an estimate. We employ the Shewhart cycle of Plan-Do-Study-Act. At end of each project we compare actuals vs. estimates and revise model. We get better at estimated. The same cannot be true of the judgments because there's no basis for improving the method. – user1704475 Nov 21 '16 at 21:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.