By "machine learning algorithm" I'm referring to basic, primarily statistical, machine learning algorithms; for concrete examples consider simple classifier algorithms like SVM or Bayesian classifier or decision trees. I'm stipulating that these machines do not have a mind.

I see a homology between these algorithms and the JTB theory of knowledge: the training set and model structure map to the justification, the results (declared class labels) map to the beliefs and the true aspect is unaffected. There is a similar homology to the "conjecture & criticism" view of knowledge (alanf) in the training and testing phases typically applied in (statistical) machine learning.

Obviously, the first objection would be that belief (and justification?) requires a mind -- something that these algorithms don't embody. Is that the only criterion that differentiates the "knowledge" of statistical machine learning systems and actual knowledge?


I'd claim that a subset of normal (human) knowledge is of the form that is amenable to representation in terms of machine learning -- an example that comes to mind is the knowledge that bird watchers employ to identify birds based on partial observation. Bird watchers have noticed which features are best observable and able to discriminate bird species from one another; that this is knowledge seems incontrovertible.

I also see something like a Sorite's paradox here: a bird watcher who id's birds by looking at them "knows his birds"; someone using a field guide to assist, still seems to have "justified true belief" when the correct id is made; what about more extensive support, like Merlin ID (which from the screen shots looks like it walks you through a decision tree)? what about just taking the result of some automated bird id algorithm at face value? (in the final case is anything new added other than that the species id has transferred from the iPhone screen to the bird watcher's mind?)

  • 1
    There seems to be a potential disanalogy in your aside about bird watchers insofar as the meaning of "notice" when applied to a human subject might be dissimilar to anything machine learning algorithms do. Or to put it another way, the "noticing" seems to need to be programmed into a program in a way at least the programmer fundamentally understands, but birdwatchers notice in a way they may not understand. – virmaior Sep 24 '15 at 0:26
  • @virmaior I interpret the fact that at the end of training the resulting model structures encode which features of the problem domain are relevant for the task at had as "noticing which features are relevant"; at least in an analogical sense. – Dave Sep 24 '15 at 12:02
  • 1
    I'm not sure I follow you on the use of "notice" there. Noticing to me seems to be a faculty that machines lack. Having a hard-coded detection sensor seems to be completely the opposite of at least one normal sense of noticing. And machines seem (and my knowledge here might be limited) to merely learn to filter out detection information that doesn't correlate. We seem (at least I think) to do the opposite at least on the level of consciousness and call this "noticing" – virmaior Sep 25 '15 at 8:09
  • @virmaior feature selection (machinelearningmastery.com/an-introduction-to-feature-selection) is an even more relevant analog to "noticing" especially when it is executed "on-line" in repsonse to a particular test case (aclweb.org/anthology/P15-1015). I'm inclined to agree that these types of algs. are probably not models or simulations of biological noticing; but in the cases where they are used, they apply, in the terms of the overall framework of learning, a similar role to noticing. – Dave Mar 27 '16 at 15:45

The OP proposal is similar in spirit to the one in Farkas's paper Belief May Not Be a Necessary Condition for Knowledge. His primary example is Otto, a guy with severe memory loss, who keeps all important information in a notebook which he carries with him at all times, and which "extends" his mind:

"There are parts of knowledge that are too tedious to acquire and retain in our head: remembering phone numbers or birthdays, for example... I shall argue that we can naturally extend the application of the concept of knowledge to such cases. Andy Clark and David Chalmers, who introduced extended mind scenarios (1998), put forward a similar argument for beliefs... My proposal is that this works better for knowledge than for belief".

The OP seems to be willing to go further in considering knowledge without a "thinking" agent altogether. Plato's dictum of knowledge as (justified true) belief remains widely held, but it is not beyond reproach. "I believe in God" and "I believe the sun will rise tomorrow" use "believe" in very different ways. One requires an active act of acceptance, while the other has an air of resignation to it, one mingles with hopes and desires, the other with assumptions and opinions. According to Radford, people may know and give correct answers without believing in them, and thinking they are guessing. If an act of acceptance is not required for knowledge in extended mind scenarios why should it be required at all? If knowledge is not a belief, at least part of the mind objection goes away.

One (Searle) might object that even if we take out the belief part, the justification part still requires "intentionality" and "understanding" to make knowledge. And only a mind can provide that. The authors of systems reply to Searle's Chinese room (Minsky, Cole) respond, however, that whatever machines have is mind enough. Cole explicitly writes about "a vast "background" of commonsense knowledge encoded in the program and the filing cabinets". Searle denies that "encoding" is possible, or that the "virtual mind" qualifies as a mind.

One can sidestep this argument over minds by discarding Plato's dictum, and describing knowledge as something like effective assumption on which actions are based. This is more or less the pragmatist theory which goes back to Peirce and James. The "grandfather" of pragmatism, Bain, defined belief as "that upon which a man is prepared to act". A machine can acquire and store information upon which it "acts", if this information is effective in making its actions "adequate" then it counts as knowledge. Arguably, that is all human knowledge amounts to as well.

For a broader perspective see SEP's Analysis of Knowledge.

  • Words are poor way to express beliefs and knowledge and to discuss their nature, but we don't have an alternative. Beware the results when using a poor tool. – user16869 Apr 28 '16 at 1:25

This borders around the idea of the "Chinese Room Thought Experiment". If you're not familiar with this experiment the following video and quote will be very helpful.

Searle's thought experiment begins with this hypothetical premise: suppose that artificial intelligence research has succeeded in constructing a computer that behaves as if it understands Chinese. It takes Chinese characters as input and, by following the instructions of a computer program, produces other Chinese characters, which it presents as output. Suppose, says Searle, that this computer performs its task so convincingly that it comfortably passes the Turing test: it convinces a human Chinese speaker that the program is itself a live Chinese speaker. To all of the questions that the person asks, it makes appropriate responses, such that any Chinese speaker would be convinced that he is talking to another Chinese-speaking human being.

The question Searle wants to answer is this: does the machine literally "understand" Chinese? Or is it merely simulating the ability to understand Chinese?[6][c] Searle calls the first position "strong AI" and the latter "weak AI".[d]

Searle then supposes that he is in a closed room and has a book with an English version of the computer program, along with sufficient paper, pencils, erasers, and filing cabinets. Searle could receive Chinese characters through a slot in the door, process them according to the program's instructions, and produce Chinese characters as output. If the computer had passed the Turing test this way, it follows, says Searle, that he would do so as well, simply by running the program manually.

Searle asserts that there is no essential difference between the roles of the computer and himself in the experiment. Each simply follows a program, step-by-step, producing a behavior which is then interpreted as demonstrating intelligent conversation. However, Searle would not be able to understand the conversation. ("I don't speak a word of Chinese,"[9] he points out.) Therefore, he argues, it follows that the computer would not be able to understand the conversation either.

Searle argues that without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and since it does not think, it does not have a "mind" in anything like the normal sense of the word. Therefore, he concludes that "strong AI" is false.

Source: https://en.wikipedia.org/wiki/Chinese_room


I hope these help and can guide you towards an answer. In my opinion as you may be able to tell, machine learning algorithms have no knowledge.

  • Searle's argument is relevant but understanding (his term) is not identical to knowledge (my focus). Or at least it is not obvious to me that these terms are interchangeable in this context; do you more clearly see how to unpack his example onto knowledge (JTB) itself? – Dave Sep 23 '15 at 17:19

Machine learning algorithms instantiate knowledge. It is perfectly possible for a system to have knowledge, but not to understand that knowledge.

Many philosophers hold the justified true belief. Knowledge that human beings have is not justified. Knowledge also need not be true, e.g. - Newtonian mechanics is false, but it is knowledge. And one of the reasons knowledge is imagined to be belief is that you need a person to justify it, but that's not true, so no belief is necessary.

Since nothing is left of the JTB theory of knowledge, that leaves the question of what separates knowledge from non-knowledge. Knowledge is information that solves some problem.

It's not necessary for anybody to know of the existence of a problem for a problem to be solved. For example, the human heart is a pump that can work continuously for decades without any human intervention or maintenance. Sparrows' wings help solve the problem of how to make small, light objects fly. The information on how to solve these problems is contained in the genes of the relevant organisms. The fact that nobody knows that information is irrelevant. Many slight variants on those structures would not solve the problems they solve. This close match between those structures and a particular problem requires an explanation. The explanation for this match has strong structural similarities to how human knowledge is created. Both human knowledge and biological knowledge is created by many rounds of variation and selection. The variation and selection take place in different media, and have some other relevant differences (see "The Beginning of Infinity" by David Deutsch, Chapters 15 and 16), but they both involve variation and selection.

Machine learning algorithms instantiate knowledge that is largely created by people. People decide what information to feed to the algorithms. People write the code that produces variations. People decide what sort of results count as success. People decide how the selection should work. The algorithms instantiate information about how to solve some problem in a form that we don't know how to read explicitly. But the algorithm may nevertheless solve some problem, facial recognition say, so people don't have to do it anymore. So the machine learning program instantiates some knowledge.

  • Example of knowledge but not understanding the knowledge: e = mc^2. – gnasher729 Mar 3 '16 at 14:02
  • Plenty of people understand E=mc^2, including me. If you don't, try reading "Special Relativity" by A. P. French. – alanf Mar 3 '16 at 14:41
  • I think that @gnasher729 's point is that we can know that e = mc^2 is "true" but not know anything more about what that means - how it became known and can be applied. I know gobs of things that I don't really understand. I know that your screenname is alanf and that you have (at this moment) 3,605 points, but why? Why? Does that mean I don't know those things? They are right in front of my face. I can even know the extent of what I don't know, to borrow the famous phrase. – user16869 Apr 28 '16 at 1:22
  • I don't think that you know e=mc^2 is true if you don't understand it. You just say it's true because it's a widely accepted slogan. The phrase means something that is true in the context of physics and is known and understood by some people. You know my screenname is alanf because you know that the browser is giving you accurate info about that name, likewise for my score. – alanf Apr 29 '16 at 10:37

Arthur Samuel, an American pioneer in the field of computer gaming and artificial intelligence, coined the term "Machine Learning" in 1959 while at IBM[12].

As a scientific endeavor, machine learning grew out of the quest for artificial intelligence.

Already in the early days of AI as an academic discipline, some researchers were interested in having machines learn from data.

They attempted to approach the problem with various symbolic methods, as well as what were then termed "neural networks"; these were mostly perceptrons and other models that were later found to be reinventions of the generalized linear models of statistics.[13] Probabilistic reasoning was also employed, especially in automated medical diagnosis.[14]:488

However, an increasing emphasis on the logical, knowledge-based approach caused a rift between AI and machine learning.

By 1980, expert systems had come to dominate AI, and statistics was out of favor.[15] Work on symbolic/knowledge-based learning did continue within AI, leading to inductive logic programming, but the more statistical line of research was now outside the field of AI proper, in pattern recognition and information retrieval.[14]:708–710; 755

Neural networks research had been abandoned by AI and computer science around the same time. This line, too, was continued outside the AI/CS field, as "connectionism", by researchers from other disciplines including Hopfield, Rumelhart and Hinton. Their main success came in the mid-1980s with the reinvention of backpropagation.[14]:25

Machine learning started to flourish in the 1990s.

The field changed its goal from achieving artificial intelligence to tackling solvable problems of a practical nature. It shifted focus away from the symbolic approaches it had inherited from AI, and toward methods and models borrowed from statistics and probability theory.[15] It also benefited from the increasing availability of digitized information, and the ability to distribute it via the Internet.

Machine learning and data mining often employ the same methods and overlap significantly, but while machine learning focuses on prediction, based on known properties learned from the training data, data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases). Data mining uses many machine learning methods, but with different goals; on the other hand, machine learning also employs data mining methods as "unsupervised learning" or as a preprocessing step to improve learner accuracy.

Much of the confusion between these two research communities (which do often have separate conferences and separate journals, ECML PKDD being a major exception) comes from the basic assumptions they work with: in machine learning, performance is usually evaluated with respect to the ability to reproduce known knowledge,

while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge.

Evaluated with respect to known knowledge, an uninformed (unsupervised) method will easily be outperformed by other supervised methods, while in a typical KDD task, supervised methods cannot be used due to the unavailability of training data.

Machine learning also has intimate ties to optimization: many learning problems are formulated as the minimization of some loss function on a training set of examples. Loss functions express the discrepancy between the predictions of the model being trained and the actual problem instances (for example, in classification, one wants to assign a label to instances, and models are trained to correctly predict the pre-assigned labels of a set of examples).

The difference between the two fields arises from the goal of generalization: while optimization algorithms can minimize the loss on a training set, machine learning is concerned with minimizing the loss on unseen samples.[16] Ref.-


  • Unfortunately, this answer fails to answer the question of whether these algorithms have knowledge. – Carl Masens Sep 26 '18 at 7:15
  • @Carl Masens- see the quote< data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases)> – drvrm Sep 26 '18 at 8:17
  • Again, this describes the discovery of knowledge by algorithms, rather than the having of knowledge, which are two very different things. – Carl Masens Sep 26 '18 at 14:34
  • @ Carl Masens-<while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge....this term 'previously unknown, new knowledge leads to the construction of knowledge... – drvrm Sep 26 '18 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.