15

Google recently updated their translation tool so that it can now translate between language pairs that it hadn't seen before, something they're calling "zero-shot translation." See here for the full paper and here for a summary.

For example, they can train a neural network to translate from Japanese to English and from English to Korean. They then ask it to perform Japanese-Korean translations, and it performs "reasonably" well, even though it was never trained to translate that particular language pair.

What stood out to me is the following conclusion from the article:

5.1 Evidence for an Interlingua:

Several trained networks indeed show strong visual evidence of a shared representation. For example, Figure 2 below was produced from a many-to-many model trained on English↔Japanese and English↔Korean. To visualize the model in action we began with a small corpus of 74 triples of semantically identical cross-language phrases. That is, each triple contained phrases in English, Japanese, and Korean with the same underlying meaning.[...] Inspection of these clusters shows that each strand represents a single sentence, and clusters of strands generally represent a set of translations of the same underlying sentence, but with different source and target languages.

In other words, Google was able to group sentences into an underlying geometrical structure, which corresponds to a meta-language, or as the authors say, an interlingua. Some of the popular articles I've read about this are going so far as to say that Google's Neural Network "invented its own language", but I feel that they're just being sensationalist.

My question: Does this evidence for a meta-language or a shared representation underlying all languages support theories like Jerry Fodor's Language of Thought Hypothesis (i.e. Mentalese) or Chomsky's claim of there being a universal grammar?

Improve this question
  • Sure it could be used as support, however, Chomsky's universal grammar has already been demonstrated to be bunk. See Sampson's "There Is No Language Instinct" as well as criticisms, particularly re: the Pirahå – Mr. Kennedy Jan 10 '17 at 23:02
  • 3
    Doesn't it support just the opposite? The "interlingua" is an interpretation by human researchers of neuro-net's global states, the net itself, on the other hand, is not based on primitives, nor combines them compositionally, as LOT and UG would have it. Not only does Google's net only emulate interlingua, it developed this emulation, which goes against all "wired language" speculations. That unification of different languages optimizes translation is no more surprising than existence of esperanto, but it does not support esperanto in the brain. – Conifold Jan 11 '17 at 0:52
  • 3
    There will never be a true language of thought until you can completely discard the word representation. As long as you are just representing something, you are always falling short of a what needs to be represented. It's just another code which requires humans to decode it, because there is no foreseeable way to free machines from the code. – user3017 Jan 11 '17 at 2:35
  • 2
    @Conifold one of the assumptions of pattern recognition (Neural Nets, SVMs, etc...) is that there exists a pattern to be discovered, if the PR algorithm can't find a natural pattern and "forces" one on the existing data, this leads to overfitting, where the algorithm works well ont the training set but fails miserably when trying to generalize to new patterns. In this case that means that Google's NNet would be able to translate existing language pairs very well but not be able to generalize to new language pairs - the fact that it was able to partition the space so efficiently is remarquable – Alexander S King Jan 11 '17 at 4:30
  • 2
    Given that Japanese and Korean have major documented similarities, this example sounds more like a case of a mentalist improving their odds of a good-looking result than anything else. This question may be a good fit for Skeptics.SO in that regard. – bright-star Jan 11 '17 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.