Is this an interlingua?
From the paper:
We propose a simple, elegant solution to use a single Neural Machine Translation (NMT) model
to translate between multiple languages. Our solution requires no change in the model architecture
from our base system but instead introduces an artificial token at the beginning of the input sentence
to specify the required target language.
From Wikipedia:
Neural machine translation (NMT) is an approach to machine translation that uses an artificial neural network to predict the likelihood of a sequence of words, typically modeling entire sentences in a single integrated model.
The word sequence modeling was at first typically done using a recurrent neural network (RNN). A bidirectional recurrent neural network, known as an encoder, is used by the neural network to encode a source sentence for a second RNN, known as a decoder, that is used to predict words in the target language. Recurrent neural networks face difficulties in encoding long inputs into a single vector. This can be compensated by an attention mechanism which allows the decoder to focus on different parts of the input while generating each word of the output.
So, maybe.
The system described in this paper translates sentences to vectors (a fixed-length string of numbers; this is a standard technique), and then translates those numbers back to sentences. This system uses the same model to translate multiple different languages, by kinda treating all languages as if they're the same language with a more complicated grammar. Previous systems used a separate model per pair of languages.
The representation for Apfel in one model can be mapped onto the representation for pomme in another, because both refer to apples! Apples are generally described as red or green, no matter what language you're using, so the structure around them is the same. (Especially so if you're using a corpus consisting of the same document translated into loads of languages – but I'd expect this even if you weren't.)
So there's an extent to which this internal representation is an interlingua. However, it's probably more accurate to describe it as a correlation. See section 5.2 of the paper (emphasis mine):
For example, Figure 3a shows a t-SNE projection of attention vectors from a model that was trained
on Portuguese→English (blue) and English→Spanish (yellow) and performing zero-shot translation from
Portuguese→Spanish (red). This projection shows 153 semantically identical triples translated as described above, yielding 459 total translations. The large red region on the left primarily contains zero-shot
Portuguese→Spanish translations. In other words, for a significant number of sentences, the zero-shot
translation has a different embedding than the two trained translation directions. On the other hand, some
zero-shot translation vectors do seem to fall near the embeddings found in other languages, as on the large
region on the right.
It is natural to ask whether the large cluster of “separated” zero-shot translations has any significance. A
definitive answer requires further investigation, but in this case zero-shot translations in the separated area
do tend to have lower BLEU scores.
Additionally, this "interlingua" (a vector) probably has no grammar. Charitably, you could describe it as an impressionist artwork of parts of a sentence, or a numerical representation of complex concepts, or a protocol, but I don't think it's really a language. It's better described by statistics than linguistics.1 If it's a language, it's an alien one.
Does this support the Language of Thought Hypothesis?
From Wikipedia:
The language of thought hypothesis (LOTH), sometimes known as thought ordered mental expression (TOME), is a view in linguistics, philosophy of mind and cognitive science, forwarded by American philosopher Jerry Fodor. It describes the nature of thought as possessing "language-like" or compositional structure (sometimes known as mentalese). On this view, simple concepts combine in systematic ways (akin to the rules of grammar in language) to build thoughts. In its most basic form, the theory states that thought, like language, has syntax.
Since this translation tool does not exhibit behaviour obviously like thought, I can't see how it supports this hypothesis at all. The LOTH is about human thought being language-like, not about human language being universal in some way. (It's not even supposing that "mentalese" is universal.)
Does this support there being a universal grammar?
From Wikipedia:
Universal grammar (UG), in modern linguistics, is the theory of the genetic component of the language faculty, usually credited to Noam Chomsky. The basic postulate of UG is that there are innate constraints on what the grammar of a possible human language could be. When linguistic stimuli are received in the course of language acquisition, children then adopt specific syntactic rules that conform to UG.
UG is a claim about human psychology; this machine translation technology is not limited to natural language, and would exhibit the same behaviour in languages that are outside a hypothetical UG (so long as they have enough locality to be comprehensible). The internal vector representation is more about the meaning than the grammar. I don't think this says anything about UG.
Except to the extent we can model human language processing as behaving like this machine translation model. But by the time we know enough about human psychology to know if our language processing works this way, UG will already be settled.
1: In fact, it's the kind of format that we convert written language into, in order to apply statistics to it. You can do statistics on written language directly, but it's usually quite limited unless you're very clever. (Not to say that Markov chains are in any way the limit of what you can do if you're very clever with statistical analysis of language.)
