Hi @trilo, interesting article. I'm going to have to follow you now! Your interest in neuroscience fits with my interest in machine learning. Maybe we can compare notes now and then. I believe that our next great leaps in machine learning will be inspired by things we learn from biology.

Your proposal sounds interesting, but I suspect that the devil will be in the details. One concern is whether fMRI will give you enough granularity. I wonder how many neurons and synapses are represented by one pixel in the image.

You might want to try the experiment without principal component analysis in the beginning. That's the advice I've heard from data scientists in the field. PCA is basically like a projection of points in multidimensional space to eliminate one dimension, then another, then another. For example, starting in 3D space, we would project the points onto some plane cutting through that space, and do it in such a way as to minimize error. But that would have to be done consistently with all of your fMRI data. If you eliminate a dimension, it has to be done consistently for the entire data set. Do you have an intuition from the study of neuroscience as to what that "immaterial" dimension might be?

The next hard part is going to be the training. I think you'll need a rather large data set of fMRI images that have been hand-labeled with a word that describes what the subject brain was thinking about at the time the image was taken. That might be expensive to generate, unless labs already working on this problem are sharing heir data.