How is neural network different from data-center? Is it just way to say 'we got a shit ton of CPU cores on this mofo'. Sounds really cool, I like Googly for stuff like this.
How is neural network different from data-center? Sounds really cool, I like Googly for stuff like this.
A data center is essentially a huge store of information (well, they actually do a lot more - but I'll keep it simple). A simple way to think about is it being full of excel spreadsheets (they aren't using excel spreadsheets, but you get the idea). People can then 'query' these data centres and it will do it's best to get you relevant information.
For example, lets say I query "pictures of cats riding roombas".
A regular ass Google query, something that is still extremely sophisticated, would look for images with labels/titles/key words near it that match 'cat(s)' 'riding' and 'roomba(s)'. Doing its best to put the most likely results (probably something that says "my cat riding a roomba") near the top.
Now, what Neural networks and deep learning does is entirely different. Again, I am going to simplify it, this time because I really don't have the intelligence to grasp all it's intricacies let alone explain them - but for the same query of "pictures of cats riding roombas":
It would 'look' at the images themselves. It would first have already
taught itself what a roomba looks like from all sorts of angles, what a cat looks like from all sorts of angles, what
riding could/should mean, and what that sentence would mean in a picture. Thus it would then find an actual image with a cat riding a roomba - regardless of what it was titled or what words are surrounding it.
How it does this is extremely complicated system work? Again, I'm not qualified to give you a good answer but basically, a neural network is designed after a biological system of neuron systems. These are massively parallel systems where (in for example the human brain) billions of nodes are connected to billions of other nodes with trillions of total analog connections. Simulating this is extremely difficult, but it is a venture a lot of different companies are pursuing.
Last year google made a 1million node practical system, which is one of the largest available. This system utilized 16000 processors, and they basically just told it to watch a shitload of youtube videos. It did, and afterward it came out the other end 'categorizing' things. It took stills of human faces, cats and other things and put them in their own categories without being told that these things are similar or even what these things were - it was just able to recognize patterns. The 'learning' process took almost all the power of the system, but replicating the search apparently only takes 100 processors (I had no idea about this until this article, but it makes sense). That technology was folded into the technology available at Google, and one of the great things that came out of it for example was a huuuuge reduction in errors regarding voice recognition, something like 30% - which is why Googles voice recognition tech is so impressive.
Hinton is working on a 1 trillion node system.
