Comments on: Low-power chips to model a billion neurons http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons Accelerating Intelligence Wed, 19 Jun 2013 09:27:15 +0000 hourly 1 http://wordpress.org/?v=3.4.1 By: asiwel http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27491 asiwel Sat, 04 Aug 2012 16:22:07 +0000 http://www.kurzweilai.net/?p=157570#comment-27491 There are many ways to define or conceptualize "intelligence" - natural or "artificial." This one, which I often prefer, is a measure of an agent's ability to recognize features of a context and selectively take advantage of those (vis-a-vis its own interent abilities) that will facilitate the achievement of a goal. The faster and "better" that spider (or for that matter a human being) can do this, the "smarter" we might conclude it is. There are many ways to define or conceptualize “intelligence” – natural or “artificial.” This one, which I often prefer, is a measure of an agent’s ability to recognize features of a context and selectively take advantage of those (vis-a-vis its own interent abilities) that will facilitate the achievement of a goal. The faster and “better” that spider (or for that matter a human being) can do this, the “smarter” we might conclude it is.

]]> By: Editor http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27219 Editor Thu, 02 Aug 2012 09:44:01 +0000 http://www.kurzweilai.net/?p=157570#comment-27219 I'll check with Furber. Sounds like his model is at a more granular level. I’ll check with Furber. Sounds like his model is at a more granular level.

]]> By: ben951 http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27217 ben951 Thu, 02 Aug 2012 09:28:52 +0000 http://www.kurzweilai.net/?p=157570#comment-27217 "With traditional digital circuits, that would require a supercomputer that’s 1000 times as powerful as the best ones we have available today. And we’d need the output of an entire nuclear power plant to run it." Really I thought Ray said IBM Sequoia would already have the power to simulate the human brain if we had the right software. “With traditional digital circuits, that would require a supercomputer that’s 1000 times as powerful as the best ones we have available today. And we’d need the output of an entire nuclear power plant to run it.”

Really I thought Ray said IBM Sequoia would already have the power to simulate the human brain if we had the right software.

]]> By: Zack http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27212 Zack Thu, 02 Aug 2012 05:53:07 +0000 http://www.kurzweilai.net/?p=157570#comment-27212 It sounds like your interests and work lie in the same domain as Jeff Hawkins. I'm sure you are familiar with his work, and his book 'On Intelligence'. He is very interested in modelling cortical columns, like yourself, and has much knowledge in the fields of neuroscience and electrical/computer engineering. It sounds like you guys are working on the same thing. Which is good... because in my opinion this task is among the most important tasks faced by humanity right now. It sounds like your interests and work lie in the same domain as Jeff Hawkins. I’m sure you are familiar with his work, and his book ‘On Intelligence’. He is very interested in modelling cortical columns, like yourself, and has much knowledge in the fields of neuroscience and electrical/computer engineering. It sounds like you guys are working on the same thing. Which is good… because in my opinion this task is among the most important tasks faced by humanity right now.

]]> By: Promethean http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27209 Promethean Thu, 02 Aug 2012 03:41:15 +0000 http://www.kurzweilai.net/?p=157570#comment-27209 Actually, some neuroscientists think a spiking neural net may be enough for mind uploading (see http://www.philosophy.ox.ac.uk/__data/assets/pdf_file/0019/3853/brain-emulation-roadmap-report.pdf pages 13-14). Even if they're wrong, there are plenty of experiments with spiking neural nets that could advance neuroscience radically. A failed attempt at mind uploading would be one; a replication of http://www.ncbi.nlm.nih.gov/pubmed/21397213 with a more humanlike model would be another. Actually, some neuroscientists think a spiking neural net may be enough for mind uploading (see http://www.philosophy.ox.ac.uk/__data/assets/pdf_file/0019/3853/brain-emulation-roadmap-report.pdf pages 13-14). Even if they’re wrong, there are plenty of experiments with spiking neural nets that could advance neuroscience radically. A failed attempt at mind uploading would be one; a replication of http://www.ncbi.nlm.nih.gov/pubmed/21397213 with a more humanlike model would be another.

]]> By: Promethean http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27204 Promethean Thu, 02 Aug 2012 03:12:13 +0000 http://www.kurzweilai.net/?p=157570#comment-27204 "The problem is that a car is not a replacement for a horse." This. Some of Toronto's police are on horseback -- not for ceremonial purposes like with the Mounties, but because horses are the best way to get through the woods and around ditches. “The problem is that a car is not a replacement for a horse.”

This. Some of Toronto’s police are on horseback — not for ceremonial purposes like with the Mounties, but because horses are the best way to get through the woods and around ditches.

]]> By: Peter van der Made http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27200 Peter van der Made Thu, 02 Aug 2012 02:01:11 +0000 http://www.kurzweilai.net/?p=157570#comment-27200 You are right, but I don't believe that computers are the way forward. Their technology is based on fetching instructions and executing them. I built a circuit that behaves like a Izhikevich neuron, but that is only a part of the solution. The real power of the brain is in its dynamic synapses. Memory stored in dynamic synapses is processed by the neuron. The output from the post-synaptic neuron updates the synapses. I built a learning machine using this method. Using this technology, a mere 25 wafers could contain the entire human brain. I added a parallel inteface to my neuron so that each synapse in the matrix can read by a computer. The entire system learns like a child learns, formats itself during early childhood, and has a 100 trillion synapses. All I need now is the money to build this machine....I simulated part of the design in FPGA with very encouraging results You are right, but I don’t believe that computers are the way forward. Their technology is based on fetching instructions and executing them. I built a circuit that behaves like a Izhikevich neuron, but that is only a part of the solution. The real power of the brain is in its dynamic synapses. Memory stored in dynamic synapses is processed by the neuron. The output from the post-synaptic neuron updates the synapses. I built a learning machine using this method. Using this technology, a mere 25 wafers could contain the entire human brain. I added a parallel inteface to my neuron so that each synapse in the matrix can read by a computer. The entire system learns like a child learns, formats itself during early childhood, and has a 100 trillion synapses. All I need now is the money to build this machine….I simulated part of the design in FPGA with very encouraging results

]]> By: Peter van der Made http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27198 Peter van der Made Thu, 02 Aug 2012 01:39:14 +0000 http://www.kurzweilai.net/?p=157570#comment-27198 Since 2004 I have been working full-time in the development of a new Artificial Intelligence paradigm. I started after the sale of my Computer Immune System vCIS to ISS (now IBM) with my simple RISC processor core that I patented in Europe in 1986. The advantage of that core is its simplicity, and that I can put many on a single chip. The problem I found in this approach is that each core needs memory, ROM, and I/O. Each core is still a sequential 'von Neumann' computer. The brain does not compute, it associates incoming information, in the shape of temporal pulse streams. These pulse streams directly address memory, which is processed by the neuron. Its kind of back to front to the method used in a computer. I came up with the idea of using many cores to simulate the brain in 2005. The overhead in programming and peripheral circuitry made me think that there must be a better way. Instead of programming a processor to behave like a digital neuron I came up with the solution to build a digital neuron and synapses from logic gates, and to put 15,000 of those on a single chip. I tested that idea in FPGA (programmable logic gates), and found that the simulated dynamic neural matrix learned very quickly. I exposed the FPGA to 10 frequecies, and the neural matirix learned to recognise those frequencies in speech patterns. This was a simple test, because the FPGA can only contain a small part of the intended chip. I concluded that the way forward lies in learning machines that are modeled on the brain's and structure - not larger and faster 'von Neumann' computers. We become an intelligent entity through learning - and I don't mean academic learning. We learn from the moment the brain is formed. A baby learns how to move its limbs through preprioception - feedback from the muscles and tendons in that limb. At the moment I am looking for funding to scale my design to emulate a single cortical column. I am also publishing a book on my findings of the last 8 years, called 'Higher Intelligence' Since 2004 I have been working full-time in the development of a new Artificial Intelligence paradigm. I started after the sale of my Computer Immune System vCIS to ISS (now IBM) with my simple RISC processor core that I patented in Europe in 1986. The advantage of that core is its simplicity, and that I can put many on a single chip. The problem I found in this approach is that each core needs memory, ROM, and I/O. Each core is still a sequential ‘von Neumann’ computer. The brain does not compute, it associates incoming information, in the shape of temporal pulse streams. These pulse streams directly address memory, which is processed by the neuron. Its kind of back to front to the method used in a computer.
I came up with the idea of using many cores to simulate the brain in 2005.
The overhead in programming and peripheral circuitry made me think that there must be a better way. Instead of programming a processor to behave like a digital neuron I came up with the solution to build a digital neuron and synapses from logic gates, and to put 15,000 of those on a single chip. I tested that idea in FPGA (programmable logic gates), and found that the simulated dynamic neural matrix learned very quickly. I exposed the FPGA to 10 frequecies, and the neural matirix learned to recognise those frequencies in speech patterns. This was a simple test, because the FPGA can only contain a small part of the intended chip. I concluded that the way forward lies in learning machines that are modeled on the brain’s and structure – not larger and faster ‘von Neumann’ computers. We become an intelligent entity through learning – and I don’t mean academic learning. We learn from the moment the brain is formed. A baby learns how to move its limbs through preprioception – feedback from the muscles and tendons in that limb. At the moment I am looking for funding to scale my design to emulate a single cortical column. I am also publishing a book on my findings of the last 8 years, called ‘Higher Intelligence’

]]> By: Spikosauropod http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27179 Spikosauropod Wed, 01 Aug 2012 23:26:13 +0000 http://www.kurzweilai.net/?p=157570#comment-27179 Mumen: "Try to compare a horse with it’s modern replacement, the car and you will have a neat image of this unbearable fact." If only I had a dollar for every time I have heard the old horse/car bromide. The problem is that a car is not a replacement for a horse. A car is merely an alternate form of transportation that is now preferred to a horse. A better example of attempting to replace a horse with a machine would be Boston Dynamics’ Bigdog. Since the success of Bigdog is actually reliant on computer programming, you could say that we are actually in the same position with respect to replacing a horse with a robot as we are with replacing a horse’s mind with a computer. They are actually the same problem. Similarly, Petman might be a good indicator of where we are with respect to duplicating a man. As with the horse, a better computer algorithm will lead to a better robot duplicate. The SpiNNaker project, if successful and sufficiently miniaturized, could transform Petman into Cyberman. Of course, Petman is not the only model of a man we have to work with. We also have Watson and a handful of other science projects which have never been integrated. We have many of the parts of a man which, when successfully miniaturized and integrated, may bring us closer to a man. Also, there is no argument that can be made that we had sufficient raw computing power in the 80’s to equal a human brain. We won’t have that kind of raw power for another decade. Mumen: “Try to compare a horse with it’s modern replacement, the car and you will have a neat image of this unbearable fact.”

If only I had a dollar for every time I have heard the old horse/car bromide.

The problem is that a car is not a replacement for a horse. A car is merely an alternate form of transportation that is now preferred to a horse. A better example of attempting to replace a horse with a machine would be Boston Dynamics’ Bigdog. Since the success of Bigdog is actually reliant on computer programming, you could say that we are actually in the same position with respect to replacing a horse with a robot as we are with replacing a horse’s mind with a computer. They are actually the same problem.

Similarly, Petman might be a good indicator of where we are with respect to duplicating a man. As with the horse, a better computer algorithm will lead to a better robot duplicate. The SpiNNaker project, if successful and sufficiently miniaturized, could transform Petman into Cyberman. Of course, Petman is not the only model of a man we have to work with. We also have Watson and a handful of other science projects which have never been integrated. We have many of the parts of a man which, when successfully miniaturized and integrated, may bring us closer to a man.

Also, there is no argument that can be made that we had sufficient raw computing power in the 80’s to equal a human brain. We won’t have that kind of raw power for another decade.

]]> By: Spikosauropod http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27177 Spikosauropod Wed, 01 Aug 2012 22:42:11 +0000 http://www.kurzweilai.net/?p=157570#comment-27177 If I had it to do over again, I would have used "expects" instead of "suspects". If I had it to do over again, I would have used “expects” instead of “suspects”.

]]> By: MrFriendly http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27168 MrFriendly Wed, 01 Aug 2012 21:28:40 +0000 http://www.kurzweilai.net/?p=157570#comment-27168 Biological neurons have millions of molecular interactions within them, so creating a spiking model is just the first, important step. The brain is "computing" even when no spikes are being sent. Also, I find it interesting that Furber is using the Izhikevich equations for his spike dynamics. That's a really successful and computationally efficient model that could, at this large scale, powerfully solve many narrow AI problems, such as object/facial recognition. Biological neurons have millions of molecular interactions within them, so creating a spiking model is just the first, important step. The brain is “computing” even when no spikes are being sent.

Also, I find it interesting that Furber is using the Izhikevich equations for his spike dynamics. That’s a really successful and computationally efficient model that could, at this large scale, powerfully solve many narrow AI problems, such as object/facial recognition.

]]> By: Mumen http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27164 Mumen Wed, 01 Aug 2012 20:45:22 +0000 http://www.kurzweilai.net/?p=157570#comment-27164 If intelligence was just a fact of "quantity of something", the AI would be real since the 80's... Looks to me brain simulation is a lot further than what we dare to imagine, and it is not only a question of computerized algorithm, it is something else. Try to compare a horse with it's modern replacement, the car and you will have a neat image of this unbearable fact. So unbearable that one prefers not to see it, and dream than our inventions might magically contain some "intelligence" not petrified that would not directly come from us, the humains. This is a naive belief. If it is only a question of power and watts, why not try first to reproduce the cleverness of any insect in a computer, with it's adaptability ? In fact, such searchers will see that really Descartes was wrong, they will see that body and mind cannot be separated, because at a certain point they will have to think about the support - its reproduction and mutation - instead of oversimplifying the reality with the Scalpel Cartesian. In saying that I am not denying those brilliant researches. I just want to emphasize the wrong vocabulary which is (it seems to me) driven by the sensationalism (and maybe the need for credits) and which have the power to mislead the searchers themselves. If intelligence was just a fact of “quantity of something”, the AI would be real since the 80′s…

Looks to me brain simulation is a lot further than what we dare to imagine, and it is not only a question of computerized algorithm, it is something else.

Try to compare a horse with it’s modern replacement, the car and you will have a neat image of this unbearable fact. So unbearable that one prefers not to see it, and dream than our inventions might magically contain some “intelligence” not petrified that would not directly come from us, the humains. This is a naive belief.

If it is only a question of power and watts, why not try first to reproduce the cleverness of any insect in a computer, with it’s adaptability ? In fact, such searchers will see that really Descartes was wrong, they will see that body and mind cannot be separated, because at a certain point they will have to think about the support – its reproduction and mutation – instead of oversimplifying the reality with the Scalpel Cartesian.

In saying that I am not denying those brilliant researches. I just want to emphasize the wrong vocabulary which is (it seems to me) driven by the sensationalism (and maybe the need for credits) and which have the power to mislead the searchers themselves.

]]> By: John http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27147 John Wed, 01 Aug 2012 19:43:24 +0000 http://www.kurzweilai.net/?p=157570#comment-27147 Well said. Well said.

]]> By: John http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27146 John Wed, 01 Aug 2012 19:41:55 +0000 http://www.kurzweilai.net/?p=157570#comment-27146 Well, the point was, this can be more effective than anyone suspects, including himself. It could be so effective, he himself would be awed and said 'God, i knew this would be big, but this is even bigger!'. No contradiction here. Well, the point was, this can be more effective than anyone suspects, including himself. It could be so effective, he himself would be awed and said ‘God, i knew this would be big, but this is even bigger!’. No contradiction here.

]]> By: someday69 http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27143 someday69 Wed, 01 Aug 2012 18:59:10 +0000 http://www.kurzweilai.net/?p=157570#comment-27143 Just another day at the office',,,Spin'naker...sounds like some'thing on ah'sail'boat..some'kind of sail''that gets' unfurl'ed out front...off the bow-- Sail'on'''brother's an''sister's... Just another day at the office’,,,Spin’naker…sounds like some’thing on ah’sail’boat..some’kind of sail”that gets’ unfurl’ed out front…off the bow–
Sail’on”’brother’s an”sister’s…

]]> By: AZryan http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27141 AZryan Wed, 01 Aug 2012 18:50:26 +0000 http://www.kurzweilai.net/?p=157570#comment-27141 "This could be more effective than anyone suspects." Impossible since you yourself apparently suspect it. Just kidding, but it is similar to how people misuse the word 'literally' literally all the time. heh “This could be more effective than anyone suspects.”
Impossible since you yourself apparently suspect it.
Just kidding, but it is similar to how people misuse the word ‘literally’ literally all the time. heh

]]> By: Spikosauropod http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27127 Spikosauropod Wed, 01 Aug 2012 17:26:18 +0000 http://www.kurzweilai.net/?p=157570#comment-27127 They may find that an efficient algorithm more than makes up for the lack of raw processing power. This could be more effective than anyone suspects. They may find that an efficient algorithm more than makes up for the lack of raw processing power. This could be more effective than anyone suspects.

]]> By: Bri http://www.kurzweilai.net/low-power-chips-to-model-a-billion-neurons/comment-page-1#comment-27122 Bri Wed, 01 Aug 2012 17:07:08 +0000 http://www.kurzweilai.net/?p=157570#comment-27122 Looks to me like whole brain simulation is a lot closer than we think. True AI could be just ahead. As I've said before, look at a jumping or tree spider, with a head the size of a pin. Able to model a constantly moving world of leaves. If you have ever watched one, it's surprising how smart they are. This chip configuration probably has many more times the neuronal capabilities of the spiders tiny brain. We are very close to true AI. Looks to me like whole brain simulation is a lot closer than we think. True AI could be just ahead. As I’ve said before, look at a jumping or tree spider, with a head the size of a pin. Able to model a constantly moving world of leaves. If you have ever watched one, it’s surprising how smart they are. This chip configuration probably has many more times the neuronal capabilities of the spiders tiny brain. We are very close to true AI.

]]>