The big picture: Hinton's Pause
Robinson’s work in neural nets, suddenly back in fashion, goes back to the 1980s and he has been cited by the “father of deep learning”, Geoffrey Hinton, as a pioneer.
“He has said that a few times, which is nice,” says Robinson. The Speechmatics founder says he was inspired by Hinton, prior to his landmark 1986 paper (pdf). "I realised I had a project to do."
Popular speech recognition systems from the big players can often struggle
What about the coming “freeze” or “pause” in AI that Hinton has predicted? The argument is that there has only really been one big breakthrough behind the advances in pattern recognition, but the pioneer of that technique now says it has hit its limit. Already there are grumblings 2017 is marked by anything much new. Have neural net techniques already picked all the low hanging fruit?
Robinson told us: “This is Geoff. He’s brilliant because he really believes in it, sticks with it, he’s not afraid to try different things.” He cites the example of seeing something once and realising its significance, whereas backpropagation needs to see it thousands or millions of times.
“There are a lot of very interesting problems that he’s thinking about like that, that we don’t quite have the tools for right now. It’s a wonderful problem to get to think about. We have to step back and investigate a lot of things that don’t work well as neural nets.
“But we’ve created a lot of progress. There is bound to be a time while before a load more advances come out, but there’s a lot of momentum.”
There’s more from “Project Omniglot to come, the company promises. ®