Spanning a vast geographic area rich with natural resources, Canada is rare among developed nations as a country whose economy relies heavily on natural resources. Departing from that mold, the country’s moves in the artificial intelligence field have captured considerable attention over the past decade.

One factor tying together Canada’s state-level AI research center and global high-tech behemoth Google’s AI research and development is a man named Geoffrey (Geoff) Hinton.

Many AI field insiders respond to the name “Geoff Hinton” with approval and admiration. Currently holding positions as Professor Emeritus at the University of Toronto and vice president of research and development at Google Brain, Hinton’s theses are cited more than any others in the field, with even more citations than the total of the next three top AI researchers.

One of his theses, published jointly by Hinton and colleagues in 1986, is credited for introducing the back-propagation algorithm, which laid the groundwork for all current deep learning technology. Hinton discovered that back propagation can be used to train two to three layers of deep neural networks.

Some 26 years later, in 2012 Hinton published a joint paper with University of Toronto students that applied deep neural networks trained through back propagation to the field of visual recognition.

They defeated the most advanced systems at the time in a visual recognition competition, launching “deep learning” into the mainstream.

Dressed in a plaid shirt and sneakers, the 69-year-old Hinton stands before a table in a tiny laboratory in Google’s downtown Toronto office, staring intently at a computer screen. A white board on the wall behind him is covered with numbers and equations that are completely opaque to laymen.

The lab, which did not even exist less than a year ago, was established expressly for Hinton. At his request, it is equipped only with a standing desk and no chairs.

 “I haven’t worked sitting down since 2005,” relates the widely acknowledged “father of deep learning” in reference to his back ailments.

He has also not traveled by airplane since 2005, confining him to Canada. If he were to visit Asia, he conjectures, he would likely have to travel by sea.

Marginalized Academic

A native of Wimbledon in the United Kingdom, Hinton studied at Cambridge and the University of Edinburgh, where his research in artificial intelligence focused on neural networks. “I’m keenly interested in how the brain works,” he says.

As a student, he made up his mind to learn all about how the nerves that transmit signals learn or operate to produce what we call intelligence - questions that scholars of conventional artificial intelligence could not answer at the time.

As a revered figure in the AI field, why did Hinton choose to settle in Canada, rather than staying at home in England or moving to the R&D hotspot of the Silicon Valley? “I study neural networks, and at the time nobody in UK academic circles would hire me,” Hinton states.

From the 1960’s to the 1990’s, neural network theory was marginalized in the AI research field. Hinton once had a paper rejected by a deep learning symposium, and mainstream artificial intelligence circles did not even take research on neural networks seriously.

Yet Hinton persisted, conducting post-doc research in the U.S. at Carnegie Mellon University.

“I was taken aback at learning that most of the funding going to AI research came from military institutions,” recalls Hinton, who promptly picked up and relocated north of the border to Canada.

Extensive Investment for Hard Times

From the 1990s through early 2004, despite having conducted over 20 years of deep learning on the back-propagation algorithm, lack of sufficient computing power prevented processing of the mass quantities of data demanded by neural networks. “Even I was very disappointed, to be honest,” Hinton admits.

In 2004, having faithfully adhered to neural network research since his student days, Hinton finally got a break. That year, the Canadian Institute for Advanced Research (CIFAR) provided funding for Hinton’s neural networks research, without even limiting researchers to working at CIFAR.

Hinton also earned the backing of two other AI experts, namely New York University’s Yann LeCun and Yoshu Bengio of the Université de Montréal. With those two heavyweights on his side, he formed an ad hoc deep learning team, hand-picking researchers and creating a system to process, understand, analyze, and respond to vast quantities of imagery, sound, and language. Over the course of their efforts, they made the deep learning algorithm more robust and capable of processing more data. 

In 2011, Hinton’s team partnered with Professor Andrew Ng of Stanford, co-founder of Google Brain, to establish Google’s deep learning project. The deep learning technology Hinton and his team came up with swept AI competitions all over the world, achieving a technical recognition rate higher than that of the human eye. Google makes extensive use of neural network technology for voice and image recognition.

 ‘Open’ Soft Power for High ROI

“With just a half-a-million-dollar-a year-investment from CIFAR, Hinton's consortium of free thinkers is set to feed countless dollars back into the economy. It's already happening at Google. The return on investment for both Canada and the rest of the world has been tremendous, says Denis Therien, CIFAR’s vice president for research and partnerships.” According to an article in Wired magazine.

At the national level, Canada’s open resources and platform, and strategy for recruiting top talent from countries around the world appears relaxed, and makes it hard for government to measure the return on its investment. Nonetheless, as noted by The Economist, “the soft benefits are clear.”

To date, Google, Facebook, Samsung, and Microsoft have all established AI laboratories in Toronto and Montreal, and rumors circulated in September that Amazon was considering setting up its second headquarters in Canada, making Canada a sort of northern Silicon Valley and earning the nickname “the Maple Valley.”

Major AI Breakthroughs Help Doctors Diagnose

Leveraging Hinton’s research momentum and connections, the Canadian government established the Vector Institute just this autumn, making AI an integral component of the medical field.

Hinton has been making the rounds at medical institutions around Toronto lately, speaking to healthcare professionals and startups, and aggressively integrating deep learning in medical applications.

“The medical field is a key focus. Five years from now, advancements in deep learning and visual recognition technology will let us predict tumors and enable us to project patients’ medical conditions,” says Hinton excitedly, adding that doctors are especially enthusiastic about the possibilities for improving diagnosis.

“People don’t know what to do with certain data, but deep learning can predict outcomes,” explains Hinton.

From the margins to the core, the stubborn scholar that once refused to take U.S. Defense Department funding could barely have imagined that one day he would become a key part of Canada’s attractiveness to the world and an important engine in the rapidly advancing field of artificial intelligence research.

Translated from the Chinese article by David Toman

---

Additional Reading

♦ Can Taiwan Find an AI Niche?
♦ AI: A Wave Taiwan Must Ride
♦ What Happened to Canada’s Economic Competitiveness?
♦ How Canada’s unique research culture has aided artificial intelligence (The Economist)