In this post, Sean Kheraj assesses his Google Home device’s performance as a “history calculator,” testing how well it can define historical terms and answer basic historical questions. He puts this in context with some history of digital history:
In their 2005 article in First Monday, Daniel J. Cohen and Roy Rosenzweig recount the story of a remarkably prescient colleague, Peter Stearns, who “proposed the idea of a history analog to the math calculator, a handheld device that would provide students with names and dates to use on exams—a Cliolator, he called it, a play on the muse of history and the calculator.” [1] Cohen and Rosenzweig took Stearns’s idea and ran with it. They set out to build the Cliolator in the form of a software algorithm called “H-Bot” which served as a history fact finder, scouring the web for information to answer questions about the past. Even with all its limitations and the limits to what was available online in 2004-05, H-Bot was remarkably accurate. It was especially adept at identifying dates and simple definitions. Where it fell short was in more complex questions, including “hows” and “whys”.
Since 2005, the web has grown well beyond the scale of information available to H-Bot, providing a much larger reservoir of data to crawl. And artificial intelligence and machine learning software have brought us much closer to the so-called Cliolator. Google, Amazon, Microsoft, and Apple have all developed quasi-artificial intelligence voice assistants that can parse natural language queries and supply answers drawn from the web… Are these the ultimate versions of the Cliolator? I decided to put one to the test to see how well this form of artificial intelligence could perform in my introductory Canadian history course.
Kheraj ultimately finds that the Google Home does a pretty good job defining terms (like Wendat Confederacy and Seigneurial System) but, unsurprisingly, not such a good job explaining their historical significance. The Google Home is limited to analysis that is already available on the web. As Kheraj puts it, “It cannot generate new ideas based on the information it finds.” From this experiment, Kheraj has a few insights for historians. The first is that historians should contribute to resources like Wikipedia and make their research openly available so that there is “more high quality historical research on the web.” The second is that, “[A]rtificial intelligence is already influencing historical practice. The Google Home is not going to replace historians, but it is an example of how advanced algorithms and search engines can influence research practice and history education.” For this reason, historians should seek to understand these algorithms. Kheraj’s third insight is that these devices are not going to be “the most important artificial intelligence tools for historical research… AI software that assists in the analysis of historical documents and other data will have far more pervasive influence on historical research in the future.”
Having picked up a Google Home of my own this Black Friday, I’ve been doing my own unofficial experiments with the device. I’ve found it most useful while I’m reading. I can get a quick answer to whatever little question I think of without taking the risk of falling into a Wikipedia hole. Most of the time, I ask about dates, like when was Pontiac’s War or when was The Middle Ground published, but I throw in the occasional (and more perilous) “what was _____” question. The advantage of the Google Home is also its disadvantage; the speed and simplicity of getting an answer to your question also makes it more difficult to evaluate the accuracy of that answer or get any sense of the complexity of and debates around that topic. You only hear the first sentence or two, but the Google Home presents it like the full story. Unlike viewing Wikipedia on a screen, there are no footnotes to click (or lack of footnotes to raise a red flag) and no warning if the article fails to meet some Wikipedia standard. I plan to keep using my Google Home for the limited circumstances when I use it, but I do have concerns about how it might be presenting history to anyone using it as more than a supplement to other forms of historical inquiry. Kheraj’s insights are valuable, especially his call for historians to understand how algorithms are influencing historical practice. All of the issues plaguing online search are amplified in the way that voice assistants treat information. If these devices prove to be more than a gimmicky fad, historians would do well to consider their implications for how we work and how we present our work.
Read the original Editors’ Choice piece here.
