Go deeper - join us!
In the past few months, Google has introduced–and progressively tweaked–generative AI responses to search queries. Its accuracy has undoubtedly improved since its inception, but some concerns about its reliability remain.
According to Search Engine Roundtable’s Barry Schwartz, the AI mostly serves to summarize the main points of any search results. This can cause some page results–such as those not indexed by Google, or ones that have been locked behind paywalls–to be excluded from the AI summary, but for the most part, generative search tends to do fairly well in terms of summarizing basic information.
Per Google, the tool is admittedly “experimental” (and says so right at the top of the search page); this, coupled with the fact that users almost always have to choose to view AI-generated results (or click to expand an existing generative response), seemingly exempts this tool from the bulk of the criticism that most might feel compelled to levy. Skeptics, nevertheless, will notice a handful of issues.
Anecdote time: A few weeks ago, I looked up a phrase that appeared in a YouTube video that one of my students had shown to me–as I often do before incorporating such a phrase into my lexicon–and found that Google’s search results yielded nothing of substance (as I had suspected, the phrase was patent nonsense). However, the generative AI response referred to the phrase’s origin as “a tweet” from a specific user.
This is a problematic answer for two reasons: It is demonstrably false–the phrase clearly originated in the video, especially given that the tweet was posted well after the video aired–and it centered an obscure (like, 20 followers) account right at the top of Google’s search results, which could have unexpected consequences for some poor fella who, like me, found an internet non sequitur worth repeating sans context.
Either one of these problems would be bad enough on its own, but the concurrence of the two is what is troubling about Google’s AI. If this service is able to treat minor iterations of obscure information with the same level of reverence as top-of-the-search-results data, it still has a long way to go before it can be considered anything other than a supplementary resource–something that Google seems to understand, but should probably clarify in more certain terms.
It is worth noting that the AI result I mentioned earlier has been corrected in the past few weeks, and the X account that posted the phrase for which I searched no longer appears in the generative response (nor did the tweet in question seem to receive an undue amount of attention). This whole situation, then, is a gentle but firm reminder about the true place of AI in research: a helpful tool, but one that cannot stand on its own.
