Testing AIs With Two Questions Related To Current Events

I’ve been testing artificial intelligence (AI) tools for more than a year. The latest is a new search engine that integrates AI. It is Perplexity.ai. This search engine wants to give Google a run for its money and is backed by early investors that include Jeff Bezos, SoftBank and Nvidia. They tout Perplexity.ai as the poster child for AI search that delivers conversational results.

Since OpenAI introduced ChatGPT I’ve studied Large Language Models (LLMs) and tested their evolving capabilities. ChatGPT created Monica as an enhancement to search engines. Others have produced competitors: Gemini (Google’s AI), Claude (Anthropic) and Grok (xAI, Elon Musk’s contribution).

When I read about two women in Brazil today dying from the Oropouche virus, one I had never heard of, and that it was mosquito-borne and spreading beyond the Amazon Basin to other areas of South America, I became somewhat concerned.

Having just read about Perplexity.ai as an AI search assistant, I decided to test it on a general question about pandemic threats. I asked:

“What viruses represent future pandemic threats?” 

This is what Perplexity wrote:

Based on the search results, several viruses represent potential future pandemic threats:

1. Novel coronaviruses: Similar to SARS-CoV-2 that caused COVID-19, new coronaviruses could emerge and cause future pandemics.
2. Influenza viruses: New strains of influenza, particularly avian influenza like H7N9, pose a significant pandemic risk.
3. Ebola virus: This has caused outbreaks and is considered a priority pathogen by the WHO.
4. Marburg virus: Another hemorrhagic fever virus that is on the WHO’s priority list.
5. Lassa fever virus: Included in WHO’s list of priority pathogens.
6. Nipah virus: Considered a threat for future outbreaks.
7. Zika virus: Has caused significant outbreaks and is on WHO’s watchlist.
8. MERS (Middle East Respiratory Syndrome) coronavirus.
9. Other zoonotic viruses: Many potential pandemic threats are zoonotic, originating in wildlife before spilling over to humans.
10. “Disease X”: This represents an unknown pathogen that could cause a serious international epidemic or pandemic.

The World Health Organization (WHO) maintains a list of priority pathogens for research and development efforts, which includes these and other potential threats. It’s important to note that while these viruses are known concerns, the next pandemic could also be caused by a currently unknown pathogen. This is why the concept of “Disease X” is included in pandemic preparedness efforts. Factors contributing to increased pandemic risk include environmental changes, human encroachment on natural habitats, and increased global travel and trade. To prepare for future pandemics, ongoing surveillance, research into these viral families, and development of vaccines and treatments are crucial.

This is a pretty comprehensive response and expressed well. What I didn’t understand was since Perplexity.ai was a search assistant, why wouldn’t it mention Oropouche when a Google search did?

I thought a fair test would be to pose the same question to other AIs. I asked ChatGPT’s Monica and it came back with two additions to the Perplexity.ai list. The one I couldn’t understand Perplexity.ai missing was Dengue, described by Monica as “A mosquito-borne virus that can cause severe illness and has been spreading to new regions due to climate change.” The other was unfamiliar to me called Hendra and was described as “similar to Nipah, it is also transmitted from bats and can infect horses and humans, leading to severe respiratory and neurological symptoms.”

I then asked Google’s Gemini. Gemini’s response grouped viruses into two categories: zoonotic and RNA-based. It also included “Disease X” as an unknown. Then it added a category neither of the previous two had mentioned: the growing threat of AMR (anti-microbial resistance) as a pandemic threat.

Gemini wasn’t finished in its response. Even though I hadn’t asked, Gemini added information about root causes including:

• Climate change: Altering ecosystems, increasing contact between humans and animals, and creating new environments for pathogens to thrive.
• Globalization: Rapid travel and trade facilitate the spread of diseases in a borderless world.
• Antimicrobial overuse: Contributing to the rise of AMR.  
• Deforestation and habitat destruction: Bringing humans into closer contact with wildlife and their pathogens.

In fairness to Perplexity and Monica, both included suggested follow-up questions to provide root-cause answers.

I went through the same process with Claude. Claude had two more to add to the list: Filoviruses, a group of viruses similar to Ebola, and Henipaviruses, a group of bat-borne viruses related to Nipah. Claude then asked me if I wanted it “to elaborate on any of these viruses or discuss pandemic preparedness strategies?”

My final pursuit in this exercise was to try and ask Grok the same question But with Grok I couldn’t because I needed to be an X Premium+ subscriber in the United States to use it.

Getting back to Perplexity.ai, it wasn’t alone in not mentioning Oropouche. But as a true search AI assistant, I couldn’t understand why it didn’t get mentioned.

I knew why the others had missed Oropouche. That’s because Monica, Gemini and Claude are AIs built with Large Language Model (LLM) algorithms that train with datasets of materials from the Internet that are often several months old.

To prove the difference, I went through a final exercise involving current events at the Paris Olympics. I asked Gemini and Claude who won medals today on August 5, 2024, and here are the two answers respectively:

Gemini: “There seems to be a slight confusion. The Paris Olympics are scheduled for July 26 to August 11, 2024. As of now, we are in the year 2023.” Whoops!!

Claude: “I apologize, but I don’t have access to real-time information about Olympic events…The most recent Olympic Games I have detailed information about were the 2022 Winter Olympics in Beijing. Sorry!

Ask Perplexity.ai the same question and here is its answer:

“Here are the key medal winners from the Paris Olympics on August 5, 2024:

• Gymnastics: Simone Biles won a silver medal in the floor exercise final, concluding her participation in the 2024 Paris Olympics without adding to her seven Olympic golds.
• Badminton: Lee Zii Jia secured Malaysia’s second medal at the Paris Games.
• Track and Field: Noah Lyles won gold in the men’s 100-meter sprint with a time of 9.784 seconds, narrowly beating Jamaica’s Kishane Thompson, who took silver.”

As the old saying goes, “One of these is not like the others” and it is Perplexity.ai. It knows what’s happening in the present. Not mentioning the Oropouche virus was the only surprise to me. Maybe the explanation is that Oropouche wasn’t seen as a pandemic threat as of yet.

Ignoring the Oropouche omission, it is pretty clear to me that as AIs go, the current frontrunners whose LLMs train on datasets from the past will soon be obsolete. If I were them, in the light of Perplexity.ai, I would be panicking.

Let’s hope Oropouche is right in its omission and that the virus doesn’t join other vector-borne diseases to spread across the planet.