The real problem with chatbots

Forget the speculation about whether chatbots can be sentient.  Assume they are what they are cracked up to be.

There are serious problems both with the basic technology and with how the technology can be used.

A search engine will take you to links on your chosen topic.

 It may be programmed to manipulate you by choosing certain links and hiding or downplaying others. This goes on all the time.

But at least you can read the links and judge for yourself.

A chatbot is an oracle.  It will give you a plausible answer in plain English (or whatever your language is).  It may be based on true information, false information or something complete imaginary.

One physician, cited by the blogger Lambert Strether, tested OpenAI’s ChatGPT to make a medical diagnosis.  It gave one that sounded plausible.  The physician asked for a link to a clinical study that would back up its claim.  ChatGPT generated a link to a study – which didn’t exist.

If something like ChatGPT is routinely used as a source of information, how many people would bother to check the correctness?

A chatbot can be programmed to generate false or misleading propaganda.   It can be programmed to follow certain rules of accuracy.  It can’t be programmed to tell the truth because, being non-sentient, it has no concept of the truth.

Artificial intelligence has many existing and potential benefits.  Whether they’re realized depends on whether is in the hands of people who are accountable to the public.  But let’s not assume that artificial intelligence is a substitute for human intelligence.  

LINKS

AI = BS by Lambert Strether for Naked Capitalism.

Google’s chatbot panic by Cory Doctorow for Pluralistic.

Tags: Chatbots, Google, OpenAI