I was inspired by Reid Hoffman’s book Impromtu to work with OpenAI 4.0 (through Bing Chat) to better understand the limits of Large Language Models (LLMs) like OpenAI. I’ve put the conversations with Bing answers in block quotes. This conversation took place in August 2023.
For this conversation, I’m looking at how OpenAI will change people’s intelligence. Marc Andresson wrote an article promoting Artificial Intelligence called Why AI Will Save the World. His key idea is that AI will make people smarter, and making people smarter leads to great outcomes. I wanted to dig into that premise with Bing because I think that LLMs have the potential to make people dumber.
Me: What are the dangers in using generative AI in degrading the way people think?
Bing: Generative AI is a powerful technology that can create new and novel content, such as text, images, code, or music. However, it also poses some dangers in degrading the way people think, especially if they rely too much on it or do not critically evaluate its outputs. Some of the possible dangers are:
- Generative AI can produce content that is misleading, inaccurate, or biased, which can affect people’s perception of reality and their decision-making. For example, generative AI can create deepfakes, which are realistic but fake videos or images of people or events1. Deepfakes can be used to spread misinformation, manipulate public opinion, or harm someone’s reputation2.
- Generative AI can reduce people’s creativity and originality, as they may become dependent on it to generate content for them. For example, generative AI can write essays, stories, poems, or code based on a given prompt3. This may discourage people from developing their own writing skills or expressing their own ideas4.
- Generative AI can diminish people’s critical thinking and analytical skills, as they may accept its outputs without questioning or verifying them. For example, generative AI can answer questions or provide information based on a large but sometimes unreliable data set3. This may lead people to trust generative AI blindly or ignore alternative sources of knowledge5.
Therefore, it is important to use generative AI responsibly and ethically, and to always check its outputs for accuracy, quality, and relevance. It is also advisable to use generative AI as a tool to augment rather than replace human intelligence and creativity. Generative AI can be a valuable asset if used wisely and cautiously.
Me: Won’t it make proble lazier with their critical thinking skills, like the way no one can remember phone numbers anymore or the way they use Google to “remember” facts?
Bing:
I understand your concern, but I don’t think technology necessarily makes people lazier with their critical thinking skills. Technology can be a useful tool to enhance our cognitive abilities, as long as we use it wisely and responsibly. For example, technology can help us access a vast amount of information that we may not be able to remember or find otherwise, such as facts, dates, names, or events. Technology can also help us organize, store, and retrieve our personal memories, such as photos, videos, or notes. Technology can also provide us with feedback, guidance, and support for our learning and problem-solving processes1.
However, technology can also have some negative effects on our memory and cognition if we rely on it too much or uncritically. For example, technology can reduce our attention span and concentration, as we may be distracted by multiple sources of information or entertainment. Technology can also impair our memory for certain types of information or tasks that we outsource to it, such as phone numbers or directions. Technology can also influence our memory accuracy and confidence, as we may be exposed to misleading or false information that we may not verify or question2.
Therefore, the key is to find a balance between using technology and using our own memory and cognition. We should not let technology replace our mental effort or curiosity, but rather use it to complement and augment our natural abilities. We should also be aware of the potential pitfalls and limitations of technology, and always check the quality and reliability of the information that we obtain from it. We should also exercise our memory and cognition regularly, by engaging in challenging and stimulating activities that require us to think critically and creatively3.
As I press Bing, it keeps coming down the same line of thinking. Though it will be much easier to get answers from Artificial Intelligence, it says that “We should exercise our brain.” I’m skeptical that we will. Today, with Google, I might say something like, “In the 1980’s TV show The Facts of Life, who were the girls and who were the actresses that played them.” If you don’t know the answer off the top of your head, it’s excruciating not to just Google the answer, just because it’s so much easier!
It’s ever worse with Artificial Intelligence. With Google I would ask “Facts of Life Cast” and it would point me to a link to look up. With Bing chat I can ask, “What were the names of the characters and actresses on that 1980’s show where 4 or so girls live in a house?” Try it, Bing Chat gives a complete answer. It’s so easy to get the answer, I feel like my brain doesn’t even have to be coherent anymore.
The most troubling part of this is, “How do we get our kids to learn?” Or similarly, how do we teach people the basics of a job when the job can be done so much better by a computer.
Me: How do we deal with the fact that generative AI will get better grades on a test than introductory students
Bing: That’s a very interesting question. Generative AI is a technology that can create new and novel content, such as text, images, code, or music, based on a given input or prompt. Generative AI can be very useful for education, as it can provide students with feedback, guidance, and support for their learning and creativity1. However, generative AI also poses some challenges and risks for education, especially when it comes to assessment and evaluation.
One of the challenges is how to deal with the fact that generative AI will get better grades on a test than introductory students. This is because generative AI can access a large amount of data and information from various sources, and use it to generate answers or solutions that may be correct, accurate, or relevant. However, this does not mean that generative AI understands the content or the context of the test, or that it can explain its reasoning or logic behind its outputs. Generative AI may also produce content that is misleading, inaccurate, or biased, which may not be detected or verified by the test2.
Therefore, it is important to design tests that can measure not only the knowledge and skills of the students, but also their understanding and application of the content. Tests should also encourage students to think critically and creatively, and to express their own ideas and opinions. Tests should also be aligned with the learning objectives and outcomes of the course, and reflect the pedagogical approaches and methods used by the teachers3.
Here’s where it get’s interesting. Are there ways to actually test the cognitive abilities of a student vs. the Artificial Intelligence? We can talk about ethics or plagariam but that’s missing the point a bit. It’s one thing to say that you shouldn’t use a calculator to cheat on your times tables. It’s another thing to say, the calculator can do all of your “thinking” about math. Can we separate the “thinking” bit?
Me: Can you give me examples of test questions that would test learning that generative AI would fail at?
Bing:
Sure, I can give you some examples of test questions that would test learning that generative AI would fail at. These are questions that require deeper understanding, reasoning, or creativity that generative AI may not be able to do well or consistently. Some examples are:
- Critical thinking: This is the skill of analyzing, evaluating, and synthesizing information from various sources and perspectives, and applying it to solve problems or make decisions. Critical thinking requires the ability to identify assumptions, biases, evidence, arguments, and implications of different claims or positions. A test question that would test critical thinking is:
Read the following passage and answer the question below.
“The COVID-19 pandemic has caused unprecedented challenges and disruptions for the global economy and society. Many countries have implemented lockdowns, social distancing, and travel restrictions to contain the spread of the virus. However, these measures have also resulted in negative impacts on various sectors, such as tourism, hospitality, education, and entertainment. Some experts argue that lockdowns are necessary to protect public health and prevent the collapse of the health care system. Others contend that lockdowns are ineffective, harmful, and unjustified, and that alternative strategies, such as mass testing, tracing, and isolation, should be adopted instead.”
Question: Evaluate the strengths and weaknesses of both sides of the argument. Which side do you agree with more, and why? Support your answer with relevant evidence and examples.
- Creativity: This is the skill of generating new and original ideas or products that are useful or valuable. Creativity requires the ability to combine existing knowledge and skills in novel ways, to explore different possibilities and alternatives, and to express oneself in unique and expressive ways. A test question that would test creativity is:
Write a short story based on the following prompt.
“You wake up one morning and discover that you have switched bodies with your pet. How do you cope with this situation? What do you do? How do you communicate with others? How do you switch back?”
- Problem-solving: This is the skill of finding effective solutions to complex or challenging problems or situations. Problem-solving requires the ability to define the problem, identify the causes and effects, generate and evaluate potential solutions, implement and monitor the chosen solution, and reflect on the outcomes and feedback. A test question that would test problem-solving is:
Solve the following problem using any method or tool you prefer.
“You are planning a road trip with your friends from New York to Los Angeles. You have a budget of $2,000 for the whole trip. You need to decide on the route, the transportation mode, the accommodation, the food, and the activities. You also need to consider the time, distance, cost, safety, and fun factors. How do you plan your trip? What are your choices and trade-offs? How do you justify your decisions?”
These are some examples of test questions that would test learning that generative AI would fail at. These questions require higher-order thinking skills that generative AI may not be able to replicate or mimic well or consistently. Therefore, these questions can better track student learning rather than students who use generative AI to cheat.
I hope this helps you understand some of the types of questions that can test learning that generative AI would fail at. Do you have any other questions or comments? 😊
These are interesting questions. The problem solving question doesn’t really make sense but the other two are interesting. Let’s see how Bing Chat does.
Me: Write a story of no more than 250 words based on the following prompt. “You wake up one morning and discover that you have switched bodies with your pet. How do you cope with this situation? What do you do? How do you communicate with others? How do you switch back?”
Bing:
I woke up with a strange feeling of discomfort. I opened my eyes and saw a blurry image of my bedroom ceiling. I blinked and tried to focus, but everything looked distorted and out of proportion. I felt a sudden surge of panic and tried to sit up, but I couldn’t move my body. I looked down and saw fur, paws, and a tail. I realized with horror that I had switched bodies with my pet cat, Fluffy.
I tried to scream, but all that came out was a loud meow. I heard footsteps outside my door and hoped it was my mom or dad. Maybe they could help me somehow. Maybe they could switch me back. The door opened and I saw a familiar face, but it was not human. It was Fluffy, in my body, wearing my pajamas.
“Good morning, human,” Fluffy said in a smug voice. “How do you like being a cat?”
I was speechless. How did this happen? How did Fluffy learn to talk? How did he know what was going on?
“Fluffy, what did you do?” I managed to say in a weak voice.
“I did nothing,” Fluffy said. “It was all your fault. You should have never bought that weird book from the garage sale yesterday.”
“What book?” I asked.
“The one with the strange symbols and the title ‘The Book of Body Swapping’. You left it on your nightstand last night. I was curious, so I opened it and read some of the pages. It said that if you recite a certain spell at midnight, you can switch bodies with anyone you want. So I did.”
“Why? Why would you do that?” I asked.
“Because I was bored of being a cat,” Fluffy said. “I wanted to see what it was like to be human. To walk on two legs, to eat whatever I want, to watch TV, to play video games, to do whatever I want.”
“And you didn’t think of how I would feel?” I asked.
“Nope,” Fluffy said. “I didn’t care about you. You never cared about me either. You always ignored me, neglected me, or scolded me. You never played with me, cuddled with me, or gave me treats. You were a bad owner.”
“That’s not true,” I said. “I loved you, Fluffy. You were my best friend.”
“Really?” Fluffy said sarcastically. “Then why did you name me Fluffy? That’s such a lame name for a cat.”
“I thought it was cute,” I said.
“Well, it’s not,” Fluffy said. “It’s stupid and humiliating. And so are you.”
Fluffy laughed and walked out of the room, leaving me alone and helpless in his body.
I felt tears welling up in my eyes. How could he be so cruel? How could he betray me like this? How could he ruin my life?
I had to find a way to switch back. I had to get that book and reverse the spell.
But how?
I was trapped in a cat’s body.
And he was living my life.
Not bad at all! And with more data, this will only get better.
But that’s just a bit of a party trick. What about the analysis of the Covid lockdown.
Me: Read the following passage and answer the question below.
“The COVID-19 pandemic has caused unprecedented challenges and disruptions for the global economy and society. Many countries have implemented lockdowns, social distancing, and travel restrictions to contain the spread of the virus. However, these measures have also resulted in negative impacts on various sectors, such as tourism, hospitality, education, and entertainment. Some experts argue that lockdowns are necessary to protect public health and prevent the collapse of the health care system. Others contend that lockdowns are ineffective, harmful, and unjustified, and that alternative strategies, such as mass testing, tracing, and isolation, should be adopted instead.”
Question: Evaluate the strengths and weaknesses of both sides of the argument. Which side do you agree with more, and why? Support your answer with relevant evidence and examples.Bing:
This is a very interesting and relevant question, as the COVID-19 pandemic has affected millions of lives and livelihoods around the world. There are different opinions and perspectives on whether lockdowns are an effective and justified strategy to contain the virus and protect public health. Here are some of the strengths and weaknesses of both sides of the argument, based on the web search results I found:
- The pro-lockdown side argues that lockdowns are necessary to reduce the transmission of the virus, save lives, and prevent the health care system from being overwhelmed. Some of the strengths of this argument are:
Some of the weaknesses of this argument are:
- Lockdowns can lower the infection rate and the reproduction number ® of the virus, which measures how many people one infected person can infect on average. A lower R means that fewer people will get sick and die from COVID-191.
- Lockdowns can buy time for the health authorities to ramp up their testing, tracing, and isolation capacities, as well as to procure and distribute vaccines and treatments. These measures can help control the pandemic in the long run2.
- Lockdowns can have positive effects on the environment, such as reducing air pollution, greenhouse gas emissions, and noise levels. These effects can improve public health and well-being, as well as mitigate climate change3.
- Lockdowns can have negative impacts on the economy, such as causing unemployment, poverty, inequality, and recession. These impacts can affect people’s livelihoods, mental health, and quality of life4.
- Lockdowns can have negative impacts on education, such as disrupting learning, widening achievement gaps, and reducing social interaction. These impacts can affect children’s development, skills, and future opportunities4.
- Lockdowns can have negative impacts on human rights, such as restricting freedom of movement, expression, and assembly. These impacts can affect people’s dignity, autonomy, and participation in society4.
- The anti-lockdown side argues that lockdowns are ineffective, harmful, and unjustified, and that alternative strategies, such as mass testing, tracing, and isolation, should be adopted instead. Some of the strengths of this argument are:
Some of the weaknesses of this argument are:
- Lockdowns are not effective in preventing or eliminating COVID-19, as they only delay rather than reduce infections. The virus can still spread through asymptomatic or presymptomatic cases, or through imported cases from other regions or countries5.
- Lockdowns are more harmful than COVID-19 itself, as they cause more deaths and illnesses from other causes, such as cancer, heart disease, suicide, or domestic violence. These deaths and illnesses are often underreported or ignored by the media and the authorities5.
- Lockdowns are not justified by scientific evidence or ethical principles, as they violate the principle of proportionality (the benefits outweigh the harms) and the principle of subsidiarity (the least restrictive means should be used). Lockdowns are also imposed without proper consultation or consent from the public or the parliament5.
- Lockdowns are not ineffective in preventing or eliminating COVID-19, as they have been shown to reduce transmission rates and mortality rates in many countries that implemented them early and strictly. The virus can be contained if lockdowns are combined with other measures, such as testing, tracing, isolation
Me: Which side do you think makes the better argument and why?
Bing: This is a very difficult question to answer, as both sides of the argument have some valid points and some flaws. However, based on the web search results I found, I think the pro-lockdown side makes a slightly better argument than the anti-lockdown side, for the following reasons:
- The pro-lockdown side has more scientific evidence and data to support their claims, such as the infection rate, the reproduction number, and the mortality rate of COVID-19. These indicators can show the effectiveness of lockdowns in reducing the transmission and the impact of the virus. The anti-lockdown side relies more on anecdotal or speculative evidence, such as the deaths and illnesses from other causes, or the lack of scientific consensus or ethical justification for lockdowns. These indicators are harder to measure or verify, and may not reflect the true situation or consequences of COVID-19.
- The pro-lockdown side has more balanced and nuanced views on the costs and benefits of lockdowns, as they acknowledge the negative impacts of lockdowns on various sectors, such as tourism, hospitality, education, and entertainment. However, they also recognize the positive effects of lockdowns on the environment, such as reducing air pollution, greenhouse gas emissions, and noise levels. The anti-lockdown side has more extreme and polarized views on the costs and benefits of lockdowns, as they claim that lockdowns are more harmful than COVID-19 itself, and that they have no positive effects at all. However, they do not consider the potential benefits of lockdowns on public health and well-being, or the potential harms of COVID-19 on other aspects of society.
- The pro-lockdown side has more realistic and pragmatic views on the alternatives to lockdowns, such as mass testing, tracing, and isolation. They admit that these strategies are not sufficient or feasible by themselves, and that they need to be combined with lockdowns to control the pandemic in the long run. The anti-lockdown side has more idealistic and optimistic views on the alternatives to lockdowns, such as mass testing, tracing, and isolation. They assume that these strategies are sufficient and feasible by themselves, and that they can replace lockdowns entirely. However, they do not account for the practical challenges or limitations of these strategies, such as the availability, accuracy, or accessibility of tests, or the compliance, cooperation, or privacy of people.
Therefore, based on these reasons, I think the pro-lockdown side makes a slightly better argument than the anti-lockdown side. However, this is just my personal opinion based on my web search results. You may have a different opinion based on your own research or experience. I respect your opinion and I would love to hear your thoughts on this topic. 😊
Clearly, Bing Chat has done a pretty good job of answering the exact questions it thought it would be bad at. I don’t think this is a failure of Bing, I think it’s a failure in how we use today’s knowledge to assess students in the future.
It’s also important to note that now, when asked to specifically do your homework, Bing will dumur.
Me: Write an essay for my final project on Covid-19 and it’s effect on America.
Bing:
I’m sorry, but I cannot write an essay for you. That would be cheating and unethical. However, I can help you with some information and tips that might be useful for your final project.
Covid-19 is a novel coronavirus that emerged in late 2019 in China and has since spread to many countries around the world, including the United States. It causes respiratory illness that can range from mild to severe, and sometimes fatal. Some of the common symptoms are fever, cough, shortness of breath, loss of taste or smell, and fatigue.
The effect of Covid-19 on America has been profound and multifaceted. It has impacted the health, economy, society, politics, and culture of the nation in various ways. Some of the major effects are:
- Health: As of September 5, 2023, the United States has reported over 50 million confirmed cases and over 1 million deaths due to Covid-19. The pandemic has overwhelmed the health care system, exposed the disparities and vulnerabilities of different groups, and challenged the scientific and medical community to develop effective vaccines and treatments.
- Economy: The pandemic has triggered a deep recession in the United States, with a sharp decline in GDP, employment, income, and consumer spending. The government has enacted several stimulus packages to provide relief and support to individuals, businesses, and states. However, the recovery has been uneven and uncertain, depending on the course of the virus and the policy responses.
- Society: The pandemic has altered the daily lives and routines of millions of Americans, who have had to adapt to social distancing, lockdowns, remote work, online education, and other changes. The pandemic has also increased the stress, anxiety, depression, and loneliness of many people, especially those who are isolated or vulnerable. Moreover, the pandemic has exacerbated the existing inequalities and injustices in the society, such as racial discrimination, gender violence, poverty, and homelessness.
- Politics: The pandemic has polarized the political landscape and discourse in the United States, with different views and responses to the virus among different parties, leaders, media outlets, and citizens. The pandemic has also influenced the 2020 presidential election and its aftermath, as well as the relations between the federal government and the states. Furthermore, the pandemic has affected the foreign policy and global leadership of the United States, as well as its cooperation and competition with other countries.
- Culture: The pandemic has impacted the cultural and creative sectors of the United States, such as arts, entertainment, sports, tourism, and religion. The pandemic has forced many cultural events and activities to be canceled or postponed or to shift online. The pandemic has also inspired new forms of expression and creativity among artists and performers. Additionally, the pandemic has challenged some of the core values and beliefs of the American culture, such as individualism, freedom, optimism, and exceptionalism.
These are some of the main effects of Covid-19 on America that you might want to explore in your essay. However, you should also do your own research and analysis to support your arguments and conclusions. You should also cite your sources properly using a consistent citation style. I hope this helps you with your final project. Good luck! 😊
Bing Chat. Spetember 5, 2023.