Open books exams are bounded by time, something an AI has plenty of, ridiculously more than a person, to say it’s giving them an advantage is missing the absurdity of the situation as it also had much more time to train and “study”, and yet it produces the results it does.
But if ChatGPT is able to pass med school it must mean AI is good?
I could pass that too if I had all the books and can sift through them in seconds.
Have you ever had an open book exam?
Open books exams are bounded by time, something an AI has plenty of, ridiculously more than a person, to say it’s giving them an advantage is missing the absurdity of the situation as it also had much more time to train and “study”, and yet it produces the results it does.
What results? We are assuming it passes med school, so the result must be good.
The results people get from prompting current models, which hallucinate, a lot, and that’s not really a fixable issue
So then ChatGPT wouldn’t pass the exam, nothing to worry about.