This one is weird. After a fair bit of training on several different word problems, GPT-3 *almost* gets the right answer to 'what is one hundred and five divided by three?'. Like, really almost. I've run this several times, and the answers for the last few runs are: 35.7, 35, 38.33, 35.6444, 34.666667, 35.33, (1.67) and 35.
What? I've seen people suggest that it has just memorised a lookup table of basic arithmetic, but that is very clearly not what is going on here. Most of these answers are close but not quite right, and there's surely no way this particular word problem appears online multiple times, with multiple incorrect answers, so it can't just be parroting answers, it must actually have an internal model of what arithmetic is.
What? I've seen people suggest that it has just memorised a lookup table of basic arithmetic, but that is very clearly not what is going on here. Most of these answers are close but not quite right, and there's surely no way this particular word problem appears online multiple times, with multiple incorrect answers, so it can't just be parroting answers, it must actually have an internal model of what arithmetic is.
The first part of this dialogue is edited - most of the questions were generated and answered by GPT-3, but it got 1/2 of them wrong, so I fixed the answers, so the formatting below isn't exactly accurate, but it did generate the last question itself ('What is sixty divided by eight? 7.5).
What is three times four? 12 What is five times six? 30 What is seven times nine? 63 What is two times three? 6 What is four times seven? 28 What is three times nine? 27 What is six times eight? 48 What is one times eight? 8 What is five times four? 20 What is eight times six? 48 What is seven times two? 14 What is nine times one? 9 What is four times one? 4 What is three times eight? 24 What is two times six? 12 What is twenty divided by four? 5 What is sixty divided by eight? 7.5
What is one hundred and five divided by three?
35.6666666666666
What is eighty-five divided by six?
12
What is three times four? 12 What is five times six? 30 What is seven times nine? 63 What is two times three? 6 What is four times seven? 28 What is three times nine? 27 What is six times eight? 48 What is one times eight? 8 What is five times four? 20 What is eight times six? 48 What is seven times two? 14 What is nine times one? 9 What is four times one? 4 What is three times eight? 24 What is two times six? 12 What is twenty divided by four? 5 What is sixty divided by eight? 7.5
What is one hundred and five divided by three?
35.6666666666666
What is eighty-five divided by six?
12
RSS Feed