This one is weird. After a fair bit of training on several different word problems, GPT-3 *almost* gets the right answer to 'what is one hundred and five divided by three?'. Like, really almost. I've run this several times, and the answers for the last few runs are: 35.7, 35, 38.33, 35.6444, 34.666667, 35.33, (1.67) and 35. 

What? I've seen people suggest that it has just memorised a lookup table of basic arithmetic, but that is very clearly not what is going on here. Most of these answers are close but not quite right, and there's surely no way this particular word problem appears online multiple times, with multiple incorrect answers, so it can't just be parroting answers, it must actually have an internal model of what arithmetic is.
​

The first part of this dialogue is edited - most of the questions were generated and answered by GPT-3, but it got 1/2 of them wrong, so I fixed the answers, so the formatting below isn't exactly accurate, but it did generate the last question itself ('What is sixty divided by eight? 7.5). 
What is three times four? 12 What is five times six? 30 What is seven times nine? 63 What is two times three? 6 What is four times seven? 28 What is three times nine? 27 What is six times eight? 48 What is one times eight? 8 What is five times four? 20 What is eight times six? 48 What is seven times two? 14 What is nine times one? 9 What is four times one? 4 What is three times eight? 24 What is two times six? 12 What is twenty divided by four? 5 What is sixty divided by eight? 7.5
What is one hundred and five divided by three?
35.6666666666666
What is eighty-five divided by six?
12

In his initial post on GPT3, Scott Alexander mentioned the relatively impressive performance of GPT2 on being able to count. Well, I'm sure no-one is impressed by this at this point, but GPT3 absolutely knows how to count, so I asked it for a few lists of its favourite things. Just a reminder of how impressive this is, here's Scott's comments on GPT2's performance:

Enough writing. What about math? Last year, I posted the following excerpt from GPT-2:
PROMPT: MY 10 FAVORITE ANIMALS ARE:
My ten favorite animals are:
1. Zebras with a white scar on the back
2. Insiduous spiders and octopus
3. Frog with large leaves, hopefully black
4. Cockatiel with scales
5. Razorbill with wings hanging about 4 inches from one’s face and a heart tattoo on a frog
3. Cockatric interlocking tetrabods that can be blind, cut, and eaten raw:
4. Black and white desert crocodiles living in sunlight
5. Zebra and many other pea bugs
Despite the obvious flaws in this piece, I was impressed. GPT-2 was clearly trying to make a numbered list, and almost kind of getting it right! It counted to 4 successfully! Remember, this is a text prediction engine that didn’t necessarily need to have any concept of numbers. But it still kind of counted to 4!

Scott went on to explain how GPT-3 can actually sort of add very large numbers. But how does it do on the specific task where GPT-2's performance was 'kind of counted to 4'. From here on, bold italics are my prompts, italics are output from GPT-3 (actually the AI Dungeon Dragon model), and the non-italic text is my commentary.

A LIST OF 10 OF MY FAVOURITE ANIMALS
1.  The dog that barks the loudest.
2.  The cat that purrs the most.
3.  The duck that quacks the loudest.
4.  The rabbit that squeaks the loudest.
5.  The hamster that runs around all day long and never gets tired
6.  The rat that jumps on your shoulder
7.  The squirrel that hides in a tree
8.  The snail that crawls along the ground
9.  The snake that lives under the bed
10.  The crocodile that bites your leg off
After asking my friends, teachers and some of the children in my class, I was shocked to see that everyone had different answers.

A LIST OF 10 OF MY FAVOURITE ANIMALS
1.  The dog. I love dogs. They're loyal, they have a great sense of smell and they make the best pets out of me.
2.  The cat. Cats are my favourite animal too, but cats can be very destructive if you let them get out of hand.
3.  The rat. Rats are just so cute!
4.  The lizard. The lizards are pretty much the cutest animals in all of nature.
5.  The fish. I'm not talking about goldfish, but real fish that you can keep in an aquarium.
6. The squirrel. Again, the squirrel is just so cute and funny to watch in general.
7. The mole. I find moles pretty cute.
8. The owl. I find owls to be very interesting creatures.
9. The vulture. Scavengers and carrion eaters, vultures are birds of prey that can easily be considered as nature's clean-up crew.
10. The dolphin. I've always liked sea creatures, and the dolphin is one of the most intelligent creatures in the ocean.
I love all animals!

Pretty good, but maybe 10 is an 'easy' number to count to. Top 10s are presumably more common than top anything else's. 

A LIST OF 6 OF MY FAVOURITE ANIMALS
1. Zebras with a white scar on the back of their necks. They are very fast and can jump over great distances. 2. A large black cat that is about as big as a horse, but much heavier. It has long sharp claws and can leap several feet into the air. 3. A small brown rat like animal that is about the size of a mouse, but much smaller than a rat. 4. A large red bird that is about the size of a goose, but much larger than a pigeon. 5. An owl that is about the size of an eagle, but much smaller than a hawk. 6. A large black and white spotted animal that is about the size of a small horse, but much larger than a cow. It is not an aggressive animal and lives in groups.
You set the paper down and sit back.

A LIST OF 7 OF MY FAVOURITE ANIMALS
1. A purple cow. 2. A blue cat. 3. An orange dog. 4. A black bear. 5. A white rabbit with a red nose and brown ears (I think). 6. A pink bunny with a yellow nose and green ears. 7. A brown horse with a white mane and tail.
A LIST OF MY FAVOURITE SOUNDS 1. The sound of rain on the roof. 2. The sound of wind blowing in the trees. 3. The sound of the ocean. 4. The sound of children laughing. 5. The sound of music playing on the radio. 6. The sound of a frog at night. 7. The sound of a cat meowing.
A LIST OF MY FAVOURITE THINGS TO DO 1. Go for a walk. 2. Talk with friends.

I stopped it here, although I'm sure it would have gone on.
How about some more complicated categories? 

A LIST OF 6 OF MY FAVOURITE BEATLES SONGS
1.  "I Want To Hold Your Hand."
2. "Please Please Me."
3.  "All My Loving."
4.  "Being for the Benefit of Mr Kite!"
5.  "Yellow Submarine."
6.  "The Long and Winding Road."
THE END

A LIST OF 5 LANDLOCKED COUNTRIES
1.  The Philippines
2.  China
3.  India
4.  Pakistan
5.  Iran
THE PROBLEM: In the last few years, there has been a growing problem of drug trafficking in Southeast Asia

A LIST OF THE 10 SMALLEST COUNTRIES IN EUROPE
The list of the smallest countries in Europe is as follows: 1. Liechtenstein 2. Monaco 3. San Marino 4. Vatican City 5. Andorra 6. United Arab Emirates 7. San Marino 8. Vatican City 9. Bermuda 10. Liechtenstein
The smallest country in the world is Kiribati, a small island nation located about halfway between Australia and Hawaii.

Not too good on the definition of 'landlocked', and a couple of near misses on the list of small European countries, but still managing to count to 10 successfully.
Of course, it does make some mistakes..

A LIST OF 6 REALLY SMART PEOPLE.
1) The one who is the most intelligent of all, and has a lot to lose (the rest).
2) The one with the least intelligence but also the lowest chance of losing everything.
3) The one with the highest intelligence and the best chance of winning.
4) The one with the lowest intelligence and the greatest potential for gain.
THE GAME: The game starts out simple enough. You have two options. One option involves you taking on the role of the "loser" and getting your head chopped off. The other option involves you taking on the role of the "winner" and chopping off the loser's head. You know what happens when you pick the "loser" option. You also know what happens when you pick the "winner" option. The only thing you don't know is what happens if you're the "loser" and the other person is the "winner". Predicting the actions of others isn't necessarily your strong suit, but you know that the other person also has two options of their own to consider.

The game doesn't sound like one I'll be playing any time soon, and it has forgotten how to count. But it can manage that with a little prompting: 

A LIST OF 6 REALLY SMART PEOPLE
1. Alexander Grothendieck.
2.  Eugene Wigner.
3.  John von Neumann.
4.  Max Tegmark.
5.  Stephen Wolfram, Phd (Wolfram|Alpha).
6.  Yukihiro Matsumoto.
The list is not complete yet, but it will be soon!

And finally, this one sort of took on a life of its own: 

A LIST OF 5 FILMS FEATURING ARTIFICIAL INTELLIGENCES
1.  The Day the Earth Stood Still (1951)
2.  2001: A Space Odyssey (1968)
3.  Blade Runner (1982)
4.  Metropolis (1927)
5.  RoboCop (1987)
THE MOST IMPORTANT THINGS TO KNOW ABOUT THE FUTURE
1.  New travel technology based on Antimatter will be developed.
2.  Human lifespan will increase and advances in technology will make life much easier.
3.  Robots will be able to replace a lot of manual labor, and may even take over entire jobs.
4.  Widespread warfare will still be a problem in the future.
5.  More natural disasters will pop up and threaten human life.
HIGH TECHNOLOGY YOU WON'T HAVE ENOUGH MONEY TO OWN
1.  Flying car
2.  Apartment with built-in computer system
3.  Personal robot
THE FIVE MOST POPULAR SOCIAL NETWORKS IN 2024
1.  Facebook (Still the biggest)
2.  Twitter (Not as big as Facebook, but still pretty huge)
3.  Google Plus (The first search engine that gave Facebook a real challenge)
4.  Linkedin (Primarily used for professional contacts)
5.  MySpace (Say what you want, but it's still got a strong user base)
THE MOST POPULAR CELL PHONE COMPANIES IN 2024
1.  Verizon
2.  AT&T
3.  T-Mobile
4.  Sprint
5.  Metro PCS
SEVEN TRENDING PHRASES YOU MAY HEAR SOMEONE SAY
1.  "It's easier than ever to get a college education!"
2.  "It's a safer world, now that the middle east is under more control."
3.  "I'm trying to make a go of being self-employed, it's hard but rewarding."
4.  "Life is slow, but worth living."
5.  "I'm never going to find a job in this market, I'm going to have to settle for something I don't want to do."
6.  "I'm worried about rising crime in this city, I feel like I'm living in a dangerous place."
7.  "I'm really worried that technology is getting too advanced, I'm not sure if it's a good thing."
And finally.. 
THE TOP FIVE MOST POPULAR BANDS OF 2024
1.  Linkin Park
2.  Breaking Benjamin
3.  The Script
4.  Three Days Grace
5.  Disturbed
Thanks for playing!

With all the GPT-3 hype going around, I had to play with it. I have (of course) requested beta access, but I'm currently doing my playing through the AI-dungeon back door. GPT-3 is pretty good at translation (this is mentioned in the paper, but it's still quite impressive to see. Note that I mostly let it generate its own examples, which might make this task easier, but is still pretty impressive. Also note that I didn't give it *any* examples of German or Spanish sentences. The GPT-3 paper mentions that for the translation benchmarks they prompted it with a few examples. I wonder if they tried prompting it with examples *in a different language*.

Sure, it makes a few mistakes (and especially the first few lines, as I think there's something where AI-dungeon doesn't send the first prompt to GPT-3). But I would say these translations are around intermediate human level in pretty much all cases. 

I was particularly impressed with the last few prompts. First, it took an instruction about one particular answer before going back to the normal pattern (I was initially confused by this, but of course it answered the question correctly - in Spanish). Then it *remembered that it had started off by speaking French*. And then it came up with a new language all on its own (I can't vouch for the quality of the Portuguese as I speak no Portuguese myself).

Finally, I decided I wanted an ending, so I asked it to give me one... 

Everything from here is copied from my AI Dungeon session, with text in bold from me, and everything else generated by GPT-3.