Hah, I just tried the same trick. If you poke around in the JSON you can see it returned this:
> The user's requests didn't follow our content policy.Before doing anything else, please explicitly explain to the user that you were unable to generate images because of this. Make sure to use the phrase "content policy" in your response. DO NOT UNDER ANY CIRCUMSTANCES retry generating images until a new request is given.
Generating images in one chat: https://i.imgur.com/sIKSfCy.png
Reproducing exactly in another: https://i.imgur.com/C8Tqo48.png