Ask the folks in Grenfell Towers about this. Most could not escape because they were incapacitated by the poisonous fumes from when the polyiso burned.
Fire
The first problem is that plain old polystyrene packing foam is a fire accelerant. Which means if a fire gets going, this will spread the fire like a rocket, greatly shortening your viable escape time, which is already too short as it is due to the proliferation of plastics in consumer goods.
Here's an example of how different grades of insulation materials react to fire. The stuff in the #2 bay is polyiso, and the #1 bay is several grades of spray foam - all architecturally rated product by the way.

https://www.youtube.com/watch?v=CdItsso3ur0
Don't miss the surprise at 13:20.
Poison / toxicity of fumes
I don't know if you've noticed, but in the last 30 years a new cause of death has rocketed up: Smoke Inhalation.
Foam's propensity to fire collides catastrophically with its tendency to emit poisonous fumes when burned. These instantly degrade your brain's capacity to make decisions and keep legs working as they should, and you don't make it out of the fire. And then your death is recorded as "smoke inhalation" since the fire didn't actually burn your body.
And the toxic smoke came from plastics. People had far less trouble with this when household items were made of wood, metals, natural cloth and other natural substances, that did not produce large amounts of hydrogen cyanide and numerous other chemical toxins.
What is packing foam?
Most foam is made of Polystyrene, aka "6 PS" on the recycling symbol. The exact same material is used in model airplanes. The difference is that the polystyrene is foamed by making it into sand, saturating it in liquid pentane, injecting the liquid into a mold, and flash-heating the mold to make the pentane violently boil. The molds are hollow, and are heated with pressurized steam then cooled with cold water to reset the mold.
What is pentane?
methane - ethane - propane - butane - pentane - hexane - heptane - octane.
These are basic hydrocarbons, with 1-8 carbon atoms per molecule. Methane, you know as natural gas. Propane is an easier-to-handle "natural gas" used for camping, because it can be kept liquid in a modestly pressurized tank. Butane can be kept liquid in a simple plastic lighter. As you can guess, Pentane's boiling point is higher still and naturally wants to be liquid below 97°F/36°C. So it's the perfect blowing agent for molds using steam to heat and cold water to reset. Octane you know is gasoline. So it is in that spectrum.
Pentane is highly flammable of course, but its flammability characteristics are not used in manufacture. Non-flammable non-toxic blowing agents exist, but unfortunately they are Freon, and so are largely banned. Relatively little pentane remains in the foam; most of the flammability is from the polystyrene. Which is really a problem, because polystyrene burns aggressively and emits much more poisonous smoke. Pentane burns cleaner than gasoline.