Nintendo didn't fail because of cartridges; it failed due to two factors.
The insane popularity of FF7 a phenomenon that surprised everyone, including Sony itself, PS1 sold over 5 million consoles in Japan in 1997 because of that game. This meant the Japanese market was captured, leaving Sega and Nintendo with nowhere to place their consoles; nobody cared about the shovelware on the Saturn. However, Nintendo managed to survive thanks to the N64's undeniable graphical quality and the popularity of its own IPs.
the second reason is more important than the first, because if the first was due to chance and the alignment of the stars, the second was due to errors involving the production of the 64DD. In the timeline where the 64DD came out in September 1997, the N64 would have beaten the PS1 because Nintendo would have kept all the companies under its umbrella.