This is an interesting question which everyone gets wrong all the time because they just repeat what they heard without looking into it, so everyone speaks of the same 4-6 ills without realizing those weren't causes but symptoms.
Sonic 1 arguably the issue, and I'm not talking about the game, I'm talking about how SoA handled it. Because it was from there that Sega suddenly wanted to be on war footing with competition when they were not before, I've seen more and more people bring this up and they are right on further inspection, and that grew more and more each year, the Genesis success everywhere but Japan prompted the captains to take control of the ship and compete their way(SoJ). Now all of a sudden Sega was reacting, they were thinking about strategies to attract new customers yet never had the infrastructure to do it, they wanted marketing but didn't know how to do it on a competitive scale without bleeding cash, they thought if they overinvest in product and arcades, the result would cause enough novelty and hype for people to swarm to stores or waste coins in the arcades in crowds, it didn't happen, none of this had a safety net either.
By 1994, Sega was already losing a lot of money and was about to lose more. they had no long term plan, no short-term solution to immediate problems, SoJ crippled SoA, SoE was stagnant, Sega was putting out products with niche or no target demosgraphics expecting to replicate earlier successes without realizing why those successes happened, and by end of 1996 Sega was already marked for death. Sega was gambling at that point, they had lost enough profit that they had no way of remedying, they decided to try and push out a bunch of product and software hoping that they would stick enough even if not big hits, but just enough to bring in enough revenue to keep going and eventually refocus the brand, it didn't happen.
The Dreamcast, if it was up to Sega by itself, never would have or could have released, Sega needed help for the Dreamcast to even become a reality, they had pretty much cut off any other revenue they were getting from anywhere else not long after, with no safety net and no plan once again. Hoping a lot of in your face marketing, a cheaper product, and poorly targeted software in droves + partnerships to take advantage of their early release headstart, but by Jan 2000 anybody that wasn't dumb saw the writing was on the wall, if they didn't see it earlier.
People like to say PS2 "hype" killed the DC, despite the DC picking up in sales during the PS2's launch and after, so no it had nothing to do with PS2 hype, it had to do with one major fundamental issue, and that is Sega could not find a way to get consumers to buy their software which impacts who buys the hardware. They also constantly fumbled in how they marketed some of the software, and if things weren't moving too hot, they would rotate the software marketing, which made it hard for long-term sustainability for certain titles (Hi Soul Calibur).
What you ended up with is the Dreamcast becoming more and more across it's life in NA, a NFL2k machine. Europe was of secondary interest to Sega and they had an uphill battle there, Sega didn't realize why they had short-lived initial success with the Saturn in Japan, didn't recognize why they lost it later (with N64 nearly passing it in short time with a less domestically catered library) and basically launched the DC there to die.
NA was the big gamble, and it became after launch more and more of an NFL machine, by millions(m) (k thousands) sold NPD:
DC Sega Sonic Adventure 1.05m
DC Sega NFL 2K1 1.01m
DC Sega NFL 2K 1m
DC Sega Crazy Taxi 1m
DC Sega NBA 2K1 741k
DC Sega NBA 2K 726k
DC NAMCO Soul Calibur 573k
DC Sega Shenmue 458k
DC Sega House of the Dead 2 339k
Now looking at NA, the biggest region for the Dreamcast and what it's fate ultimately relied on, I want to point out that Sonic Adventure was initially bundled highly, yet NFL ended up proving to me a more popular bundle long-term in NA after the initial window. Sonic Adventure came out at launch in 1999.
Let me delve into this issue and why it's not wise to dismiss software as the problem, because lack of discourse ability and critical thinking makes many fans default to "ur saying the games are bad' which isn't the problem, the problem is that people were only attracted to few software titles, this is even before piracy became an issue which is also an overplayed excuse, there was not interest in Dreamcast buyers to grab more software in the Dreamcasts library from the entry point. it's the exact same issue the Sega Saturn had (which no one doubts) outside of Japan, just not as bad because NFL2K was a major system seller and hit in NA, where as the Saturn in NA never really had "hits" at all.
Now, we know that on 9/9/99 was the Dreamcast launch.
Sonic Adventure was available at launch on 9/9/99, now let's look at the other best selling games not related to sports.
Crazy taxi was released in Jan 2000, not far from the 9/9/99 launch and at the end of the launch window period post-holiday season.
Soul Calibur was a launch title.
Shenmue Nov 8th 2000, the only one later on the time scale, marketed heavy and presented in mags and online as a graphics showcase, right on holiday-season, less than 500k units.
House of the Dead 2, Released as a launch title for the system.
Now, look at how most of these were near launch except Shenmue, which was a uniquely marketed case offline and online, yet didn't do that much better than House of the Dead 2, which was another game near launch.
Now yes, NFL2K was a launch game, did very very well, but 2K1 did even better, and sold more than 2K, and that game out in Sept 2000, not far from Shenmue. NBA2K1 was Oct 2000, one month form Shenmue. Both successors still retained high numbers, and saw an increase outselling their predecessors. after early 2001 sales collapsed, and later Sega announced the discontinuation, but before that there was a sales increase into early 2001.
When people think of Dreamcast what do you they thing about? Seaman, Street Fighter III 3rd strike, Blue Stinger, Phantasy Star, Power Stone 1-2. Fighting Vipers 2, Virtua Fighter 3TB, Mortal Kombat Gold, Skies of Arcadia, Sonic Adventure 2, Resident Evil: Code Veronica, Marvel vs. Capcom 2, Tony Hawk PS 2, Grandia II, Garou, Dead or Alive 2, Outrigger, Dynamite Cop, Virtua Cop 2, etc etc etc.
Now here's my question, where are all these games, and why could none of these sell 339k units or near it? Because people who were buying the Dreamcasts, were not in any, even marginally significant numbers, buying other games in the library, and over time the Dreamcast became a NFL2K/NBA2K machine, primarily the former. This was evident before piracy was much of a factor, and even though that did become worse in later 2000, in NA specifically, it still wasn't a major deal, and we saw an INCREASE in software for the popular sports titles not a DECREASE, an INCREASE while everything else was not charting at all outside Shenmue. So you can't possibly even use piracy as an excuse. We also have an increase on hardware buying with those sales increases. Peopel were buying Sega as a sports EA alternative, that's what it comes down too.
Like i said before, it's just like the Saturn from before with the DC, except the DC actually had something SAT did not outside of japan, and that was a killer APP, the NFL2k series, and to a lesser extent the NBA2k Series. Outside of that difference, they had the same issue, people buying Saturns and people buying Dreamcasts, were not interested in expanding their gaming libraries form when they entered in any meaningful way, and fence leaners did not think what they saw warranted a purchase whether they wanted the games or not. This combo was why Sega was doomed from the launch of the Saturn and they did not have at launch, continuation of popular genesis series, the same third parties that made people look at Sega has the "edgy" console, or a affordable price point, etc.
When an alternative to the Genesis came out 3DO, the first step of the stage was set, after surprisingly outpacing the Jaguar 3:1 even when it was still overpriced before the cost cut down, and even aided in PSX cutting the SAT off in Japan, in NA the 3DO was grabbing devs that Sega would have had, and then started sharing them with the PSX, and then all the games that were being made for 3DO (including MGS and others), all went to the PSX. yeah, SAT shared some games with 3DO as well, but not as much, and 3DO becoming a software maker after deeming the M2 too costly and canning it, also helped bring more games to PSX via third-party contracts canned after 3DO's death, and 3DO recommending these devs to go to PSX. None of them were interested in the SAT otherwise they wouldn't have been stalling about releasing on consoles until that recommendation.
In the end in NA, the Saturn had an issue attracting people with software, which was needed to increase hardware sales and that became an issue very quickly to the point that Sega's overgamble on NA prompted SoJ to ignore trying to at least salvage the Saturn as niche profit device, and cutting it off entirely with help of JP investors, and some connections SoA had, to bring out the Dreamcast in another gamble, this time by itself with no other revenue streams coming in (naomi arcade hardware low margins excluded) which was the same mistake Atari made with the Jaguar, and the Falcon, and Commodore with the CD32, and NEC with the Turbo CD and PCFX.
Now the question is, ok we know what the issue was, but what caused it, why were 1991-1993 game changes for Sega in the console space, what happened?
it's simple, Sega was never equipped to actually compete. Before Segas heel turn thanks to Sonic 1, Sega was not really competing. The best you'll see is in Japan the Sg-1000, a gimped CV(ColecoVusion) clone, seeing the Famicom/NES take off in japan, and decided to redesign their marketing and some of their games (look at the CV version of Congo Bongo and the SG-1000 version) to somewhat compete with Nintendo. but this wasn't to win, this was to provide a money making alternative to the Famicom because they were running away with it. it didn't work, even with the Mark III/SMS, instead it would be NEC coming out of nowhere that would shake things up in Japan.
However outside Japan, a Sega machine was a "hey remember those arcade games by Sega we can play those at home on the TV" and that's basically the extent of Segas gameplan. They would make bank on their game in the arcade, bring them to a Sega console, maybe downgrade some of them, make good money off modest sales due to low cost, and for the big name games they would bring those games to home consoles, and computers like Amiga, ST, C64, and so on. It wasn't unusual for Sega to partner with third-parties to adapt their games as well. This strategy worked and made Sega a lot of money (except SMS in NA that didn't work), and didn't have much cost to Sega. Consumers had access to Segas games across platforms, but with the "best" ports being on Sega's own branded machines of course.
With Sonic 1, and successful third-party partnerships like MK which was being pitted against SFII by consumers and industry media, Sega was now king of the mountain and they had to protect their throne. But when you look at Genesis software sales, seeing their best sellers, and looking at how things such as the Sega CD, even with MK and Sonic, couldn't really move many units, you could tell things were going the wrong direction even as early as 1993. Not to mention their computer and edutainment attempts among other bone headed decisions. The behavior of Sega then and before are completely different.
Sega hits were no longer on other consoles, Sega no longer had regional autonomous control for local revenue and profits, Sega no longer has a solid gameplan that was max return/low cost, low risk. Sega now had to centralize themselves corporate, marketing, and funding wise. Sega now was trying to get ahead of everyone with the next big thing, Sega was now spending crazy on in your face marketing, Sega was now saying they had a competition software IP to react to other companies hits in similar genres, Sega was now partnering with other companies to try and invest or enter into markets they had no preparation for, and so on. All with no safety net, and all without building the infrastructure to support it internally and externally.
You don't go from a sudden high you never expected to reach, and then hit turmoil in only 3 years or so for nothing, or for just one or two things, Sega never did what they needed to to make the company able to handle all the things they wanted to do, or were doing, the relationship between SoJ, and SoA, isn't even that relevant at the end of the day, yeah there was conflict, but they were losing money regardless, they could have gotten along and nothing would have changed without a completely different approach to how they handled their success from before.
Curiously, while it's significance is debated the 32X which is often blamed for Sega's demise wrongfully, was actually a great seller. The thing about the 32X that hurt Sega wasn't because they made it but because they cut it off early. Which pissed off retailers and consumers, it was actually the best selling new machine before they killed it, compared to the 3DO and Jaguar, and actually did better in the first 6 months iirc than the Saturn did in a year in NA. It also would have been a good way for developers to learn how to program for the Saturn since the 32X has similarities architecturally. But by throwing that in the trash, Sega killed their last life line of saving the Saturn in NA, which they gambled heavily on and lost.
To tackle Japan really quick, the Saturn was a fluke as much as it pains me to say, all evidence points to it being a fluke, and it was only a fluke for a short time. The PSX with help of 3DO at first nipping at Saturn sales, and then dying and shifting all it's support to PSX, helped with early PSX sales as it built-up it's library, by the time of Tobal no. 1, the trajectory has PSX ahead while the Saturn was gradually declining. Software sales were becoming notably worse for the Saturn at this time, it was effectively already over in terms of who would win, it then became a question of, how much will Sega sell in 2nd place, especially with the N64 delay? Well, the FF7 demo came along with some other popular titles and it was now clear even to skeptics Sega was not going anywhere near number one, however the 2nd place thoughts still stood. Until the N64 came out with a great launch and was outpacing the Saturn long-term without many of the game genres and series people WERE buying PSX's and SAT's for. N64 if it came out earlier, had a few months more time in the market, or if they just had one more semi-hit in Japan would have passed the Saturn in sales, launching late, in less time. Saturn had that first two years~ as viable before it ended. The change in focus to domestic from international did not work. Now, they could have kept the Saturn around for profits as a cheap game player but Sega considered the Saturn an image problem and wiped it out once the Dreamcast was available, which did worst in Japan than anywhere else, the complete opposite of the Saturn.
Curiously, Sonic which is trigger for the chain of events that ended up resulting in Sega imploding onto itself, was also not as big as Sega made people believe. After Sonic 1, which was also bundled against the wishes of SoJ, Sonic 2's marketing campaign and media coverage for Segas "killer threat" couldn't even met half of Sonic 1's performance.
One could simply brush this off as a lack of heavy bundling, even though Sonic 2 was also bundled quite a bit, and had an expansive marketing campaign, this happened again in a worse way with Sonic 3, and even worse after that with the separate release of Sonic and Knuckles. By this point Sonic, with no matter how much hype and marketing muscle, couldn't sell 2 million units. Nothing after S&K sold a million until Sonic Adventure, which out of 9 million Dreamcasts, sold around 2 million copies, most in NA as shown above, which was overlapped by the NFL bundle and various other bundles Sega wanted to see would stick. It barely did anything in Europe, which was a major Sonic Genesis region, and of course nothing in Japan, as Sonic never took off there.
Sonics sales matched nearly 1:1 Sega's trajectory which I find quite interesting.
So the issue with Sega isn't so simple as what most people think, Sega found success, did nothing to sustain that success, and fell apart. That's an oversimplified but accurate summary as to "Wtf happed" and why they ended up leaving hardware and changing into what they became.
Really, it's not that different from what saw other companies fall, Atari, Commodore, Coleco, NEC, had many symptoms that led to their deaths, but very simple explanations of what started the chain of events, All of which reached success no one including them was expecting, and screwed it up by simply not preparing for anything, having no safety net, and not building sustaining infrastructure overtime. Even Coleco, which CV was not hurt by the crash as the others, went too far with their attempt to enter the Computer market with the Adam, instead of revamping their electronics section to successful areas they closed it down and went all in on dolls, with something that was obviously a fad to compete with Barbie and other brands, and at no point had prepared for any issues.
Commodore and Atari are well known, and NEC's mistakes are similar with a mix of their mistakes and Segas.
If anything the smartest industry entrant was 3DO, there was no way to expect the multiple license manufacturer format to not work until the project started, because even then it seemed like it was making progress. It ended up making things incredibly costly in the end, makers were not making back losses, and despite some well-selling top titles, it made it expensive to invest in both hardware and software.
3DO did this strategy because they didn't have enough money to start from scratch even with all their connections, partnerships and funding. it was the then best way for a new comer to enter then then ballooning costs of the gaming industry. This was foreshadowing the amount it would take to enter from scratch, which Microsoft ended up learning 3 or so years after 3DO's hardware division sale. A $3.5-4 billion estimated total loss or Microsoft.
But otherwise the failures seem to have always been due to the companies being just dumb really. With bad decision makers and no one thinking to build support and fallbacks. The current 3 companies have been around long enough to adjust, and all 3 are able to take hits, which Sega and others were never able to do at any point (except maybe Commodore).
TL : DR Yep, I hope this ridiculously long rant on a gaming internet forum, educates you on the matter.