I was recently having an issue with what I was seeing in my preview not being what I got in the final output render when editing HDR projects. I spent a week or so scratching my head and talking with the team about it, who was doing the same trying to figure out what it could be, when I stumbled across a solution.
If you are getting 8 bit previews in HDR mode, it could be because when you enable your external monitor in Vegas, it is kicking windows out of HDR/deep color mode for that monitor. I noticed it while resetting the mode for that monitor, as soon as I clicked the little monitor icon in Vegas to go full screen on my secondary OLED display, the "enable HDR for games and movies" button in windows would toggle off. Turning it back on would flip right back off again as soon as the external preview was enabled.
How I got around it:
Simply make sure your secondary HDR display is turned on and flagged into HDR mode in windows BEFORE opening Vegas and loading into your project. This fixed the issue for me. Hope this helps anyone else who may come across it.
For most folks, this isn't much of an issue as they would have their external monitors on and in HDR mode anyway, but with my GPU (Radeon 7), when I first turn on my OLED TV, it sends it nothing but static (it's been this way since the day I bought it, I'll never buy AMD GPUs again), so I have to make a setting change somewhere to make the TV get a new signal, then it will start working. One way I would do this would be to click the external preview monitor icon in Vegas a couple times, which would get the TV a signal, but I was unaware that this was also breaking HDR compatibility by turning it off in windows.