Hello sjb1963,
this looks like custom coding resulting in JavaScript errors (conditional fields not working) and basic CSS incompatibilities (fields pop out of their containers).
First, a popup in HTML has many possible ways and a variety of different approaches to get implemented.
There are tools for this (e.g, part of the template or result of a widget kit) or you may even provide your very own JavaScript.
How should I know?
Second, a popup is no standard view mode for forms and is therefore not officially documented in any way.
As there exist so many and different tools and ways to create a popup, any official documentation does not make much sense.
Even one documented way of doing it, would not serve much of an auditorium out there.
Third - the good news, it is of course possible to have a form in a popup well functioning.
It requires decent development skills (JavaScript, CSS, HTML, employed UI framework), much experience and a good understanding of possible challenges.
Development in this context means core debugging (JavaScript, CSS, HTML) on the client side in the developer tools of one of the big web-browsers.
As to forms in Bootstrap 5 layout I have to state, that Visforms supports and renders clean in Bootstrap 5 and it works quite fine.
I have no idea what all might going on on your page.
Best regards, Ingmar