Business unit configurations can be found in the Control Center in Configurations >> Business Units.
Existing business unit configurations can be deactivated but not removed from the control center.
Business units can be disabled (and enabled back) at any time, all you need to do is follow these steps:
1) Go to Configurations >> Business Units >> click on the business unit you would like to disable.
2) Next, on the upper right side of the business unit drop-down section, you can find the "disable/enable" switch. Click on the switch and approve the deactivation. (See screenshot below)