Offline
Firstly, since we're pretending we are administrating a "real life" network I decided to prevent anyone from utilizing media such as CD-ROMs and floppy disks. You can also lock down USB drive and tape drives if needed using this path.
Computer Configuration > Policies > Administratice Policies > System > Removable Storage Access
There are a bunch of settings here, I enabled all but the "Custom Classes".
Secondly, I put a lock down on any installation processes.
User Configuration > Administrative Templates > Windows Components > Windows Installer
Here I enabled the "Always install with elevated privileges" and "Prevent removable media source for any installation" settings.
If I were an administrator that faced major issues with rogue installations I might consider using a program such as "Deep Freeze"