Cap the unused wires in both boxes and install a single-sided cover plate. Remove the switch box, install a deep single-gang box, cap the wires in both boxes, and repair the drywall. Whether you can do this option depends on how many cables come into the box, as fill rules may prohibit using a single gang box.
