To delete a Media Library, click on Media Library options icon to the right of the Library title, then click "Settings."
You can also access Media Library Settings by clicking the library title from within a Media Library.
From the Settings window, click the "Delete Media Library" link on the top-left.
You'll have a chance to verify your decision - click "Delete" if you are sure.
All media files in the deleted Media Library will also be deleted.
See also: Remove assets from Media Libraries, Delete assets from My Media.