Deleting a file completely without sending to Recycle Bin

Am I able to delete a file completely without sending it to the Recycle Bin? I understand that the point of this is so I do not accidentally delete any unwanted files, but there are sensitive work files that sometimes I just need to delete and I don't want to forget to delete them out of the Recycle Bin. Thanks for your help.
