How to run custom method when PowerShell script module is removed (to clean up after itself)

Is there a way for a PowerShell script module to run some code when it is removed?

Example:

Import-Module my-module.psm1
Remove-Module my-module
# At this point I want my-module.psm1 to be notified
# that it is being unloaded so it can run some clean up code.

My current use case is a module that adds a breakpoint and I want it call Remove-PSBreakpoint when the module is removed so it doesn't leave that clutter in scope.