A toggle button is a button which stays down when clicked once and goes up when clicked a second time.
When to use
Use a toggle button to indicate a state.
Example: A word processor should use toggle buttons to indicate the state of "Bold", "Italic" or "Underline" formatting.
When not to use
Do not use a toggle button to indicate an action.
Example: A music player should not use a toggle button to implement a Play/Pause button. It should use a normal button and adjust the icon and label to represent the action which would be performed when clicked.
How to use
Never change the icon or label of a toggle button. The fact that it is pressed down is enough to indicate the current state of the element it represents.
Since a toggle button represents a state, in most cases the label should be a noun rather than a verb.