How to set preferred audio device for common jack

Used to be a window that came up every time I plugged in something to ask whether I intend to use it as a headphone or microphone. One of these times, I ticked headphones and also ticked the "don't ask me again" box. How do I bring up this window again?