Yes, you can add events to a panel, however there is no support for it through the UI, you have to do it in the source code ... also, if you plan to support a "backwards old fuddy duddy no fun good for nothing pain in the @rse piece of crap" browser (say IE 6), then nevermind.
Easiest way to get it done if you aren't comfortable enough coding the event 100% in the src window is to create the event on an object that supports it, then just move the xp:eventHandler block inside the xp:panel