I don't think you're primarily at fault. I'm not certain you're at fault at all.

From his reply, it feels like he has little to no experience with fan-run SF conventions. So the two of you were starting from very places regarding "what is a panel?"

If he wants to be involved in fan-run SF conventions, I hope he goes to a few and attends some panels.