First you need to add Ajax assembly reference to Asp.net page,
Then Add ScriptManager and ModalPopupExtender to asp.net form shown as below
after that create Pop form as you want, i have given an example below
finally publish the solution and click Show popup button then you can get popup window like below,
if you want to access controller events inside the popup window, use asyncPostBackTrigger to access this controller in postback.
Hope you learned something throw this post,
Thanks & Regards