How to handle C# AppDomain.UnhandledException

I have read these two pages from msdn: 1, 2

So I constructed this simple Web Forms app:

public partial class Form1 : Form
{
    public Form1()
    {
        InitializeComponent();
        new Thread(new ThreadStart(throwException)).Start();           
    }

    public static void throwException()
    {
        throw new Exception();
    }

    public static void UnhandledExHandler(object sender, UnhandledExceptionEventArgs t) 
    {
        MessageBox.Show("This is exception is unhandled.");            
    }
}

static class Program
{        
    static void Main()
    {            
        Application.SetUnhandledExceptionMode(UnhandledExceptionMode.CatchException);
        AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(Form1.UnhandledExHandler);
        Application.EnableVisualStyles();
        Application.SetCompatibleTextRenderingDefault(false);
        Application.Run(new Form1());
    }
}

The problem with this is that after executing UnhandledExHandler, Forms1() constructor does not return but keeps throwing Exception, I mean it keeps calling throwException() on a new thread. The end user experience popping of MessageBox continuously. I know I must call Application.Exit()inside UnhandledExHandler to exit the app. But shouldnt this stop calling UnhandledExHandlerafter called once?