Dynamically Add Components to a JDialog

I am having trouble adding JComponents to a JDialog when the user clicks a button on the JDialog. Basically I want it to look like this:

Then, when the user clicks "Add New Field" I want it to look like this:

I cannot seem to get the dialog to add the new JLabel or JTextField. Can anyone point me in the right direction?

EDIT: This is the action for the "Add New Field" button (Just trying a label now).

@Action
public void addNewField()
{
    Container contentPane = getContentPane();
    JLabel label = new JLabel ("welkom");
    contentPane.add(label, BorderLayout.CENTER);
}

SOLUTION:

I used mre's solution and got it to work. Here is my final function:

@Action
public void addNewField()
{
    System.out.println("New Field...");
    Container contentPane = getContentPane();
    JLabel label = new JLabel ("welcome");
    label.setBounds(10,10,100,10); //some random value that I know is in my dialog
    contentPane.add(label);

    contentPane.validate();
    contentPane.repaint();
    this.pack();
}

Another one of my problems is that I am using a "Free Design" layout in NetBeans, which meant that my label was probably in some weird position rather than being in the bounds of my dialog (just a guess). I solved this problem with label.setBounds() so that it showed exactly where I wanted it to.