Assuming you're using Excel 2007, data labels are added through the "Data Labels" selection. As shown below, cells A2:A5 contain the data Items. Cells B2:B5 contain the data Values.
1) Select cells A2:B5
2) Select "Insert"
3) Select the desired "Column" type graph
4) Click on the graph to make sure it is selected, then select "Layout"
5) Select "Data Labels" ("Outside End" was selected below.)
If you don't want the Values as the Labels, you can click on the desired label, click twice so that it's the only thing selected, and then enter your new label information in the formula bar.