Double quotes are a CSV standard and must be used in certain circumstances according to the standard. If Excel (which has a history of spotty CSV file support) is displaying the quotes, it is because it is not properly parsing the file. I highly recommend using Google Doc Spreadsheet or LibreOffice Calc when working with CSV files because they can more reliable read standard CSV file.
Hope that helps!
ps. If Excel ask when opening the file what the enclosure is, choose the double quotes and choose the comma for the delimiter.