[Diane Lindquist]

It appears that my computer rebooted during an import, and it seems entries of the entire database were duplicated. Need to eliminate duplicates – Help!

[Steven Zahm]

@ Diane

The only real solution is to bulk delete the all entries and import again or manually delete the duplicates.

So this is anew one… what happened? You were in the middle of an import and the computer rebooted. When it started back up it open the browser and started a second import??? I not sure how that would even be possible.

[Diane Lindquist]

Nevermind, my error. I guess I royally screwed up. I had divided the file into pieces, using the same base filename, and then sheet1, sheet2, … I think I mistakenly imported the original large file thinking it was the last sheet, which then duplicated all the entries. Sigh…

[Steven Zahm]

@ Diane

Thanks for letting me know!

How large is the file (number of row)?

The beta version of CSV Import is much faster at importing a large number. I tested a file with over 100K and left it run until it imported 20K. The current released version, each import step takes slightly longer than the step immediately preceding it because the CSV has to be parsed and then read on each step. In do so it has to re-read rows already imported in order to get to the rows not imported. The beta version parses the file a single time and then saves the parsed results to a bunch of individual machine readable files. So, on each step, the import only needs to pick the next file in the batch and import data within. On significantly large CSV files, the make massive improvements to the time require to import.

If you are interested in trying the beta, navigate to the Connections : Settings admin page and click the Beta Versions tab. Enable beta support for CSV Import and saved. Next navigate to the WordPress Updates admin page. Nothing to do here, this should simply force WP to check for updates. Now navigate to the Plugins admin page where the beta should be available for you to install.

Hope this helps!

[Diane Lindquist]

About 9000+ rows. Would you use the beta for a live site?

[Steven Zahm]

@ Diane

Several other have used it quite successfully on their live sites. Not sure of what their file size were. They mostly wanted to use it for the improved feedback during the import process. I’ve been wanting to release it officially but I have been holding off in order to have a few others use it. Personally, I feel it is ready.

YOu would know fairly quickly if it is not working. On the mapping step, that is where the file is parsed and split into batches of machine readable file. On the import step, if you get 2/3 steps in, It should continue on to the end without issue.

During those first few import steps you could have the manage page open in a several tab and spot check a couple entries making sure it matches the CSV file contents.

[Author]

Posts