[christian Carballo]

I’m trying to import a large file and along the way, my wifi was dropped and the import was interrupted. The import was not done alphabetically, so I can’t really tell what order it was imported. Is there any way I can resume the import without creating duplicates, or do I need to wipe the database and import everything again?

[christian Carballo]

I figured out how the import is sequenced so this is no longer an issue. I do have a somewhat related question — I am in the process of uploading a large file –60k+ entries. Any tips on how to get this done?

• This reply was modified 8 years ago by christian Carballo.

[Steven Zahm]

@ christian

I tested importing about 30k entries in a single CSV file on a shared host (Bluehost) and it imported without issue. The only real limitation at this point is server resources provided by the web host. When I do imports or others, as freelance work, I usually break the CSV file in 10k chunks.

Hope that helps!

[christian Carballo]

Great. I have bluehost also. How long did it take to import 10k?

[Steven Zahm]

@ christian

It’s been awhile since I’ve since my last import of this size… it does take quite a bit of time especially on shared hosts because they will throttle CPU utilization. Honestly, when I start one this large, I leave the browser window open and do something else.

[christian Carballo]

I seem to freeze every 1k or so. I’m just under 10k now and deleted what’s been uploaded and restarted the upload again a few minutes ago.

[Steven Zahm]

@ christian

re: I seem to freeze every 1k or so.

hmmm… that is indicative of a background PHP error. If the server responds with an error instead of the expected reply, the javascript which processes the chunk importing will stop.

Suggestion would be to make sure all PHP error reporting is either disabled or logged, my preference is the latter. Best practices dictate all PHP error reporting be suppressed on a production site so it does not expose potentially sensitive server info.

It is possible the error was logged, it may be worthwhile to look in the PHP error log to see if there seems to be anything related to the import.

[christian Carballo]

I have no clue what you just wrote but I will look it up. Thanks for the tip

[christian Carballo]

I checked the error logs and didn’t see anything that indicates an error with the upload. Granted, I’m not very literate in all of this, but the times for the handful of errors listed didn’t match up. Either way, I turned off the error reporting after learning what it was and what it entailed. Downloading it again — I have about 50k left. Bluehost said not to increase the memory speed beyond 128. Do you agree?

[christian Carballo]

So I turned off the error reporting and increased the speed to 256 but it still stopped after 100 entries. Any other ideas?

[Author]

Posts