What's the best way to go about inserting data and images in bulk into MySQL database tables with PHP or MySQL?
-
I'm looking to add new rows to certain tables in a database. Can it be done via a CSV file? XML file? Are there other, or better alternatives? Is there a way to just add rows to tables or would it need to be done as a total re-write, i.e. to download existing rows and overwrite the entire table with new data included?
-
Answer:
The best way I've found to do this, input the data into a CSV with a reference to a filename, then in PHP use http://php.net/manual/en/function.fgetcsv.php to loop through each CSV row and update/insert the data (including reading the image file and inserting it)
Bryon Keck at Quora Visit the source
Other answers
While CSV is faster than XML, it is more error prone as there is no real way to validate it without writing code to count number of delimiters on each row. With XML, the script simply won't open an invalid file. I wrote this guide for my last employer on how to integrate an XML feed into a MySQL database using PHP. Hope it helps you in some way. http://files.perfiliate.com/solutions/BuyAtParseProductFeedsGuide.pdf
Simon Quick
Depend on the amount of data you will insert. As Saurabh said, LOAD DATA INFILE is the fastest way. Turning off indexes and rebuilding them at the end could be faster than updating them for each record you insert. If is really heavy the amount of data, assuming you want to optimize inserts over queries, would be having the tables using the tokudb storage engine.
Gustavo Muslera
The safest way would be to build a parser (in PHP) to validate the data, then insert as per Bryon's answers. The quickest way is via MySQL's shell using the "load data" command to append to your table. http://dev.mysql.com/doc/refman/5.1/en/load-data.html But: I recommend not inserting images into your db. It's faster to serve static files from the file-system and easier to backup small text-only databases. If you choose to insert the images into the database, watch out for problems with large images due to MySQL/PHP message-buffer-size settings.
Alex Barkan
Look into using a migration tool like <a href="https://github.com/ruckus/ruckusing-migrations">Ruckusing</a> or <a href="http://www.phinx.org">Phinx</a>. If you are using a framework, try leveraging their Migration feature if available. It makes it rediculously easy to do these sort of things.
Ryan Flores
I highly advise against storing images in a database. Store the file name reference or you will have huge bloat and decreased performance. Use the file system, it's much better at handling that sort of thing. For data, use the http://dev.mysql.com/doc/refman/5.5/en/optimizing-innodb-bulk-data-loading.html rather than PHP. Check out http://www.mysql.com/products/workbench/.
Keith Ensign
A even better way than csv is to create bulk insert queries and multiple update queries and dump them in a sql file. Then source the file in mysql. This method should be even faster than creating csv.
Jayant Kumar
Best way is LOAD DATA INFILE I was used it for daily upload about 2 TB of Data into MySql this is best & optimize way to upload heavy data . I was create a php program which extract rar files into directory then Upload these files which is in form of comma separated (log files) into MySql DB . I was testing using all techniques like bulk insert into db, upload data into csv format table then modify table format from csv to myisam .
Saurabh Goyal
Related Q & A:
- What's the best way to get into the Graphic Design industry?Best solution by Yahoo! Answers
- What's the best way of returning damaged NIKE shoes to NIKE for another pair?Best solution by Yahoo! Answers
- What's the best way to hook up an overhead projector to a laptop?Best solution by Yahoo! Answers
- What is the best way to go to the TX Motor Speedway?Best solution by Yahoo! Answers
- What are the best way to go to the Bollywood?Best solution by wikihow.com
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.