How can I query Parse data by creation date with Swift?

MySQL Monster Query Issue

  • MySQL monster query is timing out. Anything I can do about it? (Long question, possible a cure for insomnia.) I set up (using my usual mediocre coding skills) a small PHP app which allows form creation, saves the results, and outputs the results into a tab-delimited file. First two parts seem to work fine. Having some trouble with the last bit. The form information is saved into two tables: "form" which saves the name, etc., of each individual form, and "field" which saves the fields, using the 'form' row id to keep track of where they belong. A simple form might be saved as form -> form_id=1, form_name='Test', etc. field -> field_id=1; field_form=1; field_label='First Name', etc. field -> field_id=2; field_form=1; field_label='Last Name', etc. Pretty basic stuff. Likewise, when the form is filled out, the data goes into two tables, "form_save", and "field_save" which mimics the structure above. The 'field_save' row has a column pointing to both the id of the 'form_save' row and 'field' row it is associated with. In essence, for every form in the database, there is one row in 'form' and however many rows in 'field'. For every form someone fills out, there is one row in 'form_save' and however many rows in 'field_save'. This all works fine on modest forms filled out in reasonable numbers, which is mostly what my clients have. However, a client created a monster 100 question form and has had a large number of people filling it out. It can be edited on the back-end and saved on the front-end just fine, but I cannot get it to spit out a tab-delimited report because the way I'm approaching it in PHP either requires a cajillion queries or a cajillion if/then reiterations to process. This times out on the server, which has time limits on queries. The "simplest" way to grab results seems to be to: * Query grabbing all of the 'field' rows for the form type. * Query grabbing all of the 'field_save' rows which match the form type. Then it gets ugly. Step through all of the 'form_save' rows, and within that step through the 'field' rows, looking for matches. It works on reasonable results sets but on a form with 100 fields (really) with 15,000 people who have filled it out (really) we're talking about 1.5 million loops. Boom. My results file looks like this: [date] [ip] [first name] [last name] [another optional] [another required] [another required] [another optional] etc. With the date and IP coming from 'form_save'and the rest coming from 'field_save', as matched against the 'field' rows for that particular form. Fields which are optional to fill out may have no saved result in the 'field_save' table and therefore need a tab to keep the columns aligned. Is there any way I can structure a query to return results combing the 'form_save' and 'field_save' data all in a single neat row, as above, including the "empty" slots which may exist? Or is there a smarter way of doing what I'm doing? (I think I can move this client to a dedicated box, which would allow my program to run, but it bugs me that I think I'm being horribly inefficient here and would love some pointers on doing a better job with the query.)

  • Answer:

    Ermm.. MeFi ate that from me. How about this:SELECT fsf.*, ff.*FROM form_save_field AS fsfLEFT JOIN form_field AS ff ON (ff.field_id = fsf.form_save_field_field)LEFT JOIN form_save AS fs ON (fs.form_save_id = fsf.form_save_field_save)WHERE (fs.form_save_stamp >= $date1) AND (fs.form_save_stamp < $date2)ORDER BY fs.form_save_id, fsf.form_save_field_id;

maxwelton at Ask.Metafilter.Com Visit the source

Was this solution helpful to you?

Other answers

I'd think an outer join between form_save and field_save is what you're after, but I suspect I may be missing something...

pompomtom

What pompompom siad... you should be able to write a left outer join that'll do this.

SpecialK

(can you paste in, using pre tags, the, schema for these two tables?)

SpecialK

Thanks for any ideas!CREATE TABLE `form_save` ( `form_save_id` int(10) unsigned NOT NULL auto_increment, `form_save_form` int(10) NOT NULL default '0', `form_save_ip` varchar(20) NOT NULL default '', `form_save_stamp` timestamp NOT NULL default CURRENT_TIMESTAMP, PRIMARY KEY (`form_save_id`)) ENGINE=MyISAM DEFAULT CHARSET=latin1;Key: 'form_save_form': pointer to 'form' tableCREATE TABLE `form_save_field` ( `form_save_field_id` int(10) unsigned NOT NULL auto_increment, `form_save_field_save` int(10) NOT NULL default '0', `form_save_field_data` text NOT NULL, `form_save_field_field` int(10) NOT NULL default '0', `form_save_field_parent` int(10) NOT NULL default '0', `form_save_field_stamp` timestamp NOT NULL default CURRENT_TIMESTAMP, PRIMARY KEY (`form_save_field_id`)) ENGINE=MyISAM DEFAULT CHARSET=latin1Key: 'form_save_field_save': pointer to 'form_save' 'form_save_field_data': saved data 'form_save_field_field': pointer to 'field' 'form_save_field_parent': pointer to another row in this table; if the field type is "checkbox" with multiple possible answers, the various answers are added as additional rows while the original row has "checkbox" in the data field. In this way, if you parse the row and find "checkbox" you know to look for additional rows with a parent that matches your existing row. I do this to allow faster searching for specific answers of a multi-choice question.

maxwelton

Quick and dirty...SELECT fsf.*, ff.*FROM form_save_field AS fsfhttp://dev.mysql.com/doc/refman/5.0/en/join.html form_field AS ff ON (ff.field_id = fsf.form_save_field_field)WHERE fsf.form_save_field_save = $form_save_id;

sbutler

Or... suppose you want all the answers from a particular date range [$date1, $date2). You could do it like this:SELECT fsf.*, ff.*FROM form_save_field AS fsfLEFT JOIN form_field AS ff ON (ff.field_id = fsf.form_save_field_field)LEFT JOIN form_save AS fs ON (fs.form_save_id = fsf.form_save_field_save)WHERE (fs.form_save_stamp >= $date1) AND (fs.form_save_stamp ORDER BY fs.form_save_id, fsf.form_save_field_id;>The point is, JOINs are good for more than adding columns to be returned! Really, in this case that second LEFT JOIN becomes an INNER JOIN because of the WHERE clause (NULL would always exclude the rows anyway). IIRC, all versions of MySQL >= 4 perform this optimization for you... but it's something to be aware of.

sbutler

You're effectively using your database to store a database schema for your forms, as well as the data defined by that schema. That's probably going to make it hard to get query results with one form per row. But if you just do the queries you're already doing and bung in an ORDER BY to sort them according to the ID's the records are associated with, you should be able to generate your TSV output lines just by making one pass through each query, rather than looping and looking for matches.

flabdablet

On lack of preview: what sbutler said.

flabdablet

That is: if you use ORDER BY properly, you can make all the saved fields for a particular instance of a particular kind of form cluster together in your query result, and occur in a predictable order. Then all your TSV-generating code has to do is walk the query, spitting out fields in the order they occur in it, and adding an end-of-line every time the form instance ID changes. It would probably pay to add an extra numeric column that explicitly specifies the column number that the field concerned should occupy in a TSV report, and include that in the ORDER BY as well. That would make it super-easy to figure out how many tabs to emit between fields.

flabdablet

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.