The last thing I needed to do to finalise the feature set in BaseElements 3 was to generate the list of Variable references from each of the Calculations. This isn’t a minor thing though, as there are no variable reference types in the DDR, so you need to pull a list of variables out of the calculation manually and then generate the list of related records.
First get the variables.
In order to find variables inside a calculation, you do a few things : First remove the comments, both /**/ and // types, then remove any text between quotes, making sure you ignore escaped quote characters. Then you can work through the text and get the $ values, making sure you’ve got the right break characters. I’ve done this code before in FileMaker, and it’s not simple, and involved a few recursive custom functions. In other words it works, but it’s slow.
In BaseElements 3 we’ve worked around this entire issue with a single plugin function call : BE_ExtractScriptVariables. This sort of thing is much faster in plugin code, so that is a big advantage.
Next create the related records.
The second thing you need to do then is turn that list of variables into a set of records. You can’t do a simple import from one table to another as there will be multiple variables in some calculations and you need a separate reference record for each. The standard way to do this would be a loop through the records, and a loop inside that through the variable list, which generates a new record each time. The issue that makes this slow is the constant context switch. You’re switching from the Calculation layout to the Reference layout every time you want to move to the next record.
Slow : Looping New Records.
For curiosities sake I built a quick version of this code and timed it. Total for the 2500 odd BaseElements calculations was over 2 and half minutes, to generate nearly 4000 variable references.
Fast : Write to disk and Import.
The actual code I’m using in the next version of BaseElements is a loop through the calculations record that stores the contents of a csv export file in a local variable. So you do a very basic loop and just fill up a variable. No context switch, no new records, no modifying or committing of data, just reading from fields and writing to a variable, so it’s fast. Time for that section of the code is about 2 seconds for the entire 2500 records.
Then you write the file to disk ( less than one second ) and import the csv file ( about 8 seconds ). So total time : 11 seconds.
Obviously a huge difference between a loop with new records and a write to disk and import. For anyone doing any large moving of found sets, you might want to consider something similar.