Sometimes BaseElements updates come thick and fast. Other times there’s a wait between versions. This is one of the latter.
BaseElements 4.0.2 was released back in July of last year. We’ve since been working on some minor tweaks and fixes to BE that didn’t at any point seem, by themselves, to warrant an new release. But we had also been working on a change to the free BaseElements plugin that allows us to remove nodes from XML documents. This was working with the 2.0.1 update to the plugin back in February.
What’s the connection to BaseElements? Glad you asked. The biggest issue we have with the import process in BE is that on very large data sets ( many GB of XML DDR files ) we often hit limits in FileMaker. With the plugin, we don’t have access to more than 2GB of memory. Above this we get errors ( which we can trap for ). So we switch our XSLT processing to the internal Xalan based import engine. This works great, except in some cases where not only does it not work, it just stops half way. No errors, nothing is reported, but people contact our support page to say that the import only half worked.
There is no way for us to alter the internal FMP import, as we don’t control that part of the process. Until the plugin architecture moves to 64 bit in the client, there’s no way to increase the amount of memory available. And until we move to a XQuery based import model we need to load all of the XML file into memory every single time we import, and there are multiple imports for a single Analysis.
So the new 4.0.5 release of BaseElements includes and option to reduce the size of the XML files by removing some nodes it doesn’t need, and only importing the end results. We remove the HexData and PlatformData ( print config ) nodes, as well as ObjectStyle, CharacterStyleVector and ParagraphStyleVector nodes. None of this contains anything useful to BE, and the resulting XML files can be up to 90% smaller than the originals.
In all our testing so far, this has been able to fix issues with a DDR that just failed to import before in BaseElements. And in some cases it’s also reduced the import times as well. The original files aren’t touched, and the import result would be exactly the same either way.
This is a new checkbox on the setup layout inside BE :
It’s off by default, so if you’re having this issue, or would like to see if it reduces import times on your large solution, give it a try. And if you find anything useful or have any issues to report, please contact us.
There’s a whole slew of minor fixes in this version of BE, so we suggest everyone using v4 should update.