Goya Blog
BaseElements 2 is coming...
Submitted by Nicholas Orr on 13 June 2008 - 1:56pmJust a quick note to let everyone know that BaseElements 2 is coming soon. It's into beta and we'll have a public release up very soon.
In terms of pricing, this will be a paid upgrade. This version will add a lot of new features, clean up and re-organise a bunch of existing ones and is a general overhaul of large parts of the entire product. We think you'll agree that it's worth it.
There will be a free upgrade for people who purchased recently (dates to come soon), as well as discounted upgrade pricing for others with v1. We'll also be making another release of 1.6.6 very soon which will work with your v2 licence codes, if you want to keep using it.
More details coming shortly...
Goya Sponsors FMSummit
Submitted by Nicholas Orr on 14 April 2008 - 12:31pmGoya is pleased to announce it's sponsorship of FMSummit 2008, the FileMaker developer conference held in Antwerp in April.
Goya is sponsoring FMSummit to promote it's BaseElements developer tool. We are not attending the conference this year as our Dutch lessons haven't yet finished, but we look forward to future conferences in Antwerp and may attend next year.
BaseElements 1.6 and DevCon
Submitted by Nicholas Orr on 16 August 2007 - 9:05amWell FileMaker DevCon 2007 is over. We had a very successful show and had mountains of positive feedback for BaseElements. We're looking forward to beginning the process of demonstrating our UpdateManager tool next.
Also just before DevCon, we released BaseElements 1.6, which features a comparison reporting tool, allowing you to compare an old version of your solution to a new version. So it's possible to document changes to your solution over time. This report takes advantage of some of the new Conditional Formatting options available in FileMaker 9, and gives us some neat effects in highlighting parts of the report.
We're looking to have our UpdateManager demo out in the next month or so.
FileMaker Pro 9.0 and BaseElements 1.5
Submitted by Nicholas Orr on 11 July 2007 - 11:21amToday FileMaker Inc released version 9.0 of FileMaker Pro and Server. This is a huge release, probably the biggest since v7 came out a few years back.
Although the new debugger and scripting functions will be probably the biggest time saver for developers, by far the biggest feature is ESS. ESS stands for External SQL Sources. What this means is you can have a table of data in FMP that comes from an SQL ODBC data source, and treat it just like it was normal FileMaker data. So, your FMP database can add, delete, read and write to your back end SQL data source on top of it's existing FMP data.
This will change the way FileMaker is perceived in some areas, and without a doubt will add to it's user base. We're looking forward to all of the possibilities this opens up with FileMaker, and showing this to our clients.
And we've updated BaseElements for all of the new FMP 9 features, so all of the FileMaker developers can starting using 9 straight away without compromising their ability to analyse and document their solutions. We've got a couple more updates of BaseElements scheduled before devcon in just 3 weeks time, so we're going to be flat out like a lizard drinking over the next few weeks.
Goya Pty Ltd Releases BaseElements DDR 1.5 for FileMaker Pro 9.0
Submitted by Nicholas Orr on 11 July 2007 - 11:10amGoya Pty Ltd is pleased to announce the release of BaseElements 1.5 for FileMaker Pro 9.0. BaseElements has been enhanced to work with all of the new features in FileMaker Pro 9, released today by FileMaker, Inc.
BaseElements 1.5 allows developers to begin adding the new functionality in FileMaker 9.0 without compromising their ability to analyze and document their FileMaker database solutions. BaseElements 1.5 adds support for the new ESS functions such as External SQL Sources and ODBC References, as well as Conditional Formatting, Script Grouping, Layout Object Anchors and more.
What else is new in version 1.5?
- Now imports the FMP version number from the Summary file.
- Added a red highlight for mismatch counts of objects that may be errors.
- Now imports the first text object it finds for Buttons, so you can see the text of a standard button.
- Added the option to list all fields used in a relationship, so you can see all of the fields from both predicates, plus any used in a sort for that relationship.
- Updated the list layouts so you can click into fields.
- Added a view button and added the notes fields to the list layouts.
- Added error counting for sorts in relationships.
- Adds a new warning for a File Reference that points to itself.
- Adds a new warning for a relationship that is trying to sort on a disconnected TOG.
- Fixes an issue with the Script Unreferenced count.
- Fixes the header count and relationship for Tooltip Layout objects.
Goya Pty Ltd expects this release of FileMaker 9.0 will be the biggest release of FileMaker ever. The new features such as External SQL Sources (ESS) will open FileMaker to a larger audience of database administrators, while the considerable improvements to the developer and end user functionality will add to the existing advantages of working in FileMaker Pro.
"I am loving BaseElements and am using it all the time. Apart from the script debugger in FMP Advanced, it is my favorite development tool." - Daniel Kaan
About BaseElements:
BaseElements is a FileMaker developer tool to analyze and document FileMaker database solutions. BaseElements provides a cross reference of every element in a solution and helps developers find and fix errors, track issues, document changes, and plan future development.
BaseElements is an essential tool for anyone who works in FileMaker Pro as it helps you make decisions about the effects of changes before they're made, document those changes, and provide tacit alerts if something is broken.
BaseElements is available as both standalone fp7 files or as a Runtime application. The XSLT code to import the data is included and free for personal use as part of the license to BaseElements.
Licensing Options:
This is a free update for existing license holders of BaseElements 1.0.
A single user license of BaseElements is US$499, and multiple user license is US$1299. With the multiple user license, developers may use a copy of BaseElements on a FileMaker server among multiple developers, or as single user version or both at the same time.
Developers may try BaseElements completely unlocked for 30 days using the registration code available on the website.
Goya Pty Ltd: http://www.goya.com.au
BaseElements: http://www.goya.com.au/baseelements
Download BaseElements: http://www.goya.com.au/download
Purchase BaseElements: http://www.goya.com.au/purchase
Based in Mt Waverley, Australia, Goya Pty Ltd has been a leading expert in FileMaker Pro development for over 10 years, with a diverse client list covering a similarly wide range of industries. By leveraging depth of experience and a tremendous library of existing modules, Goya has established strategic relationships with other FileMaker Pro developers by making available their suite of developer tools as well. Goya Pty Ltd is Nicholas Orr, John Adams, Michelle Wild, and Darrin Southern.
###
Nicholas Orr
Director
+61 4 2575 8280
Australia
© 2007 Goya Pty Ltd. FileMaker and FileMaker Pro are registered trademarks of FileMaker, Inc.
Goya Pty Ltd Updates BaseElements DDR to 1.3
Submitted by Nicholas Orr on 29 June 2007 - 2:24pmPress Release
Goya Pty Ltd Updates BaseElements DDR to 1.3
Goya Pty Ltd is pleased to announce the release of BaseElements 1.3. BaseElements is a FileMaker developer tool to analyze and document FileMaker database solutions. BaseElements provides a cross reference of every element in a solution and helps developers find and fix errors, track issues, document changes, and plan future development.
BaseElements takes a different approach to FileMaker solution analysis, by using standard and familiar methodologies to viewing and reporting on data, so that experienced FileMaker Pro developers can quickly and easily find exactly the information they're after. Everything in the entire solution is imported, cross referenced and accessible to search, list, sort, display and relate to every other object. As well, BaseElements includes error, unreferenced and warning reports to pick up on issues and identify elements that are no longer in use. If the developer has a question about their solution they need answered, BaseElements can answer it.
New Features:
BaseElements 1.3 extends the existing ability to locate objects that are "Unreferenced" by alerting developers to when your solution uses Indirection. Indirection is the ability to refer to items in your solution via calculations that reference the name, instead of directly via internal ID. This gives developers much more confidence when making decisions about items to cleanup in your solutions. Plus, BaseElements 1.3 adds a plugin tracking function so you can see what external plugin function calls you're making and where, and link directly to them from any location.
Both the Indirection and plugin functionality has not previously been available in any other FileMaker developer tool, and this adds to the already impressive feature set of the product.
"I am loving BaseElements and am using it all the time. Apart from the script debugger in FMP Advanced, it is my favourite development tool." - Daniel Kaan
About BaseElements:
BaseElements is an essential tool for anyone who works in FileMaker Pro as it helps make decisions about the effects of changes to a solution before they're made, document those changes, and provide tacit alerts if something is broken.
BaseElements is available as both standalone fp7 files or as a Runtime application, giving the developer a choice in how they wish to work. The xslt code to import the data is included and free for personal use as part of the license to BaseElements.
Licensing Options:
This is a free update for existing licence holders of BaseElements 1.0.
As well as a single user licence at US$499, Goya offers a very flexible site license option that is both a combination site licence and multiple user licence for US$1299. Developers may use a copy of BaseElements either in a FileMaker server among multiple developers, or as single user version on an individual's machine, or both at the same time.
Developers may try BaseElements completely unlocked for 30 days using the registration code available on the website.
Goya Pty Ltd: http://www.goya.com.au
BaseElements: http://www.goya.com.au/baseelements
Download BaseElements: http://www.goya.com.au/download
Purchase BaseElements: http://www.goya.com.au/purchase
About Indirection: http://www.goya.com.au/node/77
Based in Mt Waverley, Australia, Goya Pty Ltd has been a leading expert in FileMaker Pro development for over 10 years, with a diverse client list covering a similarly wide range of industries. By leveraging depth of experience and a tremendous library of existing modules, Goya has established strategic relationships with other FileMaker Pro developers by making available their suite of developer tools as well. Goya Pty Ltd is Nicholas Orr, John Adams, Michelle Wild, Stuart McLellan and Darrin Southern.
###
Nicholas Orr
Director
+614 2575 8280
Australia
BaseElements Update
Submitted by Nicholas Orr on 28 June 2007 - 11:08amToday we've released today version 1.3.0 of our flagship FileMaker Pro developer tool, BaseElements, and I thought it might be handy to recap for those interested some of the bigger changes we've introduced since 1.0 was released in January :
- Runtime versions.
- The ability to add your own scripts and layouts.
- A plugin function tracking system.
- The ability to track variables, both global and local.
- A comprehensive Indirection system, to work with the existing Unreferenced system.
- A detailed "Warning" system for potential FileMaker issues.
- Much more detail and information about Layout Objects, including object names and tabs.
- New list layouts for Field References, CustomFunction references and ValueList references.
- New table for Sort Order Items.
- A print script function.
- An option to highlight "Web Compatible" scripts.
- And much more...
There is of course many more other features, a lot of changes and improvements, and a bunch of bug fixes. All of the changes are detailed in the full Version History page.
So if you've tried BaseElements 1.0 at any point and not kept up to date with the new releases, we think this is a good time to give it another run. There are lots of new additions to BaseElements that will help you in your FileMaker Pro development.
But don't think that's all, with DevCon coming up we've got lots of plans for more features before August, so stay tuned.
Understanding and using the Database Design Report - Part 3
Submitted by Nicholas Orr on 4 June 2007 - 3:56pmIn Parts one and two, I described XML, and more specifically, the FileMaker built in XML format that it understands and will import. In this final part, I'll actually show some XSLT to transform some of the DDR from one format to another.
So, what is XSLT?
First of all, XSLT, as explained in part 2, is a way of going from one XML format to another. It allows you to change any one XML format, such a a FileMaker export, into another, such as a SVG graphic, or an RTF document.
In our case we're wanting to do something very specific : Process the DDR report and extract from it some data that we can import back into FMP. I'm going to start with some very simple examples, and also show a little more complicated one so that you understand the basic ideas.
The example I'm going to show is extracting all of the account details out of the DDR. So we want to import a list of accounts, along with the relevant details for each account. First lets look back at that part of the DDR :
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
<BaseTableCatalog>
<RelationshipGraph>
<LayoutCatalog>
<ValueListCatalog>
<ScriptCatalog>
<AccountCatalog>
<PrivilegesCatalog>
<ExtendedPrivilegeCatalog>
<CustomFunctionCatalog>
<FileReferenceCatalog>
<CustomMenuSetCatalog>
<CustomMenuCatalog>
<Options>
</File>
</FMPReport>
As you can see there is a node in the list called "AccountCatalog". This obviously contains all of the accounts we need to access and view. So lets omit the other nodes and expand that one :
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
<AccountCatalog>
<Account id="1" name="[Guest]" status="Inactive" managedBy="FileMaker" privilegeSet="[Read-Only Access]" emptyPassword="False" changePasswordOnNextLogin="False">
<Account id="3" name="User" status="Active" managedBy="FileMaker" privilegeSet="Data Entry" emptyPassword="True" changePasswordOnNextLogin="False">
<Account id="5" name="FMP_Read" status="Active" managedBy="External" privilegeSet="Read-Only Shared">
</AccountCatalog>
</File>
</FMPReport>
I've added in some extra line breaks, because it's a bit difficult to read with the line wrapping. I wanted to show here that there is a lot of information in each line.
This particular file has only only 3 accounts, but you can see that all of the data that you need that is related to each account is there. Also note, in the actual data but not shown above, there is also a Description node that is below each of the Account nodes, but I've left it out for now. And finally each Account node has a closing tag, so that it's valid XML, which I also haven't shown above.
What path am I on?
There is one critical thing to follow in this, and that's the idea of a Path. XSLT uses the terminology of a path to describe a set of nodes and the direction you go to get from one node to another. It's based on the idea of the starting point being the "root" node, and you move down the path, by going from one to the next. Each subsequent node is contained within the previous one.
So have a look at the above XML from the DDR and consider the path to the first "Account" node. The path in this case is :
/FMPReport/File/AccountCatalog/Account
The "root" node is the FMPReport node, which contains one "File" node, which in turn contains a "AccountCatalog" node, which in turn contains an "Account" node. That is a path that we can use in our XSLT to access this list.
Also note that there are other possible paths, this isn't the only one. For example at the File node, we have a range of possible choices, there are 12 "somethingCatalog" nodes and an Options node, but we can pick either one to go down. Path isn't about the order that the nodes appear inside the parent node, only that one node is contained within another.
Get to the XSLT already!!
So now lets look more specifically at the actual XSLT.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet
version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns="http://www.filemaker.com/fmpxmlresult">
<xsl:template match="/">
snip
</xsl:template>
</xsl:stylesheet>
I've split the first node into multiple lines to make it easier to read. Remember, that XML and XSLT doesn't care about whitespace except where it's to break up one word into another. You can use a space, tab or return, it will still be ok.
So what does all this mean? Well, the first two tags are critical bits of information for the XSLT to work, but for the purposes of this article I'm not going to explain them in detail. Any of the good books on XSLT will start by talking about namespaces and will document these parts. We're keeping it relatively simple and will be only ever working within FMP, so all we need to know is that these two lines are required and will be the first thing we put in any of our XSLT documents.
The third line is the template line, which essentially gives us a starting point for the XSLT to work on. In this case it's starting point is "/", which means start at the root node of the XML we're going to process. The starting point is the path specified by the match attribute :
<xsl:template match="/">
so, it's starting at the root node.
You can't just skip over namespaces and templates!!
Anyone who knows XSLT is probably concerned I'm skipping over these areas. But I'm going to anyway. The purpose of this article isn't to explain all of the details of XSLT and how it works. I'm trying to give you enough information to be able to understand what's happening and to do some basic XSLT of your own. If you really need more information about these areas, it's probably beyond the scope of this article anyway, and you should buy yourself a good XSLT book.
Plus we have the bonus of this sort of template being available, so we can copy and paste these into a new document and get up and working in short order. In all of the XSLT we're going to write to transform the DDR we can just use one of the existing documents as a starting point, and make adjustments as required.
Back to the code.
There is one important thing that it is worth noting here. These tags are named differently from the others we've been working with. They all have "xsl" at the start, followed by a colon and another name. "xsl" is the namespace that the document is using, and I find the best way to think of these nodes is that they are like commands within XSLT. The namespaces are defined in the second node above - the "xsl:stylesheet" node. From there on, everything that starts with this namespace (xsl) is a command, and anything else is data. It will make a little more sense when I expand the XSLT a bit further :
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns="http://www.filemaker.com/fmpxmlresult">
<xsl:template match="/">
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<ERRORCODE>0</ERRORCODE>
<PRODUCT BUILD="" NAME="" VERSION=""/>
<DATABASE DATEFORMAT="M/d/yyyy" LAYOUT="" NAME="" RECORDS="0" TIMEFORMAT="h:mm:ss a"/>
<METADATA>
snip
</METADATA>
<RESULTSET>
snip
</RESULTSET>
</FMPXMLRESULT>
</xsl:template>
</xsl:stylesheet>
If you remember what the data looked like in part 2, then you'll see that starting a few nodes in, is exactly the same set of nodes that is required by FMP to import data in. We've left some of the details blank, like layout, name etc in the database node. ( If this was a format coming out of FMP, not being imported, FMP would include this information for us. )
The difference between these tags and the first two nodes is that they have no namespace. To put it in other terms, these aren't commands, they're data and the XSLT processor is going to just generate these tags in the output data exactly as they appear here. They aren't modified or changed in any way.
Also, although we're indenting these in the XSLT two tab stops further than in the real data format, it won't matter to the import. The indenting and "white space" in XML makes no difference. The only critical thing is the heirarchy of nodes, and we're maintaining that exactly.
Leaving out everything except the METADATA nodes this time, lets expand it one step further.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.filemaker.com/fmpxmlresult">
<xsl:template match="/">
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<METADATA>
<FIELD>
<xsl:attribute name="EMPTYOK">YES</xsl:attribute>
<xsl:attribute name="MAXREPEAT">1</xsl:attribute>
<xsl:attribute name="NAME">id</xsl:attribute>
<xsl:attribute name="TYPE">TEXT</xsl:attribute>
</FIELD>
<FIELD>
<xsl:attribute name="EMPTYOK">YES</xsl:attribute>
<xsl:attribute name="MAXREPEAT">1</xsl:attribute>
<xsl:attribute name="NAME">name</xsl:attribute>
<xsl:attribute name="TYPE">TEXT</xsl:attribute>
</FIELD>
<FIELD>
<xsl:attribute name="EMPTYOK">YES</xsl:attribute>
<xsl:attribute name="MAXREPEAT">1</xsl:attribute>
<xsl:attribute name="NAME">Description</xsl:attribute>
<xsl:attribute name="TYPE">TEXT</xsl:attribute>
</FIELD>
snip
</METADATA>
</FMPXMLRESULT>
</xsl:template>
</xsl:stylesheet>
In this example, we're going to import three fields : id, name and status. We can tell that by the number of FIELD nodes there are in the METADATA node. You will also see here, a new set of four nodes below each FIELD node, that starts with "xsl". Remember, this means in effect that these are "commands" or programming, and aren't sent to the output exactly, they're processed.
Attributes
The xsl:attribute node adds an attribute to a node. So for example this node :
<PRODUCT BUILD="" NAME="" VERSION=""/>
The node is a PRODUCT node. It has 3 attributes, "BUILD", "NAME" and "VERSION". In our example, they all have blank values, but they could just as easily contain real data. The xsl:attribute command tells the XSLT engine to add an attribute to the parent node ( which in this case in the FIELD node ) with the details included. So this command :
<xsl:attribute name="EMPTYOK">YES</xsl:attribute>
Turns this :
<FIELD>
into this :
<FIELD EMPTYOK="YES">
It has added the attribute, given it the name specified, and given it the value specified. So once all four of those attribute commands are done, we're left with this :
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="id" TYPE="TEXT">
</FIELD>
And that is the end result that is generated by our XSLT. We could just as easily have included them in the main tag itself like the example above. Both ways give the same result. There are other places that using the attribute command is used though, so it's a good example to understand how this is working.
Now lets go back to the XSLT and look at the RESULTSET node, leaving out the METADATA nodes this time.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.filemaker.com/fmpxmlresult">
<xsl:template match="/">
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<RESULTSET FOUND="0">
<xsl:for-each select="/FMPReport/File/AccountCatalog/Account">
snip
</xsl:for-each>
</RESULTSET>
</FMPXMLRESULT>
</xsl:template>
</xsl:stylesheet>
It doesn't actually matter in the import if the FOUND attribute on the RESULTSET node has a real value or not. FileMaker will import the data just fine anyway. In an export, it would contain an actual record count.
Loop the loop
And now we learn our next bit of XSLT programming : the "xsl:for-each" command. This is in principle very similar to a loop in ScriptMaker, with one subtle difference. In ScriptMaker we need to explicitly tell the script to go to the next record, and also when to exit the loop. In this xslt command we're being given a set of nodes, and looping through each one, one at a time. Once that is done we stop looping. There isn't any command to go to the next node, or to exit when you've got to the end, it's all built into this loop.
And so the question is : What am I looping through? Well you're looping through a set of nodes. And the set of nodes that it's looping through is specified in the "select" attribute of the for-each command. You may notice that this matches the "Path" I mentioned above. It's a way of saying, loop through every Account node in the file you find at this path, and for each one, do something.
This concept of the for-each loop and the paths to get to them is the critical learning step in understanding the DDR and XSLT. I'll get back to it in a moment, but first I'll show you what exactly the loop does. Lets expand that node further :
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns="http://www.filemaker.com/fmpxmlresult">
<xsl:template match="/">
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<RESULTSET FOUND="0">
<xsl:for-each select="/FMPReport/File/AccountCatalog/Account">
<ROW MODID="0" RECORDID="0">
<COL>
<DATA>
<xsl:value-of select="@id"/>
</DATA>
</COL>
<COL>
<DATA>
<xsl:value-of select="@name"/>
</DATA>
</COL>
<COL>
<DATA>
<xsl:value-of select="@status"/>
</DATA>
</COL>
</ROW>
</xsl:for-each>
</RESULTSET>
</FMPXMLRESULT>
</xsl:template>
</xsl:stylesheet>
The first thing in our "for-each" loop is a "ROW" node. In other words, for each Account node in the DDR, generate a ROW. Back in part 2, you will have seen that a ROW is equivalent in FileMaker to a record. So this will have one ROW for each Account, which will get transformed into an Account record once imported.
This is the biggest trick to understand in our processing of the DDR. You find a list of nodes you want to turn into records, and use a loop ( in xslt it's a for-each loop ) to generate a list of output data records.
Each ROW is a Record.
The ROW node has a couple of attributes. Again, we put dummy values in them, as we're importing not exporting. And finally we get to the part where we do all the work. Each ROW has 3 COL nodes - one for each FIELD in the metadata above.
It's important here that you make sure that things match up. For example you should have as many columns as you have setup in the METADATA section, and that the order matches. So COL # 2 in the list is the right data for FIELD # 2 in the metadata.
And we also get to the other critical part of the programming in XLST which is the xsl:value-of command. This command basically says to get this specific value and return it. More on this in a bit.
Back to the overview.
You can see in this that there are parts of our XSLT which is just like the XML that we're wanting to import. There are other parts that are commands, that manipulate the data. Anything that isn't in a namespace is exported exactly as it appears. The xsl namespace allows us to perform commands so we can loop through items, generating the data we need. As long as you can understand the two sections of the XSLT and why they're there, you're well on the way to writing your own XSLT and utilising the DDR.
Context
There is a concept in XSLT which is very similar to the idea of context in FileMaker. In FileMaker, when you're on a layout with a specific base table occurrence, the fields from that table are only one step away. Anything else requires a relationship, and will be referenced via it's related table. In XSLT, the concept is the same. When I'm inside this for-each loop, the context is the node that you specified in the "select" attribute. In this case the select attribute was "/FMPReport/File/AccountCatalog/Account", so everything from here on is relative to that node.
The select attribute.
The xsl:value-of command returns to the output some text from the DDR. We tell it which text we want by setting the select attribute. In our case above, we've chosen to select "@status". This will output a value into the output XML of whatever we want, and is how we move the data from input (DDR XML) to output (import data).
The select attribute has a whole bunch of options for how to specify the information you want to get. There are ways of looking at other nodes relative to this one, going either back or forward in the hierarchy. There are ways of getting attributes or whole blocks of text, or even whole nodes and their sub nodes. There are conditional statements, similar to IF or CASE in FileMaker, and there are functions you can perform to modify the data you're getting before you send it to the output.
In processing the DDR, there are 3 things we're going to need to do more than anything to get the data we're after :
- Get the value of an attribute of a node.
- Change context - move from one node to another.
- Get the value of an entire node.
Getting Attributes
To get an attribute value, we use the "@" symbol. So to get the id attribute of the Account node, we use "@id". So our select statment is :
<xsl:value-of select="@id"/>
This is exactly what we're doing in the example above. Our context ( the starting point for this value-of command ) is the Account node. And the Account node has a bunch of attributes, one of which is named "id". So to return the value of the id attribute in the context of the Account node, we just use "@id".
Moving around a Path
Remember that this is all context dependent. We're only able to specify this attribute directly because our previous for-each loop put us at this context. To get to the id attribute if you started from the root node (from the start, without our loop) we'd have to specify :
<xsl:value-of select=""/>
By starting with the a leading "/" we've effectively said, go back to the start. You should be able to see here that we've used a path in the select attribute in the same way we did in the select attribute of the for-each loop. What works in one select will work in another, if the nodes exist.
You can move from one point to another by using a path within the select statement. In the example I showed above, the account node doesn't show any sub nodes. But in fact it does have one in a real DDR document. Each account has a Description node like so :
<Account id="1" name="[Guest]">
<Description>Your account description is here.</Description>
</Account>
Getting Text
Because the Description node is a node within the account node, it's in a different context to the current "Account" context. So we need to move to that new node to get the data. And when we're there, we need to get the entire value of any text inside that node. To do that we use a function :
text()
So our select statement for getting the description of an account is this :
<xsl:value-of select="Description/text()"/>
This says "move from the current context to the child Destination node, and get all of the text within that node." The context being the Account, so it selects the Description node within the Account node.
The logic for paths works in much the same way as a path in any command line editor, and it follows much of the same convention. "." is the current node, ".." is the previous ( parent ) node. We follow a path by going "path/to/destination/node/".
Recap Again
So once you've found the starting point you want to work frrom, you find a set of nodes by doing a for-each loop for the nodes you need to export, and generate a ROW in the output data for each record. You then use @ and text() to get the real data, in conjunction with some different paths for accessing child or parent nodes.
There are lots more options for paths and the sorts of things you can use to locate information, if you need to know lots more, I'd suggest a book, I've found the O'Reilly ones to be generally good. Everything we need to get out of the DDR we can get with just those two options ("@" for getting attributes, and text() for getting whole nodes), and the knowledge of how paths work.
Recap
There are some concepts here worth touching on again. First, that the XSLT file is actually XML as well, so it follows the same conventions and has the same requirements that the XML files do.
Secondly, there are two distinct parts to the XSLT, regular nodes which get output exactly as they appear in the XSLT and commands, which follow the xsl:commandname syntax. The commands aren't sent to the output file, they're processed and the result, if any, is sent to the output.
The third and final thing that we use everywhere in processing the DDR is the for-each loop and various select attributes. The for-each loop allows us to work through lists of information one at a time, and the select specifies which information to get. In the case of the loop, the select tells us which node to process, and when used in a value-of node, it tells us which data to return.
So what other parts of the DDR can I get?
In short, all of it. If you take the example code above, or my example file attached to this page, you can modify it to grab whatever data you want. If you modify this code to work somewhere else, follow a few critical steps :
- First, modify the METADATA node to have one FIELD node for each field you want to bring in. Feel free to move the attributes inside the node if you want, instead of having them as separate commands.
- Second, make sure that for every FIELD node you have setup in the METADATA, that you have a matching COL node, which corresponds to the data you want to bring in. This is critical, and it's easy to add an extra COL node and forget to add the FIELD, or to get the order wrong. Everything will probably still work if you get the order wrong, but your data will be in the wrong places.
- Finally, add a for-each loop to process the list of nodes you want to get out. You can also have multiple for-each nodes, if you want to process data in multiple places at once. Each loop would generate the same output but for a different part of the DDR.
Pop Quiz
So, can you tell what each of these for-each nodes would process :
<xsl:for-each select="/FMPReport/File/ScriptCatalog/Script">
<xsl:for-each select="/FMPReport/File/LayoutCatalog/Layout">
<xsl:for-each select="/FMPReport/File/ValueListCatalog/ValueList">
Or what about this one that has a for-each inside another for-each :
<xsl:for-each select="/FMPReport/File/BaseTableCatalog/BaseTable">
<xsl:for-each select="FieldCatalog/Field">
</xsl:for-each>
</xsl:for-each>
Is it all as straightforward as this? (Said with tongue firmly in cheek.)
Ahh.. no... But this does give you some idea of the way that XSLT works and how to get data out of the DDR. You could from these examples create some XSLT to generate a list of scripts, or layouts or fields. If you're after some examples of more complex XSLT for getting at some of the details of the DDR, I suggest you download a copy of BaseElements. Included in your licence to BaseElements is a licence to use the XSLT DDR files that we're using to import all of our data. These examples cover every element in a FileMaker solution and use a few tricks to grab all of the data from every nook and cranny of the DDR.
I've attached to this post two of the more simple XSLT files that BaseElements uses. Feel free to use either of these files as they are or as templates for other parts of the DDR or other XSLT projects in FMP.
Can you explain xyz in your XSLT files?
Sure, post a comment below, or contact me directly and I'll do my best to answer whatever I can.
Haven't you just given away the keys to BaseElements?
No, not quite. BaseElements has been in development for over 2 years, and 90% of the work involves working in FMP creating the interface and making sure all of the data is related to all of the other data. So just having access to the XSLT doesn't mean you can create BaseElements any time soon.
Plus I think this is something that is helpful in other ways, not just related to BaseElements. I think there should be more open standards like this from FMI. I'd like to see them document the DDR properly for each new release, and also either endorse or generate their own set of XSLT files that you can use. This would make things better for everyone, not just Goya.
And if it means that others start to look at the same issues and come up with innovate ways to do things, then it helps everyone, not just Goya.
And we've got a lot of cool plans and ideas for BaseElements, so it's not just about the XSLT. As they say, it's not what you've got, but what you do with it that counts.
I hope you found this interesting, and if you have any more queries about XSLT or the DDR, feel free to contact us, or leave a comment on this page.
| Attachment | Size |
|---|---|
| Account.xslt | 3.66 KB |
| Layout.xslt | 4.04 KB |
Understanding and using the Database Design Report - Part 2
Submitted by Nicholas Orr on 17 May 2007 - 9:31pmIn part 1 of this series, I described the XML format of the Database Design Report, or DDR. If you haven't read that yet, go back and have another look as this section will assume you've read that and understand the basics of the XML format. If you understand the concepts of nodes and heirarchy of nodes in the DDR, then you're ready to move to the next step.
Why can't I just import the DDR directly into FileMaker?
XML just doesn't work that way. What XML does instead is allow every single application that supports XML to do so in their own special way - but all still as a standard XML document - and then add a method to move from one XML document to another very simply. So even though each format is different, each one is following certain rules and is consistent, so that I can move from any one format to any other format with relative ease. Once I've programmed a step from one format to another, I can apply that process many times for different data sets and it will always work.
XML is actually used in lots of different places, for example you can go directly from a FileMaker export to PDF, or to an SVG graphic. Or to RTF documents, or many other database formats. All from within FileMaker directly. This is the benefit of XML.
The process to move from one XML format to another is called XSLT, but we won't look at that until part 3.
What does FileMaker's XML format look like?
This is easy to answer. Take any filemaker file you've got with a few records and a few fields, and export it to XML with no XSLT file selected, then open the file in a text editor.
I'm going to go through this in much the same way as I did before, by opening the XML out a few steps at a time.
Take a lok at the code below. First of all we can see that there is a inital comment line. This is actually called the "XML Declaration" but for our purposes for the moment we can ignore it.
Then the second line is the start of the data, and contains the first node ( called the root node ).
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
snip
</FMPXMLRESULT>
The root node, in this case contains one attribute with the name of xmlns. This is an XML namespace, and although it's important, it's beyond the scope of what we're doing here. For our purposes we'll just leave these things intact. One of the big benefits of this all being in text format documents is that you can take someone else's starting point and just modify it to suit your needs and continue on your merry way.
At first glance, and with only a small amount of the data showing, this appears similar to the DDR. Opening it up one step further we get :
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<ERRORCODE>0</ERRORCODE>
<PRODUCT BUILD="" NAME="" VERSION="" />
<DATABASE DATEFORMAT="M/d/yyyy" LAYOUT="" NAME="" RECORDS="23" TIMEFORMAT="h:mm:ss a" />
snip
</FMPXMLRESULT>
So you can see all of these three new nodes are either self closing nodes, or like the first ERRORCODE node, closed already on the same line. They are just all places holders for information about the file itself. This section includes information about what version of FileMaker the file was created with, layout data and even date and time formats. Of course, you may not need this extra information, so it may be ignored, but regardless, FileMaker will generate it when you export from FileMaker, and will expect it when you import into FileMaker.
Expanding again, one step further we see the most important nodes :
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<ERRORCODE>0</ERRORCODE>
<PRODUCT BUILD="" NAME="" VERSION="" />
<DATABASE DATEFORMAT="M/d/yyyy" LAYOUT="" NAME="" RECORDS="23" TIMEFORMAT="h:mm:ss a" />
<METADATA>
snip
</METADATA>
<RESULTSET FOUND="6">
snip
</RESULTSET>
</FMPXMLRESULT>
The METADATA and RESULTSET nodes are the critical part of the XML format. METADATA is a term used in other places outside the FileMaker XML format. Metadata means literally "data about data". In the case of FileMaker and the XML it's a node set that describes the data that is going to be exported. The RESULTSET contains the actual data itself.
So lets expand the METADATA node one step further. This time I'll leave out all of the preceding and RESULTSET nodes so we can concentrate on the METADATA parts.
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<METADATA>
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Last" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="First" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Group" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Score" TYPE="TEXT" />
</METADATA>
</FMPXMLRESULT>
So does the comment about meta data being "data about data" now make sense? This is a set of nodes to describe what the actual exported data is. It's data that describes the data.
This node set tells us all of the fields that are in the export or import, all of their field types, some validation rules and repetition information - yes, the XML format understands repetitions, but just because they're there doesn't mean you should use them!
So now lets expand the RESULTSET node.
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<METADATA>
snip
</METADATA>
<RESULTSET FOUND="6">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
</RESULTSET>
</FMPXMLRESULT>
I've left out from this example all of the closing ROW nodes, but assume everyone has a closing node because other wise it wouldn't be valid XML.
Two things to note here, the first is that the FOUND attribute in the RESULTSET node matches the number of ROW lines inside the RESULTSET node. The second is that the MODID and RECORDID values are all empty. This is because the data being generated is for importing into FileMaker not exported out of FileMaker. If it was an export, there would be real values in there. The MODID is a count of the number of times the record has been modified, the RECORDID is the internal record id of the records.
RECORDID is different from any field auto-enter serial id you generate within your data set. It's an internal id that you can retreive from FileMaker using a Get(RecordID) function in a calculation.
Because we're going to import this data into FileMaker these values don't need to be included, and I'm showing them here more for reference. FileMaker will generate it's own RECORDID's for each record, we can't set them, and the MODID will be set to zero, as they're all new records.
So, now lets expand one of the ROW nodes.
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<METADATA>
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Last" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="First" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Group" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Score" TYPE="TEXT" />
</METADATA>
<RESULTSET FOUND="6">
<ROW MODID="0" RECORDID="0">
<COL>
<COL>
<COL>
<COL>
</ROW>
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
</RESULTSET>
</FMPXMLRESULT>
Again, assume there's a closing COL node for each of the COL nodes there.
This time, the thing to note, and the reason I put back all of the METADATA is that there is a COL node for each FIELD node in the METADATA. What this part of the XML is showing is the actual data. There is one column for each FIELD.
As an aside, the terminology is a bit off from what you'd expect in FileMaker. We're used to fields and records, but this is showing ROWs and COLumns. Who knows why they chose this terminology, but the concept is the same. So, lets expand the data one step further.
<?xml version="1.0"?>
<FMPXMLRESULT xmlns="http://www.filemaker.com/fmpxmlresult">
<METADATA>
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Last" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="First" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Group" TYPE="TEXT" />
<FIELD EMPTYOK="YES" MAXREPEAT="1" NAME="Score" TYPE="TEXT" />
</METADATA>
<RESULTSET FOUND="6">
<ROW MODID="0" RECORDID="0">
<COL>
<DATA>White</DATA>
<COL>
<COL>
<DATA>Mark</DATA>
<COL>
<COL>
<DATA>Red</DATA>
<COL>
<COL>
<DATA>100</DATA>
<COL>
</ROW>
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
<ROW MODID="0" RECORDID="0">
</RESULTSET>
</FMPXMLRESULT>
There's a lot of information there, but you can probably get the general idea. If you kept expanding the other ROW nodes, you'd find each one of those contains four COL nodes, each with it's own DATA node containing the relevant data.
We have a result set.
Each result set contains as many rows as there are records in the data.
Each row ( or record ) contains as many columns as there are fields in the metadata ( the header row ).
And then each column contains data.
It may seem like a really convoluted way of containing export data. And it is. In short : that's XML.
XML's advantage is that it's meant to be self describing. So this format is a way of generating a document in such a way that you wouldn't have to know anything about the origin of the data. The XML itself contains everything that is needed to document the data. All of the columns and rows are ways of describing the real data that the XML document contains.
So how do I use this information?
To use this XML format, you're either going to be exporting data from FileMaker, or importing it into FileMaker ( the CWP interface also uses XML, but that's a whole other set of articles ).
If you're exporting, this is the format that the data is going to be in. If you want to do something with it, you need to be able to read this format, and extract the data from within it.
If you're importing into FileMaker, you're going to have to generate this XML format as your data.
Fortunately there is something that can not only read this format and transform it into other formats, or can read another format, and transform the other format into this one. That thing is called XSLT.
Am I going to have to remember all of this?
Not really no. It's good to understand all of this, but you wont' have to know this by heart. What you'll more likely do is setup one XSLT document once that contains all of the parts you need, and then we'll just duplicate the various parts. At the end of this series I'll provide some templates that do just that, and you can modify and adapt these to your own purposes.
So what is XSLT?
XLST documents are actually also XML documents, with special commands in them to work with other XML. XSLT is said to "transform" one document to another. We use it in FileMaker to either generate the format above or to take the data out of the format above and produce something else from it.
To explain how this fits into the process in FileMaker, go back into your copy of FileMaker and do an XML import or export. You'll notice that there is an option for XSLT processing. Select that option and you get to select a document that is your xslt document to transform the XML you're importing.
Imports first.
In an import, if you're NOT using any XSLT, it's going to expect the xml file you choose to match FileMaker's required setup for XML import. If you're using XLST, then the file you import won't be in FileMaker's format, it will be in some other format, and the XSLT will take that format and generate a new document which FileMaker will understand, and it will import this new document. You never see the intermediate document, it's all just part of the import processing.
And exports.
In an export, if you're NOT using XSLT, the output file will match the details above. If you ARE using XSLT, then the output isn't XML at all, but will be whatever your XSLT generates. This may be an SVG image, or a PDF or RTF document, or any other format. Again, you don't get to see the intermediate "FileMaker format" xml file, you'll only see the end result.
Back to the DDR
Getting back now to where we started. We know what the DDR contains, we can get that just by looking through a sample document and looking at all of the node structure and contents. It would be really nice if FileMaker had documented the DDR somewhere in advance, but it's not too difficult to figure it out once you've written a complex enough FileMaker data file.
And we now know what we need to be importing into FileMaker. This is the document format described above. So we've just got to write a suitable XSLT file that does the step of going from one to the other. It's actually not that difficult.
And the other purpose I had for showing you all of this XML and how it works and how to use it is that the XSLT we're going to write is actually an XML file itself. So it's better to understand about XML and how it works within FileMaker to make things easier to create and write the XSLT.
Next
In the next and final article I'll show how to write XSLT, and give some examples of how to get real data out of the DDR using XSLT.
Part 3 is now available here.
Understanding and using the Database Design Report - Part 1
Submitted by Nicholas Orr on 25 April 2007 - 9:43pmIn developing our BaseElements product we had to develop an intimate knowledge of the Database Design Report ( DDR for short ) that you can produce from FileMaker Pro Advanced. However BaseElements isn't the only reason you might want to know what the DDR contains or how to interpret it. This is the first in a series of articles about what the DDR is and how to use it. We hope you find it useful.
What is the DDR?
First, you're going to need a copy of FileMaker Pro Advanced to even generate the DDR. It's one of the options in the "Tools" menu in FMPA, and there are 2 options for the type of report : HTML or XML.
In an of itself, the XML report is much harder to read, and isn't really intended to be read directly, like the HTML version is. However the HTML version has some serious limitations. To start with, it's not cross referenced across multiple files. And more importantly, there isn't any built in searching functions other than searching the HTML output via your browser.
So there isn't a way to, for example, list all of the fields of a certain type, you'd just have to go through the fields one by one.
The obvious answer to this problem is to put the data into a database, so we can search it, but we need to have something we can import first.
The XML output is actually in a format we can use. XML is a generic data storage format, and it comes with another tool - XSLT - that is used to transform XML amongst different formats. But before we get to that point, lets take a look at some XML from the DDR so we can understand what we're doing.
What does the XML look like?
What I've done here, is to take the XML version of the DDR, and put it into a text editor that allows me to hide and un-hide parts of the report (hidden parts I've replaced with a snip ). This lets me view sections of the data at a time. To start with, here is a view of the DDR collapsed down to just 4 lines :
<?xml version="1.0" encoding="UTF-16"?>
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
snip
</FMPReport>
There are some bits in that I've taken out (the snip part, but there's also others), but essentially it's as per the XML.
The first line of the XML is a comment line. It's differentiated by the <?xml at the start, and is a purely information link about what sort of XML document this is. In using the DDR we can ignore this line.
First of all, anyone who's familiar with HTML will see it uses a familiar notation. The text contains tags - which is the < and > brackets. Every < must be followed by a > somewhere, otherwise the XML won't be valid. Within each tag - between the brackets - is the data. The first word after the < character is the tag's name. As you can see from the example above, there are 2 FMPReport tags. One is an opening tag, and the other is the closing tag. The closing tag is distinguished by the / before the name.
As a side note there are also other ways to have a complete open and closed tag, and that is to use a single self closing tag. This would look like this :
<NotEmpty value="False"/>
The difference is just the position of the / character.
In XML terms, a complete opening and closing tag set, and everything inside it, is called a "node". This is the first important thing to learn about the XML format. A node may be a single line by itself, or it may contain other nodes. Either way, the nodes contain all of the information we need to know about the solution.
Other than the tag name, the other information contained inside the tag are called attributes. Some examples from the code above :
link="Summary.xml"
or
value="False"
In this case, the attributes are stored as what's called name value pairs. The attribute has a name, "link", and a value "Summary.xml". A tag can have either none, one or many attributes.
So, to review the XML DDR so far, we have :
<?xml version="1.0" encoding="UTF-16"?>
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
snip
</FMPReport>
The first line is a comment.
The second line is an opening FMPReport tag, that also contains some other attribute values.
The third line has been cut for now, but is the rest of the data.
The fourth line is the closing FMPReport tag, that finishes the FMPReport area of the XML.
All of the XML we need to understand is made up of only these elements. Tags containing either attributes or actual data, which are nested inside other tags, themselves containing other data or attributes.
So how does the XML contain information about my solution?
To better understand what the DDR contains, I've indented each of the lines in the XML examples. This is actually very similar to what is actually going on in the XML. So in the example below, the "File" node is contained inside the FMPReport node :
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
snip
</File>
</FMPReport>
So you can see that the FMPReport node contains another node : this one called "File". The File node has some useful attributes, such as name and path that we can see. There is a opening and closing node again so that the File node can also contain other data and nodes.
So now we look one level deeper into the XML :
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
<BaseTableCatalog>
<RelationshipGraph>
<LayoutCatalog>
<ValueListCatalog>
<ScriptCatalog>
<AccountCatalog>
<PrivilegesCatalog>
<ExtendedPrivilegeCatalog>
<CustomFunctionCatalog>
<FileReferenceCatalog>
<CustomMenuSetCatalog>
<CustomMenuCatalog>
<Options>
</File>
</FMPReport>
And I've left some details out here, so I hope I didn't confuse everyone. Assume in the example above that every line has a matching closing tag like :
<BaseTableCatalog>
snip
</BaseTableCatalog>
which I've left out to shorten the code a little.
Hopefully some of these node names will be familiar to FileMaker developers. BaseTables, Relationships, Layouts, Scripts and more.
If you think once more about the indenting of the nodes, and what a node contains, then you can see that the FMPReport node contains one and only one File node. And a File node contains a list of other nodes, most ending in "Catalog" as well as an Options node. So what, for example does a BaseTableCatalog node contain? : a list of BaseTables. So lets expand that node, leaving out the others this time, and have another look :
From here on, I'm going to leave out the closing tag, and the "snip" lines for the sake of clarity, so you'll have to assume they're there. Obviously there are in a real data file.
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
<BaseTableCatalog>
<BaseTable id="32797" name="Globals" records="0">
<BaseTable id="32803" name="Plugins" records="1">
<BaseTable id="32813" name="XSLT" records="1">
<BaseTable id="32814" name="Graphics" records="1">
</BaseTableCatalog>
</File>
</FMPReport>
So you can see that each BaseTableCatalog node contains a list of BaseTable nodes. And each BaseTable also contains enough information for us to identify the BaseTable, namely an ID and a name, as well as a record count. So what happens if we expand the BaseTable node in our example :
<FMPReport link="Summary.xml" type="Report" version="8.5v1">
<File name="CCP_Worksheet" path="192.168.20.4">
<BaseTableCatalog>
<BaseTable id="32797" name="Globals" records="0">
<FieldCatalog>
<Field id="3" name="Back Record ID" dataType="Number" fieldType="Normal">
<Field id="14" name="BackCounter" dataType="Number" fieldType="Normal">
<Field id="28" name="cRegistration String" dataType="Text" fieldType="Calculated">
<FieldCatalog>
</BaseTable>
</BaseTableCatalog>
</File>
</FMPReport>
So a BaseTable node contains a FieldCatalog node, which contains a list of Field nodes. Now you can see we're building up some real information about our FileMaker file. If you go to the next step, and expand the Field node, you'll see it contains all of the information about the field. In that case, it's things like field type, calculations, storage, validation, comments etc. Some example nodes from a real DDR field node :
<Comment></Comment>
<AutoEnter value="CreationDate" lookup="False" constant="False" furigana="False" calculation="False" allowEditing="True">
<ConstantData></ConstantData>
</AutoEnter>
<Validation type="OnlyDuringDataEntry" message="False" maxLength="False" valuelist="False" calculation="False" alwaysValidateCalculation="False">
<NotEmpty value="False"/>
<Unique value="False"/>
<Existing value="False"/>
<StrictValidation value="False"/>
</Validation>
<Storage index="None" global="False" autoIndex="True" indexLanguage="English" maxRepetition="1"/>
You can see there, that there is enough information to completely recreate this field in a solution. This particular field has an Auto Enter creation date, isn't being stored and isn't a global. There is a lot of extra information that appears as "false" and obviously because this isn't a calculation, we don't see any of that detail.
All of the rest of the "Catalog" nodes listed above work in the same way. For example the LayoutCatalog node contains a List of Layouts. Each Layout node contains all of the Object nodes to describe the objects in them. Or a ScriptCatalog name contains a list of Script nodes, which contains a list of ScriptStep nodes, etc etc.
There is one other part of the XML DDR I'd like to mention, these are the CDATA sections. They appear as the following :
<Calculation table="Table 1">
<![CDATA[Text1 & Number1 & Custom 1 & Trim( Text1 ) & CF_myCF]]>
</Calculation>
Essentially the CDATA is a way of having a data section that contains complex text. For example, there is a high likelyhood that some of your calculations use the < or > symbols. If they didn't appear in a CDATA section, then this would be interpreted by the XML reader as part of a tag and our data couldn't be imported. So the CDATA section can contain these characters without breaking the XML. It's bit like the way you have to escape a \ in a filemaker calculation by using \\ or a " by doing \".
I won't continue further with these examples, but if you understand the basics above you can probably follow along in any DDR file you generate. What I would suggest is that you take one of the DDR files and run it through a text editor that indents XML properly. BBEdit and TextMate on the mac have just such functions, as do any other editors that implement the unix "tidy" command, as it makes the XML much easier to read and follow.
When you're looking at the XML, just think in terms of the hierarchy and structure of the file. Each node contains other nodes, which in turn contain either data, or other nodes. Some nodes are purely there so that we can read all of the attributes, other nodes like the
Where to from here?
The most important thing to know at this point is that you don't have to know the entire DDR XML by heart in order to make use of it. In terms of us working with the data, all you need to understand are the concepts of tags and nodes and the way the nodes are structured in a hierarchy with some nodes contained inside others. When you're working with the DDR, we'll be tracing a path through that XML from one node to another, but once you figure out what the path is, you don't need to remember it. And you won't need to read the XML directly, we have another tool called XSLT that does that for us.
So the next issue we need to tackle is getting this information into a FileMaker database so we can work with it. Can FileMaker import XML? Yes, it can. Can FileMaker import the DDR directly? No. If you're thinking that's a huge oversight, it's not. It's a function of the way XML works. Every app has it's own XML format that it understands, and for FileMaker, the DDR isn't it.
In part 2, I'll look at what FileMaker can actually import and how you'd setup a file specifically for that. And then in part 3, I'll demonstrate some XSLT that does the magic of going from one format to another.
Part 2 is now available here.