feature: add field

[cpetz]

Feature Suggestion/Request

Support for inserting/adding Field Codes in a word document. They are a handy feature for report generation type applications (originally intended for automatic mailout merges I believe).

In Office, they make it easy to add dynamic features to a document without getting your fingers all slimey with macros/VBA (although, if designed properly they are accessible from VBA using custom DocProperties and clever references). You work with them using text "markups" that form a restricted scripting framework (hit ctrl+F9 to get started within Word).

They can easily work with DocProperties, named elements (tables, lists, headings), and external text documents that will drive the dynamic content. You can still use styles to drive a document, but with field codes you can adjust/apply syles conditionally.But field codes are admittedly ackward to work with (odd syntax, updating, poor UI tools). Thats where python-docx needs to come in.

I think with a little love, working with field codes could actually be neat, organized, readable, and very functional. They would slip in just lile your other elements...

document.add_fcode("ASK", "chap1_caption", "Type in caption for Chaper 1")

Nesting field codes arbitrarily would be important requirement.

Instead of making custom routines in python-docx that help a user hack together structured portions of a document or specific output patterns, let us use field codes to define those structures expliciltly. And then when we export back into an Office driven workflow, all of the glue is still intact and fully functional.

The main weakness of field codes is that they are typically hidden/disabled by default in Word, but in that regard, its not much different than shipping a document with embedded macros: it is understood that you know how to interact with the extended features.

[cpetz]

Ive notice youve not given up on the idea of field codes, and I was wondering if there were any particular reasons they have been getting pushed back, and if there is anything I can do to help.

It seems like the XML for some of the simpler field codes, is..., quite simple. I imagine some of the more advanced codes get incrementally more difficult, especially when used recursively.

< w : fldSimple w:instr=" DOCPROPERTY "customer_name" * MERGEFORMAT " >
(code woudlnt show correct in comment, so I had to pollute it with a few whitespaces.)

And to be clear, my feature request was NOT FOR YOU to evaluate these codes, only help me inject them. (Look how cumbersome they are to type, honestly, its easier in SML, because you have to wrap them in a special *hidden character * when your in Word.

Help me strategically place these in my Word docs and you will have made me smile very large. I have no intention of you attempting to go through and evaluate them for me when I save it or something. If there is any grunt work regarding field codes that is slowing you down, tell me, and I will see if I help absorb some of it.

[scanny]

@cpetz The first step is to understand the feature well enough to be able to design the series of steps to implement it fully. That means documenting an analysis of the features and the XML representation well enough to ground those design choices. You can see examples of analyses for other features here: http://python-docx.readthedocs.org/en/latest/dev/analysis/index.html, the restructureText source is in the source code under the /docs/dev/analysis/features directory.

There are several places to look for the inputs to that analysis; the behavior of the Word app itself, the Open XML spec, the XML schema is particularly important, and the MS API are toward the top of that list.

This particular feature is tricky in the general case because it can involve a fairly arbitrary range of text, either block elements, inline elements, or both; as the body of the field. The MS API seems to use a Range object to specify this. There is no counterpart to the Range object yet in python-pptx, so it's not clear yet how fields would be implemented in the general case.

If you want to move it along the analysis is the place to start. Once that gets far enough along to provide a solid foundation for an API proposal (sometimes called a Candidate Protocol in the analysis pages), then things can move toward implementation. Seems like the analysis pages usually take me about an hour. This one might take a little longer because of the complexities. I've kind of developed a sense for where to look for things so it might take someone beginning a bit longer.

[cpetz]

cant promise anything, but I will do some digging and see if I come up with anything worthwhile. thanks for direction

[scanny]

One idea that occurred to me after reflecting on this a bit is that "complex" fields might be made a lot easier to insert if the "boundary" begin and end elements were inserted at the same time as the bits that they bounded. I'm not sure how that would work, and it might not work in all cases, but it might be a way to add capability beyond what could be done with a <w:fldSimple> element. Just a thought as you're poking around. This is one of the things the XML Specimens part of the analysis page is good for judging :)

[cpetz]

@scanny One question, and a few followup thoughts (I have not gotten into Open XML, merely digging through MSDN for ~official docs):

Are we looking for COMPLETE support for ALL field codes, or is it acceptable to only work with a subset? (Many Field types have been replace with more modern tools, are probably redundant several times over, and could be excessively complex/undocumented)

---

It seems that many simple Field types can be represented with basic XML tags:

(from Word2007 XML)
<w:fldXXX attr1="value1" attr2="document relative text"></w:fldXXX>

( According to 2003 MSDN ).

I don't believe many Fields need to utilize a Range, and even from VBA the Range for these simple cases seems to be the current insertion point. These would be ideal starting candidates for analysis and implementation.

Following the previous, glancing at the MSDN link,

• note the attribute instr. That value is the actual CODE that Word will execute. This is done with raw text (in the simple, non-embedded cases). This is perhaps a good starting place to draw the line of involvement that python-docx will undertake, leaving the rest to a properly formatted user string (which may or may not work when it gets to Word, python-docx might not care)
• If I am not mistaken, child elements can be excluded as far as python-docx is concerned. These would become important if we wanted to read an existing Field value, or after running document wide updates, checking for live calculated values.

There is some to be learned from the VBA handling of Fields, but honestly, I don't want to read VBA. If I get time, I might see if Visual Studio will give me anything useful if I go around inspecting certain Field-related Objects.

[cpetz]

Also have a look at the field special character. I believe with this element, you would run it inline like other text, and may be able to stay out of XML schema (if I am understanding its usage correctly), as if I were manually typing everything through Word. This gives easy access to the special bracket character that Fields are defined within.

I will test this further to be sure, but it could mean that python-docx provides little more than a string templating interface that helps handle some of the more ackward/repetative syntax. Leaving all other functional responsibilty on the user (poorly written field code tends to fail silently in many simple cases).

[scanny]

It would be okay to start with a subset. The implementation would need to be complete enough to be useful to some folks and provide a foundation for future implementation. It wouldn't necessarily have to implement the general case, but its API should be consistent with that for the general case. In other words, the API shouldn't need to change to accommodate the general case. Needing to extend the API would be okay though.

Be careful around Word 2003 and by extension MSDN 2003. That version of XML for Word is completely different from Word 2007 and later. The links you provided are for that old WordML XML dialect.

The XML Schema is really the critical link for the analysis. You can find it in the code base here:
https://github.com/python-openxml/python-docx/blob/master/ref/xsd/wml.xsd#L1231

I usually load it up in my editor so I can use search to move back and forth. The other critical tool is opc-diag to examine the XML Word produces for simple examples and minor changes. Those end up in the XML Specimens part of the analysis page and basically demonstrate what you're trying to achieve with a feature.