Read text inside <w:sdt> tag

[SandeR2012]

Is it possible read text inside w:sdt tag and replace it with Run or remove completely?

[scanny]

There is no support in the API just yet for structured document tags, or "content controls" as I believe they show up in the Word UI.

Can you say a little about your use case so we have something concrete to go on?

I think we might be able to help you whip up a quick function that uses the python-docx internals to get what you need.

We'll eventually get to adding this, I've had my eye on it for a while, but you're actually the first one to show up with a use case, so I'm keen to know more so we can better work out what kind of support we should actually build for these.

[SandeR2012]

User provides a template with placeholders that I change to the text or table or remove otherwise. Some of them created using the <w:sdt>, it is within a paragraph and text inside is unreadable.
paragraph: https://gist.github.com/SandeR2012/f5e7a56bc27374356d35
table row: https://gist.github.com/SandeR2012/1285facb32fbdf477ea4

[scanny]

If you're willing to dig into the internals a bit and use the underlying lxml interface, you can probably get this done.

Once you have a paragraph, you can get access to any <w:sdt> elements inside it using something like this:

paragraph = ... # however you get the paragraph, maybe with `for paragraph in document.paragraphs`
p = paragraph._element
sdts = p.xpath('w:sdt')
for sdt in sdts:
    print('found an sdt element')

You can extend the XPath expression to get more detailed items, like `w:sdt/w:sdtPr/w:alias/@w:val'. And when you have an element you can use the lxml API for its _Element object to get its text, attributes, etc. The lxml API is documented here: http://lxml.de/api/index.html, although you'll need to click through a few links to get to the lxml.etree._Element class. This is the direct link: http://lxml.de/api/lxml.etree._Element-class.html, but it's designed in an old-style multi-pane web page so if you want to navigate around the first link is probably better. The lxml tutorial is pretty good too.

You can delete an sdt element like this, once you've found it.

parent = sdt.getparent()
parent.remove(sdt)

So one approach might be to loop through the paragraphs, look for ones with an sdt element inside, delete the element, and then add text to the paragraph.

Let us know how you go if you decide to pursue that, would love to see the code you come up with as it will likely be of help to someone else while we're waiting for the API feature to arrive.

[SandeR2012]

I'll try it tomorrow and I will write about the result

[chaudhary238]

Hi Steve, i want to extract the text from content control ( w:sdt) but was not able to extract it. I have tried your suggested method but its not giving me any "sdt element within paragraph but if i am able to extract the "sdt" element if i am using xpath on document._element.xpath("//w:sdt"). Also i cant delete the "sdt" as i need to keep the formatting of the word document same. Can you please suggest any method where i can extract the "sdt" text do some translation of text and put it back on same location. My "sdt" has table inside so i need to process them as well.

[flutefreak7]

I definitely have a use case and was disappointed to see that this isn't well supported already. For clean, robust reading of content controls I was forced to use the COM API which is significantly slower.

I've got hundreds of files on a SharePoint server whose controls have been populated. I basically need to compile all that data into a table. I'm completely flabbergasted that they didn't just use SharePoint forms and lists to do this, but someone needed it to be Word documents so now I have a "fun" project to process 1000 word documents.

Part of why I wish the python-docx route worked is that the COM method requires Word opening the files which gets messy (of course there are macros... why not) and is much slower. Right now my script takes 35 minutes to process all the files where an XML solution should be much faster (based on previous evidence).

I may try the above recommendations if speed becomes my biggest problem, but until then I just wanted to report in and cast my vote for this feature.

[vuonghungvinh]

doc_elm = document._element
contentControls = doc_elm.xpath('.//w:sdt')
for contentControl in contentControls:
print(contentControl)
same question, I got that elemnt, how can I edit font-size or line spacing?

[Fellfrosch]

Hi, here is a snippet from my try.
@scanny can you help me?

I need to update a content control element.
I manage to find the content control with the desired tag using .xpath.
Then I create a new paragraph element but I fail to clear the existing information inside the Content Control and also fail to add the new text inside the Content Control.
As you can see from the image, I manage only to add text next to the content Control.

doc = Document('A_Template_Chelen List+Sudurjanie_2022.docx')

elms=[]

for p in doc.paragraphs:

	result=p._element.xpath("w:sdt/w:sdtPr/w:alias/@w:val")

	for i in result:

		if i=='ИМЕ_ОБЕКТ':

			sdt=i.getparent().getparent().getparent() #Get w:sdt Element

	
			new_p = OxmlElement("w:p")        #new w:p Element
			new_para = Paragraph(new_p,sdt)	  #new paragraph element child of w:sdt Element 			
			new_para.add_run('ddddddddd')	#add run to new paragraph	

			sdt.getparent().append(new_para._element)



doc.save('demo.docx')

[maddymhs]

Hello @scanny! I'm parsing word files and using find and replace with docx. The word files use rich text regularly. Rich text controls also use the "sdt" tags in xml like checkboxes and some other features. Using this thread, and this other thread: #295 I'm able to put out some text in paragraphs that are within w:sdt tags, but not all of them. I would love to do find and replace using the python-docx module like I can on runs and paragraphs! For now, is there a way I can use lxml to do find and replace or text extraction for the paragraphs (and/or runs) within sdt tags? thanks!

[davinoregan]

@scanny I have used the same parent.remove(sdt) you did to replace all sdt in my document (i just want to remove them entirely, not preserve any embedded text or tags). however, when I write to a word doc and try to open it in MS Word to review the content, I get a "Word experienced an error trying to open the file..." prompt. If I use the same code to remove sdtPr and sdtContent nodes, the Word doc opens but it still has the sdt tags, they're just empty now. Any idea what the error might be?

[scanny]

@davinoregan better open a separate issue referencing this one. You'll need to dump the XML so we can see before and after. Something like print(document._element.xml) will do the trick although you'll need to pare down your document to a reasonable size. We shouldn't need more than say 50 lines of XML to see what's happening. Then also show your exact source code to process it, trimmed down to a minimal reproducible example.

[charlie2clarke]

@scanny I'd be interested in API support for w:sdt tags. My use case is trying to read and then remove bibliography sections (using Word's built-in functionality).

Once you have a paragraph, you can get access to any <w:sdt> elements inside it using something like this:
paragraph = ... # however you get the paragraph, maybe with for paragraph in document.paragraphs
p = paragraph._element
sdts = p.xpath('w:sdt')
for sdt in sdts:
print('found an sdt element')

When using the above work-around, the document's xml updates and deletes the w:sdt tags as I want it to. However, when I save the new document the output .docx file still includes the bibliography section.

Why is it that despite the python-docx document's xml not including the w:sdt elements does this not reflect once the document is saved and opened in Microsoft Word. I have read here that document relationships could have something to do this with but given that I get no error when opening the new word document, I don't think this is the problem.

Here is the xml of the test doc I am looking at:

<w:p xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:w14="http://schemas.microsoft.com/office/word/2010/wordml" xmlns:wpc="http://schemas.microsoft.com/office/word/2010/wordprocessingCanvas" xmlns:cx="http://schemas.microsoft.com/office/drawing/2014/chartex" xmlns:cx1="http://schemas.microsoft.com/office/drawing/2015/9/8/chartex" xmlns:cx2="http://schemas.microsoft.com/office/drawing/2015/10/21/chartex" xmlns:cx3="http://schemas.microsoft.com/office/drawing/2016/5/9/chartex" xmlns:cx4="http://schemas.microsoft.com/office/drawing/2016/5/10/chartex" xmlns:cx5="http://schemas.microsoft.com/office/drawing/2016/5/11/chartex" xmlns:cx6="http://schemas.microsoft.com/office/drawing/2016/5/12/chartex" xmlns:cx7="http://schemas.microsoft.com/office/drawing/2016/5/13/chartex" xmlns:cx8="http://schemas.microsoft.com/office/drawing/2016/5/14/chartex" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" xmlns:aink="http://schemas.microsoft.com/office/drawing/2016/ink" xmlns:am3d="http://schemas.microsoft.com/office/drawing/2017/model3d" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:oel="http://schemas.microsoft.com/office/2019/extlst" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:m="http://schemas.openxmlformats.org/officeDocument/2006/math" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:wp14="http://schemas.microsoft.com/office/word/2010/wordprocessingDrawing" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:w10="urn:schemas-microsoft-com:office:word" xmlns:w15="http://schemas.microsoft.com/office/word/2012/wordml" xmlns:w16cex="http://schemas.microsoft.com/office/word/2018/wordml/cex" xmlns:w16cid="http://schemas.microsoft.com/office/word/2016/wordml/cid" xmlns:w16="http://schemas.microsoft.com/office/word/2018/wordml" xmlns:w16sdtdh="http://schemas.microsoft.com/office/word/2020/wordml/sdtdatahash" xmlns:w16se="http://schemas.microsoft.com/office/word/2015/wordml/symex" xmlns:wpg="http://schemas.microsoft.com/office/word/2010/wordprocessingGroup" xmlns:wpi="http://schemas.microsoft.com/office/word/2010/wordprocessingInk" xmlns:wne="http://schemas.microsoft.com/office/word/2006/wordml" xmlns:wps="http://schemas.microsoft.com/office/word/2010/wordprocessingShape" w14:paraId="2BE25C04" w14:textId="0C685DB1" w:rsidR="00572630" w:rsidRDefault="006A14EA">
  <w:r>
    <w:t xml:space="preserve">Hey there </w:t>
  </w:r>
  <w:sdt>
    <w:sdtPr>
      <w:id w:val="-39898552"/>
      <w:citation/>
    </w:sdtPr>
    <w:sdtContent>
      <w:r>
        <w:fldChar w:fldCharType="begin"/>
      </w:r>
      <w:r>
        <w:instrText xml:space="preserve"> CITATION Fra69 \l 2057 </w:instrText>
      </w:r>
      <w:r>
        <w:fldChar w:fldCharType="separate"/>
      </w:r>
      <w:r>
        <w:rPr>
          <w:noProof/>
        </w:rPr>
        <w:t>(Herbert, 1969)</w:t>
      </w:r>
      <w:r>
        <w:fldChar w:fldCharType="end"/>
      </w:r>
    </w:sdtContent>
  </w:sdt>
</w:p>

[scanny]

@charlie2clarke I'm not sure, but one hypothesis is that like a table of contents, the bibliography is generated on demand based on these sdt elements. So getting rid of the elements by itself does not change the bibliography. If that's the case I expect you'll find a w:instrText element (with its <w:fldChar> buddies) where the bibliography appears in the document and that would be the element you need to delete or modify.

I think you'll need to do some research to see what's what with bibliography features in Word. I've never looked into it myself.

This would have nothing to do with relationships if there are no elements involved that have an attribute like w:embed="rId{N}" where N is any smallish integer and rId{N} is a key to a relationship ((r)elationship-{Id}). Such an attribute is how an element references a relationship to another part in the package.

[charlie2clarke]

@scanny Thanks for your help. I think you're right about the auto regeneration given the w:instrText elements. I realised my problem was I was only deleting the w:sdt elements that were children of paragraph elements and not the main bibliography w:sdt element. Deleting both are required to be fully rid of all the bibliography and its references.

For reference - below is for deleting bibliography refs:

paragraph = ... # however you get the paragraph, maybe with `for paragraph in document.paragraphs`
p = paragraph._element
sdts = p.xpath('w:sdt')
for sdt in sdts:
    parent = sdt.getparent()
    parent.remove(sdt)

And below will delete the main bibliography element:

body = doc._body._element
sdts = body.xpath("w:sdt")
if sdts:
    for sdt in sdts:
        parent = sdt.getparent()
        parent.remove(sdt)