Each Answer to this Q is separated by one/two green lines.
I have an object that can build itself from an XML string, and write itself out to an XML string. I’d like to write a unit test to test round tripping through XML, but I’m having trouble comparing the two XML versions. Whitespace and attribute order seem to be the issues. Any suggestions for how to do this? This is in Python, and I’m using ElementTree (not that that really matters here since I’m just dealing with XML in strings at this level).
First normalize 2 XML, then you can compare them. I’ve used the following using lxml
obj1 = objectify.fromstring(expect) expect = etree.tostring(obj1) obj2 = objectify.fromstring(xml) result = etree.tostring(obj2) self.assertEquals(expect, result)
This is what I finally came up with:
from doctest import Example from lxml.doctestcompare import LXMLOutputChecker class XmlTest(TestCase): def assertXmlEqual(self, got, want): checker = LXMLOutputChecker() if not checker.check_output(want, got, 0): message = checker.output_difference(Example("", want), got, 0) raise AssertionError(message)
This also produces a diff that can be helpful in case of large xml files.
If the problem is really just the whitespace and attribute order, and you have no other constructs than text and elements to worry about, you can parse the strings using a standard XML parser and compare the nodes manually. Here’s an example using minidom, but you could write the same in etree pretty simply:
def isEqualXML(a, b): da, db= minidom.parseString(a), minidom.parseString(b) return isEqualElement(da.documentElement, db.documentElement) def isEqualElement(a, b): if a.tagName!=b.tagName: return False if sorted(a.attributes.items())!=sorted(b.attributes.items()): return False if len(a.childNodes)!=len(b.childNodes): return False for ac, bc in zip(a.childNodes, b.childNodes): if ac.nodeType!=bc.nodeType: return False if ac.nodeType==ac.TEXT_NODE and ac.data!=bc.data: return False if ac.nodeType==ac.ELEMENT_NODE and not isEqualElement(ac, bc): return False return True
If you need a more thorough equivalence comparison, covering the possibilities of other types of nodes including CDATA, PIs, entity references, comments, doctypes, namespaces and so on, you could use the DOM Level 3 Core method isEqualNode. Neither minidom nor etree have that, but pxdom is one implementation that supports it:
def isEqualXML(a, b): da, db= pxdom.parseString(a), pxdom.parseString(a) return da.isEqualNode(db)
(You may want to change some of the DOMConfiguration options on the parse if you need to specify whether entity references and CDATA sections match their replaced equivalents.)
A slightly more roundabout way of doing it would be to parse, then re-serialise to canonical form and do a string comparison. Again pxdom supports the DOM Level 3 LS option ‘canonical-form’ which you could use to do this; an alternative way using the stdlib’s minidom implementation is to use c14n. However you must have the PyXML extensions install for this so you still can’t quite do it within the stdlib:
from xml.dom.ext import c14n def isEqualXML(a, b): da, bd= minidom.parseString(a), minidom.parseString(b) a, b= c14n.Canonicalize(da), c14n.Canonicalize(db) return a==b
Use xmldiff, a python tool that figures out the differences between two similar XML files, the same way that diff does it.
Why are you examining the XML data at all?
The way to test object serialization is to create an instance of the object, serialize it, deserialize it into a new object, and compare the two objects. When you make a change that breaks serialization or deserialization, this test will fail.
The only thing checking the XML data is going to find for you is if your serializer is emitting a superset of what the deserializer requires, and the deserializer silently ignores stuff it doesn’t expect.
Of course, if something else is going to be consuming the serialized data, that’s another matter. But in that case, you ought to be thinking about establishing a schema for the XML and validating it.
I also had this problem and did some digging around it today. The
doctestcompare approach may suffice, but I found via Ian Bicking that it is based on
formencode.doctest_xml_compare. Which appears to now be here. As you can see that is a pretty simple function, unlike
doctestcompare (although I guess
doctestcompare is collecting all the failures and maybe more sophisticated checking). Anyway copying/importing
xml_compare out of
formencode may be a good solution.
The Java component
dbUnit does a lot of XML comparisons, so you might find it useful to look at their approach (especially to find any gotchas that they may have already addressed).
def xml_to_json(self, xml): """Receive 1 lxml etree object and return a json string""" def recursive_dict(element): return (element.tag.split('}'), dict(map(recursive_dict, element.getchildren()), **element.attrib)) return json.dumps(dict([recursive_dict(xml)]), default=lambda x: str(x)) def assertEqualXML(self, xml_real, xml_expected): """Receive 2 objectify objects and show a diff assert if exists.""" xml_expected_str = json.loads(self.xml_to_json(xml_expected)) xml_real_str = json.loads(self.xml_to_json(xml_real)) self.maxDiff = None self.assertEqual(xml_real_str, xml_expected_str)
You could see a output like as:
u'date': u'2016-11-22T19:55:02', u'item2': u'MX-INV0007', - u'item3': u'Payments', ? ^^^ + u'item3': u'OAYments', ? ^^^ +
It can be easily done with
class XmlTest(TestCase): def assertXmlEqual(self, got, want): return self.assertEqual(parseString(got).toxml(), parseString(want).toxml())
in my case doesn’t work for python3.
from lxml.doctestcompare import LXMLOutputChecker, PARSE_XML class XmlTest(TestCase): def assertXmlEqual(self, got, want): checker = LXMLOutputChecker() if not checker.check_output(want.encode(), got.encode(), PARSE_XML): message = checker.output_difference(Example(b"", want.encode()), got.encode(), PARSE_XML) raise AssertionError(message)