The fix for entity processing changes the order of opening and closing
of entities. The reason is the new iterative approach that closes
and removes entities as soon as they are fully processed, unlike
the recursive approach that, as a side effect of the recursion, waits
the inner entities before closing and removal.
This commit delays the removal and the call to entityTrackingOnClose
until the current entities inner entities are processed, which in turn
allows us to have the correct debug order again.
This commit introduces a new nextPtr parameter to storeEntityValue.
After finishing its execution, storeEntityValue function sets this
parameter in way that it points to the next token to process.
This is useful when we want to leave and reenter storeEntityValue during
its execution since nextPtr will point where we left.
This commit is base to the following commit.
During processing attributes with entity references,
appendAttributeValue can reach high recursion depths that can lead to
a crash.
This commit switches the processing to an iterative approach similar to
the fix for internal entity processing. A new m_openAttributeEntities
list is introduced to keep track of entity references that need
processing. When a new entity reference is detected, instead of calling
appendAttributeValue recursively, the entity will be added to open
entities list and the execution will return to storeAttributeValue,
where newly added entity will be handled. After the entity processing
is done, appendAttributeValue will be called by using the next token.
This commits extends appendAttributeValue by introducing a new parameter
that will be set to the next token to process.
Having such a parameter allows us to reenter the function after an exit
and continue from the last token pointed by the pointer.
The new test asserts that internalEntityProcessor does not loop forever while
processing entities where external entity content references back to internal
entities from the parent document (see &e3; and &e4; below).
We ensure that progress is made after moving the parser from recursive
invocation to a state based processing within function callProcessor.
A version of this test case (originally external-to-Expat, "make run-xmltest")
failed earlier, so we wanted to have a variant of this test (that proved
itself relevant) included within the core test suite ("make check").
Co-authored-by: Jann Horn <jannh@google.com>
Add a new reenter flag. This flag acts like XML_SUSPENDED,
except that instead of returning out of the library, we
only return back to the main parse function, then re-enter
the processor function.
