Pgml convert improvements revised

[drgrice1]

The loadMacros parsing is now much more versatile. In addition to basic single and double quoting, and qw quotes, it can handle any quotelike quotes. There is a minimal attempt to keep comments inside of a loadMacros call. Basically, if a macro is removed from the list, then a comment after it is also removed. If a macro is kept, then so is the comment that was after it. If there is a line that only has a comment on it, then if the preceding line had a comment it is assumed that the comment continues that one, and if that is kept so is the next line. Otherwise the comment line is dropped.

Previously the MathObjects.pl macro was removed if loaded, since it is loaded by PGML.pl. Now niceTables.pl is also removed, since PGML.pl also loads that.

The naive approach of replacing ans_rule calls with PGML has been removed. Instead any ans_rule call is just wrapped in [@ ... @]*, and the corresponding ANS calls are left alone. One advantage of this approach is that after the conversion is completed, the problem actually works. In addition, the previous approach was clobbering the ans_rule calls in many cases that give information about what needs to be done to properly convert the answers into the PGML answer syntax. The con is that the conversion into PGML syntax must be done manually. However, since that really was the case before other than basically just putting a non-functional PGML answer template into place, that is not really a tradeoff.

Instead of handling only two very specific forms of heredoc/nowdoc syntax SOLUTION(EV3(<<'END_SOLUTION')); and TEXT(EV2(<<EOT)) where the heredoc/nowdoc terminator was fixed and must be exactly what is shown above, any such form is handled. The heredoc/nowdoc terminator can be anything the author specified and an unquoted, singled quoted, or double quoted terminator can be used. In addition both EV2 and EV3 are covered, and the HINT form is also handled. In addition spaces can be in those constructs as allowed by Perl. Note that there are OPL problems that do all of those things, so it was really quite naive to only literally cover those two cases.

Variable interpolation now actually works much better, including for hash and array access, and hash reference and array reference access.

Note that a file that already uses PGML and does not have any of the old style PG constructs is actually completely left alone. @pstaapb's #1425 still modifies those files in some cases.

The bin/convert-to-pgml.pl script no longer accepts -s|--suffix argument. Instead it accepts a -e|--extension-prefix argument. That is much the same except that its value is prepended to .pg instead of used as the extension itself. It defaults to pgml as the suffix option did before. So if problem.pg is converted, then the result will be written to problem.pgml.pg. Thus the resulting file has the correct pg problem file extension.

A new option for the bin/convert-to-pgml.pl, -p|--output-path was added. When given, the converted file is written into that directory with the same name as the original file. The -b|--backup and -e|--extension-prefix options are ignored, unless the output path resolves to the directory containing the original file, in which case those options will still apply.

TRhere were some other minor things fixed along the way as well that I did not list here, such as an issue with the conversion of the $HR variable into a PGML --- in which the --- was added, but the $HR variable left and changed into [$HR] resulting in

---

[$HR]

in the converted file.

Note that this is built on top of #1425, and is intended to replace that pull request.

This was done in collaboration with Claude.