Return to the home page of Upreckon
Since the Upreckon 2.00 release is shaping to be an almost complete rewrite and I want it to be devoid of any peculiar deficiencies test.py 1.x currently has, I have decided to create this page to keep track of changes that I want to implement, am implementing or have implemented, as well as changes users want to see and bugs that all of us want to get fixed.
If you want to suggest another feature or report a bug (which you should do whenever you encounter a bug), please create a new issue on Bitbucket (which is preferred) or directly contact me using one of the means of communication I have listed on my homepage (email is included).
| Short description | Status |
|---|---|
| CPU time should be able to be measured instead of wall-clock time | implemented in 2.001 |
| memory usage of the program being tested should be able to be limited | implemented in 2.001 |
| output validators should be able to be given as Python callables | implemented in 2.00 |
| points should be able to be awarded for whole test groups rather than just for individual test cases | implemented in 2.00 |
| run-time overhead should be significantly reduced | implemented in 2.00 |
| signal names, when available, should be shown in addition to their numbers | implemented in 2.00 |
| single-test-case groups should print totals if at least one group spans multiple test cases | implemented in 2.00 |
| test.py should have a serious name | implemented in 2.00 (Upreckon) |
| testing within a test group should be stopped as soon as one test case fails | implemented in 2.00 (--skim) |
| the ansname configuration variable should always be optional | implemented in 2.00 |
| the output of output validators should be cleaned from text like ‘OK’, ‘correct’ and ‘wrong’ | implemented in 2.00 |
| the program being tested should be able to be manually killed by pressing Escape | implemented in 2.00 |
| Upreckon should be fully compatible with Python 3 | implemented in 2.00 |
| Win32 errors caused by the program being tested should be reported in hexadecimal | implemented in 2.00 |
| Windows Error Reporting should be suppressed | implemented in 2.00 |
| absent output files should not crash the whole Upreckon | implemented in 2.00.1 |
| CPU time should be natively limited on platforms with posix_spawn | implemented in 2.01 (in C only) |
| output-only problems should be supported | implemented in 2.01 |
| time-sensitive and simply very platform-dependent code should be re-implemented in C+POSIX | implemented in 2.01 |
| Upreckon should be installable using distutils | implemented in 2.01 |
| test case identifier sets should be able to be defined by regular expressions | implemented in 2.03 |
command-line test case identifier overrides should work with match='re' | implemented in 2.03.1 |
| pressing Escape should cancel test input unarchiving | implemented _unix-only in 2.04 |
| time-sensitive and simply very platform-dependent code should be re-implemented in C+Win32 | being implemented for 2.0x |
| Upreckon should be fully compatible with Python 2.5 | being implemented for 2.0x |
| Upreckon should follow PEP 8 | being implemented for 2.0x |
| all exceptions at least somewhat expected should show a friendly message | set to be implemented in 2.0x |
| configuration for multiple problems should be able to be given in the same file | set to be implemented in 2.0x |
| configuration variable types should be prescribed and validated | set to be implemented in 2.0x |
| different time limits should be able to be given for different test cases | set to be implemented in 2.0x |
| problem.load_testcases and problem.Problem.test should communicate using a sensible protocol | set to be implemented in 2.0x |
| test and configuration data should be always searched in the same locations and in the same order | set to be implemented in 2.0x |
| test case identifier sets should be able to be defined by glob wildcards | set to be implemented in 2.0x |
| Upreckon should be fully compatible with IronPython | set to be implemented in 2.0x |
| Upreckon should be fully compatible with Jython | set to be implemented in 2.0x |
| Upreckon should be fully compatible with PyPy | set to be implemented in 2.0x |
| Upreckon should be in PyPI | set to be implemented in 2.0x |
| Upreckon should be well-documented | set to be implemented in 2.0x |
| Upreckon should follow PEP 257 | set to be implemented in 2.0x |
| output validators should be able to be supplied in archives | hoped to be implemented in 2.0x |
| partial score should be allowed to be given to slow but correct solutions | hoped to be implemented in 2.0x |
| tape archives should be allowed to contain testconf.py files with non-LF line endings | hoped to be implemented in 2.0x |
| differences in whitespace should be able to be ignored when validating output | taken into consideration for 2.0x |
| custom pre-testing actions should be supported | hoped to be implemented in 2.x |
| interactive (reactive) problems should be supported | hoped to be implemented in 2.x |
| output of programs using standard I/O should be piped to output validators | hoped to be implemented in 2.x |
| directories named taskname in the current working directory should not be checked for test and configuration data | taken into consideration for 2.x |
| the first test case should be run twice if reasonable with only the second run valued (think caching/whatever) | taken into consideration for 2.x |
| Web judges should be supported (just uploading and proxying the results) | taken into consideration for 2.x |
| the front-end (reading and parsing the command line and configuration files) should be easily swappable | hoped to be implemented in 3.00 |
| programs that append to the output file instead of overwriting it should be granted Wrong Answer | rejected (impossibly objective) |
1 Support varies from platform to platform. See the wiki on Bitbucket for more details.