Upreckon: testcases.py annotate

annotate testcases.py @ 123:90c002c960cb

Fixed CPU time display on UNIX Previously, the total CPU time spent by the testee on all test cases up to and including the current one was displayed.

author	Oleg Oshmyan <chortos@inbox.lv>
date	Sun, 24 Apr 2011 19:28:40 +0100
parents	218b8c28549c
children	42c8f5c152a5

rev	line source
77 69eadc60f4e2 Memory limit is now applied to the RSS when os.wait4 is available Oleg Oshmyan <chortos@inbox.lv> parents: 76 diff changeset	1 # Copyright (c) 2010-2011 Chortos-2 <chortos@inbox.lv>
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	2
43 81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	3 # TODO: copy the ansfile if not options.erase even if no validator is used
81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	4
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	5 from __future__ import division, with_statement
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	6
91 c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	7 from compat import *
c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	8 import files, problem, config
c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	9 from __main__ import options
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	10
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	11 import glob, re, sys, tempfile, time
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	12 from subprocess import Popen, PIPE, STDOUT
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	13
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	14 import os
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	15 devnull = open(os.path.devnull, 'w+')
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	16
85 741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	17 if options.autotime:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	18 # This is really a dirty hack that assumes that sleep() does not spend
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	19 # the CPU time of the current process and that if clock() measures
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	20 # wall-clock time, then it is more precise than time() is. Both these
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	21 # assumptions are true on all platforms I have tested this on so far,
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	22 # but I am not aware of any guarantee that they will both be true
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	23 # on every other platform.
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	24 c = time.clock()
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	25 time.sleep(1)
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	26 c = time.clock() - c
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	27 if int(c + .5) == 1:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	28 clock = time.clock
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	29 else:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	30 clock = time.time
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	31
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	32 class DummySignalIgnorer(object):
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	33 def __enter__(self): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	34 def __exit__(self, exc_type, exc_value, traceback): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	35 signal_ignorer = DummySignalIgnorer()
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	36
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	37 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	38 from win32 import *
72 7520b6bb6636 Windows Error Reporting is now suppressed (at least the dialogs) Oleg Oshmyan <chortos@inbox.lv> parents: 71 diff changeset	39 except Exception:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	40 from unix import *
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	41
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	42 __all__ = ('TestCase', 'load_problem', 'TestCaseNotPassed',
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	43 'TimeLimitExceeded', 'CanceledByUser', 'WrongAnswer',
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	44 'NonZeroExitCode', 'CannotStartTestee',
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	45 'CannotStartValidator', 'CannotReadOutputFile',
81 24752db487c5 Fixed errors in the win32 module Oleg Oshmyan <chortos@inbox.lv> parents: 79 diff changeset	46 'CannotReadInputFile', 'CannotReadAnswerFile',
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	47 'MemoryLimitExceeded', 'CPUTimeLimitExceeded',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	48 'WallTimeLimitExceeded')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	49
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	50
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	51
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	52 # Exceptions
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	53
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	54 class TestCaseNotPassed(Exception): __slots__ = ()
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	55 class TestCaseSkipped(TestCaseNotPassed): __slots__ = ()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	56 class TimeLimitExceeded(TestCaseNotPassed): __slots__ = ()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	57 class CPUTimeLimitExceeded(TimeLimitExceeded): __slots__ = ()
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	58 class WallTimeLimitExceeded(TimeLimitExceeded): __slots__ = ()
77 69eadc60f4e2 Memory limit is now applied to the RSS when os.wait4 is available Oleg Oshmyan <chortos@inbox.lv> parents: 76 diff changeset	59 class MemoryLimitExceeded(TestCaseNotPassed): __slots__ = ()
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	60 class CanceledByUser(TestCaseNotPassed): __slots__ = ()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	61
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	62 class WrongAnswer(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	63 __slots__ = 'comment'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	64 def __init__(self, comment=''):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	65 self.comment = comment
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	66
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	67 class NonZeroExitCode(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	68 __slots__ = 'exitcode'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	69 def __init__(self, exitcode):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	70 self.exitcode = exitcode
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	71
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	72 class ExceptionWrapper(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	73 __slots__ = 'upstream'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	74 def __init__(self, upstream):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	75 self.upstream = upstream
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	76
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	77 class CannotStartTestee(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	78 class CannotStartValidator(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	79 class CannotReadOutputFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	80 class CannotReadInputFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	81 class CannotReadAnswerFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	82
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	83
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	84
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	85 # Helper context managers
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	86
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	87 class CopyDeleting(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	88 __slots__ = 'case', 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	89
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	90 def __init__(self, case, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	91 self.case = case
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	92 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	93 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	94
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	95 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	96 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	97 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	98 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	99 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	100 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	101 self.__exit__(None, None, None)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	102 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	103 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	104 raise
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	105
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	106 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	107 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	108 self.case.files_to_delete.append(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	109
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	110
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	111 class Copying(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	112 __slots__ = 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	113
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	114 def __init__(self, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	115 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	116 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	117
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	118 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	119 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	120 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	121
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	122 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	123 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	124
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	125
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	126
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	127 # Test case types
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	128
f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	129 class TestCase(object):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	130 __slots__ = ('problem', 'id', 'isdummy', 'infile', 'outfile', 'points',
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	131 'process', 'time_started', 'time_stopped',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	132 'realinname', 'realoutname', 'maxcputime', 'maxwalltime',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	133 'maxmemory', 'has_called_back', 'files_to_delete',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	134 'cpu_time_limit_string', 'wall_time_limit_string',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	135 'time_limit_string')
106 aa0378ea1f93 Replaced constant properties with regular attributes Oleg Oshmyan <chortos@inbox.lv> parents: 105 diff changeset	136 has_ansfile = has_iofiles = False
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	137 needs_realinname = True
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	138
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	139 if ABCMeta:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	140 __metaclass__ = ABCMeta
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	141
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	142 def __init__(case, prob, id, isdummy, points):
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	143 case.problem = prob
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	144 case.id = id
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	145 case.isdummy = isdummy
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	146 case.points = points
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	147 case.maxcputime = case.problem.config.maxcputime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	148 case.maxwalltime = case.problem.config.maxwalltime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	149 case.maxmemory = case.problem.config.maxmemory
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	150 if case.maxcputime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	151 case.cpu_time_limit_string = '/%.3f' % case.maxcputime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	152 else:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	153 case.cpu_time_limit_string = ''
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	154 if case.maxwalltime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	155 case.wall_time_limit_string = '/%.3f' % case.maxwalltime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	156 else:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	157 case.wall_time_limit_string = ''
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	158 if not isdummy:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	159 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	160 case.realinname = case.problem.config.testcaseinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	161 case.realoutname = case.problem.config.testcaseoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	162 else:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	163 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	164 case.realinname = case.problem.config.dummyinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	165 case.realoutname = case.problem.config.dummyoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	166
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	167 @abstractmethod
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	168 def test(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	169 raise NotImplementedError
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	170
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	171 def __call__(case, callback):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	172 case.has_called_back = False
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	173 case.files_to_delete = []
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	174 case.time_limit_string = case.wall_time_limit_string
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	175 try:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	176 return case.test(callback)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	177 finally:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	178 now = clock()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	179 if getattr(case, 'time_started', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	180 case.time_started = case.time_stopped = now
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	181 elif getattr(case, 'time_stopped', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	182 case.time_stopped = now
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	183 if not case.has_called_back:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	184 callback()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	185 case.cleanup()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	186
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	187 def cleanup(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	188 #if getattr(case, 'infile', None):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	189 # case.infile.close()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	190 #if getattr(case, 'outfile', None):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	191 # case.outfile.close()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	192 if getattr(case, 'process', None) and case.process.returncode is None:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	193 # Try KILLing after three unsuccessful TERM attempts in a row
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	194 for i in range(3):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	195 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	196 terminate(case.process)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	197 except Exception:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	198 time.sleep(0)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	199 case.process.poll()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	200 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	201 case.process.wait()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	202 break
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	203 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	204 # If killing the process is unsuccessful three times in a row,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	205 # just silently stop trying
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	206 for i in range(3):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	207 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	208 kill(case.process)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	209 except Exception:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	210 time.sleep(0)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	211 case.process.poll()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	212 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	213 case.process.wait()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	214 break
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	215 if case.files_to_delete:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	216 for name in case.files_to_delete:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	217 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	218 os.remove(name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	219 except Exception:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	220 # It can't be helped
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	221 pass
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	222
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	223 def open_infile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	224 try:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	225 case.infile = files.File('/'.join((case.problem.name, case.realinname.replace('$', case.id))))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	226 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	227 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	228 raise CannotReadInputFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	229
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	230 def open_outfile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	231 try:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	232 case.outfile = files.File('/'.join((case.problem.name, case.realoutname.replace('$', case.id))))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	233 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	234 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	235 raise CannotReadAnswerFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	236
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	237
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	238 class SkippedTestCase(TestCase):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	239 __slots__ = ()
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	240
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	241 def test(case, callback):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	242 raise TestCaseSkipped
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	243
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	244
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	245 class ValidatedTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	246 __slots__ = 'validator'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	247
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	248 def __init__(case, *args):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	249 TestCase.__init__(case, *args)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	250 if not case.problem.config.tester:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	251 case.validator = None
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	252 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	253 case.validator = case.problem.config.tester
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	254
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	255 def validate(case, output):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	256 if not case.validator:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	257 # Compare the output with the reference output
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	258 case.open_outfile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	259 with case.outfile.open() as refoutput:
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	260 for line, refline in zip_longest(output, refoutput):
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	261 if refline is not None and not isinstance(refline, basestring):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	262 line = bytes(line, sys.getdefaultencoding())
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	263 if line != refline:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	264 raise WrongAnswer
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	265 return 1
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	266 elif callable(case.validator):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	267 return case.validator(output)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	268 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	269 # Call the validator program
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	270 output.close()
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	271 if case.problem.config.ansname:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	272 case.open_outfile()
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	273 case.outfile.copy(case.problem.config.ansname)
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	274 try:
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	275 case.process = Popen(case.validator, stdin=devnull, stdout=PIPE, stderr=STDOUT, universal_newlines=True, bufsize=-1)
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	276 except OSError:
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	277 raise CannotStartValidator(sys.exc_info()[1])
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	278 with signal_ignorer:
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	279 comment = case.process.communicate()[0].strip()
26 5bbb68833868 Output text improvements Oleg Oshmyan <chortos@inbox.lv> parents: 25 diff changeset	280 match = re.match(r'(?i)(ok\|(?:correct\|wrong)(?:(?:\s\|_)answer)?)(?:$\|\s+\|[.,!:]+\s)', comment)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	281 if match:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	282 comment = comment[match.end():]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	283 if not case.problem.config.maxexitcode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	284 if case.process.returncode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	285 raise WrongAnswer(comment)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	286 else:
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	287 return 1, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	288 else:
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	289 return case.process.returncode / case.problem.config.maxexitcode, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	290
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	291
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	292 class BatchTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	293 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	294
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	295 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	296 def has_iofiles(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	297 return (not case.problem.config.stdio or
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	298 case.validator and not callable(case.validator))
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	299
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	300 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	301 def has_ansfile(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	302 return case.validator and not callable(case.validator)
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	303
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	304 def test(case, callback):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	305 case.open_infile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	306 if case.problem.config.stdio:
54 1914ae9cfdce Bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 50 diff changeset	307 if options.erase and not case.validator or not case.problem.config.inname:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	308 # TODO: re-use the same file name if possible
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	309 # FIXME: 2.5 lacks the delete parameter
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	310 with tempfile.NamedTemporaryFile(delete=False) as f:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	311 inputdatafname = f.name
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	312 contextmgr = CopyDeleting(case, case.infile, inputdatafname)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	313 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	314 inputdatafname = case.problem.config.inname
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	315 contextmgr = Copying(case.infile, inputdatafname)
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	316 with contextmgr:
79 ee8a99dcaaed Renamed configuration variable tasknames to problems Oleg Oshmyan <chortos@inbox.lv> parents: 77 diff changeset	317 with open(inputdatafname) as infile:
83 37c4ad87583c Several small fixes Oleg Oshmyan <chortos@inbox.lv> parents: 82 diff changeset	318 with tempfile.TemporaryFile('w+') if options.erase and (not case.validator or callable(case.validator)) else open(case.problem.config.outname, 'w+') as outfile:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	319 call(case.problem.config.path, case=case, stdin=infile, stdout=outfile, stderr=devnull, universal_newlines=True, bufsize=-1)
62 593ad09cd69b Multiple exit code handling fixes Oleg Oshmyan <chortos@inbox.lv> parents: 61 diff changeset	320 if config.globalconf.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	321 raise NonZeroExitCode(case.process.returncode)
103 4e6f231f055f Fixed race condition resulting in calling back twice from TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 99 diff changeset	322 case.has_called_back = True
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	323 callback()
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	324 outfile.seek(0)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	325 return case.validate(outfile)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	326 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	327 case.infile.copy(case.problem.config.inname)
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	328 call(case.problem.config.path, case=case, stdin=devnull, stdout=devnull, stderr=STDOUT)
62 593ad09cd69b Multiple exit code handling fixes Oleg Oshmyan <chortos@inbox.lv> parents: 61 diff changeset	329 if config.globalconf.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	330 raise NonZeroExitCode(case.process.returncode)
103 4e6f231f055f Fixed race condition resulting in calling back twice from TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 99 diff changeset	331 case.has_called_back = True
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	332 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	333 try:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	334 output = open(case.problem.config.outname, 'rU')
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	335 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	336 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	337 with output as output:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	338 return case.validate(output)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	339
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	340
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	341 # This is the only test case type not executing any programs to be tested
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	342 class OutputOnlyTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	343 __slots__ = ()
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	344 needs_realinname = False
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	345
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	346 def cleanup(case):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	347 pass
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	348
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	349 def test(case, callback):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	350 case.time_stopped = case.time_started = 0
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	351 case.has_called_back = True
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	352 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	353 try:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	354 output = open(case.problem.config.outname.replace('$', case.id), 'rU')
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	355 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	356 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	357 with output as output:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	358 return case.validate(output)
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	359
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	360
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	361 class BestOutputTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	362 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	363
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	364
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	365 # This is the only test case type executing two programs simultaneously
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	366 class ReactiveTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	367 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	368 # The basic idea is to launch the program to be tested and the grader
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	369 # and to pipe their standard I/O from and to each other,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	370 # and then to capture the grader's exit code and use it
26 5bbb68833868 Output text improvements Oleg Oshmyan <chortos@inbox.lv> parents: 25 diff changeset	371 # like the exit code of an output validator is used.
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	372
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	373
71 1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	374 class DummyTestContext(problem.TestGroup):
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	375 __slots__ = ()
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	376 def end(self):
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	377 say('Sample total: %d/%d tests' % (self.ncorrect, self.ntotal))
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	378 return 0, 0, self.log
71 1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	379
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	380 def load_problem(prob, _types={'batch' : BatchTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	381 'outonly' : OutputOnlyTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	382 'bestout' : BestOutputTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	383 'reactive': ReactiveTestCase}):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	384 # We will need to iterate over these configuration variables twice
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	385 try:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	386 len(prob.config.dummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	387 except Exception:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	388 prob.config.dummies = tuple(prob.config.dummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	389 try:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	390 len(prob.config.tests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	391 except Exception:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	392 prob.config.tests = tuple(prob.config.tests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	393
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	394 if options.legacy:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	395 prob.config.usegroups = False
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	396 newtests = []
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	397 for i, name in enumerate(prob.config.tests):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	398 # Same here; we'll need to iterate over them twice
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	399 try:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	400 l = len(name)
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	401 except Exception:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	402 try:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	403 name = tuple(name)
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	404 except TypeError:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	405 name = (name,)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	406 l = len(name)
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	407 if l > 1:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	408 prob.config.usegroups = True
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	409 newtests.append(name)
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	410 if prob.config.usegroups:
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	411 prob.config.tests = newtests
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	412 del newtests
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	413
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	414 # Even if they have duplicate test identifiers, we must honour sequence pointmaps
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	415 if isinstance(prob.config.pointmap, dict):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	416 def getpoints(i, j, k=None):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	417 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	418 return prob.config.pointmap[i]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	419 except KeyError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	420 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	421 return prob.config.pointmap[None]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	422 except KeyError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	423 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	424 elif prob.config.usegroups:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	425 def getpoints(i, j, k):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	426 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	427 return prob.config.pointmap[k][j]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	428 except LookupError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	429 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	430 else:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	431 def getpoints(i, j):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	432 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	433 return prob.config.pointmap[j]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	434 except LookupError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	435 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	436
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	437 # First get prob.cache.padoutput right,
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	438 # then yield the actual test cases
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	439 for i in prob.config.dummies:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	440 s = 'sample ' + str(i).zfill(prob.config.paddummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	441 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	442 if prob.config.usegroups:
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	443 if not isinstance(prob.config.groupweight, dict):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	444 prob.config.groupweight = dict(enumerate(prob.config.groupweight))
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	445 for group in prob.config.tests:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	446 for i in group:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	447 s = str(i).zfill(prob.config.padtests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	448 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
89 3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	449 if prob.config.dummies:
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	450 yield DummyTestContext()
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	451 for i in prob.config.dummies:
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	452 s = str(i).zfill(prob.config.paddummies)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	453 if (yield _types[prob.config.kind](prob, s, True, 0)):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	454 yield
89 3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	455 yield problem.test_context_end
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	456 for k, group in enumerate(prob.config.tests):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	457 if not group:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	458 continue
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	459 yield problem.TestGroup(prob.config.groupweight.get(k, prob.config.groupweight.get(None)))
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	460 case_type = _types[prob.config.kind]
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	461 for j, i in enumerate(group):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	462 s = str(i).zfill(prob.config.padtests)
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	463 if not (yield case_type(prob, s, False, getpoints(i, j, k))):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	464 if options.skim:
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	465 case_type = SkippedTestCase
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	466 else:
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	467 yield
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	468 yield problem.test_context_end
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	469 else:
f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	470 for i in prob.config.tests:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	471 s = str(i).zfill(prob.config.padtests)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	472 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	473 for i in prob.config.dummies:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	474 s = str(i).zfill(prob.config.paddummies)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	475 if (yield _types[prob.config.kind](prob, s, True, 0)):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	476 yield
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	477 for j, i in enumerate(prob.config.tests):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	478 s = str(i).zfill(prob.config.padtests)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	479 if (yield _types[prob.config.kind](prob, s, False, getpoints(i, j))):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	480 yield

Mercurial > ~astiob > upreckon > hgweb

annotate testcases.py @ 123:90c002c960cb