`symeval`

from symeval import *

API Reference

source

EvaluatorMathBatch

 EvaluatorMathBatch (strict_extract:bool=True,
                     include_percentage:bool=True, rel_tol:float=1e-09,
                     abs_tol:float=1e-08, percent_rel_tol:float=0.001,
                     ascii_only:bool=True, timeout:int=5, n_procs:int=2,
                     use_tqdm:bool=True)

Batch evaluator for math problems, capable of extracting answer segment from complex resp and processing various mathematical objects (e.g. fractions, symbolic expressions, matrices, vectors) and special text (e.g. bool values).

	Type	Default	Details
strict_extract	bool	True
include_percentage	bool	True	Whether to include percentage comparisons.
rel_tol	float	1e-09	The relative tolerance for numerical comparisons.
abs_tol	float	1e-08	The absolute tolerance for numerical comparisons. Necessary for precision issues.
percent_rel_tol	float	0.001	The absolute tolerance for percentage comparisons.
ascii_only	bool	True	Only allowing ASCII characters
timeout	int	5	The timeout for each evaluation.
n_procs	int	2
use_tqdm	bool	True

source

EvaluatorMath

 EvaluatorMath (strict_extract:bool=True, include_percentage:bool=True,
                rel_tol:float=1e-09, abs_tol:float=1e-08,
                percent_rel_tol:float=0.001, ascii_only:bool=True)

Evaluator for math problems, capable of extracting answer segment from complex resp and processing various mathematical objects (e.g. fractions, symbolic expressions, matrices, vectors) and special text (e.g. bool values).

	Type	Default	Details
strict_extract	bool	True
include_percentage	bool	True	Whether to include percentage comparisons.
rel_tol	float	1e-09	The relative tolerance for numerical comparisons.
abs_tol	float	1e-08	The absolute tolerance for numerical comparisons. Necessary for precision issues.
percent_rel_tol	float	0.001	The relative tolerance for percentage comparisons. Relative for different surface forms (e.g. 99% v.s. 0.99).
ascii_only	bool	True	Only allowing ASCII characters

source

EvaluatorBase

 EvaluatorBase (strict_extract:bool=True)

Base class for evaluators.