Comparing Results
=================

Usage
-----
The final results obtained by the evaluation script can now be compared among all
trained models. To do this interactively, use the following commands::

	cd models
	ipython3 compare_models.py
	
Review Results
--------------
The results of performance metrics are stored to Pandas data frames. Results are stored
in the following *Pandas* dataframes:

- **global_res:** Global performance metrics.
- **class_res:** Class-wise performance metrics.

- **global_class_res:** Global performance metrics by using optima class-wise thresholds.

By accessing it interactively, it is possible to find the global performance 
metrics with respect to the data set, model, aggregation strategy and averaging 
method used.

For example::

	global_res.loc['irmas_1024'].loc['model_baseline'].loc['s2'].loc['micro']
	class_res.loc['jazz_db_1024'].loc['model_baseline'].loc['s1'].loc['voi']
	global_class_res.loc['youtube_1024'].loc['model_leaky'].loc['s1'].loc['macro']

will print the precision, recall, and f-scores of these cases.

Finally, you can uncomment sections of the script to plot the improvements by comparing
the models in between them.