File size: 2,391 Bytes
db5a87e 34246fc |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
Directory Overview: This directory contains all the evaluation modules and results: ################################################################################################################################ ## 1. evaluation.py The `evaluation.py` module provides a comprehensive framework for evaluating Knowledge-Based Visual Question Answering (KB-VQA) models. This module includes methods for syntactic and semantic evaluation, leveraging both exact match and VQA scores. The evaluation results can be saved to an Excel file for further analysis. Notes Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config" file. The module requires the following dependencies: pandas, fuzzywuzzy, nltk, ast, typing, streamlit, and openai. ################################################################################################################################ ## 2. demo.py The `demo.py` module includes the `ResultDemonstrator` class, which provides a comprehensive framework for visualizing and demonstrating the results of Knowledge-Based Visual Question Answering (KB-VQA) models. The class utilizes Streamlit for displaying data and Altair for creating interactive visualizations. this class is the backbone for the results demo tool of the application. Notes Ensure that the necessary configuration settings are correctly set in "my_model/config/evaluation_config". The module requires the following dependencies: os, altair, streamlit, PIL, pandas, random. ################################################################################################################################ ## 3. evaluation_results.xlsx This file contains all the evaluation results based on the evaluation data (~ 1,000 samples), the results are in the sheet names "Main Data". running the evaluation module will read the main columns (Model Answers for all model's configurations) from this sheet and regenerate and save the evaluation rsults. This Main Data sheet is the result of running the evaluation module (with the same file name chosen for save). ############################################################################################################################### ## 4. Demo_Images This folder contains a pool of images used for the demo app, to visualize and demonstrate the images together with the model answers. |