Facilitating model verification with automatic robustness and precision checks
Presenting author:
Simulations are used to calibrate and experiment with a mathematical model that has been created to reproduce the observations of one or more phenomena.
Two concepts have been developed and used since the 1960s to check that (i) the simulation correctly simulates the model (Verification) and (ii) the model correctly reproduces the associated experimental observations (Validation). With the increasing amount of shared scientific knowledge in all fields of study, Verification and Validation (V&V) steps have become a necessity, especially in the scope of model sharing and reuse.
Developing and performing V&V tests is often time consuming, and in neuroscience is not always carried out systematically. We are therefore developing tools to increase the automation of V&V, and so facilitate increased use of this approach, leading to increased reliability, robustness and reuse of models. One approach to simulation verification is trying to reproduce the expected results of a simulation. This can give clues about the robustness and precision of the model's implementation. This particular check is not in general sufficient to completely verify a simulation but is potentially the easiest check to automate.
The main objective of this work is to investigate the feasibility of automatic verification of reproducibility and robustness of a large number of models shared via the EBRAINS Data and Knowledge Services (https://ebrains.eu).
Two concepts have been developed and used since the 1960s to check that (i) the simulation correctly simulates the model (Verification) and (ii) the model correctly reproduces the associated experimental observations (Validation). With the increasing amount of shared scientific knowledge in all fields of study, Verification and Validation (V&V) steps have become a necessity, especially in the scope of model sharing and reuse.
Developing and performing V&V tests is often time consuming, and in neuroscience is not always carried out systematically. We are therefore developing tools to increase the automation of V&V, and so facilitate increased use of this approach, leading to increased reliability, robustness and reuse of models. One approach to simulation verification is trying to reproduce the expected results of a simulation. This can give clues about the robustness and precision of the model's implementation. This particular check is not in general sufficient to completely verify a simulation but is potentially the easiest check to automate.
The main objective of this work is to investigate the feasibility of automatic verification of reproducibility and robustness of a large number of models shared via the EBRAINS Data and Knowledge Services (https://ebrains.eu).