diff --git a/doc/design_choices.md b/doc/design_choices.md new file mode 100644 index 0000000..b688f6a --- /dev/null +++ b/doc/design_choices.md @@ -0,0 +1,30 @@ +# Design Choices + +This section describes the guiding design choices that are attempted to be +implemented in the NUISANCE v3 rewrite. Any perceived failure to adhere to these design choices should be regarded as a bug and submitted to the developers. + +## Modularity of components + +The various moving parts that go into a NUISANCE analysis should be as independent as possible (and no more). For example, as in previous versions, 'input handlers' are implementations of an `InputHandler` ABC, they are responsible for converting a specific event format into the NUISANCE event format. This event format should be generator agnostic as far as possible, however, for the generator-based-event reweighting to perform efficiently, the original generator event must be associated to the NUISANCE event. Instead of having some per-generator event manager, or reweight engines being allowed specific access to known InputHandler subclasses, it was decided that a pointer to the generator event would be carried around by the NUISANCE event base-class. This design choice has been kept. + +In NUISANCE v3, the design of the 'sample', the basic implementation unit of a comparison, has been further modularized. The previous Measurement{Base,1D,2D} base classes have been surplanted by the ISample and IDataComparison interfaces. IDataComparison being a subclass of ISample. These specify the interface that any NUISANCE sample must implement. The implementation of any subclass is left up to the user, however, a helper base class that should be used for almost all simple data comparisons can be found in SimpleDataComparison. It attempts to provide a fully functional base class, for data comparisons that previously subclassed from Measurement1D or 2D, that needs minimal specializing for specific data samples---most importantly the signal definition, the kinematic projection, and optional-but-encouraged data set metadata. As the interface suggests, the responsibility of looping over events and determining event variations is now fully that of the ISample, rather than a finely tuned collaboration between the sample and some calling method. However, subclasses of SimpleDataComparison can still be fully implemented by providing only per-event methods as the base class handles the event looping. + +Related to this, it may be useful to extend the 'reweight' concept, which was assumed to be the only form of event response in previous versions to a more general event 'variation'. However, since the majority of variations will still be weight-based, and as weight-based variations can often be calculated and applied fully with just the `nuis::event::MinimalEvent` format, the InputHandler will expose convenience methods that allow the current event weight to be without direct interrogation of any IWeightProvider instances. However, because general event variations can be arbitrarily complex and can require signal definitions to be rechecked (requiring a `nuis::event::FullEvent`), no such convenience method exists and samples looking to make use of such variations must explicitly call the VariationManager. + +## Extensibility without recompilation + +In previous versions of NUISANCE, the addition of new studies by non-experts was somewhat involved. Even though well-documented in multiple tutorials the addition of new studies required multiple source code changes, for opaque reasons outside of the study implementation itself. It also required recompilation of the entire set of NUISANCE binaries, this makes sharing a NUISANCE release on a cluster problematic and limits the entry barrier for non-expert users. V3 of NUISANCE is built from the ground up to allow for use and extensibility by non-experts, this is primarily facilitated by the `Instantiate` template method that implements a plugin factory, somewhat inspired by the `art::make_tool` utility from the ART framework. While the architecture of c++ means that plugins compiled with different toolsets than the main NUISANCE binaries are unlikely to work (and thus checked and disallowed before attempting any c++ object instantiation), multiple users working on a cluster should now be able to have local analysis implementations that are dynamically instantiated at runtime and require no recompilation of a central NUISANCE install. + +## Flexibility of configuration + +Originally NUISANCE used an inflexible DSL for configuration, in v2, a more-flexible XML-based configuration file was used. The change increased the scope of specific extensions to the core configuration, but the configuration API was clunky. In v3, NUISANCE uses an implementation of the FHiCL-c++ language +bindings to provide both configuration parsing and validation, as well as programatic access to the runtime configuration. FHiCL is particularly adept at providing a naturally hierarchial configuration, whereby global defaults can be +specified but overridden at runtime by user-supplied configuration files without special handling of the in-memory global configuration document. The FHiCL language bindings also specify how to retrieve configuration elements with specified types, reducing the need for comprehensive runtime checks for missing or incomplete configuration. + +## Exceptional circumstances + +Previously, when NUISANCE encountered exceptional circumstances an exception was thrown. This exception was usually unqualified and sometimes preceeded by an error message. This made debugging and reporting of runtime errors fiddly for non-experts. In v3, all exceptions should be sensibly named, derived from `nuis::nuis_except` and be supplied with a verbose message diagnosing the problem encountered. + +## Documentation + +The inline documentation of the code in previous versions of NUISANCE has been acceptable. The barrier to entry for new users was significantly lowered by external tutorials and documents. For v3, the hope is that the interface documentation can be fully comprehensive, and a specific set of instructions for common and less-common tasks can be distributed with the code. In addition to this, applications and scripts that verbosely aid users in standard NUISANCE workflows should be provided, and all usage prompts should be kept informative and fully up-to-date. diff --git a/doc/glossary.md b/doc/glossary.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/inputs.md b/doc/inputs.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/main.md b/doc/main.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/samples.md b/doc/samples.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/structure.md b/doc/structure.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/variations.md b/doc/variations.md new file mode 100644 index 0000000..e69de29 diff --git a/doc/Doxyfile.in b/doc_old/Doxyfile.in similarity index 100% rename from doc/Doxyfile.in rename to doc_old/Doxyfile.in diff --git a/doc/addingsamples.dox b/doc_old/addingsamples.dox similarity index 100% rename from doc/addingsamples.dox rename to doc_old/addingsamples.dox diff --git a/doc/building.dox b/doc_old/building.dox similarity index 100% rename from doc/building.dox rename to doc_old/building.dox diff --git a/doc/mainpage.dox b/doc_old/mainpage.dox similarity index 100% rename from doc/mainpage.dox rename to doc_old/mainpage.dox diff --git a/doc/samplelist.dox b/doc_old/samplelist.dox similarity index 100% rename from doc/samplelist.dox rename to doc_old/samplelist.dox diff --git a/doc/statisticalmethods.dox b/doc_old/statisticalmethods.dox similarity index 100% rename from doc/statisticalmethods.dox rename to doc_old/statisticalmethods.dox diff --git a/doc/tutorial.dox b/doc_old/tutorial.dox similarity index 100% rename from doc/tutorial.dox rename to doc_old/tutorial.dox diff --git a/doc/tutorial_dialtuning.dox b/doc_old/tutorial_dialtuning.dox similarity index 100% rename from doc/tutorial_dialtuning.dox rename to doc_old/tutorial_dialtuning.dox diff --git a/doc/tutorial_inputs.dox b/doc_old/tutorial_inputs.dox similarity index 100% rename from doc/tutorial_inputs.dox rename to doc_old/tutorial_inputs.dox diff --git a/src/generator/variation/CMakeLists.txt b/src/generator/variation/CMakeLists.txt new file mode 100644 index 0000000..e69de29 diff --git a/src/generator/variation/NEUTReWeight.cxx b/src/generator/variation/NEUTReWeight.cxx new file mode 100644 index 0000000..4e61a9c --- /dev/null +++ b/src/generator/variation/NEUTReWeight.cxx @@ -0,0 +1,3 @@ +#include "generator/variation/NEUTReWeight.hxx" + +DECLARE_PLUGIN(IWeightProvider, NEUTReWeight); diff --git a/src/variation/IWeightProvider.hxx b/src/generator/variation/NEUTReWeight.hxx similarity index 68% copy from src/variation/IWeightProvider.hxx copy to src/generator/variation/NEUTReWeight.hxx index dfaf751..fd2c330 100644 --- a/src/variation/IWeightProvider.hxx +++ b/src/generator/variation/NEUTReWeight.hxx @@ -1,34 +1,35 @@ // Copyright 2018 L. Pickering, P Stowell, R. Terri, C. Wilkinson, C. Wret /******************************************************************************* * This file is part of NUISANCE. * * NUISANCE is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * NUISANCE is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with NUISANCE. If not, see . *******************************************************************************/ -#ifndef VARIATION_IWEIGHTPROVIDER_HXX_SEEN -#define VARIATION_IWEIGHTPROVIDER_HXX_SEEN +#ifndef GENERATOR_VARIATION_NEUTREWEIGHT_HXX_SEEN +#define GENERATOR_VARIATION_NEUTREWEIGHT_HXX_SEEN -#include "variation/IVariationProvider.hxx" +#include "variation/IWeightProvider.hxx" -class IWeightProvider { +class NEUTReWeight : public IWeightProvider { public: - double GetEventWeight(nuis::event::MinimalEvent) = 0; - - virtual ~IWeightProvider() {} + double GetEventWeight(nuis::event::MinimalEvent const &); + void Initialize(fhicl::ParameterSet const &); + paramId_t GetParameterId(std::string const &); + void SetParameterValue(paramId_t, double); + bool ParametersVaried(); + void Reconfigure(); }; -DECLARE_PLUGIN_INTERFACE(IWeightProvider); - #endif diff --git a/src/parameters/CMakeLists.txt b/src/parameters/CMakeLists.txt new file mode 100644 index 0000000..eb19019 --- /dev/null +++ b/src/parameters/CMakeLists.txt @@ -0,0 +1,11 @@ +SET(parameters_implementation_files +ParameterManager.cxx) + +SET(parameters_header_files + ParameterManager.hxx) + +add_library(nuis_params SHARED ${parameters_implementation_files}) +target_link_libraries(nuis_params) + +install(TARGETS nuis_params DESTINATION lib) +install(FILES ${parameters_header_files} DESTINATION include/parameters) diff --git a/src/parameters/ParameterManager.cxx b/src/parameters/ParameterManager.cxx new file mode 100644 index 0000000..76b17d6 --- /dev/null +++ b/src/parameters/ParameterManager.cxx @@ -0,0 +1,133 @@ +#include "parameters/ParameterManager.hxx" + +namespace nuis { +namespace params { + +ParameterManager *ParameterManager::_global_inst = nullptr; + +ParameterManager::ParameterManager() : locked(false) {} + +ParameterManager &ParameterManager::Get() { + if (!_global_inst) { + _global_inst = new ParameterManager(); + } + return *_global_inst; +} + +void ParameterManager::ValidateParamId(paramId_t pid) { + if (pid >= Parameters.size()) { + throw invalid_parameter_id() + << "[ERROR]: Passed parameter id " << pid + << ", but the ParameterManager only knows about " << Parameters.size() + << " parameters."; + } +} + +void ParameterManager::LockParameterList() { locked = true; } +void ParameterManager::UnlockParameterList() { locked = false; } + +paramId_t +ParameterManager::EnsureParameterRegistered(fhicl::ParameterSet const &ps) { + + if (locked) { + throw parameter_list_is_locked() + << "[ERROR]: Attempted to register parameter: " << ps.to_string() + << " when global ParameterManager was locked in state: " + << StateString(); + } + + NamedParameter np; + np.name = ps.get("name"); + np.type = ps.get("type"); + paramId_t pid = GetParameterId(np.name, np.type); + if (pid != kParamUnhandled) { + return pid; + } + + np.start = ps.get("start"); + np.value = np.start; + np.min = ps.get("min", kDefaultLimit); + np.max = ps.get("max", kDefaultLimit); + np.step = ps.get("step"); + + pid = Parameters.size(); + Parameters.push_back(np); + + return pid; +} +paramId_t ParameterManager::GetParameterId(std::string const &name, + std::string const &type) { + paramId_t pid = kParamUnhandled; + for (size_t p_it = 0; p_it < Parameters.size(); ++p_it) { + if (name != Parameters[p_it].name) { + continue; + } + if (type.size() && (type != Parameters[p_it].type)) { + continue; + } + // matches search, check if it is the first to match the search. + if (pid != kParamUnhandled) { + throw ambiguous_parameter_specified() + << "[ERROR]: When searching for parameter by name-only, found at " + "least two matching parameters: { PID: " + << pid << ", name: " << Parameters[pid].name + << ", type: " << Parameters[pid].type << " } and { PID: " << p_it + << ", name: " << Parameters[p_it].name + << ", type: " << Parameters[p_it].type << " }"; + } + pid = p_it; + } + return pid; +} +void ParameterManager::SetParameterValue(paramId_t pid, double val) { + ValidateParamId(pid); + + if (!IsValidParameterValue(pid, val)) { + throw param_value_out_of_bounds() + << "[ERROR]: Attempting to set parameter { PID: " << pid ", name: " + << Parameters[pid].name << ", type: " << Parameters[pid].type + << " } to " << val << ", but this is out of the allowed range [" + << ((Parameters[pid].min == kDefaultLimit) + ? "unbounded" + : std::to_string(Parameters[pid].min)) + << "," + << ((Parameters[pid].max == kDefaultLimit) + ? "unbounded" + : std::to_string(Parameters[pid].max)) + << "]"; + } + + Parameters[pid].value = val; +} +double ParameterManager::GetParameterValue(paramId_t pid) { + ValidateParamId(pid); + return Parameters[pid].value; +} +double ParameterManager::GetParameterStep(paramId_t pid) { + ValidateParamId(pid); + return Parameters[pid].step; +} +double ParameterManager::GetParameterStart(paramId_t pid) { + ValidateParamId(pid); + return Parameters[pid].start; +} +double ParameterManager::GetParameterMin(paramId_t pid) { + ValidateParamId(pid); + return Parameters[pid].min; +} +double ParameterManager::GetParameterMax(paramId_t pid) { + ValidateParamId(pid); + return Parameters[pid].max; +} +bool ParameterManager::IsValidParameterValue(paramId_t pid, double val) { + ValidateParamId(pid); + return ( + ((Parameters[pid].min == kDefaultLimit) || (Parameters[pid].min < val)) && + ((Parameters[pid].max == kDefaultLimit) || (Parameters[pid].max > val))); +} + +std::string ParameterManager::StateString() { + return "Parameter Manager state:"; +} +} // namespace params +} // namespace nuis diff --git a/src/parameters/ParameterManager.hxx b/src/parameters/ParameterManager.hxx new file mode 100644 index 0000000..07f8651 --- /dev/null +++ b/src/parameters/ParameterManager.hxx @@ -0,0 +1,91 @@ +// Copyright 2018 L. Pickering, P Stowell, R. Terri, C. Wilkinson, C. Wret + +/******************************************************************************* + * This file is part of NUISANCE. + * + * NUISANCE is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * NUISANCE is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with NUISANCE. If not, see . + *******************************************************************************/ + +#ifndef PARAMETERS_PARAMETERMANAGER_HXX_SEEN +#define PARAMETERS_PARAMETERMANAGER_HXX_SEEN + +#include "exception/exception.hxx" + +#include +#include +#include + +namespace fhicl { +class ParameterSet; +} + +namespace nuis { +namespace params { + +typedef size_t paramId_t; +static paramId_t const kParamUnhandled = std::numeric_limits::max(); +static double const kDefaultLimit = 0xdeadbeef; + +class ParameterManager { + struct NamedParameter { + std::string name; + std::string type; + double value; + double start; + double min; + double max; + double step; + }; + std::vector Parameters; + // TMatrixD describing parameter covariance. + + ParameterManager(); + + static ParameterManager *_global_inst; + + void ValidateParamId(paramId_t); + + bool locked; + +public: + static ParameterManager &Get(); + + NEW_NUIS_EXCEPT(invalid_parameter_id); + NEW_NUIS_EXCEPT(param_value_out_of_bounds); + NEW_NUIS_EXCEPT(ambiguous_parameter_specified); + NEW_NUIS_EXCEPT(parameter_list_is_locked); + + ///\brief Lock the parameter list so that subsequent calls to + ///EnsureParameterRegistered cause an exception to be thrown. + /// + /// Useful for ensuring that plugins do not attempt to add parameters mid-fit. + void LockParameterList(); + void UnlockParameterList(); + + paramId_t EnsureParameterRegistered(fhicl::ParameterSet const &); + paramId_t GetParameterId(std::string const &, std::string const &type = ""); + void SetParameterValue(paramId_t, double); + double GetParameterValue(paramId_t); + double GetParameterStep(paramId_t); + double GetParameterStart(paramId_t); + double GetParameterMin(paramId_t); + double GetParameterMax(paramId_t); + bool IsValidParameterValue(paramId_t, double); + + std::string StateString(); +}; +} // namespace params +} // namespace nuis + +#endif diff --git a/src/variation/IVariationProvider.hxx b/src/variation/IVariationProvider.hxx index 5f66584..a7ccbb4 100644 --- a/src/variation/IVariationProvider.hxx +++ b/src/variation/IVariationProvider.hxx @@ -1,60 +1,60 @@ // Copyright 2018 L. Pickering, P Stowell, R. Terri, C. Wilkinson, C. Wret /******************************************************************************* * This file is part of NUISANCE. * * NUISANCE is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * NUISANCE is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with NUISANCE. If not, see . *******************************************************************************/ #ifndef VARIATION_IVARIATIONPROVIDER_HXX_SEEN #define VARIATION_IVARIATIONPROVIDER_HXX_SEEN #include "plugins/traits.hxx" -#include "exception/exception.hxx" +#include "parameters/ParameterManager.hxx" namespace fhicl { class ParameterSet; } namespace nuis { namespace event { class MinimalEvent; +class FullEvent; } // namespace event } // namespace nuis class IVariationProvider { public: - typedef int paramId_t; - static paramId_t const kParamUnhandled = - std::numeric_limits::max(); - virtual void Initialize(fhicl::ParameterSet const &) = 0; - paramId_t GetParameterId(std::string const &) = 0; + virtual nuis::params::paramId_t GetParameterId(std::string const &) = 0; bool HandlesParameter(std::string const ¶m_name) { - return (GetParameterId(param_name) != kParamUnhandled); + return (GetParameterId(param_name) != nuis::params::kParamUnhandled); } - void SetParameterValue() = 0; - bool ParametersVaried() = 0; - void Reconfigure() = 0; + virtual void SetParameterValue(nuis::params::paramId_t, double) = 0; + virtual bool ParametersVaried() = 0; + virtual void Reconfigure() = 0; + + virtual nuis::event::FullEvent + VaryFullEvent(nuis::event::FullEvent const &) = 0; virtual ~IVariationProvider() {} }; DECLARE_PLUGIN_INTERFACE(IVariationProvider); #endif diff --git a/src/variation/IWeightProvider.hxx b/src/variation/IWeightProvider.hxx index dfaf751..2feabbb 100644 --- a/src/variation/IWeightProvider.hxx +++ b/src/variation/IWeightProvider.hxx @@ -1,34 +1,41 @@ // Copyright 2018 L. Pickering, P Stowell, R. Terri, C. Wilkinson, C. Wret /******************************************************************************* * This file is part of NUISANCE. * * NUISANCE is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * NUISANCE is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with NUISANCE. If not, see . *******************************************************************************/ #ifndef VARIATION_IWEIGHTPROVIDER_HXX_SEEN #define VARIATION_IWEIGHTPROVIDER_HXX_SEEN #include "variation/IVariationProvider.hxx" -class IWeightProvider { +class IWeightProvider : public IVariationProvider { public: - double GetEventWeight(nuis::event::MinimalEvent) = 0; + virtual double GetEventWeight(nuis::event::MinimalEvent const &) = 0; + + /// For weight providers, the full variation is just the application of the reweight_weight. + nuis::event::FullEvent VaryFullEvent(nuis::event::FullEvent const &fe) { + nuis::event::FullEvent fe_clone = fe.clone(); + fe_clone.RWWeight = GetEventWeight(fe_clone); + return fe_clone; + } virtual ~IWeightProvider() {} }; DECLARE_PLUGIN_INTERFACE(IWeightProvider); #endif diff --git a/src/variation/VariationManager.hxx b/src/variation/VariationManager.hxx index 773d1c6..79fac54 100644 --- a/src/variation/VariationManager.hxx +++ b/src/variation/VariationManager.hxx @@ -1,76 +1,69 @@ // Copyright 2018 L. Pickering, P Stowell, R. Terri, C. Wilkinson, C. Wret /******************************************************************************* * This file is part of NUISANCE. * * NUISANCE is free software: you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation, either version 3 of the License, or * (at your option) any later version. * * NUISANCE is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with NUISANCE. If not, see . *******************************************************************************/ #ifndef VARIATION_VARIATIONMANAGER_HXX_SEEN #define VARIATION_VARIATIONMANAGER_HXX_SEEN #include "variation/IWeightProvider.hxx" #include "plugins/traits.hxx" #include "exception/exception.hxx" #include #include namespace fhicl { class ParameterSet; } namespace nuis { -namespace input { +namespace variation { class VariationManager { - struct NamedVariationProvider { - NamedVariationProvider( +public: + typedef size_t VarProv_id_t; + +private: + struct NamedWeightProvider { + NamedWeightProvider( std::string const &, plugins::plugin_traits::unique_ptr_t &&); std::string name; plugins::plugin_traits::unique_ptr_t handler; }; - std::vector VarProvs; + std::vector VarProvs; VariationManager(); static VariationManager *_global_inst; public: - typedef size_t VarProv_id_t; - typedef size_t paramId_t; - - struct VariationProviderParameter { - std::string name; - VarProv_id_t providerId; - IVariationProvider::paramId_t providerParameterId; - }; - - std::vector VarParams; - static VariationManager &Get(); paramId_t EnsureParameterHandled(fhicl::ParameterSet const &); void SetParameterValue(paramId_t, double); void GetParameterPull(paramId_t); - double GetEventWeight(); + double GetEventWeight(nuis::event::MinimalEvent const &); }; -} // namespace input +} // namespace variation } // namespace nuis #endif