diff --git a/INSTALL b/INSTALL --- a/INSTALL +++ b/INSTALL @@ -1,208 +1,173 @@ /** @page install Installation instructions @tableofcontents @section downloads Source file downloads The source files can be downloaded from https://www.hepforge.org/downloads/lhapdf @section quickstart Quick start instructions If you have a C++11 compiler, building LHAPDF >= 6.2 should be straightforward: wget http://www.hepforge.org/archive/lhapdf/LHAPDF-6.X.Y.tar.gz tar xf LHAPDF-6.X.Y.tar.gz cd LHAPDF-6.X.Y ./configure --prefix=/path/for/installation make make install You will then need to install PDF data files, most easily using the "lhapdf" manager script but also possible by manual download. See the LHAPDF website for details. On systems where you want to use non-system compilers and libraries, the configure command will need to be a bit more complicated: see the following for more information. Version 6.2.0 of LHAPDF, and later, have no external library dependencies. Earlier versions rely on the Boost C++ header library. We recommend upgrading to the current series to avoid that complication (and benefit from other improvements, too). @section compilers Build tools LHAPDF6 just needs your system to have a copy of Make and a C++11 compiler: there is no Fortran code and hence no need for gfortran. We have tested with both the g++ and clang++ compilers. Note that fairly recent compiler versions are needed, to get C++11 support: g++ >= 4.8.5 and clang++ >= 3.3. The `-std=c++11` flag, if needed, will be added automatically by the LHAPDF configure script. Building LHAPDF is typically straightforward on Linux systems, including CC7 and Ubuntu. Mac OS X, however, can cause problems due to inconsistent compiler and Python versions, and other such fun. If you want to build LHAPDF on a Mac, please see @ref osx. @subsection pyext Python If you want to build the Python interface to LHAPDF (which is very nice!), you will need the Python development headers to be installed (e.g. via the `python-dev` Ubuntu package). If the `Python.h` header is not found by `configure`, no Python extension module will be built. @section lhapdf Building LHAPDF If you have downloaded a release tarball for LHAPDF 6.X.Y, unpack it with `tar xf LHAPDF-6.X.Y.tar.gz`, then `cd` to the newly-created directory. @note If checking out from version control rather than unpacking a tarball, again `cd` to the new directory, but you must then also run `autoreconf -i` before proceeding to the instructions below. There will also be more requirements for external packages if you build this way, since this is the "developer" route to building LHAPDF and requires a bit more expertise. Now you should run the `configure` script to analyse your machine, compiler, etc. and set up the Makefiles. You will probably need to provide the `--prefix` argument to `configure` to tell it where you want to install LHAPDF (probably you don't want to install to `/usr/local`, which is the default). For example, ./configure --prefix=$HOME/local An example build script for LHAPDF6 on the CERN lxplus6 system is shown at the end of these instructions. @subsection altcomp Alternative compilers If you want to use an alternative C++ compiler, then you can specify the CXX variable on the command line. This is essential on OS X Mavericks and later, where the consistent compiler suite is clang rather than gcc -- in that situation, use: ./configure --prefix=... CXX=clang++ The configure script will run and produce quite a bit of output from its various tests. Hopefully everything will be successful: if it gets to the end without stopping due to an error then all is well. Then just call `make` to build the library (or e.g. `make -j4` to compile 4 files in parallel -- if your machine has enough processor cores to do so, even `-j2` will speed up the build quite a bit). To install LHAPDF to the `--prefix` location that you specified, call `make install`. You will (or at least *should(!) find installed files in `$prefix/lib`, `$prefix/include/LHAPDF`, and `$prefix/share/LHAPDF`. @section lxplus Building on lxplus / LCG -CERN's lxplus6 shared system is always an awkward environment to build packages, -since the system compiler etc. are not part of the LCG supported list of -architectures. +CERN's lxplus7 system runs CC7 Linux, whose system compiler and Python installations are +sufficiently modern to compile LHAPDF (although the system compiler is not an official +LCG platform). No special environment setup is needed on lxplus7 unless you want to +interface with precompiled LCG software. -The SLC6 system compiler is too old to support C++11, and hence -setting up a non-system compiler (and consistent Python) is mandatory even if not -wishing to interact with LCG/experiment tools; on CC7 the system compiler is (just) -new enough for C++11, and so you can use the system environment and make a completely -standard build if you don't need to be LCG-compatible. - -Here is an example of how to build LHAPDF using LCG tools on lxplus6 and lxplus7 -- -although you can of course use other compilers, Python versions, build flags, etc. as you wish. +By contrast the older lxplus6 system runs SLC6, for which the system compiler +is neither supported by LCG nor LHAPDF: on this system you will need to set up a more +modern LCG environment capable of compiling C++11 source code. Here is an example of how +o build LHAPDF using LCG tools on lxplus6 (and lxplus7) -- although you can of course +use other compiler and Python versions, build flags, etc. if you wish. ## Set up LCG compiler & Python (required on SLC6/lxplus6, optional on CC7/lxplus7) source /cvmfs/sft.cern.ch/lcg/releases/LCG_87/gcc/4.9.3/x86_64-slc6/setup.sh source /cvmfs/sft.cern.ch/lcg/releases/LCG_87/Python/2.7.10/x86_64-slc6-gcc49-opt/Python-env.sh ## Make an install directory mkdir local ## Build LHAPDF wget http://www.hepforge.org/archive/lhapdf/LHAPDF-6.X.Y.tar.gz -O- | tar xz cd LHAPDF-6.X.Y ./configure --prefix=$PWD/../local make -j2 && make install cd .. ## Set environment variables export PATH=$PWD/local/bin:$PATH export LD_LIBRARY_PATH=$PWD/local/lib:$LD_LIBRARY_PATH export PYTHONPATH=$PWD/local/lib64/python2.6/site-packages:$PYTHONPATH ## Test the scripts lhapdf-config --help lhapdf list @section osx Building on Mac OS X Builds are typically straightforward on Linux, but Mac OS X unfortunately has a long history of incoherent system compiler setups, which have been worked around manually by users' private installations of Fink, MacPorts, HomeBrew and manual tarball installations of required tools. These work-arounds can themselves be the source of problems when the native compilers or Python libraries get updated, and due to the ad hoc nature of such installations we are restricted in how much we can help to get LHAPDF to compile on a broken system: it is the user's responsibility to make sure that their machine has a consistent set of build tools! From experience, the simplest reliable route seems to be to run a fresh copy of OS X 10.9 Mavericks (or later) without any additional manual compiler installations: if you use the clang++ compiler on such a system, LHAPDF6 building should "just work". @note At the time of writing there is a bug in the Mac Python version which requires that you call `export CPPFLAGS=-Qunused-arguments` and `export CFLAGS=-Qunused-arguments` before building. Alternatively you can run the configure script with `--disable-python`, which avoids the bug at the cost of not building the very useful Python interface to LHAPDF. The Mac OS X "Homebrew" system (http://brew.sh/) comes recommended by several LHAPDF developers. Many HEP packages are already available for Homebrew via the homebrew-hep project: http://davidchall.github.io/homebrew-hep/ . Success has also been reported with the MacPorts system (http://www.macports.org/): please see HepForge's information about MacPorts at https://www.hepforge.org/docs/macosx . With both these approaches, you should set your environment to *only* use compilers and Python from the Brew/Ports area and to ignore the system packages: a hybrid approach will only cause unnecessary pain. - -@section boost Boost (for LHAPDF < 6.2) - -Until LHAPDF 6.2.0, we made use of header files from the Boost C++ utility -library (http://www.boost.org). This made building LHAPDF a bit more complex, -and for reference we include some troubleshooting information here... for -now. The much better solution, however, is to use the current version which has -no such dependency. - -If Boost is installed in the system `/usr` directory tree, via your machine's -packaging system, it should be discovered automatically. If not, you will need to use -something like - - ./configure --prefix=$HOME/local --with-boost=$HOME/pkgs/boost-1_58 - -for your LHAPDF configuration step. - -On Linux machines, Boost this should typically be available via -your system's packaging mechanism, e.g. the `libboost-all-dev` package on Ubuntu -and other Debian derivatives. On Scientific Linux 6 the system installation of -Boost is sufficient. - -Building Boost by hand is not particularly simple and should not be needed in -most cases: we recommend avoiding this! If you have access to the CERN AFS -filesystem, you can find builds of Boost for various platforms in the -`/afs/cern.ch/sw/lcg/external/Boost/` directory. If you really want or need to -do it manually, version 6.0.5 and later of LHAPDF only use Boost headers -(previous ones used compiled libraries) and rather than fully building Boost, -you can just use the header files direct from its source tarball: this is not -wonderfully neat, but is a lot easier than doing a full manual build and -installation of the Boost libraries. - - */ diff --git a/doc/main.dox b/doc/main.dox --- a/doc/main.dox +++ b/doc/main.dox @@ -1,246 +1,245 @@ /** @mainpage @tableofcontents @section intro Introduction LHAPDF is a general purpose C++ interpolator, used for evaluating PDFs from discretised data files. Previous versions of LHAPDF were written in Fortran 77/90 and are documented at http://lhapdf.hepforge.org/lhapdf5/. LHAPDF6 vastly reduces the memory overhead of the Fortran LHAPDF (from gigabytes to megabytes!), entirely removes restrictions on numbers of concurrent PDFs, allows access to single PDF members without needing to load whole sets, and separates a new standardised PDF data format from the code library so that new PDF sets may be created and released easier and faster. The C++ LHAPDF6 also permits arbitrary parton contents via the standard PDG ID code scheme, is computationally more efficient (particularly if only one or two flavours are required at each phase space point, as in PDF reweighting), and uses a flexible metadata system which fixes many fundamental metadata and concurrency bugs in LHAPDF5. Compatibility routines are provided as standard for existing C++ and Fortran codes using the LHAPDF5 and PDFLIB legacy interfaces, so you can keep using your existing codes. But the new interface is much more powerful and pleasant to work with, so we think you'll want to switch once you've used it! LHAPDF6 is documented in more detail in http://arxiv.org/abs/1412.7420 @section installsec Installation The source files can be downloaded from https://www.hepforge.org/downloads/lhapdf Full installation and troubleshooting details are given at @ref install . @section sets Official PDF sets In the move to LHAPDF6 we have migrated the most recent PDF set families of each major PDF fitting group from LHAPDF5 to the new unified LHAPDF6 format. With a few exceptions for particularly significant older PDF sets, this means an effective age cutoff of ~2005 on the migrated sets. If you need older sets you are encouraged to convert the LHAPDF5 version to LHAPDF6's data format: see the @ref migration page for details of this procedure. We do not provide support for LHAPDF5, and a very strong case will be needed for further "official" set migrations beyond the many already available in LHAPDF6. The following link takes you to the PDF sets which are currently available and officially supported: - @ref pdfsets The PDF set data files can be downloaded from https://www.hepforge.org/archive/lhapdf/pdfsets/ and the latest versions are -installed (as tarballs and expanded into directories) on CERN AFS at -`/afs/cern.ch/sw/lcg/external/lhapdfsets/current/` and CERN CVMFS at +installed (as tarballs and expanded into directories) on CERN CVMFS at `/cvmfs/sft.cern.ch/lcg/external/lhapdfsets/current/`. Many users will find it easiest -to just set their `LHAPDF_DATA_PATH` environment variable to point at one of these -distributed filesystem areas and use the preinstalled latest sets there. +to just set their `LHAPDF_DATA_PATH` environment variable to point at this +distributed filesystem area and use the preinstalled latest sets there. We also provide a script called `lhapdf` which can be used to query the catalogue of PDF sets and to install and update them from the command line. The user interface of this script is inspired by the Debian Linux `apt` package management system -- it accepts commands `list`, `update`, `install` and `upgrade`. Please run `lhapdf help` for full usage instructions. Please note that this script currently requires the LHAPDF library to have been built and installed with Python support. If you want to add new PDF sets to the standard collection, please contact the LHAPDF authors at lhapdf@projects.hepforge.org. You are expected to create and validate these PDF data files yourself, and to fix and update them in the case of user-reported bugs in the public files. We will allocate appropriate PDF ID codes, which will need to be added to your new sets' `.info` files, and manage the indexing and distribution of the new files. Thanks for your support! @section usage Usage As for previous versions, LHAPDF6 is a programmatic library. It can be accessed via user code written in C++, and to a more limited extent from Python and Fortran programs. PDF sets (each of which is stored in a unique filesystem directory) should usually be installed in the `$prefix/share/LHAPDF/` directory (i.e. the PDF dirs are at the same level as the global `lhapdf.conf` file). To make use of PDF sets installed in other places, those search paths should be listed in the `LHAPDF_DATA_PATH` environment variable. [`LHAPDF_DATA_PATH` is an extension of the old `LHAPATH` variable which supports multiple search paths separated by colon (`:`) characters, cf. standard system paths like `PATH`, `LD_LIBRARY_PATH`, etc.] Here are some reference documents on the library design and the system of PDF/set/config metadata flags: - Design rationale: @ref design - PDF metadata flags: @ref config Many code usage examples are collected here: - Code examples: @ref codeexamples @subsection buildagainst Building against LHAPDF Using LHAPDF through Python just requires that the LHAPDF library and module be installed, then you can use it: there are no compilation or linking complexities. For C++ and Fortran, however, you need to compile your code against LHAPDF, which introduces some extra technical hurdles. Building your own program that uses LHAPDF6 is aided by the `lhapdf-config` script which can among other things produce the compiler flags needed to use LHAPDF. These include the `-I/path/to/include` flag to indicate the position of installed LHAPDF header files, and `-L/path/to/lib -lLHAPDF` flags for the library linker. The script is used in a compilation/linking command like this: g++ mycode.cc -o myexe `lhapdf-config --cflags --ldflags` If you are compiling a Fortran program against LHAPDF, the header files are not relevant, but the library details are. You will also need to explicitly link against the C++ standard library, which is done implicitly by C++ compilers: gfortran mycode.f90 -o myexe `lhapdf-config --ldflags` -lstdc++ @subsection runtimeld Runtime symbol resolution Finally, note that LHAPDF is primarily available as a "dynamic" shared library, i.e. a `.so` or `.dylib` (Mac) rather than a static `.a` library. The dynamic library needs to be found at runtime as well as at compile/link time. If your code compiled fine, but you get errors about "missing symbol" when you try to run your program, you probably need to add the LHAPDF lib directory path to your `LD_LIBRARY_PATH` environment variable (or `DYLD_LIBRARY_PATH` on Macs): export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/lib @subsection strip Trick to remove unwanted PDF members A typical LHAPDF6 PDF set with error members is between 20 and 100 MB in size. If you only intend to use a subset of these (e.g. in MC production where you will only use the central member), you can simply remove the unneeded `.dat` files from the PDF set directory. It will work fine, as long as you don't try to load one of the missing members of course! In grid job submissions this can reduce the tarball of data that you send with your job by a large amount, since PDF sets often dominate such job tarballs. @subsection zip Trick to use zipped data files For special applications where PDF initialization speed or disk space are _very_ critical, you may wish to use zipped PDF data files. Since the data files are plain text, they compress well, by a factor of 4 or so, but for most applications this is largely irrelevant since you will usually not need to install more than about 100 MB of data files to use the main modern PDF sets, and set initialisation time is less than one second... and that happens only once per run. Actually reading zipped data files would make LHAPDF's code and (more importantly) build process much more awkward, so we haven't done that. However, if you really need to squeeze the maximum out of limited space and time then there is a trick you can play -- at least on Linux systems -- using `LD_PRELOAD`. This environment variable specifies libraries to be loaded before starting a process, and is a mechanism for replacing system library functions with modified versions. In this case, we can override the `open("foo")` function with a version which will attempt to transparently read `foo.gz`, uncompressed into the `/tmp` directory, by setting `LD_PRELOAD=/lib/uncompress.so` in your job. See the [zlibc](http://www.delorie.com/gnu/docs/zlibc/zlibc.3.html) documentation for more details. LHAPDF6 itself needs no modification, but you will need to `cd` into your PDF sets and run `for i in *.dat; do gzip $i; done` in each one that you want to compress. This is rather a hack and may not work on Macs... but it can be useful in special circumstances. Please let us know your experiences if you use it! @section authors Authors @include AUTHORS @section support Support and bug reporting If you need assistance with LHAPDF, please contact the authors at lhapdf@projects.hepforge.org . Please also send feature suggestions to this address: please don't contact individual developers if you can avoid it. We also accept compliments ;-) You are _very_ strongly advised to make sure that you are running the latest version of the LHAPDF library, since issues are often fixed in later releases. Please supply some information about which version you are using, what type of system and compiler you are using, a copy of the LHAPDF config.log file, a reasonably full copy of the errors you are getting, and the output of downloading and running this script: https://users.hepforge.org/~buckley/sysdebug.sh . Please do not send us screenshots of your terminal! To capture the output of a command, use a shell redirection like this: `make &> makelog.txt` or `bash sysdebug.sh &> sysdebug.txt` (you may prefer to use a redirection like `|& tee foolog.txt` but probably you don't need our advice if you have such preferences!) Lastly, please try to _read_ the configure/compiler/runtime error message: once decoded it may tell you something simple e.g. you have not got the required compiler, you are compiling against a 32 bit library on a 64 bit system, or you have not installed a PDF data file. If you can't parse the error messages then try Googling for generic-looking bits of the message before asking for direct assistance: there are often good explanations and solutions online. @section dev For developers To check out LHAPDF6 from Mercurial, use this command for read-only: `hg clone https://phab.hepforge.org/source/lhapdfhg/`, and this for read-write: `hg clone ssh://vcs@phab.hepforge.org/source/lhapdfhg` (requires a HepForge login account.) See the following pages: - Code browser: https://lhapdf.hepforge.org/files.html - @ref todolist - @ref todo - @ref migration - @ref codingstyle - @ref design @todo Add some developer build guides, including the autotools, Cython, etc. gotchas. @example ../examples/testpdf.cc @example ../examples/pythonexample.py @example ../examples/compatibility.cc @example ../tests/testgrid.cc @example ../tests/testinfo.cc @example ../tests/testindex.cc @example ../tests/testpaths.cc @example ../tests/testalphas.cc @example ../examples/analytic.cc @example ../examples/testpdfunc.cc */