Release Notes for QSNETMPI.1.24-xx ================================== This is an Open Source release of the Quadrics QsNet enhanced version of the ANL MPICH MPI implementation For installation instructions and notes please see the README in the MPI.1.24-2X directory. For licensing information please see the ANL MPICH documentation. ========================================================================= This kit contains the following components: qsnetmpi - the QsNet enabled/enhanced version of MPICH ========================================================================= Known issues with this release: Binary Compatibility -------------------- The lack of binary compatibility between libraries built with different versions of the compilers makes it impossible for Quadrics to provide a generic binary mpi release and it is increasingly difficult to provide all the permutations and combinations required by all our customers. We currently offer the following binary releases and customers who need versions built with other compilers will have to recompile from the src rpm. * MPI built with gcc 2.X and Intel 9.0 for Linux_i686, Linux_ia64 and Linux_x86_64 * MPI built with ccc Compaq C T6.5-002 for Linux_alpha As of MPI release MPI.1.24-41 we have made some changes to the build process which will make rebuilding from the source and with different versions of the Intel compilers easier. Rebuilding MPI -------------- To rebuild the MPI rpm you need the src rpm which can be found in the SRPMS subdirectory of the MPI software product (SRPMS/qsnetmpi.1.24-49.intel9.0.src.rpm for example). If you don't have root permission or would prefer not to rebuild as root (it is recommended that you don't rebuild as root) you will need to create a .rpmmacros file. This should be in your home directory and should set the following two variables (replacing /home/user/rpms with the path of your build directory) %_topdir /home/user/rpm %_tmppath /home/user/rpm/tmp You will need to create a SOURCES, BUILD, RPM and SRPMS directories in %_topdir before executing the rpmbuild command: mkdir -p /home/user/rpm/SOURCES mkdir -p /home/user/rpm/BUILD mkdir -p /home/user/rpm/RPMS mkdir -p /home/user/rpm/SRPMS rpmbuild --rebuild [--target=i686] qsnetmpi.1.4.24-49.intel9.0.src.rpm You will end up with an rpm in %_topdir/RPMS/ (/home/user/RPMS/ in the example given). The rpm will have the appropriate intel version number (e.g. .intel81 or intel9.0) as part of the rpm version number. TIP: Locating these directories in /tmp can improve build times versus using a network filing system such as NFS. Support has been added for rebuilding with a different version of the Intel compilers. The default is to build with Intel 9.0 compilers from the default Intel C and Fortran installation directories (i.e. /opt/intel/[f|c]c[e]/9.0). You can set the following two variables in your environment to build with a different compiler version or to indicate a different install path for the compilers: * INTEL_HOME - this should be set to the base of the Intel compiler installation directory, i.e. the directory containing the 'bin' and 'lib' directories. For example /opt/intel/compiler81 where that directory contains both the Intel C and Fortran compiler installations. For Intel compiler versions 9.X and later this environment variable should not be set if the compilers have been installed in their default locations. * INTEL_COMPILER_VERSION - should be set to 80 for Intel compiler 8.0 or 81 for Intel compiler 8.1 etc. For Intel compiler versions 9.X and later, which have been installed in the default locations, it should be set to 9.X, i.e. the version number retains the '.' Dependencies ------------ * The elan _[un]pin_region_ change in MPI.1.24-43 introduces a dependency on QSNETLIBS.1.4.31 and QSNET2LIBS 1.8.11 or later. ====================================================================== Release notes for MPI.1.24-49 Aug 2006 ====================================================================== * GNAT 8349: Fortran 'C' stubs were not being compiled with '-g' compiler flag and hence stack traces were broken. * GNATs 7816, 7878, 8352, 8247: Backported ROMIO from MPICH 1.2.7p1 code base. ====================================================================== Release notes for MPI.1.24-48 Dec 2005 ====================================================================== * GNAT 7942: Ensure that the correct totalview library is pointed to by the i8 library. * GNAT 7816: MPI_File_write_all hang with multiple procs per node fixed by backport from MPICH 1.2.27 * Add a symbolic link from libmpich.so.0.1 to libmpich.so so programs linked against mpich externally can work. * Checking for valid ops inside MPI_Reduce and MPI_AllReduce ====================================================================== Release notes for MPI.1.24-47 1st Aug 2005 ====================================================================== * Re-instated hard-coded path to the totalview library to workaround problems seen at LLNL. * Renamed the totalview library from libtvmpich.so to libtvelan.so to avoid naming conflicts. * Added support for building with PGI compilers. * Added support for the new MPI-2 type MPI_UNSIGNED_LONG_LONG. * Fixed some issues when mapping MPI datatype and operations to the optimised elan_reduce() implementation. * Modified MPI_Wtick() to parse the env var 'MPI_SET_WTICK' so that a user can overload the default. Also changed the default to be 1us (1e-6) as that's more realistic due to PCI read overheads. ====================================================================== Release notes for MPI.1.24-45 16th Jun 2005 ====================================================================== * The Quadrics version of 'mpirun' has been removed from the MPI release. It is now released in the qsnetlibs rpm (QSNETLIBS software product). * Fix for GNAT 7208, don't specify a full path for the Totalview callback library but rely on the standard linking mechanism to find it for us. This introduces a potential problem if attaching to a process and the environment is different between the application and totalview itself. * Fix for GNAT 7577, put/get with target self and non-contiguous types gives wrong answer. * Fix for GNAT 7560, AlltoAll, Gather, AllGather get the wrong answer if the target datatype is a non-contiguous datatype but the source datatype is contiguous. This situation will now provoke an Elan exception message so that users can find out if their code was experiencing this bug. This exception can be disabled by setting MPI_REPORT_COLLECTIVE_TYPE_MISMATCH=0. * Fix for GNAT 7232, Onesided put/get doesn't work for non-contiguous remote datatypes. ====================================================================== Release notes for MPI.1.24-43 21st Mar 2005 ====================================================================== * Support has been added for the Intel version 8.1 compiler for EM64T. The x86_64 release now contains /usr/lib/mpi/mpi_intel as well as /usr/lib/mpi/mpi_gnu as for the i686 and ia64 releases. * Added '-fp' to intel build flags to fix a problem with incomplete back traces (IA32). * GNAT 7321: MPI_Request_f2c() was broken to due missing declaration of MPIR_ToPointer() * Added Patch Free support to MPI-2 Onesided operations by pinning and unpinning memory in MPI_Win_create/MPI_Win_destroy. To avoid undefined reference linkage errors about 'elan_[un]pin_region' when compiling MPI programs, QSNETLIBS 1.4.31 and QSNET2LIBS 1.8.11 or later should also be installed. * Added =mpirun= implementation, using RMS, PDSH, SLURM and falling back to hand-crafted capabilities where available. This supports the standard =-np= and =-machinefile= options, and provides additional options. mpirun [-hvs] [-m method] [-d level] [-o offset] [-np nproc] [-machinefile file] exe [args ...] -v verbose -s use ssh over rsh when prun, srun and pdsh unavailable -t show what commands will be executed -d level debugging level -o offset netid offset for handcrafted capabilities -m method force method to one of prun, srun, pdsh, handcraft -np nproc number of processes to spawn -machinefile file machine file specifying hosts Note: files need to be creatable in the current directory for prun support * GNAT 6025: Implemented orphan list code to prevent descriptor leaks when MPI_Request_free() is used in conjunction with MPI_Isend(). * GNAT 4771: Fixed problem wit MPI_Get_elements() returning the wrong value as was including zero-sized datatypes. * GNAT 4764: MPI_Bsend() should return an error if no buffer is attached. ====================================================================== Release notes for MPI.1.24-42 17th Jan 2005 ====================================================================== * GNATs 7100,7247: Fixed a bug where the mpi_wtime_ function prototype was incorrect. This broke weak symbol generation on x86_64 systems resulting in 'undefined reference to `mpi_wtime_' messages. * Fixed a couple of problems in the Onesided code where put/get operations being performed on self were using an uninitialised variable. This could lead to buffer overruns and possible corruptions. * Enabled bug fix for Fortran 32-bit mpi_address() on LP64 architectures. Code now represents 64-bit C pointer values as 32-bit Fortran integers and converts them back again using a lookup cache. Cache supports 4 regions of 1GB in size, if this is insufficient for the application then the MPI2 mpi_get_address() function should be used instead. * Added correct weak prototypes for MPI2 mpi_get_address() function. * GNAT 7115: When MPI communicator creation failed due to MPIR_ERR_TOO_MANY_CONTEXTS it displayed an incorrect text error message due to the MPICH error message table being messed up. Quite a few other error codes were also broken in this way. * Added Patch Free support to MPI-2 Onesided operations by pinning and unpinning memory in MPI_Win_create/MPI_Win_destroy * Made MPI-2 onesided operations MPI_Win_complete and MPI_Win_wait honour the specification by ensuring that all RMA operations have been completed in the origin and target windows respectively. * Added support for the Portland compiler. This involved removing MEMBARS from the one-sided code, using new kernel headers and disabling MEMBARS when compiling the totalview code. * The open source build mechanism has been changed so that the intallfiles.sh script and the rpm filelist are generated as part of the build rather than being fixed files. This fixes some problems where the filelist was architecture dependent and so the .src.rpm was not generic. ====================================================================== Release notes for MPI.1.24-41 16th Nov 2004 ====================================================================== * We now release a version of MPI built with version 8.1 of the Intel compilers (MPI.1.24-41.intel81). Note that we will no longer release a version built with version 7.0 of the Intel compilers after the end of December 2004. * Added initial totalview support for elan4 systems. * Made some changes to the top level Makefile and the build process which means that the Intel compiler version can be set in the environment when executing 'rpmbuild' on the src rpm. The two variables which can be set are: * INTEL_COMPILER_VERSION - the default is 81 (for version 8.1) but it can be set to 70 (for version 7.0) or 80 (for version 8.0). * INTEL_HOME - this should be set to the base of the Intel compiler installation directory, ie. the directory containing the 'bin' and 'lib' directories. The default is /opt/intel/compiler81. Instructions on rebuilding with different versions of the compiler are given in the 'Rebuilding MPI' section above. * Made the qsnetmpi rpm relocatable, with the relocatable prefix set to /usr/lib/mpi. This means that at install time '/usr/lib/mpi' can be replaced with an alternative installation directory. This allows more than one version of the rpm to be installed. For example, if the version compiled with version 8 of the Intel compilers is installed in the normal way into /usr/lib/mpi the version compiled with version 7 could be installed into /usr/lib/mpi-intel70 by executing: rpm -i --prefix=/usr/lib/mpi-intel70 ./qsnetmpi... Users who wished to use the version 7 Intel compilers would then need to add /usr/lib/mpi-intel70/mpi-intel/bin to their PATH and /usr/lib/mpi-intel70/mpi-intel/lib to their LD_LIBRARY_PATH. * Fixed a bug which meant that the mpi-intel version of mpiCC failed to link correctly. * GNAT 6818: Made 'mpidefault' more robust to always correctly report the default mpi compiler. ====================================================================== Release notes for MPI.1.24-40 30th Sep 2004 ====================================================================== No functional change - just a rebuild on SLES 8 for x86_64 to ensure backwards compatibility. ====================================================================== Release notes for MPI.1.24-39 1st Sep 2004 ====================================================================== . GNAT 6777: Backported a fix from the ANL MPICH 1.2.5 release to the bsendutil.c code to try and avoid crashes we have seen in the Bsend code. . Merged in MULTI_DEV code from Bull. Only relevant to Bull machines running with their own optimised MPI stack. . Modified the MPICH Alltoall and Alltoallv algorithms to post the receives in reverse rank order as this is more likely to match the message arrival order. . GNAT 6744: Defined MPID_MAX_CONTEXT to be 12-bits for Elan4 as it requires the upper 4-bits of the sender id space for internal TPORT use. Enabled the Jim Cownie bitmask based context id allocation (MPIR_Context_alloc) and made sure any error code is propogated up. ====================================================================== Release notes for MPI.1.24-38 19th Jul 2004 ====================================================================== . Added a %verify option to the rpm filelist so that rpm --verify does not object when the /usr/lib/mpi/default symlink points to mpi_intel rather than mpi_gnu. . Fixed a problem with the C++ build on i686 which resulted in incorrect definition of MPI2CPP_HAVE_BOOL (GNAT 6752). . Fixed a problem with the x86_64 build which resulted in no C++ support in the previous x86_64 release (MPI.1.24-37 only, MPI.1.24-36 was OK). ====================================================================== Release notes for MPI.1.24-37 18th Jun 2004 ====================================================================== . GNAT 6686: Fixed a build problem which resulted in the mpi_intel version of the mping binary having a hardwired rpath to the location of the intel libraries on the build machine. . GNAT 6696: Downgraded elan_checkVersion() to be a silent warning (-mpichdebug) rather than a fatal exception. . GNAT 6697: Modified code to set elanGroup pointer to NULL if elan_createSubGroup() fails. The prevents subsequent crashes in elan_createSubGroup() when it is passed a ug == -1 ====================================================================== Release notes for MPI.1.24-36 27th May 2004 ====================================================================== . Build changes for Linux_alpha and Linux_x86_64 only. ====================================================================== Release notes for MPI.1.24-35 20th May 2004 ====================================================================== . Made to work with a changed ELAN_BASE structure to fix a problem with using unnamed unions when compiled with Intel compilers on ia32. ====================================================================== Release notes for MPI.1.24-34 19th May 2004 ====================================================================== . Changed mpidefault script to output current setting, when executed with no arguments (Linux). . Fixed mpidefault script to give the usage when an unrecognised option argument is given (Linux). . Fix for GNAT 6528,6599: Creation of Elan subgroup may return -1 when out of memory and hence needs to be handled carefully. . Fix for GNAT 6244. The released i8r4 (ie lib_i8) and i4r8 (ie. lib_r8) libraries were promoting reals and integers to 8 bytes respectively. mpich is designed to follow the fortran standard which requires sizeof(INTEGER) == sizeof(REAL). Four flavours of the libraries was therefore not realistic. The i4r8 and i8r4 should be bitwise identical to the i4r4 and i8r8 libraries respectively. The r8 and non r8 usage should only be reflected in the header files. In this release four flavours of header files are released but only two flavours of the libraries - the release will have symbolic links from lib_r8 to the vanilla and lib_i8 to i8r8 so that the usage remains the same. f90 modules for i8r4 and i4r8 are not supported. . Fix for GNAT 6257 (OSF1) - /usr/shlib/libtvelanmpi.so was incorrectly symbolically linked to /usr/opt/mpi/lib/libtvmpich.so. when it should have been /usr/opt/mpi/shlib/libtvmpich.so. . Fix for GNAT 6307 - the mpicxx script was missing from the release. . Fix for GNAT 6240 - Inter-communicators (i.e. created by MPI_Intercomm_merge) can have a NULL adiCollCtx. ====================================================================== Release notes for MPI.1.24-33 1st Mar 2004 ====================================================================== Note: The mpi_intel part of MPI.1.24-33 was compiled for i686 and ia64 with Intel version 8 compilers and will only work with programs also compiled with Intel version 8 compilers. . Added support for V8 Intel compilers. . Fixed the mpidefault script to use bash (Linux). ====================================================================== Release notes for MPI.1.24-32 9th Feb 2004 ====================================================================== . The C++ compiler for Linux_alpha was incorrectly set to be g++. This has been changed to cxx (GNAT 6059). ====================================================================== Release notes for MPI.1.24-31 30th Jan 2004 ====================================================================== . Fixed a problem where libmpio functions were unresolved if using mpicc which was missing '-lmpio'. GNAT 6106. . The shared versions of libmpio and libpmio were missing from the release. This has been fixed. . Added the link path to the QsNet libraries to the C++ configure script. This fixed a problem where checking for the C++ build failed with the message 'configure: error: cannot continue -- cannot link to -lmpi'. . Modified mping.c example program to produce output consistant with other *ping programs. Reduced default rep count to 1000 and tidied up use of size_t throughout . . Added the linkpath to the QsNet libraries to the C++ configure script. This fixed a problem where checking for the C++ build failed with the message 'configure: error: cannot continue -- cannot link to -lmpi'. . Added '--disable-f90' to the configure options when building with gnu compilers as there isn't a gnu version of f90. No mpif90 is now released in /usr/lib/mpi/mpi_gnu/bin (GNAT 5914). . Added 'F90INC=-I' to the build so that mpif90 uses -I to pick up correct mpif.h header file rather than making a symbolic link to it in the current directory (GNAT 5918). . Added test in MPID_Recv for maxlen being < 0 as PMB2 was broken and posting receives with -ve sized counts which eventually lead to MPI_ERR_TRUNCATED errors. Incorrect code will now get MPI_ERR_COUNT errors instead. . Removed incorrect HasDashC=1 line from mpif77 script as it breaks compilation (GNAT sw-mpi/6032). . Back ported weak symbol fixes from MPICH 1.2.5.2 release. These should fix the problems seen with Opterons and/or GCC 3.x (GNAT sw-mpi/5593). . Modified mping example program to do a memset of the rbuf during initialisation. This causes the pages to be created and be made visible to the Elan improving performance significantly. ====================================================================== Release notes for MPI.1.24-30 7th Jan 2004 ====================================================================== . Implemented ELAN_TPORT_RXLOCAL hint for optimised intra-node MPI. Will enable Shared Memory optimisations in post QSNET2LIBS 1.5.4 releases, ignored in QSNETLIBS and older releases. . GNAT sw-mpi/5960: Mirrors of COMM_WORLD must also have an identical mapping of lrank_to_grank. This fixes data corruption problems in the ANL testsuite 'allred' test. ====================================================================== Release notes for MPI.1.24-29 19th Dec 2003 ====================================================================== . The build procedure has been changed so that src rpm and source tarball are output as they are for other Quadrics open source products. . The build has been changed so that when building with the gnu compilers F90 is not set. It was being set to f77 which resulted in spurious f90 module files being produced. . The mpif.f90 file is now released into /usr/lib/mpi/src. When using a 3rd party f90 compiler 'mpif.f90' should be compiled first (with -c). This produces a valid .MOD file for the f90 compiler. You can then build F90 applications using 'use mpi' and setting your include path (-I ) to the directory the .mod file is in. All this can be done in user space. . Added workaround for MPI subgroup creation memory exhaustion. Now when a subgroup cannot be created due to lack of Elan global memory the code will backoff to the MPICH pt2pt collective algorithms. . Worked around a bug in the Intel 7.1 compiler where the MPI_Dims_create() weak function stub was being incorrectly linked causing failures in the B_EFF benchmark. . Changed MPID_TRANSPORT string to be "ELAN (QsNet/QsNetII)". ====================================================================== Release notes for MPI.1.24-28 25th Sept 2003 ====================================================================== . Fixed yet another problem with the Linux upgrade from releases prior to MPI.1.24-20 which left the install missing most of the libraries and binaries if you upgraded rather than removed and installed (GNAT 5549). . Added a symbolic link in the Linux release from /usr/lib/mpi/man to /usr/lib/mpi/default/man (GNAT 5621). . Incorporated the following changes from HP (for OSF1 and Linux): . romio/adio/common/cb_config_list.c: Un-initialized variable "found" is set to 0 (false) to prevent potential problems. . romio/mpi-io/open.c: Changed the initial call to ADIO_Open to use MPI_COMM_SELF rather than the original communicator. Internally, ADIO_Open uses attribute caching to define characteristics which it adds to both communicators which are passed to it. Since the first call to ADIO_Open is only from peer 0, if the original communicator is used then only peer 0 in that communicator will have the attribute cache values attached to it's communicator. All additional peers will *NOT* have the cached attribute and will hang in an MPI collective while peer 0 continues on... eventually blocking on a different MPI collective! . mpich/src/onesided/alloc_mem.c: Added the weak symbol approach to defining the profiling entry-points. This replaces the #ifdef RMA_DO_PSHIFT approach that was previously used. This avoids having undefined symbols when compiling/running. . Incorporated the following changes from HP (for OSF1 only): . romio/mpi-io/Makefile.in: Added -DDEC to the CFLAGS environment. . mpich/src/fortran/include/mpi_fort.h: Added #ifdef USE_FORTRAN_STACK_INFO declaration to reintroduce comparison of a fortran buffer address to the internal variable MPIR_F_MPI_BOTTOM. This allows the 'C' version of MPI_BOTTOM to be utilized when fortran wrapper make a call to the real function (See related mpich/src/fortran/Makefile.in change). . mpich/src/fortran/src/Makefile.in: Added -DMPI2_ONESIDED, -E_FORTRAN_STACK_INFO, etc. definitions so that the fortran interface gets correctly defined. . mpich/src/fortran/src/addressf.c: Added an address caching mechanism to allow a 32 bit fortran interface to work correctly with datatype constructors which require 64 bit pointers. Unlike the earlier SC implementation, this approach is not "platform specific", i.e. relying on a specific layout of the stack and heap in the address space. A more complete explanation is included in the code. In addition to the above fix for MPI-1, I added the MPI-2 MPI_Get_address wrapper so that fortran functions can utilize the newer interface which doesn't restrict the return value to a 32 bit integer. . mpich/src/fortran/src/type_[hindf|structf].c: Added code to call the new caching mechanism in addressf.c to retreive the actual 'C' pointer values prior to calling the 'C' implementation. . mpich/src/pt2pt/address.c: Added the MPI-2 version of this function (Along with the profiling interfaces). . mpich/util/makesharedlib: Since the current build utilized weak symbols for defining the profiling interface, it's unnecessary (and evidently causes problems) to link against both -lpmpi and -lmpi (since the latter defines both). Thus when creating libmpio.so (and libpmpio.so), the link against -lpmpi has been removed. ====================================================================== Release notes for MPI.1.24-27 28th July 2003 ====================================================================== Note that the use of subgroups in MPICH has been enabled in QSNETLIBS.1.4.12-1 . Fixed MPICH build defines to include -DUSE_ELAN_NANOSEC_CLOCK so that MPI_Wtime() uses the elan clock and not gettimeofday() (GNAT sw-mpi/5438). . Applied changes from HP to support new ROMIO structures and ADIO changes to what was previously known as CFS in the romio adio directory. This has been renamed SCPFS (ad_scpfs) for SC Parallel file system. . libmpio.a and libmpio.so (and libpmpio.a/libpmio.so) had become bundled with libmpi. They are now released separately again (GNAT 5452). . Fixed the mpi_intel mpiCC script to use icc or ecc rather than g++ (GNAT 5419). ====================================================================== Release Notes for QSNETMPI.1.24-26 13th June 2003 ====================================================================== * Support for x86_64 added (for GNU compilers only). Note that libg2c must be installed for the x86_64 release to work. * Fixed a problem with the error message when the version of MPI used differed across the job ====================================================================== Release Notes for QSNETMPI.1.24-25 21st May 2003 ====================================================================== * Shared library search paths now use -rpath-link rather than -rpath in all the scripts (mpicc, mpif77 etc). * Added a "Compile stamp" or version string to the library, at startup all processes compare the version string and refuse to run if the versions differ across the job. atlas0:ashley> mping --elan-version MPI: Compile stamp is 'robin@fab0 Wed May 21 15:47:33 BST 2003' MPI: compiled against libelan version '1.4.10' MPI: linked against libelan version '1.4.10' MPI: Compatible libelan versions found atlas0:ashley> ====================================================================== Release Notes for QSNETMPI.1.24-24 16th May 2003 ====================================================================== * A symbolic link from /usr/lib/mpi/src to default/src has been added (GNAT 5230). * The 'setmpidefault' script has been renamed to 'mpidefault' and it now has a '--help' option which shows the usage string (GNAT 5226). * The mping being released was erroneously being statically linked, this has been fixed. * mpicc and mpiCC have been changed to use shared libraries (UseSharedLib=yes) by default. * Added some "--elan*" command line argunments to MPI programs to allow the user to change the behaviour of MPI from the command line. atlas0:ashley> mping --elan-help Commmand line argunments that MPI parses: --elan-help Show this help --elan-version Display version information --elan-no-check-version Don't abort when linked with incompatable libelan --elan-abort-with-core Wether to dump core file --elan-base-flags Flags to pass to elan_baseInit() --elan-tport-flags Flags to pass to elan_tportInit() --elan-group-flags Flags to use then creating elan groups --elan-lock-flags Flags to pass to elan_lockCtrlInit() --elan-mqueue-flags Flags to pass to elan_mQueueInit() --elan-global-alloc Globally allocate q --elan-rma Enable/Disable rma --elan-coll Enable/Disable optimised collectives --elan-coll-sub Enable/Disable optimised collectives --elan-coll-simple Enable/Disable optimised collectives atlas0:ashley> ========================================================================= Release Notes for QSNETMPI.1.24-23 11th April 2003 ========================================================================= * Rebuild for compatibility with latest Elan libraries and header files. ========================================================================= Release Notes for QSNETMPI.1.24-20 10th March 2003 ========================================================================= Changes since MPI.1.24-8 (5 November 2002) ------------------------------------------ * The structure of the release has been changed to support versions of MPICH built with alternative compilers for some architectures. The following notes have been added to the MPI product README: Compiler Support ---------------- As of version MPI.1.24-20 the Quadrics product now contains versions of MPICH built with alternative compilers for some architectures. The compiler versions supported are: Linux_alpha : Compaq C (ccc 6.4) and Fortran (cfal 1.1.1) compilers Linux_i686 : GNU (gcc/g77 2.96) and Intel (icc/ifc 7.0) Linux_ia64 : GNU (gcc/g77 2.96) and Intel (icc/ifc 7.0) The qsnetmpi rpm still installs into /usr/lib/mpi but this directory now contains a subdirectory for each supported compiler vendor. For example, on Linux_i686 and Linux_ia64 /usr/lib/mpi contains mpi_gnu and mpi_intel. Which version is currently in use is controlled by the /usr/lib/mpi/default file which is a symbolic link to one of the mpi_ subdirectories. On Linux_alpha systems only the Compaq compilers are supported. /usr/bin/mpicc, /usr/bin/mpif77, /usr/bin/mpif90 and /usr/bin/mpiCC are symbolic links to the equivalent script in /usr/lib/mpi/default/bin. The default value of /usr/lib/mpi/default on Linux_i686 and Linux_ia64 systems is set at installation to be mpi_gnu. If you have Intel compilers (V.7.0) installed and wish to use the Intel compiled version of Quadrics MPI to be used by default '/usr/lib/mpi/bin/setmpidefault mpi_intel' on every node. Individual users who wish to use the version which is not the default should put /usr/lib/mpi/mpi_/bin in their path ahead of /usr/bin and should set their LD_LIBRARY_PATH to be /usr/lib/mpi/mpi_/lib. For example, if /usr/lib/mpi/default points to mpi_gnu and a user wishes to use the Intel compiled version they should do the following: export PATH=/usr/lib/mpi/mpi_intel/bin:$PATH export LD_LIBRARY_PATH=/usr/lib/mpi/mpi_intel/lib * Support has been added for Fortran compilers which support the -i8 and -r8 flags. The following notes have been added to the MPI product README: Support for -i8 and -r8 Fortran compiler flags ---------------------------------------------- As of version MPI.1.24-20 support has been added for Fortran compilers which support the -i8 and -r8 options (Compaq, Intel). Passing -i8 and/or -r8 to the mpif77 or mpif90 scripts will cause these flags to be passed on to the underlying compile commands and the appropriate include and library paths to be set. For users not using mpif77 and mpif90 the include paths and library paths will have to be set manually. For compilers which support these flags the mpi_ directory (eg. mpi_intel, mpi_compaq) will contain include_i8, include_r8, include_i8r8, lib_i8, lib_r8 and lib_i8r8 directories and the -I, -L and -Wl,rlink-path flags should be set appropriately to pick up header files and libraries from these directories rather than /usr/lib/mpi/default/include or /usr/lib/mpi/default/lib. These include directories contain only the mpif.h header file and the f90 module directories so you need to add the default include directory to your include path as well. For example, taking the Intel compiler as an example, if compiling without -i8 or -r8 you would use: ifc -I/usr/lib/mpi/mpi_intel/include -o ftest ftest.f \ -L/usr/lib/mpi/mpi_intel/lib -lmpi -lmpifarg -Vaxlib If you added the -i8 flag you would prepend the include_i8 directory to the include path and replace lib with lib_i8: ifc -i8 -I/usr/lib/mpi/mpi_intel/include_i8 \ -I/usr/lib/mpi/mpi_intel/include -o ftest ftest.f \ -L/usr/lib/mpi/mpi_intel/lib_i8 -lmpi -lmpifarg -Vaxlib And similarly for r8 and i8r8: ifc -r8 -I/usr/lib/mpi/mpi_intel/include_r8 \ -I/usr/lib/mpi/mpi_intel/include -o ftest ftest.f \ -L/usr/lib/mpi/mpi_intel/lib_r8 -lmpi -lmpifarg -Vaxlib ifc -i8 -r8 -I/usr/lib/mpi/mpi_intel/include_i8r8 \ -I/usr/lib/mpi/mpi_intel/include -o ftest ftest.f \ -L/usr/lib/mpi/mpi_intel/lib_i8r8 -lmpi -lmpifarg -Vaxlib * Added a MPI_NO_RMA environment variable to disable calling the rma_init code. This is the only code that uses elan3 so is the only bit that ever breaks when you upgrade qsnetlibs. This is intended as a temporary workaround until the rma code is rewritten to use libelan rather than libelan3. * Added support for the environment variable 'MPI_ABORT_WITH_CORE'. If this variable exists and its value is set to 0 or the string 'FALSE' then the 'MPI_Abort()' function will set the core file size limit to 0 before calling 'abort(). The program will terminate, but no core file will be generated. For example ... export MPI_ABORT_WITH_CORE=0 prun -N2 ./prog [0] MPI Abort by user Aborting program ! prun: no core file to analyse prun: atlas0 ./mprog process 0 (pid 13693) killed by signal 6