At 8:30 this morning all compute nodes on fimm was shutdown due to cooling failure in machine room.
Now cooling is back to normal , we will take up all compute nodes within next 20 minutes.
Hexagon: down due to cooling failure
Hexagon was shutdown due to a cooling failure in the building at 06:30.
We are investigating.
Update: 08:00 We will do some already planned maintenance while the machine is down.
Update: 13:05 Machine is back online. Maintenance done to disk-system firmware and some Lustre config checks, as well as a couple of hardware replacements.
We are investigating.
Update: 08:00 We will do some already planned maintenance while the machine is down.
Update: 13:05 Machine is back online. Maintenance done to disk-system firmware and some Lustre config checks, as well as a couple of hardware replacements.
Hexagon: Updated software/libraries
Hexagon has updated software/libraries.
MPI / compilers
xt-mpt 5.1.1 -> 5.1.2
xt-asyncpe 4.4 -> 4.5
Math libraries
xt-libsci 10.4.8 -> 10.4.9
PETSc 3.1.03 -> 3.1.04
Trilinos 10.2.0 -> 10.6.0
libfast 1.0.7 -> 1.0.8
NOTES:
xt-mpt
Bugfixes.
xt-asyncpe
Bugs fixed in this release:
744483 linux ayncpe drivers fail to handle verbose flag correctly
for GNU and PGI
750934 On a CADE system, using only the '-V' option does not pass
target option
754620 cc -help does not give individual compiler options.
758200 cc -V and ftn -V no longer work properly
765639 compiling w/ -Bdynamic (icc) requires PrgEnv-intel at
run time.
765956 pgi compiler wrapper with xt-libsci/10.4.8 has no openmp
765957 CPR - "-l mpich_cpr" could be added by a module.
765991 The CC driver is not setting the proper -hcpu type.
766162 Change lib search path for trilinos 10.6.
766779 xtpe-mc8 module missing istanbul_mp setting
766805 Intel compiler problem with Trilinos-10.6
766847 PrgEnv-cray needs to point to Trilinos built by GCC/4.4
Differences:
Beginning with xt-asyncpe 4.4, the f77 script for pgi is now
aliased to
ftn. In a future release, it will be removed altogether.
xt-libsci
The following features were added to Libsci 10.4.9:
- Now includes new explicit entry-points for faster complex linear
solvers using the 3m algorithm for complex matrix-matrix
multiplication.
The faster matrix multiplication can also be used without code
modification by setting an environment variable LIBSCI_USE_3M in
the job
execution script. Note that in previous libsci versions this
algorithm was
used by default, and therefore users may require setting the
LIBSCI_USE_3M
to obtain previous performance levels. This affects the LAPACK driver
routines ZPOSV, CPOSV, ZGESV, CGESV and LAPACK factorization routines
ZPOTRF, CPOTRF, ZGETRF, CGETRF. See the intro_lapack man page for
usage
information.
PETSc
Purpose:
--------
PETSc-3.1.04 includes new Cray Adaptive Sparse Kernels (CASK) for
triangular solution that allows further performance improvement for
very sparse matrices. This module is equivalent to the official
version of PETSc-3.1 with patch level 5 that include several
bug fixes.
Product and OS Dependencies:
----------------------------
xt-asyncpe 4.4
xt-libsci 10.4.9 or later
MPT 5.0.2 or later for gcc 4.5 support
PETSc 3.1.04 is not supported with the PathScale compiler.
Documentation:
--------------
http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html
http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview
Trilinos
Purpose:
--------
Trilinos-10.6 includes many new features and bug fixes. Detailed
information is available at:
http://trilinos.sandia.gov/release_notes-10.6.html
Cray Trilinos provides 33 packages. Please see the man page,
intro_trilinos for the package names and descriptions.
Cray's Epetra package introduces the new version of Cray Adaptive
Sparse Kernels (CASK), including improved multiple-vector sparse
matrix vector multiplication and triangular solution kernels. These
kernels improve the performance of the operations with very sparse
matrices.
Cray's Amesos package provides an interface to the sparse direct
solvers from SuperLU-4.0, SuperLU_DIST-2.3 and MUMPS-4.9.2 available
in the Cray petsc module. Cray's Zoltan package provides interface
to ParMetis-3.1.1 in Cray petsc module. These interfaces allow
users to call these popular sparse matrix and graph partitioning
packages with ease and interoperate with the other packages of
Trilinos.
Product and OS Dependencies:
----------------------------
xt-asyncpe 4.5 or later
xt-libsci 10.4.9 or later
petsc-3.1.04 or later
PGI 10.0.0 or later
Known Problems:
---------------
PGI:
Due to several template handling problems of PGI C++ compiler, PGI
compiler
users might have link-time or run-time errors when using relatively new
capabilities based on C++ template such as Tpetra and Teuchos packages.
As a workaround, we recommend using another compiler environment: Cray,
GNU or Intel instead of PGI or avoid using these new capabilities.
Intel:
To avoid link-time problems due to a missing libstdc++.a library, a gcc
module needs to be loaded. Please load gcc version 4.2, 4.3 or 4.4
only.
Example
module load PrgEnv-intel
module load petsc
module load trilinos
module load gcc/4.2.3
Documentation:
--------------
References and API guide are available at
http://trilinos.sandia.gov/index.html
To see descriptions of each individual Trilinos package, go to
http://trilinos.sandia.gov/capabilities.html
libfast
Fastmv 1.0.8 extends the domain of the sin, sincos and cos intrinsics to
all finite reals.
MPI / compilers
xt-mpt 5.1.1 -> 5.1.2
xt-asyncpe 4.4 -> 4.5
Math libraries
xt-libsci 10.4.8 -> 10.4.9
PETSc 3.1.03 -> 3.1.04
Trilinos 10.2.0 -> 10.6.0
libfast 1.0.7 -> 1.0.8
NOTES:
xt-mpt
Bugfixes.
xt-asyncpe
Bugs fixed in this release:
744483 linux ayncpe drivers fail to handle verbose flag correctly
for GNU and PGI
750934 On a CADE system, using only the '-V' option does not pass
target option
754620 cc -help does not give individual compiler options.
758200 cc -V and ftn -V no longer work properly
765639 compiling w/ -Bdynamic (icc) requires PrgEnv-intel at
run time.
765956 pgi compiler wrapper with xt-libsci/10.4.8 has no openmp
765957 CPR - "-l mpich_cpr" could be added by a module.
765991 The CC driver is not setting the proper -hcpu type.
766162 Change lib search path for trilinos 10.6.
766779 xtpe-mc8 module missing istanbul_mp setting
766805 Intel compiler problem with Trilinos-10.6
766847 PrgEnv-cray needs to point to Trilinos built by GCC/4.4
Differences:
Beginning with xt-asyncpe 4.4, the f77 script for pgi is now
aliased to
ftn. In a future release, it will be removed altogether.
xt-libsci
The following features were added to Libsci 10.4.9:
- Now includes new explicit entry-points for faster complex linear
solvers using the 3m algorithm for complex matrix-matrix
multiplication.
The faster matrix multiplication can also be used without code
modification by setting an environment variable LIBSCI_USE_3M in
the job
execution script. Note that in previous libsci versions this
algorithm was
used by default, and therefore users may require setting the
LIBSCI_USE_3M
to obtain previous performance levels. This affects the LAPACK driver
routines ZPOSV, CPOSV, ZGESV, CGESV and LAPACK factorization routines
ZPOTRF, CPOTRF, ZGETRF, CGETRF. See the intro_lapack man page for
usage
information.
PETSc
Purpose:
--------
PETSc-3.1.04 includes new Cray Adaptive Sparse Kernels (CASK) for
triangular solution that allows further performance improvement for
very sparse matrices. This module is equivalent to the official
version of PETSc-3.1 with patch level 5 that include several
bug fixes.
Product and OS Dependencies:
----------------------------
xt-asyncpe 4.4
xt-libsci 10.4.9 or later
MPT 5.0.2 or later for gcc 4.5 support
PETSc 3.1.04 is not supported with the PathScale compiler.
Documentation:
--------------
http://www.mcs.anl.gov/petsc/petsc-as/documentation/index.html
http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview
Trilinos
Purpose:
--------
Trilinos-10.6 includes many new features and bug fixes. Detailed
information is available at:
http://trilinos.sandia.gov/release_notes-10.6.html
Cray Trilinos provides 33 packages. Please see the man page,
intro_trilinos for the package names and descriptions.
Cray's Epetra package introduces the new version of Cray Adaptive
Sparse Kernels (CASK), including improved multiple-vector sparse
matrix vector multiplication and triangular solution kernels. These
kernels improve the performance of the operations with very sparse
matrices.
Cray's Amesos package provides an interface to the sparse direct
solvers from SuperLU-4.0, SuperLU_DIST-2.3 and MUMPS-4.9.2 available
in the Cray petsc module. Cray's Zoltan package provides interface
to ParMetis-3.1.1 in Cray petsc module. These interfaces allow
users to call these popular sparse matrix and graph partitioning
packages with ease and interoperate with the other packages of
Trilinos.
Product and OS Dependencies:
----------------------------
xt-asyncpe 4.5 or later
xt-libsci 10.4.9 or later
petsc-3.1.04 or later
PGI 10.0.0 or later
Known Problems:
---------------
PGI:
Due to several template handling problems of PGI C++ compiler, PGI
compiler
users might have link-time or run-time errors when using relatively new
capabilities based on C++ template such as Tpetra and Teuchos packages.
As a workaround, we recommend using another compiler environment: Cray,
GNU or Intel instead of PGI or avoid using these new capabilities.
Intel:
To avoid link-time problems due to a missing libstdc++.a library, a gcc
module needs to be loaded. Please load gcc version 4.2, 4.3 or 4.4
only.
Example
module load PrgEnv-intel
module load petsc
module load trilinos
module load gcc/4.2.3
Documentation:
--------------
References and API guide are available at
http://trilinos.sandia.gov/index.html
To see descriptions of each individual Trilinos package, go to
http://trilinos.sandia.gov/capabilities.html
libfast
Fastmv 1.0.8 extends the domain of the sin, sincos and cos intrinsics to
all finite reals.
FIMM: file system crash
Yesterday, 18th November 2010 around 14:00 , GPFS file system on fimm cluster is crashed , we were replacing switch which should be down without taking down GPFS file system , but unfortunately file system crashed.
Problem resolved around 15:30 same day, hopefully that will fix the continues GPFS file system crash on fimm.
Sorry for inconvenience.
Problem resolved around 15:30 same day, hopefully that will fix the continues GPFS file system crash on fimm.
Sorry for inconvenience.
FIMM: file system crash
We are still experiencing problems with our new 10GB internal
network, today around 18:30 GPFS file system crashed, and all
running jobs was killed.
We took up our file system at 21:15, and
now you can submit your job again.
Sorry for inconvenience.
network, today around 18:30 GPFS file system crashed, and all
running jobs was killed.
We took up our file system at 21:15, and
now you can submit your job again.
Sorry for inconvenience.
FIMM: file system crash
We are still experiencing problems with our new 10GB internal
network, yesterday around 21:30 GPFS file system crashed, and all
running jobs was killed.
We took up our file system at 22:15, but we put reservation on most
part of the cluster. this morning this reservation is removed, and
you can submit your job again.
Sorry for inconvenience.
network, yesterday around 21:30 GPFS file system crashed, and all
running jobs was killed.
We took up our file system at 22:15, but we put reservation on most
part of the cluster. this morning this reservation is removed, and
you can submit your job again.
Sorry for inconvenience.
Hexagon: new default permission settings from November 15th
By the new policy we are going to change default security bits for your
home and work directories. ($HOME and /work/$USER)
The default set will be: full control by owner, no access for group
and others (numeric: 700).
The change is going to be performed on Monday November 15th, at 12:00.
home and work directories. ($HOME and /work/$USER)
The default set will be: full control by owner, no access for group
and others (numeric: 700).
The change is going to be performed on Monday November 15th, at 12:00.
Hexagon: Updated software/libraries
Hexagon has updated software and libraries. To get the performance and stability improvements the applications needs to be recompiled.
MPI
xt-mpt 5.1.0 -> 5.1.1
Compilers/wrappers/debug
PGI 10.8.0 -> 10.9.0
chapel 1.1.1 -> 1.2.0
xt-asyncpe 4.3 -> 4.4
stat 1.1.0 -> 1.1.1
Scientific libraries
PETSc 3.1.02 -> 3.1.03
NOTES:
xt-mpt
- New SHMEM environment variable, SHMEM_MEMINFO_DISPLAY, to
display the layout of the SHMEM memory regions.
Several bugs were also fixed in this release and are listed below.
742474,744363,754252,763505,764480 - Prevent attempted
oversubscription
of memory during shmem_init() on XT systems
765571, 760251, 761685 - compiler warning about gethostbyname
for all MPI programs
xt-asyncpe
Bugs fixed in this release:
747448 f77 command for PGI should be deprecated
759747 Add LIBSCI_X86_64 variable for cce PE testing
762235 Intel ftn ld -openmp vs xtpe-barcelona for libsci library
- mult
def libifcore.a & libifcoremt.a
764431 Left over (-p acml) failures from the EDG-MERGE with both
ftn and cc
764491 Add accelerator targeting module
764504 linux-f77 script tries to link in seastar libraries on XE
system
764942 Legal C++ code fails with multiple instance error on esLogin
765602 xt-asyncpe module should define -D__x86_64__ when
PrgEnv-gnu is
loaded
Differences:
------------
Beginning with xt-asyncpe 4.4, the f77 script for pgi is now aliased to ftn. In a future release, it will be removed altogether.
PETSc
PETSc-3.1.03 module includes ParMetis-3.1.1 library, which can be
called
through PETSc's API. The Cray PETSc-3.1.03 is equivalent to the
official
patch release of PETSc-3.1-p4 by Argonne National Laboratory, and
allows
users to install SLEPc-3.1 package (an eigenvalue solver package from
Universidad Politecnica de Valencia).
Bugs fixed in this release:
763837 petsc-complex will not link -dynamic
PGI
Features of PGI 10.9.0 are documented at:
http://www.pgroup.com/doc/pgiwsrn109.pdf
The following bugs are fixed in the PGI 10.9.0 release.
763009 pgcc -O3 PGC-S-0000-Internal compiler error [TPR 17173]
763428 cc20 option on -ta=nvidia causes error [17191]
764082 PGI internal compiler error with COPYIN and THREADPRIVATE
clause [17213]
MPI
xt-mpt 5.1.0 -> 5.1.1
Compilers/wrappers/debug
PGI 10.8.0 -> 10.9.0
chapel 1.1.1 -> 1.2.0
xt-asyncpe 4.3 -> 4.4
stat 1.1.0 -> 1.1.1
Scientific libraries
PETSc 3.1.02 -> 3.1.03
NOTES:
xt-mpt
- New SHMEM environment variable, SHMEM_MEMINFO_DISPLAY, to
display the layout of the SHMEM memory regions.
Several bugs were also fixed in this release and are listed below.
742474,744363,754252,763505,764480 - Prevent attempted
oversubscription
of memory during shmem_init() on XT systems
765571, 760251, 761685 - compiler warning about gethostbyname
for all MPI programs
xt-asyncpe
Bugs fixed in this release:
747448 f77 command for PGI should be deprecated
759747 Add LIBSCI_X86_64 variable for cce PE testing
762235 Intel ftn ld -openmp vs xtpe-barcelona for libsci library
- mult
def libifcore.a & libifcoremt.a
764431 Left over (-p acml) failures from the EDG-MERGE with both
ftn and cc
764491 Add accelerator targeting module
764504 linux-f77 script tries to link in seastar libraries on XE
system
764942 Legal C++ code fails with multiple instance error on esLogin
765602 xt-asyncpe module should define -D__x86_64__ when
PrgEnv-gnu is
loaded
Differences:
------------
Beginning with xt-asyncpe 4.4, the f77 script for pgi is now aliased to ftn. In a future release, it will be removed altogether.
PETSc
PETSc-3.1.03 module includes ParMetis-3.1.1 library, which can be
called
through PETSc's API. The Cray PETSc-3.1.03 is equivalent to the
official
patch release of PETSc-3.1-p4 by Argonne National Laboratory, and
allows
users to install SLEPc-3.1 package (an eigenvalue solver package from
Universidad Politecnica de Valencia).
Bugs fixed in this release:
763837 petsc-complex will not link -dynamic
PGI
Features of PGI 10.9.0 are documented at:
http://www.pgroup.com/doc/pgiwsrn109.pdf
The following bugs are fixed in the PGI 10.9.0 release.
763009 pgcc -O3 PGC-S-0000-Internal compiler error [TPR 17173]
763428 cc20 option on -ta=nvidia causes error [17191]
764082 PGI internal compiler error with COPYIN and THREADPRIVATE
clause [17213]
Fimm: GPFS file system on fimm cluster crashed
GPFS file system on fimm cluster crashed , all file system is not available, we are working on it, mean while login node is blocked for maintenance.
10:21 fimm.bccs.uib.no is back online again, user ssh block is removed. We are investigating the issue.
12:17 We are still having problem with GPFS file system.all user connection is blocked.
14:00 The issue is resolved but we are doing test. cluster will be accessible soon.
10:21 fimm.bccs.uib.no is back online again, user ssh block is removed. We are investigating the issue.
12:17 We are still having problem with GPFS file system.all user connection is blocked.
14:00 The issue is resolved but we are doing test. cluster will be accessible soon.
Hexagon: #PBS -S /bin/sh again supported
The PBS option #PBS -S /bin/sh is again supported. We recommend to replace it with the /bin/bash interpreter.
Users with the /bin/tcsh default shell which are having job scripts with #!/bin/bash or #!/bin/sh header will need to add into their scripts #PBS -S /bin/bash to have working correctly module functions.
Users with the /bin/tcsh default shell which are having job scripts with #!/bin/bash or #!/bin/sh header will need to add into their scripts #PBS -S /bin/bash to have working correctly module functions.