Old netcdf-pgi-cnl and netcdf-pathscale-cnl modules have been removed from the module list on Hexagon.

Use instead Crays version, available as module netCDF, or netcdf-cnl which we have compiled.

Both netCDF and netcdf-cnl checks which PrgEnv you have loaded, and loads the appropriate version.

netcdf-cnl/4.0 is currently not netcdf4 enabled.

Thursday Oct. 16th at 09:00 will /migrate and /bcmhsm be mounted read only as /migrate-old and /bcmhsm-old on both fimm and hexagon.

A new EMPTY version will be mounted as /migrate and /bcmhsm on fimm and hexagon which will be writeable, but empty (except for directories). We will then start the process of moving the files of every user from the old file system to the new. This will be done one user at the time. Each user will be informed individually when all hers/his files has been moved to the new file system.

We would appreciate if users of /migrate and /bcmhsm would clean up their directories before we mount them read only. This will reduce the amount of data to move, hence, speed up the time consuming moving process.

We will keep the /migrate-old and /bcmhsm-old for some time after the moving has been completed. We will come back with a date for the termination of these later.

If you have any questions regarding this, please send your request to support-uib@notur.no

Update Oct. 16th 09:30: The old /migrate and /bcmhsm are now mounted read only under /migrate-old and /bcmhsm-old, while the new empty version are mounted under /migrate and /bcmhsm. We will now start the moving of data from the old to the new.

We have now activated Quota on /var/spool/torque on hexagon.
The soft limit is 3 GB and the hard limit is 6GB.

If the output of your job exceeds these limits the job will stop. You must then redirect the output of a job to a file.
I.e:
aprun -n 1 ./program > /work/$USER/output.txt

Hexagon now has activated the accounting/allocation manager "Gold", this means that all jobs will need to have a valid cpu-account with enough cpu-hours to run the submitted job.
At the time of job-submission a "credit-check" against the specified cpu-account is done and the asked for cpu-hours are reserved until the job ends - at which time the real amount of cpu-hours is subtracted from the account. The number of cpu-hours reserved and subtracted is calculated as follows:

cpuhours = 4 * blocked nodes * wallclock time

For reservations "wallclock time" is the specified "wallclock" parameter used in the PBS script or on the command line (or the default 1 hour).
For job-end account subtraction "wallclock time" is the actual used wallclock (start-time -> end-time).
The number "4" comes from 4 cores per node. A node is considered blocked if one or more cores on the node is reserved for the user since only one job can run at any time on a node.

This means that setting e.g. mppnppn=1 and mppwidth=12 for 1 hour the actual cpu-hour usage will be calculated as:

4 * 12 * 1 = 48 cpuhours

whereas a job with mppnppn=4 (the default) and mppwidth=12 for 1 hour will have cpu-hour usage calculated as:

4* 3 * 1 = 12

If your job fails to start, you should use the command:

checkjob -v jobnumber

where jobnumber is the PBS jobnumber given to you upon job-submission. If the command returns "Cannot debit account" you need to check for correct "-A mycpuaccount" specification for your job as well as enough credits to reserve and run the job.
You can check the names and balance of your available cpu-accounts with the "cost" command.

Note also that the version of Moab scheduler was updated. Users currently logged in needs to do a "module swap moab/5.2.1 moab/5.2.2" or log out and in again to have the moab client commands use the correct version.