NOCSCOMBINE
Warning
NOCSCOMBINE is old and slow. Use REBUILD_NEMO instead.
NOCSCOMBINE is a tool to combine the per-processor NetCDF results files that are generated when NEMO is run in parallel using MPI. One description of its use can be found at http://www.hector.ac.uk/cse/distributedcse/reports/nemo/nemo_notes/node18.html. The NOCSCOMBINE code was downloaded from ftp://ftp.soc.soton.ac.uk/omfftp/NEMO/.
The nocscombine.F90
file contains this notice regarding distribution and use of the package:
Unless otherwise indicated, all other software has been authored
by an employee or employees of the National Oceanography Centre,
Southampton, UK (NOCS). NOCS operates as a collaboration
between the Natural Environment Research Council (NERC) and
the University of Southampton. The public may copy and use
this software without charge, provided that this Notice and
any statement of authorship are reproduced on all copies.
Neither NERC nor the University makes any warranty, express
or implied, or assumes any liability or responsibility for the
use of this software.
Usage
From the README.nocscombine
file:
NOCSCOMBINE
FORTRAN program that assembles a global NetCDF archive from a series of
NEMO output files.
USAGE :: nocscombine -f inputfile [-o outfile] [-v] [-s]
[-d list of variables] [-ts tstart] [-te tend]
-f inputfile
Mandatory argument. inputfile is the filename of
one of the NEMO restart or diagnostic files.
This should be the whole filename of one of the individual
processor files (i.e. include the
node number, eg. ORCA1_00010101_restart_0012.nc).
This program works on NetCDF NEMO output files. The program will
create a NetCDF archive covering the entire global domain.
[-o outputfile ]
Optional argument. outputfilename is the required name
of the outputfile. By default this name will be constructed from
the inputfilename by stripping off the processor number. I.e.:
inputfile : ORCA1_00010101_restart_0012.nc
outputfile: ORCA1_00010101_restart.nc
The -o option permits the user to override this behaviour.
[-c coordinatefile ]
Optional argument. coordinatefile is the name
of the global coordinate file relevant to the NEMO model which
produced the individual processor files. This file is required to
ensure complete nav_lon and nav_lat fields in cases where wholly
land areas have been omitted from a parallel run (i.e.
jpnij .ne. jpni*jpnj). The corrections are applied after
the collation phase. If this argument is supplied and the output
file exists then the collation phase is not performed but the
coordinate corrections are still applied. In the latter scenario
the (otherwise) mandatory inputfile argument is not required.
[-n processor_offset ]
Optional argument. Start collating from this processor number and
process domains in order, cycling back through 0 if necessary. This
is useful to prevent multiple simulataneous accesses when running
nocscombine in a task-farming environment
[-v]
Optional argument. Switches on verbose reporting for debugging
[-s]
Optional argument. Normal behaviour is to show a limited amount
of progress information. This option runs the utility without any
output to stdout.
[-showhaloes]
Optional argument. Early versions of nocscombine erroneously
copied across the halo rows and columns too. Potentially producing
lines of unset values if the halo points were not set in the
individual restart files (as could happen for some purely
diagnostic fields). Use -showhaloes to reproduce this feature
if required.
[-d] comma-separated list of variables
Collate only the named datasets
[-ts] time-slice (integer) to begin extraction at
[-te] time-slice (integer) to end extraction at. If the -te option
is not used then only one time-instance is extracted (as set
by the -ts option)
Examples:
---------
nocscombine -f ORCA025-N10_1m_19670913_19680410_grid_T_0220.nc \
-o N10_1967_Tvars.nc \
-d "sosstsst,sosaline,sossheig,sowaflep,sowaflup,sorunoff,sowaflcd,
sosalflx,sohefldo,soshfldo,somxl010,somixhgt,soicecov,sohefldp,
sowafldp,sosafldp,sobowlin" \
-ts 1 -te 4
On Hector the Lustre filesystem can be used to speed up i/o by "striping"
the named output file:
lfs setstripe N10_nov1967_pt.nc 0 -1 3
nocscombine -f ORCA025-N10_1m_19670913_19680410_grid_T_0220.nc \
-o N10_nov1967_pt.nc -d votemper -ts 2
N10_nov1967_pt.nc
[ 0] [ 1] [ 2] ..........
Completed in 148.805 seconds
lfs setstripe N10_nov1967_s.nc 0 -1 3
nocscombine -f ORCA025-N10_1m_19670913_19680410_grid_T_0220.nc \
-o N10_nov1967_s.nc -d vosaline -ts 2
N10_nov1967_s.nc
[ 0] [ 1] [ 2] ..........
Completed in 147.961 seconds
Building nocscombine
The NOCSCOMBINE.tar
tarball includes several Makefiles
but it is almost certainly necessary to create your own Makefile
to set the
FC,
LDR,
NCHOME,
and LIBS variables appropriately.
This repository contains Makefile.jasper
to build nocscombine on jasper.westgrid.ca using the values:
FC = ifort -fixed -80 -O3 -DLARGE_FILE -c
LDR = ifort -O3 -fixed -o nocscombine
NCHOME = /lustre/jasper/software/netcdf/netcdf-4.1.3
LIBS = -L$(NCHOME)/lib -I$(NCHOME)/include -lnetcdf -lnetcdff -lhdf5_hl -lhdf5 -lz -lsz