Lower java security setting

Filename font size change

press "Window button + r "

type "control", then enter

 search for "metrics"

do the highlights in the picture below. 

matrix multiplication

kalman filter from student Dave

Methods and scores used for verifying ensemble forecasts

http://www.cawcr.gov.au/projects/EPSverif/scores/scores.html

http://www.eumetcal.org/resources/ukmeteocal/temp/msgcal/www/english/msg/ver_categ_forec/uos2/uos2_ko4.htm

		FCST yes	FCST no	Total
	OBS yes	hits	misses	observed yes
	OBS no	false alarms	correct negatives	observed no
Total		forecast yes	forecast no	total

Root mean square error


Answers the question: What is the magnitude of the forecast errors?
Range: 0 to infinity.  Perfect score: 0.


Equitable threat score



where
Answers the question: How well did the forecast occurrence of events correspond to the actual (observed) occurrence of events?
Range: -1/3 to 1, 0 indicates no skill.   Perfect score: 1.


Bias score


Answers the question: How does the forecast frequency of events compare to the actual (observed) frequency of events?
Range: 0 to infinity.  Perfect score: 1.


Brier score

Answers the question: What is the magnitude of the probability forecast errors?
Range: 0 to 1.  Perfect score: 0.


Brier skill score

Answers the question: What is the relative skill of the probabilistic forecast over that of climatology, in terms of predicting whether or not an event occurred?
Range: minus infinity to 1, 0 indicates no skill when compared to the reference forecast. Perfect score: 1.


Ranked probability score

Answers the question: How well did the probability forecast predict the category that the observations fell into?
Range: 0 to 1.  Perfect score: 0.


Ranked probability skill score 

Answers the question: What is the relative skill of the probabilistic forecast over that of climatology, in terms of getting close to the actual outcome?
Range: minus infinity to 1, 0 indicates no skill when compared to the reference forecast. Perfect score: 1.


 Relative value (value score)

Answers the question: For a cost/loss ratio C/L for taking action based on a forecast, what is the relative improvement in economic value between climatalogical and perfect information?
Range: minus infinity to 1.  Perfect score: 1.

NWP models

The NWP models evaluated are:  

GFS: The U.S. NCEP Global Forecast System;     
FNO: The U.S. Navy Fleet Numerical Meteorology and Oceanograpy Center;    
ECM: European Center for Medium-Range Weather Forecasts;    
UKM: The United Kingdom Met Office;    
CMC: The Canadian Meteorological Center;    
JMA: Japan Meteorological Agency;    
CDAS: The frozen version of GFS used for the NCEP/NCAR Reanalysis Project. 


The regions referred to are:  
G2 (GLB): Globe    
NHX: 20N-80N     
SHX: 20S-80S     
TRO: 20S-20N


Pattern correlations for all NWP models are computed using anomalies respective to a 30-year (1959-1988) climatology of the NCEP/NCAR reanalysis.

nail gun

Really like to have a nail gun, but feel confused all the time when shopping for them. 
The choices on display in store always seem to be too many that it makes you have no idea of what to choose. So this blog is meant to remove that kind of confusion.
 
Three major types of nail guns: ( usage, thickness and length )

brad : molding
gauge 23 , length [3/8 - 1 1/4 ] inch
gauge 18 , length [5/8 - 2     ] inch

finish : softwood & trim
gauge 16-15,  length [5/8 - 2 1/2] inch 

frame : frame & fence
gauge < 15  , length [2 - 3 1/3] inch

BFF:( brad, finish and frame) sounds like "beef"

gauge : the smaller number, the thicker of nail.

rsync

Check http://www.tecmint.com/rsync-local-remote-file-synchronization-commands/ for more details about rsync.

 Basic syntax of rsync command

$ rsync options source destination

Some common options used with rsync commands

1. -v : verbose
2. -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
3. -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
4. -z : compress file data
5. -h : human-readable, output numbers in a human-readable format

1. Copy/Sync Files and Directory Locally

$ rsync -zvh backup.tar /tmp/backups/ 

2. Copy/Sync Files and Directory to or From a Server

Send file to remote server 

$ rsync -avz rpmpkgs/ root@1.2.3.4:/home/

Get file from remote server 

$ rsync -avzh root@1.2.3.4:/home/tarunika/rpmpkgs  /tmp/myrpms 

3. Rsync Over SSH

To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using “ssh” with “-e” option and perform data transfer.

$ rsync -avzhe ssh  root@192.168.0.100:/root/install.log   /tmp/

4. Show Progress While Transferring Data with rsync

To show the progress while transferring the data from one machine to a different machine, we can use ‘–progress’ option for it. It displays the files and the time remaining to complete the transfer.

$ rsync   -avzhe    ssh    --progress   /home/rpmpkgs    root@1.2.3.4:/root/rpmpkgs 

5. Use of –include and –exclude Options

These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.
Here in this example, rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory.

$ rsync -avzh --include R*  root@1.2.3.4:/var/lib/rpm/ 

6. Use of –delete Option

If a file or directory not exist at the source, but already exists at the destination, you might want to delete that existing file/directory at the target while syncing .
We can use ‘–delete‘ option to delete files that are not there in source directory.
Source and target are in sync. Now creating new file test.txt at the target.
$ rsync    -avzh   --delete     root@1.2.3.4:/var/lib/rpm/       .

 

 

 

NCEP/EMC Global Model Precipitation Verification

http://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/

Input Data: Ying Lin's G211/RFC FHO VSDB data saved at tempest:/mmb/wd22yl/vsdb.
Monte Carlo Significance Test: Fanglin Yang's presentation
NCEP/EMC MMB QPF: Ying Lin's web

• Definition of contingency table used for computing ETS and Bias Scores:
  • Hits (a): occasions where both forecast and observation are greater than or equal to a threshold;
  • False alarms (b): occasions where forecast is above a threshold whereas observation is under the same threshold;
  • Misses (c): occasions where the observation is above a threshold and forecast is under the same threshold;
  • No forecasts (d): occasions where both forecast and observation are under the threshold.
• Bias Score: BS=(a + b)/(a + c)
  measures over-forecasts (BS>1) or under-forecasts (BS<1) precipitation frequency over an area for a selected threshold.
• Threat Score: TS=a/(a + b + c)
  Measures the fraction of observed and/or forecast events that were correctly predicted. It is sensitive to hits, penalizes both misses and false alarms, does not distinguish source of forecast error, depends on climatological frequency of events (poorer scores for rarer events) since some hits can occur purely due to random chance. TS=1 means a perfect forecast.
• Equitable Threat Score: EQ_TS=(a - ar)/(a + b + c - ar)
  where ar is the expected number of correct forecasts above the threshold in a random forecast where forecast occurrence/non-occurrence is independent from observation/non-observation, ar=(a + b)*(a + c)/(a + b + c + d). EQ_TS=1 means a perfect forecast. EQ_TS <=0 means the forecast is useless.

Met words

adiabatic : 绝热 relating to or denoting a process or condition in which heat does not
                 enter or leave the system concerned.
advection : 平流  the transfer of heat or matter by the flow of a fluid, especially horizontally
                  in the atmosphere or the sea.
analogous : 类似 comparable in certain respects, typically in a way that makes
                  clearer the nature of the things compared.
ambient : 环境
anelastic : 滞弹性 quasi-Boussinesq approximation. i.e.,
         assuming that the mass weighted three-dimensional 
         divergence is zero.  
anomaly correction : correlation between (F-C) and (A-C), which are anomaly
        from climatology.  F: forecast ,  A: analysis ,   C: climatology
baric : 气压
centrifugal : 离心 moving or tending to move away from a center.
convective mass flux : an average vertical transport of mass over for a field of
         cumulus clouds or thermal.
cross section : In weather analysis and forecasting, a graphic representation of a
             "vertical surface" in the atmosphere, along a given horizontal line
             or path, and extending from the earth's surface to a given altitude.
             ( vertical cross-section )
           Eg: Model Forecast Time-Height Vertical Cross-Section 
dispersion : 分散 the action or process of distributing things or people over 
                  a wide area 
divergence : In vector calculus, divergence is a vector operator that measures 
                 the magnitude of a vector field's source or sink at a given point, in 
                 terms of a signed scalar. More technically, the divergence represents 
                 the volume density of the outward flux of a vector field from an 
                 infinitesimal volume around a given point. 

entropy : 化學及热力学中所指的熵^[3]，是一種測量在動力學方面
       不能做功的能量總數，也就是當總體的熵增加，其做功能力也下降，
       熵的量度正是能量退化的指標。熵亦被用於計算一個系統中的失序現象，
       也就是計算該系統混亂的程度。 
       熵的概念是由德國物理學家克勞修斯於1865年所提出。克氏定義
       一個熱力學系統中熵的增減：在一個可逆性程序裡，被用在恆溫的熱的總數（δQ），
      並可以公式表示為：
       
       克勞修斯對變數S予以「熵」（英语：entropy）一名，希臘語源意
       為「內向」，亦即「一個系統不受外部干擾時往內部最穩定
       狀態發展的特性」^[4]。
geocentric : 地心 having or representing the earth as the center, as in former
         astronomical systems.
geographic : 地理
geopotential : 位势
geostrophic : 地转 relating to or denoting the component of a wind or current that arises 
             from a balance between pressure gradients and Coriolis forces 
geostrophy :  地转
graupel : Heavily rimed snow particles, often called snow pellets; often
                   indistinguishable from very small soft hail except for the size convention
                   that hail must have a diameter greater than 5 mm.

Sometimes distinguished by shape into conical, hexagonal, and lump (irregular) graupel.

gravitational : 引力
gravity : 重力 
gravitational potential :  is equal to the work (energy transferred) per unit mass that is 
        done by the force of gravity to move an object to a fixed reference location. 
       The reference location, where the potential is zero, is by 
        convention infinitely far away from any mass, resulting in a negative potential 
        at any finite distance.
Hecto Pascal (hPa) :  a unit for pressure, and, in this web site, used mainly for
             representing the central pressure of a typhoon. In Japan, the unit of
            "millibar" (mb) was used through November 1992, but since December 1992,
             the unit of "hectopascal" (hPa) has been used to comply with the International
            System of Units. Conversion between those two units is 1hPa = 1mb,
            however, so the value itself is the same as before.
homogeneous : 同质 of the same kind; alike.
hydrometeor : 水凝   an atmospheric phenomenon or entity involving water or water vapor,
             such as rain or a cloud.
inhibition : 抑制
isobar : A line of equal or constant pressure; an isopleth of pressure.
isobaric : 等压

iostherm : A line of equal or constant temperature.

isoheight / contour line : (Also called contour, isohypse, isoheight.) A line of
            constant elevation above a certain reference level (usually mean sea level) on a
            previously defined surface, which may be the earth's surface,
           a constant-pressure surface, an isentropic surface, etc.
iso : 异
isotropic : 各向同性 (of an object or substance) having a physical property that has
              the same value when measured in different directions.

isobaric :  等压 equal or constant pressure, with respect to either space or time.
isotherm : 等温线 equal temperature
isotropic : 各向同性

kinematic : (noun.) movement

Knot : a unit for speed. One knot means a speed of moving one nautical mile (nm) in one hour.
           Knot is used for representing the maximum wind speed at the center of a typhoon,
           or a movement speed of a typhoon on this website. Because 1nm = 1.852km,
          1kt = 1.852km/h = 0.5144m/s. Roughly speaking, halving the knot makes the speed
           in meter per second, while doubling it makes the speed in kilometer per hour.

linear : we are estimating a value for intercept (a) and the slop (b) that are raised only to
            the power 1. ( unbiased ) if we repeat the estimation with new samples, we
            will likely find different values for b.  If we do so, we can then calculate the
            average of all of these bs. If it is true that the average of the bs is equal to the
            population's true beta (standardized b), then the estimator is said to be unbiased;
           similarly for a, the intercept term.
LTE : local thermodynamic equilibrium

mean sea level : (Abbreviated MSL; popularly called sea level.) The arithmetic mean of
           hourly heights observed over some specified period. In the United States, mean
           sea level is defined as the mean height of the surface of the sea for all stages
            of the tide over a 19-year period. Selected values of mean sea level serve
           as the sea level datum for all elevation surveys in the United States. In 

           meteorology, mean sea level is used as the reference surface for all altitudes in upper-

           atmospheric work; in aviation it is the level above which altitude is measured by a  

           pressure altimeter. Along with mean high water, mean low water, and mean lower low 

           water, mean sea level is a type of tidal datum ( standard ).
moist : 潮湿

occluded front :  http://www.athensgaweather.com/meteorology-101-pressure-fronts/
perpendicular : 垂直 at an angle of 90° to a given line, plane, or surface.pendicular : 相垂直
predictor : x in regression equation y = ax + b
predictand : y in regression equation y = ax + b 
prognostic : 前兆serving to predict the likely outcome of a disease or 
           ailment; of or relating to a medical prognosis. 
quasi : 准 seemingly; apparently but not really.
quasi-geostropic : 准地转
Quantitative Precipitation Forecasts: QPF:
            The Quantitative Precipitation Forecast (abbreviated QPF) is the expected
            amount of melted precipitation accumulated over a specified time period over a
            specified area. A QPF will be created when precipitation amounts reaching a
            minimum threshold are expected during the forecast's valid period.
radian : 弧度 a unit of angle,  equal to an angle at the center of a circle
             whose arc is equal in length to the radius.  s = r . theta
radiance : 1. light or heat as emitted or reflected by something.
                2. the flux of radiation emitted per unit solid angle in a given direction
                   by a unit area of a source.
regression : a measure of the relation between the mean value of one variable
           (e.g., output) and corresponding values of other variables (e.g., time and cost).
regression analysis : In statistics, it is a statistical process for estimating the relationships
             among variables. It includes many techniques for modeling and analyzing
            several variables, when the focus is on the relationship between a dependent variable
            and one or more independent variables. More specifically, regression analysis
            helps one understand how the typical value of the dependent variable
            (or 'criterion variable') changes when any one of the independent variables is
            varied, while the other independent variables are held fixed.
sea level pressure : The atmospheric pressure at mean sea level, either directly          measured or, most commonly, empirically determined from the observed
         station pressure. In regions where the earth's surface is above sea level, it is
         standard observational practice to reduce the observed surface pressure to the
        value that would exist at a point at sea level directly below if air of a temperature
        corresponding to that actually present at the surface were present all the way
        down to sea level. In actual practice, the mean  temperature for the preceding
        12 hours is employed, rather than the current temperature. This
        "reduction of pressure to sea level" is responsible for many anomalies

         in the pressure field in mountainous areas on the surface synoptic chart.

standardized coefficient : Before solving a multiple regression problem, all variables
         (independent and dependent) can be standardized. Each variable can be
          standardized by subtracting its mean from each of its values and then
         dividing these new values by the standard deviation of the variable.
         Standardizing all variables in a multiple regression yields standardized
         regression coefficients that show the change in the dependent variable
         measured in standard deviations.
stratification : 分层 demixing, lamination
thermal : 热 
thermodynamics :  热力学 the branch of physical science that deals with the relations
            between heat and other forms of energy (such as mechanical, electrical,
           or chemical energy), and, by extension, of the relationships between all forms of
           energy.
trajectory : 轨迹 the path followed by a projectile flying or an object moving under
                the action of given forces.


Greek alphabet:
Letter  Name   
Α α      alpha
Β β      beta
Γ γ      gamma
Δ δ      delta
Ε ε      epsilon
Ζ ζ      zeta
Η η      eta
Θ θ      theta
Ι ι        iota
Κ κ      kappa
Λ λ      lambda
Μ μ      mu
Ν ν      nu
Ξ ξ      xi
Ο ο      omicron
Π π      pi
Ρ ρ       rho
Σ σ/ς    sigma
Τ τ       tau
Υ υ      upsilon
Φ φ      phi
Χ χ      chi
Ψ ψ     psi
Ω ω     omega 

SLURM

Scheduling jobs with SLURM

by Jesse Stroik — last modified Aug 07, 2014 10:29 AM

SLURM usage guide for scientists on systems at SSEC.

Contents

1. SLURM Description and Design
2. Parallel Job Submission
   1. Setting Job Flags in a Script
   2. Example MPI Script 
   3. Job Limits and Resources Requested
3.  Submitting Hybrid jobs
4. Serial Job Submission
5. Overview of SLURM commands and arguments
   1. Primary user-facing commands
   2. Nomenclature
   3. Common Arguments for sbatch and srun
6. Job Dependencies
   1. Example job dependency script
7. Additional Resources
   1. System and Scheduler Related Assistance
   2. Scientific Software Related Assistance

SLURM Description and Design

SLURM schedules jobs and manages nodes in similar manner compared with PBS, LSF and Gridengine. Whereas other schedulers submit all jobs with qsub or bsub, SLURM uses sbatch.
SLURM is designed to submit parallel (MPI / Hybrid) jobs from control jobs. A control job is just like a regular sbatch submission, but within the script it contains an srun line, similar to mpiexec or mpirun commands used in scripts for other schedulers.

PBS, LSF, Gridengine : qsub/bsub,        mpiexec/mpirun
SLURM                       : sbatch,              srun

Parallel Job Submission

Parallel job examples include WRF variants and various stages of the GFS which use MPI and/or OpenMP. We start by explaining the scheduler pre-processed flags.

Setting Job Flags in a Script

You can specify flags in your script with #SBATCH. For example, if you wanted to submit a job to the serial partition with the name YOUR_JOB_NAME and output in your home directory, you'd add this:

#!/bin/bash
#
#SBATCH --job-name=YOUR_JOB_NAME
#SBATCH --partition=serial
#SBATCH --output=${HOME}/output/YOUR_JOB_NAME-control.%j
#SBATCH --ntasks=1

Lines in your script that begin with:

#SBATCH

Are pre-processed by the scheduler. They do not affect other parts of your script but the scheduler takes them as modifiers to your job submission. 

Example MPI Script 

This is an sbatch script that you'd use to submit to the scheduler. Contained within is a srun command which runs a MPI parallel job:

#!/bin/bash
#SBATCH --job-name=YOUR_JOB_NAME
#SBATCH --partition=s4
#SBATCH --export=NONE
#SBATCH --ntasks=180
#SBATCH --mem-per-cpu=6000
#SBATCH --time=02:00:00
#SBATCH --output=/scratch/${USER}/output/YOUR_JOB_NAME-control.%j
source /etc/bashrc
module purge
module load licence_intel intel/14.0-2
module load impi
module load hdf hdf5
module load netcdf4/4.1.3

# Way1:within a script  

# here you could call a script that creates your srun jobs and manages them
# or you could just run srun like this
srun --cpu_bind=core --distribution=block:block $HOME/path/to/mpi-executable

For those who have previously used mpiexec or mpirun, you will notice that srun is used similarly to those commands.

# Way2:   from command line

srun --output=/scratch/${USER}/output/YOUR_JOB_NAME.%j --cpu_bind=core --distribution=block:block \
 --mem-per-cpu=6000 --time=2:00:00 --ntasks=120 $HOME/path/to/mpi-executable

'srun' is considered a job step within the SLURM job. The overall job needs to have sufficient resources allocated to it to execute the srun -- that is, if you want to run an MPI job with 200 CPU cores, you must specify that in your initial sbatch.
In this srun example, you see that it is issuing 120 MPI tasks. This will automatically go across multiple nodes. If you are submitting a job for which you anticipate using fewer than 10 CPU cores, please submit with --shared

--cpu_bind=core --distribution=block:block

This setting is ideal for MPI jobs and especially for Hybrid jobs

Job Limits and Resources Requested

It is important to understand how much memory your job will consume and how long it will take your job to run. Submitting jobs with accurate estimates for both will improve job scheduling for you.
Accurately specifying --time benefits you because jobs with a lower --time are likely to start sooner. The system will be configured with a low default.

--time: The lower your estimation, the quicker your job will be scheduled. Jobs must complete within the time allotted.

--mem-per-cpu: This is important to ensure jobs do not use more memory than is available on a node. SLURM will schedule accordingly if you need higher amounts of memory.


module commands are best begun fresh with a 'module purge' at the beginning of the job to ensure a consistent state. Then load the modules your job needs. This way, regardless of what happens with your environment, your job always gets the right modules. If you want very specific versions, be specific -- default for modules can change! In the example above, all module versions are specified explicitly.
If you do not yet understand module then please visit our documentation on module HERE before submitting jobs.

 Submitting Hybrid jobs (dxu: hybrid is about OMP threads)

S4-Cardinal supports Hybrid jobs with MPI tasks running OpenMP threads. The scheduler needs to know how many threads you will issue per MPI task. MPI needs the same information with the environment variable OMP_NUM_THREADS.
dxu: 

Hybrid jobs can drastically increase the amount of memory available to your tasks, but must be written with OpenMP loops.

The amount of CPUs you need is ntasks * threads. So if you have an MPI job that normally would run on 320 tasks, and you wanted to try 5 OpenMP threads (5 threads per MPI task) , you'd tell slurm --ntasks=64 ( number of MPI tasks). The following is a table to illustrate the relationship between tasks, threads and nodes.

 dxu: 
OpenMP : Open Multi Processing  (OMP)
MPI : Message Passing Interface ( A standard API for message passing communication and process information lookup, registration, grouping and creation of new message data types.)

dxu: 
--ntasks = 64                               :  number of MPI tasks
 OMP_NUM_THREADS = 5    : 5 threads per MPI tasks

OMP_NUM_THREADS	ntasks	Nodes consumed
1 (pure MPI)	200	10 nodes
2	100	10 nodes
4	50	10 nodes
5	40	10 nodes
10	20	10 nodes

If you wished to run 5 OpenMP threads per MPI task, you would issue the following arguments to srun within your sbatch script:

export OMP_NUM_THREADS=5
srun --cpus-per-task=5 --ntasks=40 --distribution=block:block --cpu_bind=core --time=02:00:00 \
--mem-per-core=5500 <script>

The sockets in S4-Cardinal are 10-core, so we recommend issuing job that use 2, 5 or 10 threads. In our experience, 5 is the fastest GFS fcst, but it may vary for you.
IMPORTANT: --mem-per-core has a multiplicative effect on your memory available when running OpenMP. If you run 5 threads, for example, that MPI task has approximately 5x the memory requested per core at its disposal.

Serial Job Submission

An example serial job might be a job that just manipulates I/O, such as combining files, or a job-step that is not written to take advantage of MPI or OpenMP.

#!/bin/bash
#SBATCH --job-name=my-job
#SBATCH --partition=serial
#SBATCH --share
#SBATCH --time=1:20:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=4500
#SBATCH --output=$HOME/job-output/my-serial-output.txt
module purge
module load license_intel
module load impi/4.1.3.049
module load intel/14.0-2
module load hdf/4.2.9
module load hdf5/1.8.12
module load netcdf4/4.1.3
export I_MPI_JOB_STARTUP_TIMEOUT=10000
$HOME/myscript.scr  # job runs here

Note the use of --share. This ensures that you do not run exclusively on a node with 20 CPU cores available and use only one.(dxu: not to waste resources !!! ) If you have a multhreaded program that requires multiple CPUs, you could use this same script but change --cpus-per-task=X where X is the number of processors you can use.

Overview of SLURM commands and arguments

Primary user-facing commands

Job submission is done via two commands: sbatch and srun.

sbatch: Handles serial jobs. Also used as a first step for MPI jobs.
srun: MPI-capable job submission, usually run from within a script run via sbatch.

Additionally, for examining the queue you can run sinfo and squeue. 

sinfo: Displays nodes/and partitions. 
squeue: Shows you the list of jobs running and yet to be run. It also will have state, including R(unning) and PD(pending)

To manage your jobs, you will need scancel and sstat:

scancel: delete a job
sstat: get the status of a job
scontrol: manipulate or view details on a job. For example 'scontrol -dd show job <jobid>'

Nomenclature

SLURM uses new nomenclature to help make important distinctions. Taking time to understand the nomenclature is fundamental to SLURM fluency.

Queue: the list of slurm jobs being executed and yet to be executed

Task: a job or a job-step (sub-job). This distinction is helpful because 'job' in SLURM now means an overall job, whereas a task can be a job or a job-step.

Node: a computer typically with multiple CPUs. Many computers make up the cluster and each have limited CPU cores and Memory.

JobID: the numeric identifier that can be used to specify a job (as shown in squeue)

Partition: a set of hardware with similar rules and priorities -- referred to as a queue by other schedulers

NOTE: CPUs/Cores are sometimes used interchangeably. If the system means an actual CPU, it will usually be denoted as socket.

Common Arguments for sbatch and srun

--ntasks=<NUMBER> the number of tasks your job needs. Often, you will run 1 CPU core per task
--mem=<NUMBER> the amount of memory your job will use per node. Usually, you are better of requesting --mem-per-cpu
--mem-per-cpu=<NUMBER> The memory (in MB) you job needs per CPU Core. This defaults to 6GB
--cpus-per-task=<NUMBER> The number of CPUs each of your processes might use. An example is if you were using MPI + OpenMP hybrid models
--job-name=<STRING> please describe your job with appropriate acronyms, such as FSCT, GSI, WRF, etc
--shared if you plan to issue jobs that will use fewer than 10 CPU cores, we recommend adding --shared

Job Dependencies

For complex multi-step sbatch jobs, job depenencies are very useful.  The basic syntax is simply

sbatch --dependency=afterok:<JOBID> myjob

This says run 'myjob' after job with id <JOBID> completes successfully. If submitting multiple jobs by hand you simply look at the job id and enter it manually. However, for multi-step jobs it is likely preferable to submit them automatically.

To do this, one technique is to submit your dependent job *inside* the first job. This works well because $SLURM_JOB_ID is available.

Example job dependency script

In this example we run the MPI job in the earlier example, but then have an "file copy" job that copies data after the job has completed.

#!/bin/bash
#SBATCH --job-name=YOUR_JOB_NAME
#SBATCH --partition=s4
#SBATCH --export=NONE
#SBATCH --ntasks=180
#SBATCH --mem-per-cpu=6000
#SBATCH --time=02:00:00
#SBATCH --output=/scratch/${USER}/output/YOUR_JOB_NAME-control.%j
source /etc/bashrc
module purge
module load licence_intel intel/14.0-2
module load impi
module load hdf hdf5
module load netcdf4/4.1.3

#Submit the "file_copy" dependent job
#  Note you could also pass $SLURM_JOB_ID as a parameter to file_copy

sbatch --dependency=afterok:$SLURM_JOB_ID file_copy

# here you run your job as before, this is srun, but it could be anything.
srun --cpu_bind=core --distribution=block:block $HOME/path/to/mpi-executable

file_copy - would be it's own sbatch script. If desired, that script could issue another job dependency, and so on.

Additional Resources

The SLURM site at schedmd.com has user documentation, as well as a translation article they refer to the rosetta stone of schedulers which can sometimes help users who came from different schedulers. 

System and Scheduler Related Assistance

Please contact the S4 support team at SSEC using the following email address:

s4.admin@ssec.wisc.edu

thermodynamic diagram

inter-compute t, td and rh

radiosonde : rawinsonde , dropsonde

A radiosonde (Sonde is French and German for probe) is a battery-powered telemetry instrument package carried into the atmosphere usually by a weather balloon that measures various atmospheric parameters and transmits them by radio to a ground receiver. Radiosondes may operate at a radio frequency of 403 MHz or 1680 MHz.

Special radiosonde: 

1. rawinsonde: A radiosonde whose position is tracked as it ascends to give wind speed and direction information is called a rawinsonde ("radar wind -sonde").^[1][2] Most radiosondes have radar reflectors and are technically rawinsondes.

2. dropsonde: A radiosonde that is dropped from an airplane and falls, rather than being carried by a balloon is called a dropsonde.

Radiosondes are an essential source of meteorological data, and hundreds are launched all over the world daily.