Tuesday, March 11, 2014

histogram

What is it?
A histogram is a bar graph of raw data that creates a picture of the data distribution. The bars represent the frequency of occurrence by classes of data. A histogram shows basic information about the data set, such as central location , width of spread , and shape. Use histograms to assess the system’s current situation and to study results of improvement actions. The histogram’s shape and statistical information help you decide how to improve the system. If the system is stable, you can make predictions about the future performance of the system. After improvement action has been carried out, continue collecting data and making histograms to see if the theory has worked.
Descriptive statistics, such as chi-square , kurtosis , and skewness  can help you interpret the histogram and can show you if the data distribution is normal.

Central location
Central location is the center of a set of data points. Mean, median, and mode are the statistics used to describe it.

Chi-square
A goodness-of-fit-test statistic used to test the assumption that the distribution of a set of data is similar to the expected distribution, such as a normal distribution.

Kurtosis
Kurtosis is a statistic that is used to measure the "flatness" or "peakedness" of a set a of data. It represents a measure of the combined weight of the tails relative to the rest of a distribution. As the tails of a distribution become heavier, the kurtosis will increase. As the tails become lighter, the kurtosis value will decrease.

Skewness
Skewness is a statistic that is used to measure the symmetry of the distribution for a set of data. A process that is skewed tails off to the left or to the right.

 


Thursday, March 6, 2014

Physical Inversion and Data Assimilation Pre-Processing Using the MiRS Variational System

Dr. Sid Boukabara, NOAA/NESDIS/STAR: 
"Physical Inversion and Data Assimilation Pre-Processing Using the MiRS Variational System. Application to the Microwave Sensors Constellation (SNPP, POES, Metop, DMSP, GCOM-W, GPM, M-T and TRMM)"



We present in this seminar the mathematical basis, the technical implementation and the performances assessment of an iterative physical algorithm based on a Bayesian variational approach. This algorithm, called the Microwave Integrated Retrieval System (MiRS), is used operationally in NOAA to generate sounding, surface, hydrometeor and cryospheric parameters from a variety of microwave sensors including AMSU/MHS, SSMIS and ATMS onboard POES/Metop, DMSP and SNPP platforms, respectively. It is also applied routinely in a research mode (non-operationally) to data from AMSR-2, TMI and SAPHIR onboard GCOM-W, TRMM and Megha-Tropiques satellites, respectively.
The algorithm relies on the Community Radiative Transfer Model (CRTM), developed in the Joint Center for Satellite Data Assimilation (JCSDA), to (1) simulate brightness temperatures and (2) generate Jacobi (dxu: weighting function) with respect to all geophysical parameters. These two components, along with the (3) background covariance matrix used, are critical for the physical inversion.  In order to ensure a stable and fast processing, the inversion is undertaken after projecting it into a reduced space using the Empirically Orthogonal Functions (EOF). The state vector parameters are retrieved simultaneously, which ensures that the resulting geophysical solution fits the observations consistently, which is a necessary, although sometimes overlooked, condition for the inversion process.

Obviously, the performances obtained by MiRS when applied to different satellite data, will depend on the information content of those sensors and their configurations (frequencies, polarizations, viewing angles, etc). We will present a snapshot of the performances, both as obtained internally using a locally developed testbed and by independent assessments provided by users and outside teams (such as the International Precipitation Working Group IPWG). We will in particular look at the performances of (1) the total precipitable water over all surfaces including ocean, land, sea ice, snow and coastal surfaces, (2) the surface rainfall rate, (3) the atmospheric temperature and moisture vertical profiles and (4) the surface emissivity and temperature.
In addition to the inversion aspect, we will highlight a few recent JCSDA-led developments that aim at applying the same technology used in MiRS to develop a uniform Quality Control (QC) and pre-processing system called the Multi-Instrument Inversion and Data Assimilation pre-Processing System (MIIDAPS), used as a tool to pre-process all satellite data (to apply to microwave and IR sensors) before they are assimilated into the GSI system. This tool allows to (1) optimize the QC filtering and spatial thinning of the data, as well as (2) provide an estimate of the dynamic surface emissivity as well as (3) provide estimates of sounding profiles in cloudy and rainy conditions, situations where the data is currently rejected. Future directions and collaborative efforts will be presented as part of the presentation.
Location M-Square Building #950 Room # 4102 (Large Conference Room) 5825 University Research Court, College Park, MD 20740
Contact Isaac Moradi, imoradi@umd.edu

ESSIC seminars

The Earth System Science Interdisciplinary Center
The goal of ESSIC is to enhance our understanding of how the atmosphere,
ocean, land, and biosphere components of the Earth interact as a coupled
system and the influence of human activities on this system.

This is accomplished via studies of the interaction between the physical climate
system (e.g., El Nino) and biogeochemical cycles (e.g., greenhouse
gases, changes in land use and cover).



Seminars: 
http://essic.umd.edu/joom2/index.php/calendar/essic-seminar/eventdetail/552/-/dr-sid-boukabara-noaanesdisstar-qphysical-inversion-and-data-assimilation-pre-processing-using-the-mirs-variational-system-application-to-the-microwav


DTC

Developmental Testbed Center

To serve as a bridge between research and operations to facilitate the activities of both halves of the NWP Community in pursuit of their own objectives:
  • Research community gets a functionally equivalent operational environment to test and evaluate new NWP methods in retrospective extended period tests using advanced tools
  • Operational community benefits from DTC testing and evaluation of strengths and weaknesses of new NWP advances prior to consideration for operational implementation

 The focuses seem be in this order with heaviest weight on top.
  • Mesoscale modeling
  • hurricane
  • Data Assimilation
  • Verification
  • Ensembles
  • Mesoscale Modeling: Jamie Wolff | Ph: 303.497.2812 | email: jwolff@ucar.edu
    Hurricanes: Ligia Bernardet | Ph: 303.497.4315 | email: Ligia.Bernardet@noaa.gov
    Data Assimilation: Hui Shao | Ph: 301.683.3794 | EMAIL: huishao@ucar.edu
    Ensembles: Isidora Jankov | Ph: 303.497.4919 | email: Isidora.Jankov@noaa.gov
    Testbed Collaborations: Tara Jensen | Ph: 303.497.8479 | email: jensen@ucar.edu
Community code:

GSI Documentation: 
GSI user guide: 
   http://www.dtcenter.org/com-GSI/users/docs/index.php
   http://www.dtcenter.org/com-GSI/users/docs/users_guide/GSIUserGuide_v3.2.pdf
workshop:
   http://www.dtcenter.org/com-GSI/users/docs/workshop_presentations_2011.php
   http://www.dtcenter.org/com-GSI/users/docs/workshop_presentations_2013.php

 

 

 

Monday, March 3, 2014

binary file structure holding data for a list of radiance files

1. Radiance files:
This file structure applied to both observed radiance file and simulated radiance file.

A radiance file has both a header, which has summary information about the data, and a body,
which holds data itself.

A binary file can contain data for a list of radiance files, and the file structure we are talking about here refers to this kind of binary files.

If a binary files contains data for a list of  radiance files, then headers are stacked in the front while
bodies are group at the end of this binary file.
































LoadRadFile,nfilesRad,filesRad,Rad,Idtop
   - nfilesRad          I           Number of radiance files to read
   - filesRad           I           List of files of read
   - Rad                O           Structure containing
   - Idtop              X           Dummy (not needed here)

      ReadRadHdr,
         - TopId              I              Not used
         - fileRad            I              Name of radiance file
         - iu                 O              Unit number
h1        - nMeasurData        O              Number of measuremnts in file
h2        - nchan              O              Number of channels
h6        - cfreq              O              Central frequencies
h7        - polar              O              Poalrizations
h5        - nqc                O              Size of QC
h3        - nPosScan           O              Number of scan positions
h4       == nScanLines   ;; missing in description


      ReadRad
         - IdTop              I              Not used.
         - iu                 I              Unit number
         - nprof              I              Number of profiles
         - nchan              I              Number of channels
         - nqc                I              Size of QC array
b12       - tb                 O              TB array
b13       - qc                 O              QC array
b11       - Angle              O              Viewing angle
b3        - RelAziAngle        O              Relative Azimuth Angle
b4        - SolZenAngle        O              Solar Zenith Angle
b1        - lat                O              Latitude
b2        - lon                O              Longitude
b5        - direc              O              Orbit mode
b6        - iscanPos           O              Scan position
b7       == iscanLine    ;; missing in description
b8        - Year               O              Year
b9        - Day                O              Day
b10       - Time               O              Time
         - nprofsEff          O              Efective number of profiles

Data structure:
  Rad={                                             $
       ;----Header
       nfilesRad:   nfilesRad,                      $  ;Number of files
  constants:
  h1     nProf:       nprof,                          $  ;Number of profiles
  h2     nchan:       nchan,                          $  ;Number of channels
  h3     nPosScan:    nPosScan,                       $  ;Number of scan positions
  h4     nScanLines:  nScanLines,                     $  ;Number of scan lines
  h5     nqc:         fltarr(nfilesRad),              $  ;Size of QC vector
  1-d:  per channel
  h6     Cfreq:       cfreq (fltarr(nchan))           $  ;Central frequencies
  h7     Polarity:    polar (lonarr(nchan))           $  ;Polarizations
;; ---------------------------------------------------------------------------------------------
  3-d:  per file, per fov , per chan
  b12    tb:          fltarr(nfilesRad,nProf,nchan),  $  ;Array of TBs
  b11    Angle:       fltarr(nfilesRad,nProf,nchan),  $  ;Array of viewing angles

  2-d:  per file, per fov
  b3     RelAziAngle: fltarr(nfilesRad,nProf),        $  ;Array of relative azimuth angles
  b4     SolZenAngle: fltarr(nfilesRad,nProf),        $  ;Array of solar zenith angles
  b1     Lat:         fltarr(nfilesRad,nProf),        $  ;Array of latitude
  b2     Lon:         fltarr(nfilesRad,nProf),        $  ;Array of longitude
  b5     Direc:       intarr(nfilesRad,nProf),        $  ;Array of orbit mode flags
  b6     ScanPos:     intarr(nfilesRad,nProf),        $  ;Scan positions
  b7     ScanLine:    intarr(nfilesRad,nProf),        $  ;Scan lines
  b8     Year:        lonarr(nfilesRad,nProf),        $  ;Year
  b9     Day:         lonarr(nfilesRad,nProf),        $  ;Day
  b10    Time:        fltarr(nfilesRad,nProf),        $  ;UTC Time
         Hours:       fltarr(nfilesRad,nProf),        $  ;Hours
         Mins:        fltarr(nfilesRad,nProf),        $  ;Minutes
         Secs:        fltarr(nfilesRad,nProf),        $  ;Seconds
  b13    qc:          intarr(nfilesRad,nProf,14)      $  ;QC information
      }

what is read in Header:  (7)
constant:
    nMeasurData
    nchan
    nPosScan
    nScanLines
    nqc
1-d : fltarr(nchan)
    cfreq ; 1d(nchan)
    polar ; 1d(nchan)


what is read in Body:  (13)
1-d: lonarr(nprof)
  1  lat
  2  lon
  3  raa
  4  sza
  5  direc
  6  iscanPos
  7  iscanLine
  8  Year
  9  Day
  10 time
  11 angle
2-d: fltarr(nprof,nchan)
  12 tb : 2-d
  13 qc : 2-d

2. Scene files: (EDR format)
readu order when reading scene data file: 

header:  18 elements
   iType
   algSN
   nProf
   nLayer
   nLevel
   nChan
   nPosScan
   nScanLines
   nAbsorb
0  nParmCLW
   nParmRain
   nParmSnow
   nParmIce
   nParmGrpl
   absorbID
   cFreq
   polar
8  nQC

body:   28 + [7] ==> up to 35 elements
   profIndx
   presLayer
   presLevel
   tempLayer
   absorbents
   xCLW
   xRain
   xGrpl
   emiss
0  angle
   windSpeed
   tSkin
   sfcPressure
   sfcType
   windU
   windV
   RAA
   SZA
   snowDepth
0  QC
   lat
   lon
   node
   scanUTC
   scanYear
   scanDay
   iScanPos
8  iScanLine
   IF (Scene.iType eq 1) THEN BEGIN
       nAtt
       nIter
       chiSquare
       yFwd
       chanSelected
       ym
7      ymCorr
   ENDIF 
 
scene data structure: 
  Scene={                                        
    ;----Header
       algSN:algSN,                                readu,iu,algSN
       iTyp:iTyp,                                  readu,iu,iTyp
       nProf:nPrf,                                 readu,iu,nPrf
       nProfsProcessed:0L, not from scene file,    Num of profs read successfully                       nLay:nLay,                                  readu,iu,nLay
       nLev:nLev,                                  readu,iu,nLev
       nChan:nChan,                                readu,iu,nChan
       nScanPos:nPosScan,                          readu,iu,nPosScan
       nScanLines:nScanLines,                      readu,iu,nScanLines
       nAbsorb:nAbsorb,                            readu,iu,nAbsorb
       nParmCLW:nParmCLW,                          readu,iu,nParmCLW
       nParmRain:nParmRain,                        readu,iu,nParmRain
       nParmSnow:nParmSnow,                        readu,iu,nParmSnow
       nParmIce:nParmIce,                          readu,iu,nParmIce 
       nParmGrpl:nParmGrpl,                        readu,iu,nParmGrpl
       absorbID:absorbID2use4Declaration, A        AbsorbID=lonarr(nAbsorb), readu,iu,absorbID
       cFreq:cFreq,                       A        cfreq=fltarr(nChan), readu,iu,cFreq
       polarity:polar,                    A        polar=lonarr(nChan), readu,iu,polar
       nQC:nQC,                                    readu,iu,nQC
       declarN_Prf:nPrf,    duplicate value        nPrf
    ;----Body of Scene file  ( all FOVs )
       profIndxVec:  intarr(nPrf),        A    readu,iu, profIndx
       PresLayVec:   fltarr(nPrf,nLay),   AA   fltarr(Nlay), readu,iu, pressLay
       presLevVec:   fltarr(nPrf,nLev),   AA   fltarr(nLev), readu,iu, pressLev
       tempLayVec:   fltarr(nPrf,nLay),   AA   fltarr(Nlay), readu,iu, tempLay
       absorbLayVec: fltarr(nPrf,nLay,2), AA   fltarr(nAbsorb), readu,iu,absorbents
       tpwVec:       fltarr(nPrf),        A    DERIVED. !!!
       rhVec:        fltarr(nPrf,nLay),   A    DERIVED. !!!
       clwLayVec:    fltarr(nPrf,nParmCLW)AA   fltarr(nParmCLW), readu,iu, xCLW 
       clwVec:       fltarr(nPrf),        A    DERIVED. !!!
       rwpVec:       fltarr(nPrf),        A    DERIVED  !!!
       iwpVec:       fltarr(nPrf),        A    NOT SET  !!!
       gwpVec:       fltarr(nPrf),        A    DERIVED  !!!
       swpVec:       fltarr(nPrf),        A    NOT SET  !!! 
       rainLayVec:   fltarr(nPrf,nParmRain),AA fltarr(nParmRain), readu,iu, xRain
       snowLayVec:   fltarr(nPrf,nParmSnow),AA 0.0  (hard-coded)
       IceLayVec:    fltarr(nPrf,nParmIce), AA 0.0  (hard-coded)
       grplLayVec:   fltarr(nPrf,nParmGrpl),AA fltarr(Scene.nParmGrpl), readu,iu, xGrpl 
       anglVec:      fltarr(nPrf),          A  readu,iu, angl
       relAziAnglVec:fltarr(nPrf),          A  readu,iu, relAziAngl
       solZenAnglVec:fltarr(nPrf),          A  readu,iu, solZenAngl
       emissVec:     fltarr(nPrf,nChan),    AA fltarr(Scene.nchan), readu,iu, emiss
       reflVec:      fltarr(nPrf,nChan),    AA NOT SET !!!
       windSpVec:    fltarr(nPrf),          A  readu,iu, windSp
       windUVec:     fltarr(nPrf),          A  readu,iu, windU
       windVVec:     fltarr(nPrf),          A  readu,iu, windV 
       tskinVec:     fltarr(nPrf),          A  readu,iu, tSkin
       snowDepthVec: fltarr(nPrf),          A  readu,iu, snowDepth
       sfcPressVec:  fltarr(nPrf),          A  readu,iu, sfcPress
       sfcTypVec:    intarr(nPrf),          A  readu,iu, sfcTyp
       QC:           intarr(nPrf,nqc),      AA intarr(Scene.nqc), readu,iu, qc
       lat:          fltarr(nPrf),          A  readu,iu, lat
       lon:          fltarr(nPrf),          A  readu,iu, lon 
       direc:        intarr(nPrf),          A  readu,iu, node  (asc, desc)
       time:         fltarr(nPrf),          A  readu,iu, scanUTC
       year:         intarr(nPrf),          A  readu,iu, scanYear 
       day:          intarr(nPrf),          A  readu,iu, scanDay
       hours:        fltarr(nPrf),  DERIVED A  fix(scanUTC/3600.)  !!!
       mins:         fltarr(nPrf),  DERIVED A  fix( ( time - fix(Hours) * 3600. )/60.)  !!!
       secs:         fltarr(nPrf),  DERIVED A  (Time - Hours*3600. - Mins*60.)   !!!
       nAttempt:     intarr(nPrf),    COD   A  readu,iu, nAtt
       nIter:        intarr(nPrf),    COD   A  readu,iu, nIter
       chiSq:        fltarr(nPrf),    COD   A  readu,iu, chiSq
       yFwd:         fltarr(nPrf,nChan),COD AA fltarr(Scene.nchan), readu,iu,yFwd
       chanSel:      lonarr(nPrf,nChan),COD AA lonarr(Scene.nchan), readu,iu,chanSel
       ym:           fltarr(nPrf,nChan),COD AA fltarr(Scene.nchan), readu,iu,ym
       ymCorr:       fltarr(nPrf,nChan),cOD AA fltarr(Scene.nchan), readu,iu,ymCorr
       scanPos:      lonarr(nPrf),          A  readu,iu, iScanPos
       scanLine:     lonarr(nPrf)           A  readu,iu, iScanLine
  }

NOTE: 
1. Condition : COD
   COD: if Scene.iTyp eq 1

2. Derived data
   ;---compute TPW  (1)
   ind = WHERE(scene.absorbLayVec(iprof,0:Scene.nLay-1,0) ge 0.,ncount)
   IF (ncount gt 0) THEN BEGIN
      presLevFiltered = FLTARR(nCount+1)
      presLevFiltered(0:nCount-1) = scene.presLevVec(iProf,ind)
      presLevFiltered(nCount) = scene.presLevVec(iProf,ind(nCount-1)+1)
      columIntegr_LayW, nCount,     $
                        presLevFiltered,     $
                        sfcPress,     $
                        scene.absorbLayVec(iProf,ind,0),     $
                        water
      scene.tpwVec(iprof)  = water      ; (derived 1)
   ENDIF ELSE BEGIN
      scene.tpwVec(iprof)  = 0
   ENDELSE

   ;---compute ozon amt  ( NOT used ! ) 
   IF (WHERE(scene.absorbID(0:scene.nAbsorb-1) EQ 3) GE 0) THEN BEGIN
       columIntegr_LayW, scene.nLay,     $
                         scene.presLevVec(iProf,0:scene.nLev-1),   $
                         sfcPress,$
                         scene.absorbLayVec(iProf,0:scene.nLay-1,1),   $
                         ozon          ;  ( NOT USED ! )
   ENDIF

   ;---compute hydrometeors integrated amounts 
   columIntegr, scene.nParmCLW,     $
                scene.presLevVec(iProf,0:scene.nLev-1),   $
                sfcPress,                                 $
                scene.clwLayVec(iProf,0:scene.nParmCLW-1),$
                clw
   scene.clwVec(iprof) = clw        ; (derived 2)

   columIntegr, scene.nParmRain,     $
                scene.PresLevVec(iprof,0:Scene.nLev-1),  $
                sfcPress,                                $
                Scene.RainLayVec(iprof,0:Scene.nParmRain-1), $
                rwp
   scene.rwpVec(iprof) = rwp              ; (derived 3)

   columIntegr, scene.nParmGrpl,   $
                scene.PresLevVec(iProf,0:scene.nLev-1),  $
                sfcPress,$
                scene.GrplLayVec(iprof,0:Scene.nParmGrpl-1),  $
                gwp
   scene.gwpVec(iprof) = gwp           ; (derived 4)

   ;---compute RH
   FOR iLay=0L,scene.nLay-1 DO BEGIN
       IF (scene.absorbLayVec(iProf,iLay,0) GT 0) THEN BEGIN
           T=scene.tempLayVec(iProf,iLay)
           P=scene.presLayVec(iProf,iLay)
           Q=scene.absorbLayVec(iProf,iLay,0)/1000.
           scene.rhVec(iProf,iLay) = mixingratio_to_RelHum(Q,T,P)*100.   ; (derived 5)
       ENDIF ELSE BEGIN
           scene.rhVec(iprof,iLay) = -999.
       ENDELSE
   ENDFOR
 
 
 

Friday, February 28, 2014

IDL experience accumulated

1. Output device
1) SET_PLOT : specify the output device.
    Normally it's 'X' that is screen, however, you can specify other devices such as PostScript.
    Normal screen is black background. PostScript is white background.
2) DEVICE : access or control abilities a device provides.

2. Variables have be defined somewhere, whether it's in procedure/function or it's in the main program. What it mean by "defined" is that a variable needs to be assigned with a value.
You can feed less parameters to a procedure/function than what the procedure/function expects as long as you don't refer to these missing variables in your code.
eg:
  PRO proc1,  i1, i2, o1, o2
    o1=i1 + i2   ; o1 and o2 are defined within proc1
    o2=i1 * i2  
  END

  i1=1       ; i1 and i2 are defined within main.
  i2=2 
  proc1 i1, i2, o1   ; you don't feed o2 when calling proc1
  print o1              ;  and that's okay to miss o2 as long as you don't refer o2
  print o2              ; This is an error because you refer to o2.
                            ;  to correct the issue, you need to define o2 either in main or in proc1.


3. multiple statements can be joined together using "&"
eg:
    if deyong eq 1 then print, 1 &  print , 2 & print, 3
is the same as :
    if deyong eq 1 then begin
        print, 1
        print, 2
        print, 3
    endif


4. summary about "SIZE" function: return type info of a variable
  t1 = size(deyong, /type)
  print , "t1 is " , t1          ;  undefined
  deyong=3L
  t1 = size(deyong, /type)
  print , "t1 is " , t1          ;  3 -> long int
  deyong=4.5          
  t1 = size(deyong, /type)
  print , "t1 is " , t1           ; 4   -> float

5 "POSITION"
Allows direct specification of the plot window.
POSITION is a 4-element vector giving, in order, the coordinates [(X 0 , Y 0 ), (X 1 , Y 1 )], of the lower left and upper right corners of the data window. Coordinates are expressed in normalized units ranging from 0.0 to 1.0, unless the DEVICE keyword is present, in which case they are in actual device units. The value of POSITION is never specified in data units, even if the DATA keyword is present.
   Y /\ 
 1.0 |
       |
0.0  |________________\    X
      0.0                      1.0  /

6. PLOT
PLOT, [X,] Y
X :  A vector argument. If X is not specified, Y is plotted as a function of point number (starting at zero). If both arguments are provided, Y is plotted as a function of X .
Y :  A vector argument.

7. !PATH.MULTI
In IDL, the !P.Multi system variable can be used to create multiple plots in a display window. !P.Multi is a five element vector defined as follows:


!P.Multi(0) Contains the number of plots remaining on the page. Start with this as 0 to clear the page.
!P.Multi(1) The number of plot columns on the page.
!P.Multi(2) The number of plot rows on the page.
!P.Multi(3) The number of plots stacked in the Z direction.
!P.Multi(4) If 0, plots are displayed from left to right and top to bottom, i.e., the plots are displayed in rows. If 1, plots are displayed from top to bottom and from left to right, (i.e., the plots are displayed in columns).

To display four plots on a page in two columns and two rows, and the plots should appear in columns, the !P.Multi array should look like:
IDL> !P.Multi = [0, 2, 2, 0, 1]
8. COMMON block 
Common blocks are useful (a) when there are variables that need to be accessed by several IDL procedures or (b) when the value of a variable within a procedure must be preserved across calls.

Variables in a common statement have a global scope within procedures defining the same common block. Unlike local variables, variables in common blocks are not destroyed when a procedure is exited.

There are two types of common block statements:
    1. definition statements
          common block_name, v1, v2
    2. reference statements.      
          dxu: duplicates the COMMON block and variable names from a previous definition.
          common block_name            ; same as   common block_name, v1, v2

Variables in IDL COMMON blocks do not actually have names.

The number of variables appearing in the common block cannot change after the common block has been defined. ( Fixed once defined:  )


The "first program unit" (the one which gets compiled first : main program, function, or procedure, NOT order who gets executed ) to define the common block sets the number of included variables;
eg:
    common block_a , v1, v2, v3       ; there are 3 vars in block.



Other program units can reference the common block with any number of variables up to the number originally specified.
eg:
    common block_a , v1, v2         ; number could be less than the original number.

Different program units can give the variables different names.
eg:
    common block_a ,  M1, M2    ; name is NOT important, position IS.  so M1 = v1, M2=v2.

9. Array assignment. 
a=make_array(2,3, /integer, value=100)
b=make_array(2,3, /integer, value=200)
a) slice operation 
   b(0,0:1) = a(0, 0:1)   ; copy first two element in row 0
  b(0:1, 0) = a(0:1, 0)   ; copy first two element in column 0 
b) copy 1 row or 1 column
    b(*,1) = a(*,1)         ; copy column 1
   b(1,*) = a(1,*)          ; copy row 1
c) copy entire array
   b(*,*) = a(*,*)          ; copy all the rows / copy all the columns
   b = a                           ; same as above
d) shriek array
   a = a (*, 0:1)          ; array shrieked to first two columns.
e) Playing an array is easy.eg: 
   a = a + a     ; doubling all the elements in an array
   a = 3* a      ; tripling all the elements in an array
   a= a * 3      ; same as above
   a= a / 3       ; divide elements by 3
   a= a ^ 3      ; square elements

 f) print, a                     ; column-base: print column1 , then print column2, so on.
 g) raise initial value of array to 1-base
eg:
   a = INDGEN(3)+1
   print, a
output: 
   1       2       3

10. Index variable used in for-statement will still incremented/decremented one more time after for-statement exits.
eg:
   for i=2, 1, -1 do begin
      j=i    &   print, i , j
   endfor
   print, i , j

result: 
       2       2     ; i, j
       1       1     ; i, j
             1     ; i, j    , note : i will be decremented one more time after for-statement.

11. Conversion to string
Use strtrim or string:  str
eg:
a=strtrim("  34  ", 2)    ; 2nd parameter 2 means removal of both leading and trailing spaces.
help ,a
a=strtrim(23, 2)            ;  convert int to string
help ,a
a=string(23.234, format='(f5.2)')      ; convert float to string,
help ,a                                                ; f5.2 : 5 digits in total including dot with 2 digits after dot.
a=string(232.232)                 
help ,a
output:
A               STRING    = '34'
A               STRING    = '23'
A               STRING    = '23.23'
A               STRING    = '      232.232'

12. Catch message from IDL code when running IDL code from bash.
eg:
runIDL.sh 
#!/bin/bash
idl <<EOF
   .run  ${mainCode}
   7
   1
EOF

$ ./runIDL.sh   >log  2>> log2
Note:   a) Normal prints from IDL commands go to log.
            b) Messages from compiling IDL code go to log2.
$ tf log       # to monitor the process of IDL code. 

13. Logical/Mathematical operations involving array 
Basically it applies an operation to each element of an array and generates a new array. 
eg: 
a=indgen(5)
b=a>2          ; max value (mathematical operation)
c=a ge 2       ; logical operation.
print, a
print, b
print, c
       0       1       2       3       4
       2       2       2       3       4
      0   0   1   1   1

14. Device must close at the end of  plotting, otherwise, missing data could happen because these data are not flushed into the graphics if Device is not closed.
So remember to do "DEVICE, /CLOSE" explicitly when needed.

15. Make font better in PS output
aspect_ratio=1.5  ; rectangle shape
xsize=9
ysize=xsize/aspect_ratio
set_plot, 'ps'
!p.font=0
device, filename='fig_better.eps', encapsulated=1, /helvetica
device, xsize=xsize, ysize=ysize
plot, a, b, xtitle='X Title', ytitle='Y Title'
device, /close
set_plot, 'x'
!p.font=-1
 
16. where 
If "where" doesn't find any match, then what "where" returns is -1L, not an long array. 
In this case, using the filter return from "where" actually will copy the last 
element of input array to the new array. 
eg: 
var.a=[1,2,3,4] 
fil=where(var.a gt 4)
help, fil
print, "filter " , fil, n_elements(fil)
a.a = var.a(fil) 
print , a.a
 
output: 

FIL             LONG      =           -1
filter           -1           1
       4       4       4       4

This action could be used as a trick to initialize an array with the last element 
of an input array. 
 
eg: 
a = indgen(10) + 1 
b = make_array(10, /int, value=-999) 
filt = where ( a GT 3 ) 
help, b
print, b 
b = a(filt) ;; What this does is to copy data that satisfies the condition 
            ;; to beginning of array b and truncate tail of b off. 
help, b
print, b
 
Output:  
B               INT       = Array[10]
    -999    -999    -999    -999    -999    -999    -999    -999    -999    -999
B               INT       = Array[7]
       4       5       6       7       8       9      10
 
  
16. Create array as 1-based, or 2-based, etc. 
Eg: 
  a = INDGEN(10) + 1  ; 1-based array of 10 elements.   
  a = INDGEN(10) + 2  ; 2-based array of 10 elements
 
17. Read a line as string. 
   ; Either way, it works to read a line as string. 
   READF, iu, a
   ;READF, iu, form='(a0)',  a
   b=strsplit(a,/extract) ; /extract to return sub-string, w/o it it returns 
                          ; an array of the position of the substrings is returned.
   print, b(-1)    ; -1: print last element of an array
 
18. [] operator 
   ; This is so wrong. Here is the reason why.
   ; [1, nPos] is 2-element array
   ; [bias(*,iChan,0), bias(*,iChan,0)] is 60-element array.
   ; because * represents 30 scan positions.
   ; What happens is it gets the two values from the first
   ; two scan positions. I don't know what the initial intent is for.
   ; but this doesn't seem to do whatever the intent is for.
   ;;; PLOTS,[1, nPos], [bias(*,iChan,0), bias(*,iChan,0)]
   ; Change to below to connect first and last
;     PLOTS,[1, nPos], [bias(0,iChan,0), bias(nPos-1,iChan,0)]
 
19. if-else 
Doing so to simplify complicated if_else_statement. 
if (3 gt 4 ) then    $
    print, 'big '    $           ; concatenation is NOT allowed 
else                 $
    print, 'small' & $           ; concatenation IS allowed in else-statement
    print, 'small' & $
    print, 'small' & $
    print, 'small'
 
20. CD command 
The CD procedure is used to set and/or change the current working directory.  
CD, new_dir, CURRENT=old_dir   ; save current directory inot var old_dir 
CD, 'data'
 
; how to get the current working directory info 

IDL> cd , current=old          
IDL> print, old
/data/home001/dxu/graphic

IDL> CD, CURRENT=c & PRINT, c    ; do it in one line

IDL> cd, 'graphic'     ; take relative path
IDL> cd , '../../'          ; take relative path


IDL> cd , '/data/home001/dxu  '                ; Do NOT have whitespace in the directory string.
% CD: Unable to change current directory to /data/home001/dxu .
  No such file or directory
% Execution halted at: $MAIN$
IDL> cd , '/data/home001/dxu'       ; Only work when there is NO white-space.

21. Plot map with data 
eg: 
LOADCT, 39
MAP_SET, 0,0, charsize=1,  /label, latlab=-180, lonlab = -90, latdel=30, londel=60
for i =0, num-1 do begin
   OPLOT, [x(i)], [y(i)], psym=2, symsize = 5, color=y[i]
endfor
MAP_CONTINENTS, /hires, fill_continents=0,  MLINESTYLE=0, MLINETHICK=1, color=210

22. index in where. 
a=indgen(30)+1
print, a
print,'---------------'

ff=where(a ge 20)    ; ff is index of a.
b= a(ff)
print, ff
print,'====='
print, b
print,'---------------'

ff2 = where (b ge 25)  ; ff2 is index of new array b
c= b(ff2)
print, ff2
print,'====='
print, c


23. Set IDL path 
Rather than having to type that !path statement every time you start IDL, you can put it in your .idlstartup file.

$ vi   ~/.idl/.idlstartup

Just add:
!PATH=!PATH+':'+Expand_Path('./src/idl/coyote/')

$ cd  ~/.idl/itt/pref-10-idl_8_0-unix/idl.pref
$ vi     idl.pref

You add:
IDL_STARTUP : ~/.idl/.idlstartup

Maybe there's an easier way to do it, but at least you don't have to assign !PATH every time you start IDL.

If you want to access coyote from other IDL instances, then just replace that path with the absolute path.

24. Here is another way thread about setting up IDL path

Adding Programs to Your IDL Path

IDL looks for programs (*.pro  or *.sav) in a list of directories called a "path". If a program is in a folder that is in your path then IDL knows to look for it when you try to call it in your program or from the command line. By default your present working directory (pwd a.k.a. ./) are in your path as well as some default IDL folders. I suggest that you create a folder called "idl" to keep all your programs in to keep it organized. You then need to add that folder to your path in one of the ways described below.

dxu: Note that this is only to specify the location where IDL can fine a IDL code (*.pro or *.sav). It has nothing to do with a specific subroutine/function within an IDL file.
For example, if you are calling a subroutine called "generateOutput", which is in a.pro file,  then you still need to include @a.pro in your main IDL code, so the module "generateOutput" is available to your main code to use. When IDL looks for "a.pro" file, it will go through a list of directories called a '!path' to find it.
1) So to make a module (function/subroutine) available to your main code, you need to use "@" statement to include a IDL code (eg: a.pro)  that contains the module. 
2) To let IDL find that specific file (eg: a.pro) , you need to specify the location where that file is located in !path, which is used by IDL to find all the IDL code (*.pro and *.sav)

The easiest way to do add a folder to the path that is independent of architecture is the following:
1) Make an IDL startup file. If you have already done this then skip to step 2. You can make an IDL startup file by
     a) creating a file called .idlstartup  in /path/to folder.
     b) within IDL type:
     IDL> pref_set, 'IDL_STARTUP', '/path/to/.idlstartup',/commit
         where you replace '/path/to/.idlstartup' with the path to the file you created for step 1a

2) Now edit your .idlstartup file to include the following lines to include a folder which for example is located in ~/example/
           !PATH=!PATH+':'+Expand_Path('~/example/')
If I wanted to include all sub-directories I would add the following line instead (notice the + sign in front):
     !PATH=!PATH+':'+Expand_Path('+~/example/')

You can also do the same actions in your bash or csh profiles without the use of the .idlstartup file as follows:
bash:
      export IDL_PATH=$IDL_PATH:+'~/example/'
csh:
     setenv IDL_PATH +$IDL_PATH:~/examples

where you can again include all subdirectores by leading the name with a + sign



IDL array

Here are arrays I'm interested at:
1. Create array with initialization to subscription
B-INDGEN       : byte                         1
U-INDGEN       : unsigned int            2
INDGEN          : int                            2 
UL-INDGEN     : unsigned long int    4 
L-INDGEN        : long int                    4
 
UL64-INDGEN : unsigned 64-bit int   8
L64-INDGEN    : 64-bit int                  8
 
F-INDGEN        : float                          4
D-INDGEN        : double                      8
S-INDGEN         : string                       2.1GB

Eg:
I = INDGEN(5)

2. Create array only
BYTARR   
UINTARR   
INTARR   
 
ULONARR   
LONARR
   
ULON64ARR
LON64ARR
 
FLTARR
DBLARR
STRARR

Eg: Create I, a 3-element by 3-element integer array with each element set to 0 by entering:
I = INTARR(3, 3)

3. MAKE_ARRAY
The MAKE_ARRAY function returns an array of the specified type, dimensions, and initialization. This function enables you to dynamically create an array whose characteristics are not known until run time.
Eg: 
 To create M, a 3-element by 4-element, integer array with each element set to the value 5, enter:
M = MAKE_ARRAY(3, 4, /INTEGER, VALUE = 5)


4. REPLICATE 
   Creates an array of given dimensions, filled with specified value.
Eg: Create D, a 5-element by 5-element array with every element set to the string "IDL" by entering:
D = REPLICATE('IDL', 5, 5)