Monday, June 30, 2014

how to run setpdy.sh

1. Two vars must be exported first : cycle and PDY
$ export cycle=06

$ export PDY=20130719

2. Run setpdy.sh
$ /usr/local/jcsda/nwprod_gdas_2014/util/ush/setpdy.sh

This is generate a file called "PDY" 

$ lr  PDY
-rwxrwxr-x  1 dxu domain users     328 Jun 30 19:35 PDY
$ cat  PDY
export PDYm7=20130712
export PDYm6=20130713
export PDYm5=20130714
export PDYm4=20130715
export PDYm3=20130716
export PDYm2=20130717
export PDYm1=20130718             # <== 7 days pasted                
export PDY=20130719                  # <== current day
export PDYp1=20130720              # <== 7 days coming
export PDYp2=20130721
export PDYp3=20130722
export PDYp4=20130723
export PDYp5=20130724
export PDYp6=20130725
export PDYp7=20130726

Tuesday, June 24, 2014

chi-square test

chi-square test

A statistical significance test based on frequency of occurrence; it is applicable both to qualitative attributes and quantitative variables.

Among its many uses, the most common are tests of hypothesized probabilities or probability distributions (goodness of fit), statistical dependence or independence (association), and common population (homogeneity). The formula for chi square (χ2) depends upon intended use, but is often expressible as a sum of terms of the type (f - h)2/h where f is an observed frequency and h its hypothetical value.
 
 chi-square = Sum (  (f - h)2/h  ) 
 

f is an observed frequency and 
h its hypothetical value.
 
=================
More detailed explanation: 

Chi-Square Calculator: Online Statistical Table

The chi-square distribution calculator makes it easy to compute cumulative probabilities, based on the chi-square statistic. For help, read the Frequently-Asked Questions or review the Sample Problems.
To learn more about the chi-square distribution, read Stat Trek's tutorial on the chi-square distribution.
  • Enter a value for degrees of freedom.
  • Enter a value for one, and only one, of the remaining text boxes.
  • Click the Calculate button to compute a value for the blank text box.
Degrees of freedom
Chi-square critical value (CV)
Cumulative probability: P(Χ2 < CV)
    


Instructions: To find the answer to a frequently-asked question, simply click on the question. If you don't see the answer you need, read Stat Trek's tutorial on the chi-square distribution or visit the Statistics Glossary.
What are degrees of freedom?
Degrees of freedom can be described as the number of scores that are free to vary. For example, suppose you tossed three dice. The total score adds up to 12. If you rolled a 3 on the first die and a 5 on the second, then you know that the third die must be a 4 (otherwise, the total would not add up to 12). In this example, 2 die are free to vary while the third is not. Therefore, there are 2 degrees of freedom.
In many situations, the degrees of freedom are equal to the number of observations minus one. Thus, if the sample size were 20, there would be 20 observations; the the degrees of freedom would be 20 minus 1 or 19.
What is a chi-square critical value?
The chi-square critical value can be any number between zero and plus infinity. The chi-square calculator computes the probability that a chi-square statistic falls between 0 and the critical value.
Suppose you randomly select a sample of 10 observations from a large population. In this example, the degrees of freedom (DF) would be 9, since DF = n - 1 = 10 - 1 = 9. Suppose you wanted to find the probability that a chi-square statistic falls between 0 and 13. In the chi-square calculator, you would enter 9 for degrees of freedom and 13 for the critical value. Then, after you click the Calculate button, the calculator would show the cumulative probability to be 0.84.
What is a cumulative probability?
A cumulative probability is a sum of probabilities. The chi-square calculator computes a cumulative probability. Specifically, it computes the probability that a chi-square statistic falls between 0 and some critical value (CV).
With respect to notation, the cumulative probability that a chi-square statistic falls between 0 and CV is indicated by P(Χ2 < CV).
What is a chi-square statistic?
A chi-square statistic is a statistic whose values are given by
Χ2 = [ ( n - 1 ) * s2 ] / σ2
where σ is the standard deviation of the population, s is the standard deviation of the sample, and n is the sample size. The distribution of the chi-square statistic has n - 1 degrees of freedom. (For more on the chi-square statistic, see the tutorial on the chi-square distribution.)
What is a probability?
A probability is a number expressing the chances that a specific event will occur. This number can take on any value from 0 to 1. A probability of 0 means that there is zero chance that the event will occur; a probability of 1 means that the event is certain to occur. Numbers between 0 and 1 quantify the uncertainty associated with the event. For example, the probability of a coin flip resulting in Heads (rather than Tails) would be 0.50. Fifty percent of the time, the coin flip would result in Heads; and fifty percent of the time, it would result in Tails.

  1. The Acme Widget Company claims that their widgets last 5 years, with a standard deviation of 1 year. Assume that their claims are true.

    If you test a random sample of 9 Acme widgets, what is the probability that the standard deviation in your sample will be less than 0.95 years?

    Solution:

    We know the following:

    • The population standard deviation is equal to 1.
    • The sample standard deviation is equal to 0.95.
    • The sample size is equal to 9.
    • The degrees of freedom is equal to 8 (because sample size minus one = 9 - 1 = 8).

    Given these data, we compute the chi-square statistic:

    Χ2 = [ ( n - 1 ) * s2 ] / σ2
    Χ2 = [ ( 9 - 1 ) * (0.95)2 ] / (1.0)2 = 7.22

    where σ is the standard deviation of the population, s is the standard deviation of the sample, and n is the sample size.

    Now, using the Chi-Square Distribution Calculator, we can determine the cumulative probability for the chi-square statistic. We enter the degrees of freedom (8) and the chi-square statistic (7.22) into the calculator, and hit the Calculate button. The calculator reports that the cumulative probability is 0.49. Therefore, there is a 49% chance that the sample standard deviation will be no more than 0.95.
  2. Find the chi-square critical value, if the cumulative probability is 0.75 and the sample size is 25.

    Solution:

    We know the following:

    • The cumulative probability is 0.75.
    • The sample size is 25.
    • The degrees of freedom is equal to 24 (because sample size minus one = 25 - 1 = 24).

    Given these data, we compute the chi-square statistic, using the Chi-Square Distribution Calculator. We enter the degrees of freedom (24) and the cumulative probability (0.75) into the calculator, and hit the Calculate button. The calculator reports that the chi-square critical value is 28.2.

    This means that if you select a random sample of 25 observations, there is a 75% chance that the chi-square statistic from that sample will be less than or equal to 28.2.
 
 
 

liquid water path

liquid water path

A measure of the weight of the liquid water droplets in the atmosphere above a unit surface area on the earth, given in units of kg m-2, for example.

The liquid water path may be defined as
ams2001glos-Le22
where 
ρair is the density of the (wet) air,
and the integral is from the bottom to the top of the column.

Monday, June 16, 2014

Document needed to travel.



Close Window

What identification will I need at check-in?


The documentation you'll need will vary depending upon your citizenship and your destination.
Travel within the U.S. and U.S. Territories
  • U.S. citizens (18 and over) traveling within the U.S. will require a government issued photo ID. U.S. citizens under the age of 18 are not required to provide an ID at the airport security checkpoint.
  • Non-US/Canadian citizens are not required to carry their passports if they have documents issued by the U.S. government such as Permanent Resident Cards. Those who do not should carry their passports while visiting the U.S.

International Travel
All International travelers will need a valid passport, and may also need to show additional documentation at the destination and/or in connecting countries. In addition, all travelers must have the proper documents for entry/re-entry into a country (i.e. correct visa, valid passport, onward/return ticket). For international one-way travel, proof of return travel or onward travel may be required and children travelling without both parents may also require additional documentation. If you are unsure of the travel requirements for your destination, please contact the embassy or consulate of all countries you are traveling to and connecting through.
Close Window

Friday, June 6, 2014

open statements in Radiance Monitoro (radmon)

-------------------------------------------------
 make_base : construct base history files for each
             entry in the SATYPE list from the
             time data files (*.ieee_d) for the specified source (suffix)
-------------------------------------------------
File 1 is :  ./make_base.fd/make_base.f90
  open( lunchn, file=channel_file, form='formatted',iostat=ierror )
  open( luncyc, file=cycle_file, form='formatted',iostat=ierror )
  open( lungrd, file=data_file, form='unformatted', iostat=ierror )
  open(lunout,file=out_file,form='formatted')

-------------------------------------------------
radmon_bcor :
           extract bcor (bias correction) radiance data from
          a diagnostic file from a global run.
-------------------------------------------------
File 16 is :  ./verf_radbcor.fd/bcor.f90
  open(lndiag,file=diag_rad,form='unformatted')
  open(lungrd,file=data_file,form='unformatted')
  open(lungrd,file=data_file,form='unformatted')
File 17 is :  ./verf_radbcor.fd/create_ctl_bcor.f90
  open(lunctl,file=ctl_file,form='formatted')

-------------------------------------------------
radmon_bcoef :
   extract bcoef radiance data from a diagnostic
   file from a global run.
-------------------------------------------------
File 2 is :  ./verf_radbcoef.fd/create_ctl_bcoef.f90
  open(lunctl,file=ctl_file,form='formatted')
File 3 is :  ./verf_radbcoef.fd/bcoef.f90
  open(lndiag,file=diag_rad,form='unformatted')
  open(lncoef,file='biascr.txt',form='formatted')
  open(lungrd,file=data_file,form='unformatted')
  open(lungrd,file=data_file,form='unformatted')


-------------------------------------------------
radmon_time : extract time radiance data from
         a diagnostic file from a global run
-------------------------------------------------
File 4 is :  ./verf_radtime.fd/bad_penalty.f90
  open( UNIT=funit, FILE=fname, STATUS='NEW', IOSTAT=fios )
  open( UNIT=funit, FILE=fname, STATUS='OLD', POSITION='APPEND', IOSTAT=f
File 5 is :  ./verf_radtime.fd/create_ctl_time.f90
  open(lunctl,file=ctl_file,form='formatted')
File 6 is :  ./verf_radtime.fd/bad_obs.f90
  open( UNIT=funit, FILE=fname, STATUS='NEW', IOSTAT=fios )
  open( UNIT=funit, FILE=fname, STATUS='OLD', POSITION='APPEND', IOSTAT=f
File 7 is :  ./verf_radtime.fd/time.f90
  open(lndiag,file=diag_rad,form='unformatted')
  call open_bad_penalty_file( date, cycle, ios )
  call open_bad_chan_file( date, cycle, ios )
  open(lungrd,file=data_file,form='unformatted')
  open(lungrd,file=data_file,form='unformatted')
File 8 is :  ./verf_radtime.fd/bad_chan.f90
  open( UNIT=funit, FILE=fname, STATUS='NEW', IOSTAT=fios )
  open( UNIT=funit, FILE=fname, STATUS='OLD', POSITION='APPEND', IOSTAT=f
File 9 is :  ./verf_radtime.fd/valid.f90
  open( UNIT=funit, FILE=fname, IOSTAT=fios )

-------------------------------------------------
radmon_angle : extract angle radiance data from
            a diagnostic file from a global run.
-------------------------------------------------
File 14 is :  ./verf_radang.fd/angle_bias.f90
  open(lndiag,file=diag_rad,form='unformatted')
  open(iscan,file='scaninfo.txt',form='formatted')
  open(lungrd,file=data_file,form='unformatted')
  open(lungrd,file=data_file,form='unformatted')
File 15 is :  ./verf_radang.fd/create_ctl_angle.f90
  open(lunctl,file=ctl_file,form='formatted')

Virtual temperature (TV)


virtual temperature
(Also called density temperature.) The virtual temperature Tv = T(1 + rv/ ε)/(1 + rv),
where rv is the mixing ratio and
ε is the ratio of the gas constants of air and water vapor, ≈ 0.622.

The virtual temperature allows the use of the dry-air equation of state for moist air,
except with T replaced by Tv. Hence the virtual temperature is the temperature that dry air
would have if its pressure and density were equal to those of a given sample of moist air.

For typical observed values of rv, the virtual temperature may be approximated by
Tv = (1 + 0.61 rv) T.

Some authors incorporate the density increment due to liquid or solid water into virtual temperature,
in which case the definition becomes Tv = T(1 + rv/ε)/(1 + rv + rl) ≈ T(1 + 0.61rv - rl),
where rl is the liquid or liquid plus solid water mixing ratio.

HS     surface orography (m)
PS     surface pressure (Pa)
P     pressure (Pa)
DP   delta pressure (Pa)
TV   virtual temperature (K)
T     sensible temperature (K)
Q     specific humidity (kg/kg)
RH  relative humidity (%)
U     zonal wind (m/s)
V     meridional wind (m/s)
DIV  divergence (m/s**2)
VOR   vorticity (m/s**2)
Q2    tracer 2 (kg/kg)
Q3    tracer 3 (kg/kg)

Thursday, June 5, 2014

Run GSI tag version on badger

Here is how to run a GSI tag version on badger:

Part 1: how to compile GSI
1. Make a branch out of GSI tag version REL-5.0.0
$ svn copy https://svnemc.ncep.noaa.gov/projects/gsi/tags/REL-5.0.0   https://svnemc.ncep.noaa.gov/projects/gsi/branches/NESDIS-JCSDA/users/dxu/gsi_tag_rel-5.0.0   -m "branch from tag version REL-5.0.0"

2. Check out my own tag version. Remember to use flag "--ignore-externals" to NOT check out CRTM, which has its own trunk and is just a link within GSI trunk.
$ cd /data/dxu
$ svn co --ignore-externals https://svnemc.ncep.noaa.gov/projects/gsi/branches/NESDIS-JCSDA/users/dxu/gsi_tag_rel-5.0.0

3. Copy Makefile.conf from Krishna's version of gsi(modified for s4).
  (Krishna checked out https://svnemc.ncep.noaa.gov/projects/gsi/tags/REL-5.0.0 )
 This Makefile.conf includes information about CRTM and libraries setting.
$ cp  /data/kkumar/nwprod_v2014/REL-5.0.0/src/Makefile.conf .

4. Compile my own tag GSI version to create the only one executable file: global_gsi (32M bytes)
$ cd /data/dxu/gsi_tag_rel-5.0.0
$ cd src
$ make -f Makefile


It should create an executable file global_gsi under "src" directory.
-rwxrwxr-x 1 dxu domain users 32M Jun  5 15:19 global_gsi


Part 2: how to run GSI
1. Copy a sample script from Krishna $  cd /data/dxu/iat/gsi
$ cp /data/kkumar/for_deyong/standalone_gsi62_test.sh  . 
2. Save it locally.
$ cp standalone_gsi62_test.sh   standalone_gsi62_test_using_my_gsi.sh 
3. Modify file to customize (only two places need to be changed). 
$ vi   standalone_gsi62_test_using_my_gsi.sh 
   #  Create working directory /scratch5/dxu/tmp62/test_gsi62_standalone1.0.2_06052014 

   exp=test_gsi62_standalone1.0.2_06052014    
   # Point to GSI executable created in part 1. 
   gsipath=/data/dxu/gsi_tag_rel-5.0.0

4. Submit a job to run GSI
$  qsub standalone_gsi62_test_using_my_gsi.sh > running_gsi_0605.log 2> running_gsi_0605_err.log &

5. Check the job status
$ qstat 
6. Check GSI output 
$ cd /scratch5/dxu/tmp62/test_gsi62_standalone1.0.2_06052014
$ tf  stdout