Tuesday, July 29, 2014

sigma vertical coordinate

Two links below seem to on the same page. 

http://glossary.ametsoc.org/wiki/Sigma_vertical_coordinate

http://www.met.tamu.edu/class/metr452/models/2001/vertres.html

sigma vertical coordinate

A vertical coordinate for atmospheric models defined as pressure normalized by its surface value, or as the difference in pressure and its value at the top of the model atmosphere normalized by the surface value of this difference.
 
Thus, σ = (p - pT)/(pS - pT) 
 
where p is pressure, and the subscripts T and S stand for the top and the ground surface values of the model atmosphere, respectively. 
 
pS :  pressure on the ground surface
pT :  pressure on the top of atmosphere
 
Range: [ 0 , 1 ]

Benefits:
With the sigma coordinate, the lowest coordinate surface follows the model terrain, resulting in significant simplification of the equations compared to pressure or to an unmodified geometric height coordinate.  
 
 
What does "normalized" mean???
normalized : multiply (a series, function, or item of data) by a factor that makes the norm or some associated quantity such as an integral equal to a desired value (usually 1).

bilinear interpolation

In mathematics, bilinear interpolation is an extension of linear interpolation for interpolating functions of two variables (e.g., x and y) on a regular 2D grid.
The key idea is to perform linear interpolation first in one direction, and then again in the other direction. Although each step is linear in the sampled values and in the position, the interpolation as a whole is not linear but rather quadratic in the sample location (details below).

Suppose that we want to find the value of the unknown function f at the point P = (x, y). It is assumed that we know the value of f at the four points Q11 = (x1, y1), Q12 = (x1, y2), Q21 = (x2, y1), and Q22 = (x2, y2).
1) We first do linear interpolation in the x-direction. This yields
 f(R_1) \approx \frac{x_2-x}{x_2-x_1} f(Q_{11}) + \frac{x-x_1}{x_2-x_1} f(Q_{21})
where point R_1 = (x,y_1),

 f(R_2) \approx \frac{x_2-x}{x_2-x_1} f(Q_{12}) + \frac{x-x_1}{x_2-x_1} f(Q_{22})
where point R_2 = (x,y_2).

where
f(Q11) is the function value at point (x1,y1)
f(Q12) is the function value at point (x1,y2)
f(Q21) is the function value at point (x2,y1)
f(Q22) is the function value at point (x2,y2)



2) We proceed by interpolating in the y-direction.
 f(P) \approx \frac{y_2-y}{y_2-y_1} f(R_1) + \frac{y-y_1}{y_2-y_1} f(R_2).
This gives us the desired estimate of f(x, y).
 
\begin{array}{ l l}
f(x,y) \approx & \, \frac{f(Q_{11})}{(x_2-x_1)(y_2-y_1)} (x_2-x)(y_2-y) \, + \\
               & \, \frac{f(Q_{21})}{(x_2-x_1)(y_2-y_1)} (x-x_1)(y_2-y) \, + \\
               & \, \frac{f(Q_{12})}{(x_2-x_1)(y_2-y_1)} (x_2-x)(y-y_1) \, + \\
               & \, \frac{f(Q_{22})}{(x_2-x_1)(y_2-y_1)} (x-x_1)(y-y_1) \\
   \qquad          = & \, \frac{1}{(x_2-x_1)(y_2-y_1)} \Big(   f(Q_{11})(x_2-x)(y_2-y) \, + \\
               & \, \qquad \qquad \qquad \qquad \; \;    f(Q_{21})(x-x_1)(y_2-y) \, + \\
               & \, \qquad \qquad \qquad \qquad \; \;    f(Q_{12})(x_2-x)(y-y_1) \, + \\
               & \, \qquad \qquad \qquad \qquad \; \;    f(Q_{22})(x-x_1)(y-y_1) \quad \Big)
\end{array}
 
 
DXU: This bi-linear interpolation could be extended to N-linear interpolation. 
 

Tuesday, July 22, 2014

compiler

I really feel little dizzy by the variety of compilers, always confuse them here or there. So this is 
is the motivation to put them together,  so I, hope you too, will never have that awful feeling anymore. In the cases of dealing with scientific applications, fortran and c/c++ are the ones we are dealing with the most. 

1. GCC  -- GNU Compiler Collection
GNU Compiler Collection (gcc) : C, C++, Objective-c, Fortran, Java, Ada and Go.
First how to pronounce GNU??? : g'noo, sounds like glue.

gcc is not a compiler that can only compile c++ code, which could be mislead by its name, instead it's an abbreviate name. It can compile lots of languages.

The standard compiler releases since 4.6 include front ends
for
C (gcc),  ( sometimes, cc is an alias for gcc on your system. )
C++ (g++),
Objective-C,
Objective-C++,
Fortran (gfortran),
Java (gcj),
Ada (GNAT), and
Go (gccgo)

2. Intel compiler: 
icc : C
icpc : C++
ifort : Fortran
idb : debugger  ( another debugger besides gdb I use. )

3. IBM AIX compiler:

Introducing XL Fortran

IBM® XL Fortran for AIX®, V12.1 is an advanced, high-performance compiler that can be used for developing complex, computationally intensive programs, including interlanguage calls with C and C++ programs.

Introducing XL C/C++

IBM® XL C/C++ for AIX®, V10.1 is an advanced, high-performance compiler that can be used for developing complex, computationally intensive programs, including interlanguage calls with Fortran programs.


3.1 fortran code 
To compile a source program, use one of the xlf90, xlf90_r, xlf90_r7, f90, xlf95, xlf95_r, xlf95_r7, f95, xlf2003, xlf2003_r, f2003, xlf, xlf_r, xlf_r7, f77, or fort77 commands, which have the form:
Read syntax diagramSkip visual syntax diagram
                  .----------------------------------.   
                  V                                  |   
>>-+-xlf90-----+----+--------------+----input_file---+---------><
   +-xlf90_r---+    '-cmd_line_opt-'                     
   +-xlf90_r7--+                                         
   +-f90-------+                                         
   +-xlf95-----+                                         
   +-xlf95_r---+                                         
   +-xlf95_r7--+                                         
   +-f95-------+                                         
   +-xlf2003---+                                         
   +-xlf2003_r-+                                         
   +-f2003-----+                                         
   +-xlf-------+                                         
   +-xlf_r-----+                                         
   +-xlf_r7----+                                         
   +-f77-------+                                         
   '-fort77----'  

3.2 c / c++ code 
xlc Invokes the compiler for C source files. This command supports all of the ISO C99 standard features, and most IBM® language extensions. This invocation is recommended for all applications. xlc_r, xlc128_r4, xlc128_r7, xlc128, xlc128_r, xlc128_r4, xlc128_r7
c99 Invokes the compiler for C source files. This command supports all ISO C99 language features, but does not support IBM language extensions. Use this invocation for strict conformance to the C99 standard. c99_r, c99_r4, c99_r7, c99_128, c99_128_r, c99_128_r4, c99_128_r7
c89 Invokes the compiler for C source files. This command supports all ANSI C89 language features, but does not support IBM language extensions. Use this invocation for strict conformance to the C89 standard. c89_r, c89_r4, c89_r7, c89_128, c89_128_r, c89_128_r4, c89_128_r7
cc Invokes the compiler for C source files. This command supports pre-ANSI C, and many common language extensions. You can use this command to compile legacy code that does not conform to standard C. cc_r, cc_r4, cc_r7, cc128, cc128_r, cc128_r4, cc128_r7
gxlc Invokes the compiler for C source files. This command accepts many common GNU C options, maps them to their XL C option equivalents, and then invokes xlc. For more information, refer to Reusing GNU C/C++ compiler options with gxlc and gxlc++.
xlc++, xlC Invokes the compiler for C++ source files. If any of your source files are C++, you must use this invocation to link with the correct runtime libraries. Files with .c suffixes, assuming you have not used the -+ compiler option, are compiled as C language source code. xlc++_r, xlc++_r4, xlc++_r7, xlc++128, xlc++128_r, xlc++128_r4, xlc++128_r7, xlC_r, xlC_r4, xlC_r7, xlC128, xlC128_r, xlC128_r4, xlC128_r7
xlc++core, xlCcore Invokes the compiler as described above for xlc++ and xlC, but links only to the core of the runtime library. Use this invocation if you want to link your application to a runtime library other than that supplied with XL C++. xlc++core_r, xlc++core_r7, xlc++core128, xlc++core128_r, xlc++core128_r7, xlCcore_r, xlCcore_r7, xlC128core, xlC128core_r, xlC128core_r7
gxlc++, gxlC Invokes the compiler for C++ files. This command accepts many common GNU C/C++ options, maps them to their XL C/C++ option equivalents, and then invokes xlc++. For more information, refer to Reusing GNU C/C++ compiler options with gxlc and gxlc++.

4. PGI compiler: 
  • PGFORTRAN™ native OpenMP, OpenACC and auto-parallel Fortran 2003 compiler with CUDA extensions
  • PGCC® OpenMP, OpenACC and auto-parallel ANSI and K&R C99 compiler
  • PGC++® OpenMP, OpenACC and auto-parallel GNU 4.8 g++ compatible C++ compiler with CUDA-x86 extensions
  • PGDBG® OpenMP and MPI parallel graphical debugger
  • PGPROF® OpenMP and MPI parallel graphical performance profiler

tar

       tar <operation> [options]

       Operations:
       [-]A --catenate --concatenate
       [-]c --create
       [-]d --diff --compare
       [-]r --append
       [-]t --list
       [-]u --update
       [-]x --extract --get
       --delete

       Common Options:
       -C, --directory DIR
       -f, --file F
       -j, --bzip2
       -p, --preserve-permissions
       -v, --verbose        # detailed info will be popped out. No need to use it.
       -z, --gzip




# Add three files into a tar file
$ tar -cf   a.tar   file1 file2 file3

# List files in the tar file
$ tar -tf   a.tar  

# ONLY extract two files out of the tar file
$ tar -xf   a.tar   file1 file2 

# Extract out all the files in the tar file
$ tar -xf   a.tar  

Monday, June 30, 2014

how to run setpdy.sh

1. Two vars must be exported first : cycle and PDY
$ export cycle=06

$ export PDY=20130719

2. Run setpdy.sh
$ /usr/local/jcsda/nwprod_gdas_2014/util/ush/setpdy.sh

This is generate a file called "PDY" 

$ lr  PDY
-rwxrwxr-x  1 dxu domain users     328 Jun 30 19:35 PDY
$ cat  PDY
export PDYm7=20130712
export PDYm6=20130713
export PDYm5=20130714
export PDYm4=20130715
export PDYm3=20130716
export PDYm2=20130717
export PDYm1=20130718             # <== 7 days pasted                
export PDY=20130719                  # <== current day
export PDYp1=20130720              # <== 7 days coming
export PDYp2=20130721
export PDYp3=20130722
export PDYp4=20130723
export PDYp5=20130724
export PDYp6=20130725
export PDYp7=20130726

Tuesday, June 24, 2014

chi-square test

chi-square test

A statistical significance test based on frequency of occurrence; it is applicable both to qualitative attributes and quantitative variables.

Among its many uses, the most common are tests of hypothesized probabilities or probability distributions (goodness of fit), statistical dependence or independence (association), and common population (homogeneity). The formula for chi square (χ2) depends upon intended use, but is often expressible as a sum of terms of the type (f - h)2/h where f is an observed frequency and h its hypothetical value.
 
 chi-square = Sum (  (f - h)2/h  ) 
 

f is an observed frequency and 
h its hypothetical value.
 
=================
More detailed explanation: 

Chi-Square Calculator: Online Statistical Table

The chi-square distribution calculator makes it easy to compute cumulative probabilities, based on the chi-square statistic. For help, read the Frequently-Asked Questions or review the Sample Problems.
To learn more about the chi-square distribution, read Stat Trek's tutorial on the chi-square distribution.
  • Enter a value for degrees of freedom.
  • Enter a value for one, and only one, of the remaining text boxes.
  • Click the Calculate button to compute a value for the blank text box.
Degrees of freedom
Chi-square critical value (CV)
Cumulative probability: P(Χ2 < CV)
    


Instructions: To find the answer to a frequently-asked question, simply click on the question. If you don't see the answer you need, read Stat Trek's tutorial on the chi-square distribution or visit the Statistics Glossary.
What are degrees of freedom?
Degrees of freedom can be described as the number of scores that are free to vary. For example, suppose you tossed three dice. The total score adds up to 12. If you rolled a 3 on the first die and a 5 on the second, then you know that the third die must be a 4 (otherwise, the total would not add up to 12). In this example, 2 die are free to vary while the third is not. Therefore, there are 2 degrees of freedom.
In many situations, the degrees of freedom are equal to the number of observations minus one. Thus, if the sample size were 20, there would be 20 observations; the the degrees of freedom would be 20 minus 1 or 19.
What is a chi-square critical value?
The chi-square critical value can be any number between zero and plus infinity. The chi-square calculator computes the probability that a chi-square statistic falls between 0 and the critical value.
Suppose you randomly select a sample of 10 observations from a large population. In this example, the degrees of freedom (DF) would be 9, since DF = n - 1 = 10 - 1 = 9. Suppose you wanted to find the probability that a chi-square statistic falls between 0 and 13. In the chi-square calculator, you would enter 9 for degrees of freedom and 13 for the critical value. Then, after you click the Calculate button, the calculator would show the cumulative probability to be 0.84.
What is a cumulative probability?
A cumulative probability is a sum of probabilities. The chi-square calculator computes a cumulative probability. Specifically, it computes the probability that a chi-square statistic falls between 0 and some critical value (CV).
With respect to notation, the cumulative probability that a chi-square statistic falls between 0 and CV is indicated by P(Χ2 < CV).
What is a chi-square statistic?
A chi-square statistic is a statistic whose values are given by
Χ2 = [ ( n - 1 ) * s2 ] / σ2
where σ is the standard deviation of the population, s is the standard deviation of the sample, and n is the sample size. The distribution of the chi-square statistic has n - 1 degrees of freedom. (For more on the chi-square statistic, see the tutorial on the chi-square distribution.)
What is a probability?
A probability is a number expressing the chances that a specific event will occur. This number can take on any value from 0 to 1. A probability of 0 means that there is zero chance that the event will occur; a probability of 1 means that the event is certain to occur. Numbers between 0 and 1 quantify the uncertainty associated with the event. For example, the probability of a coin flip resulting in Heads (rather than Tails) would be 0.50. Fifty percent of the time, the coin flip would result in Heads; and fifty percent of the time, it would result in Tails.

  1. The Acme Widget Company claims that their widgets last 5 years, with a standard deviation of 1 year. Assume that their claims are true.

    If you test a random sample of 9 Acme widgets, what is the probability that the standard deviation in your sample will be less than 0.95 years?

    Solution:

    We know the following:

    • The population standard deviation is equal to 1.
    • The sample standard deviation is equal to 0.95.
    • The sample size is equal to 9.
    • The degrees of freedom is equal to 8 (because sample size minus one = 9 - 1 = 8).

    Given these data, we compute the chi-square statistic:

    Χ2 = [ ( n - 1 ) * s2 ] / σ2
    Χ2 = [ ( 9 - 1 ) * (0.95)2 ] / (1.0)2 = 7.22

    where σ is the standard deviation of the population, s is the standard deviation of the sample, and n is the sample size.

    Now, using the Chi-Square Distribution Calculator, we can determine the cumulative probability for the chi-square statistic. We enter the degrees of freedom (8) and the chi-square statistic (7.22) into the calculator, and hit the Calculate button. The calculator reports that the cumulative probability is 0.49. Therefore, there is a 49% chance that the sample standard deviation will be no more than 0.95.
  2. Find the chi-square critical value, if the cumulative probability is 0.75 and the sample size is 25.

    Solution:

    We know the following:

    • The cumulative probability is 0.75.
    • The sample size is 25.
    • The degrees of freedom is equal to 24 (because sample size minus one = 25 - 1 = 24).

    Given these data, we compute the chi-square statistic, using the Chi-Square Distribution Calculator. We enter the degrees of freedom (24) and the cumulative probability (0.75) into the calculator, and hit the Calculate button. The calculator reports that the chi-square critical value is 28.2.

    This means that if you select a random sample of 25 observations, there is a 75% chance that the chi-square statistic from that sample will be less than or equal to 28.2.
 
 
 

liquid water path

liquid water path

A measure of the weight of the liquid water droplets in the atmosphere above a unit surface area on the earth, given in units of kg m-2, for example.

The liquid water path may be defined as
ams2001glos-Le22
where 
ρair is the density of the (wet) air,
and the integral is from the bottom to the top of the column.