Blender for Data Visualization Part 2 – Render Output as Vector Graphics

In my last post, I showed you the basics to use Blender and Python for data visualization. Python is a powerful language which is very suitable for (large) data set processing, we may further explore it later. Blender is one of the most useful tools for 3D stuffs. In this post, I give a simple example of using Blender’s vector rendering function for a better output.

There are several vector render extensions for Blender. One of them I will use here is the “VRM” rendering script, which supports to export Flash’s “SWF” and “SVG” (Scalable Vector Graphics) file formats. Blender 2.49b should contain this extension. You can check that in your Blender’s menu (Render->VRM) or the script folder (“C:\Documents and Settings\Administrator\Application Data\Blender Foundation\Blender\.blender\scripts\py_render_249\vrm.py“). Anyway, I suggest you to download the latest release here:

http://vrm.ao2.it/section/en/files/download.html

Just download the snapshot, unzip, copy the “vrm.py” to Blender’s script folder and replace the old one.

Here is the snippet for “SVG” output, a simple modified version of the source code in my earlier post, so only the different parts are listed below:

...
import sys
sys.path.append( "C:\Documents and Settings\Administrator\Application Data\Blender Foundation\Blender\.blender\scripts\py_render_249" )
import vrm
from vrm import vectorize,ConsoleProgressIndicator,GraphicalProgressIndicator,progress,config
...
if __name__ == '__main__':
	w=World.New('BarWorld')
	w.setHor([1,1,1])
	w.setZen([1,1,1])
	csv = sys.argv[-1]#get the input file name
	if csv.endswith('.csv'):
		sc=Scene.New('BarScene')
		sc.world=w
		sc.makeCurrent()
		center = barchart(sys.argv[-1])
		addcamera(center)
		addlamp()

		#render and save the image

		config.edges['WIDTH'] = 1
		config.edges['COLOR'] = [255,0,0]
		config.edges['SHOW'] = True
		#config.edges['SHOW_HIDDEN'] = False
		#config.edges['STYLE'] = 'MESH'  # MESH or SILHOUETTE
		#config.edges['SHOW_HIDDEN'] = 1
		config.polygons['SHOW'] = False
		config.polygons['SHADING'] = 'FLAT'  # FLAT or TOON
		config.polygons['HSR'] = 'PAINTER'  # PAINTER or NEWELL
		config.polygons['EXPANSION_TRICK'] = True
		config.polygons['TOON_LEVELS'] = 2
		config.output['FORMAT'] = 'SVG'
		#config.output['ANIMATION'] = False
		#config.output['JOIN_OBJECTS'] = True

		vrm.progress = ConsoleProgressIndicator()
		#vrm.progress = GraphicalProgressIndicator()
		vectorize("C:/test.svg")

	else:
		print 'input file has no .csv extension'
else:
	print __name__

It simply imports the vrm functions and replaced the “PNG” image rendering part using the “vectorize” function provided by VRM. There are several lines for configuring the render, such as the edges and fill. You can find what those configure parameters mean by reading the source file “vrm.py” or by testing them directly in Blender through the user interface (Blender->Render->VRM), so I won’t explain them here. After execution, you can find the file “test.svg” created in your “C:/” folder; you can open and edit this file using softwares like Inkscape or GIMP, and convert it into “PS” and “PDF” formats.

To export the result as a SWF file, which can be easily embedded online, you need to install “libming” for Python first. You can find libming on its official site: http://www.libming.org/.
For windows users, precompiled installers can be downloaded here:
http://opensourcepack.blogspot.com/2011/03/pming-python-binding-for-libming.html, just choose the right version (2.5/2.6/2.7) for you Python and install. Here is the snippet for SWF output:

...
		config.edges['WIDTH'] = 2
		config.edges['COLOR'] = [0,0,0]
		config.edges['SHOW'] = False
		#config.edges['SHOW_HIDDEN'] = False
		#config.edges['STYLE'] = 'MESH'  # MESH or SILHOUETTE
		#config.edges['SHOW_HIDDEN']=1
		config.polygons['SHOW'] = True
		config.polygons['SHADING'] = 'TOON'  # FLAT or TOON
		config.polygons['HSR'] = 'PAINTER'  # PAINTER or NEWELL
		config.polygons['EXPANSION_TRICK'] = True
		config.polygons['TOON_LEVELS'] = 2
		config.output['FORMAT'] = 'SWF'
		#config.output['ANIMATION'] = False
		#config.output['JOIN_OBJECTS'] = True

		vrm.progress = ConsoleProgressIndicator()
		#vrm.progress = GraphicalProgressIndicator()
		vectorize("C:/test.swf")
...

You can find the full source code here:

https://bzstat.googlecode.com/svn/trunk/DataVisualization/Blender/Part2_VectorOutput

And the result:

Pretty nice, right?

Some useful links:

http://vrm.ao2.it/section/en/news.html
http://severnclaystudio.wordpress.com/bluebeard/pantograph-download/
http://en.wikibooks.org/wiki/Blender_3D:_Noob_to_Pro/2D,_Toon,_%26_CAD_Rendering
http://www.libming.org/
http://svg2swf.sourceforge.net/
http://www.blender.org/documentation/249PythonDoc/
http://www.scipy.org
Blender 2.49 Scripting (Book)

Posted in Uncategorized | Leave a comment

Central Limit Theorem, Normal Distribution and Entropy

A simple form of The Central Limit Theorem:

For identically independent distributed random vectors Y_{1},Y_{2},\cdots; let \bar{Y}_{n}=\Sigma_{i=1}^{n}Y_{i}/n.

If E||Y_{1}||^{2}\le\infty, \sqrt{n}(\bar{Y}_{n}-EY_{1})\to N(0,CovY_{1}).

When I first learn the central limit theorem, I’m always curious why the sum of a large number of random variables is distributed approximately as a normal/Gaussian distribution. The answer is simple, that’s where normal distribution comes from. Normal distribution is found when we first try to find the limitation of the sum of a large number of random variables. In other words, normal distribution is defined as the limit distribution of a large number of random variables.

Firstly, using the i.i.d. condition, it’s easy to check that E(\bar{Y}_{n}-EY_{1})=0 and Cov(\bar{Y}_{n}-EY_{1})=\frac{CovY_{1}}{n}.

Before we move on, let’s review the basic idea of entropy. The entropy of words s is H(s)=-\Sigma_{\omega\in s}p(\omega)logp(\omega). Entropy is a measure of uncertainty. The less information we have about something, the larger its entropy will be.

Now, ignore the central limit theorem for a while, let’s image what the limit distribution of a large number of random variables can be.

Note that \bar{Y}_{n} is a sample mean. The more we “mean”, the more individual characters will be lost. Mean will hide individual characters, make individuals indistinctive, and hence reduce the information we have. So it’s natural that the more we “mean”, the larger its entropy will be.

OK, that’s enough. From the point above, we can expect that the limit distribution has a large entropy. In other words, the normal distribution should have a large entropy.

The fact is that:

Theorem 8.6.5 ([1], P254)

Let the random vector X\in R^{n} have zero mean and covariance K=EXX^{t}. Then h(X)\leq\frac{1}{2}log(2\pi e)^{n}|K|, with equality iff X\sim N(0,K).

Theorem 8.8.6 ([1], P255, Estimation error and differential entropy)

For any random variable X and estimator \hat{X}, E(X-\hat{X})^{2}\geq\frac{1}{2\pi e}e^{2h(X)}, with equality if and only if X is Gaussian and \hat{X} is the mean of X.

Corollary: Given side information Y and estimation \hat{X}(Y), it follows that E(X-\hat{X}(Y))^{2}\geq\frac{1}{2\pi e}e^{2h(X|Y)}.

8a.6 ([2], P532) N_{p} as a Distribution with Maximum Entropy

The multivariate normal distribution N_{p} has the maximum entropy

(H=-\int P(U)logP(U)dv) subject to the condition that mean and covariance are fixed.

For given mean and covariance, the Gaussian distribution is the maximum entropy distribution. “It gives the lowest log likelihood to its members on average. That means Gaussian distribution is the safest assumption when the true distribution is unknown” ([3]). That also explains why many people tend to “abuse” Gaussian assumption so much.

References:

[1] Elements Of Information Theory, Second Edition, Thomas M. Cover, Thomas M. Cover, 2006

[2] Linear Statistical Inference and its Applications, Second Edition, C. Radhakfushna Rao, 2002

[3] A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL), Volker Nannen, 2003

Posted in entropy, Uncategorized | Leave a comment

Blender for Data Visualization

There are many tools and languages for visualizing data, such as R (1
, 2, 3), Processing, Flash (1, 2, 3). And the powerful open source 3D editor Blender is of course one of them. The advantage of using Blender is you can use the high-level scripting language Python and the handy Blender API.

Here is a simple example showing how to draw a bar-chart from a “.csv” file using Blender (Source code in Python, GPL, by Michel Anders, adapted from the “Chapter02/README.txt” of the source code package for the book “Blender 2.49 Scripting“):

__author__    = "Michel Anders (varkenvarken)"
__version__   = "1.0 2009/07/28"
__copyright__ = "copyright 2009,2010 Michel J. Anders."
__url__       = ["author's site, http://www.swineworld.org"]

#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

from Blender import Scene,Mesh,Window,Camera,Lamp,Text3d,World
from csv import DictReader
import sys

from math import pi

def label(text,position,orientation='z'):
	"""
	draw a label
	"""
	txt=Text3d.New('label')
	txt.setText(text)
	ob=Scene.GetCurrent().objects.new(txt)
	ob.setLocation(*position)
	if orientation=='x':
		ob.setEuler(-pi/2.0,0,0)
	elif orientation=='z':
		ob.setEuler(0,0,pi/2.0)
	print 'label %s at %s along %s' %(text,position,orientation)

def bar(value,position,shape='Cube'):
	"""
	draw a 3D bar
	"""
	print 'bar'
	me = Mesh.Primitives.Cube(1.0)
	print 'mesh added'
	sc = Scene.GetCurrent()
	print 'got scene'
	ob = sc.objects.new(me,'Mesh')
	print 'object added'
	ob.setLocation(position[0],position[1]+float(value)/2,position[2])
	print 'location set'
	ob.setSize(0.9,float(value),0.9)
	print '%s-bar height %s at %s' %(shape,value,position)

def barchart(filename):
	"""
	draw the bar-chart and return the center position of the chart
	"""
	csv = open(filename)
	data = DictReader(csv)#standard csv Python module to read the data

	xlabel = data.fieldnames[0]#column headers
	print xlabel
	rows = [d for d in data]#other data
	print rows
	#find the extremes of the data
	maximum = max([float(r[n]) for n in data.fieldnames[1:] for r in rows])
	minimum = min([float(r[n]) for n in data.fieldnames[1:] for r in rows])
	print maximum
	print minimum

	for x,row in enumerate(rows):
	    lastx=x
	    label(row[xlabel],(x,10,0))#draw row label
	    for y,ylabel in enumerate(data.fieldnames[1:]):
	        bar(10.0*(float(row[ylabel])-minimum)/maximum,(x,0,y+1))#draw the bar for each column of row x
	x = lastx+1
	for y,ylabel in enumerate(data.fieldnames[1:]):
	    label(ylabel,(x,0,y+0.5),'x')#draw column label

	#Window.RedrawAll()

	return (lastx/2.0,5.0,0.0)#return the center position of the chart

def removeobjects():
	sc = Scene.GetCurrent()
	for ob in sc.objects:
		sc.objects.unlink(ob)

def addcamera(center):
	sc = Scene.GetCurrent()
	ca = Camera.New('persp','Camera')
	ca.angle=75.0
	ob = sc.objects.new(ca)
	ob.setLocation(center[0],center[1],center[2]+12.0)
	sc.objects.camera=ob

def addlamp(loc=(0.0,0.0,10.0)):
	sc = Scene.GetCurrent()
	la = Lamp.New('Lamp')
	ob = sc.objects.new(la)
	ob.setLocation(*loc)

if __name__ == '__main__':
	w=World.New('BarWorld')
	w.setHor([1,1,1])
	w.setZen([1,1,1])
	csv = sys.argv[-1]#get the input file name
	if csv.endswith('.csv'):
		sc=Scene.New('BarScene')
		sc.world=w
		sc.makeCurrent()
		center = barchart(sys.argv[-1])
		addcamera(center)
		addlamp()
		#render and save the image
		context=sc.getRenderingContext()
		context.setImageType(Scene.Render.PNG)
		context.render()
		context.setRenderPath('')
		context.saveRenderedImage(csv[:-4]+'.png')

	else:
		print 'input file has no .csv extension'
else:
	print __name__

The result:

Download the source code: https://bzstat.googlecode.com/svn/trunk/DataVisualization/Blender/

And read the notes on how to run the script: https://bzstat.googlecode.com/svn/trunk/DataVisualization/Blender/ReadMe.txt

Links:

Blender 2.49 Scripting (Book)

Blender 2.49 Python API: http://www.blender.org/documentation/249PythonDoc/

http://datavisualization.ch

Posted in data visualization, programming | 1 Comment

Using SDL and OpenGL in R

SDL(Simple DirectMedia Layer) is “A cross-platform multimedia library designed to provide fast access to the graphics framebuffer and audio device”. OpenGL is “The Industry Standard for High Performance Graphics”.

With rdyncall package, which provides "a cross-platform framework for dynamic binding of C libraries using a flexible Foreign Function Interface", we can use SDL and OpenGL in R!

Testing Notes:

1. Install rdyncall package from CRAN: http://cran.r-project.org/web/packages/rdyncall/index.html

2. Download required DLLs for the test from here: http://dyncall.org/bindings/rdyncall/rdyncall-demo-dlls.zip

Unzip it and copy the DLL files to the system path location (e.g. C:\windows\system32).

3. Test Code (Copied from “rdyncall/demo/SDL.R”)
In R:

# Package: rdyncall
# File: demo/SDL.R
# Description: 3D Rotating Cube Demo using SDL,OpenGL and GLU. (dynport demo)

dynport(SDL)
dynport(GL)
dynport(GLU)

# Globals.

surface <- NULL

# Init.

init <- function()
{
  err <- SDL_Init(SDL_INIT_VIDEO)
  if (err != 0) error("SDL_Init failed")
  surface <<- SDL_SetVideoMode(512,512,32,SDL_DOUBLEBUF+SDL_OPENGL)
}

# GL Display Lists

makeCubeDisplaylist <- function()
{
  vertices <- as.double(c(
  -1,-1,-1,
   1,-1,-1,
  -1, 1,-1,
   1, 1,-1,
  -1,-1, 1,
   1,-1, 1,
  -1, 1, 1,
   1, 1, 1
  ))

  colors <- as.raw( col2rgb( rainbow(8) ) )

  triangleIndices <- as.integer(c(
    0, 2, 1,
    2, 3, 1,
    1, 3, 7,
    1, 7, 5,
    4, 5, 7,
    4, 7, 6,
    6, 2, 0,
    6, 0, 4,
    2, 7, 3,
    2, 6, 7,
    4, 0, 5,
    0, 1, 5
  ))

  glEnableClientState(GL_VERTEX_ARRAY)
  glVertexPointer(3, GL_DOUBLE, 0, vertices )

  glEnableClientState(GL_COLOR_ARRAY)
  glColorPointer(3, GL_UNSIGNED_BYTE, 0, colors )

  displaylistId <- glGenLists(1)
  glNewList( displaylistId, GL_COMPILE )
  glPushAttrib(GL_ENABLE_BIT)
  glEnable(GL_DEPTH_TEST)
  glDrawElements(GL_TRIANGLES, 36, GL_UNSIGNED_INT, triangleIndices)
  glPopAttrib()
  glEndList()

  glDisableClientState(GL_VERTEX_ARRAY)
  glDisableClientState(GL_COLOR_ARRAY)

  return(displaylistId)
}

# Mainloop.

mainloop <- function()
{
  displaylistId <- makeCubeDisplaylist()
  evt <- new.struct(SDL_Event)
  blink <- 0
  tbase <- SDL_GetTicks()
  quit <- FALSE
  while(!quit)
  {
    tnow <- SDL_GetTicks()
    tdemo <- ( tnow - tbase ) / 1000

    glClearColor(0,0,blink,0)
    glClear(GL_COLOR_BUFFER_BIT+GL_DEPTH_BUFFER_BIT)

    glMatrixMode(GL_PROJECTION)
    glLoadIdentity()
    aspect <- 512/512
    gluPerspective(60, aspect, 3, 1000)

    glMatrixMode(GL_MODELVIEW)
    glLoadIdentity()
    gluLookAt(0,0,5,0,0,0,0,1,0)
    glRotated(sin(tdemo)*60.0, 0, 1, 0);
    glRotated(cos(tdemo)*90.0, 1, 0, 0);

    glCallList(displaylistId)

    glCallList(displaylistId)

    SDL_GL_SwapBuffers()

    SDL_WM_SetCaption(paste("time:", tdemo),NULL)
    blink <- blink + 0.01
    while (blink > 1) blink <- blink - 1
    while( SDL_PollEvent(evt) != 0 )
    {
      if ( evt$type == SDL_QUIT ) quit <- TRUE
      else if (evt$type == SDL_MOUSEBUTTONDOWN )
      {
        button <- evt$button
        cat("button ",button$button," at ",button$x,",",button$y,"\n")
      }
    }
    glerr <- glGetError()
    if (glerr != 0)
    {
      cat("GL Error:", gluErrorString(glerr) )
      quit <- 1
    }
    SDL_Delay(30)
  }
  glDeleteLists(displaylistId, 1)
}

cleanup <- function()
{
  SDL_Quit()
}

run <- function()
{
  init()
  mainloop()
  cleanup()
}

run()

You will see a window showing a rotating cube:

Links:

SDL: http://www.libsdl.org/

OpenGL: http://www.opengl.org/

Dyncall: http://dyncall.org/index.shtml

Rdyncall Package: http://cran.r-project.org/web/packages/rdyncall/index.html

Posted in programming, r | 1 Comment

Spaces and Convergences

Metric spaces: a set where a notion of distance (called a metric) between elements of the set is defined.

Vector spaces or Linear space: a mathematical structure formed by a collection of vectors.

Normed vector spaces: a Vector space with a norm defined. The norm is an abstraction of our usual concept of length.

Any normed vector space is a metric space by defining d(x,y)=||y-x||.

Metric space may not be a vector space.

Inner product spaces: Vector spaces endowed with inner product.

An inner product naturally induces an associated norm, thus an inner product space is also a normed vector space.

A normed vector spaces is also an inner product spaces if and only if its norm satisfies the parallelogram law.

Banach spaces: complete normed vector spaces.

Hilbert spaces: complete inner product spaces.

Weak convergence(in normed vector space)

Suppose X is a normed vector space, X^{*} is the continuous dual of X , and x_{1},x_{2},\cdots is a sequence in X, x_{o}\in X. Then we say that {x_{n}} converges weakly to x_{0} if \lim_{n\to\infty}f(x_{n})=f(x_{0}) for every f\in X^{*}. The notation for this is x_{n}\stackrel{w}{\to}x_{0}.

Strong convergence(in normed vector space)

||x_{n}-x_{0}||\to0 (n\to\infty)

The notation for this is x_{n}\to x_{0} or x_{n}\stackrel{s}{\to}x_{0}. We also call it convergence in norm.

Weak* convergence

Suppose X is a normed vector space, \{f_{n}\}\subset X^{*}, f_{0}\subset X^{*}(n=1,2,\cdots).

It is the same as strong convergence for a series of operators.

For any x\in X, f_{n}(x)\to f_{0}(x). The notation for this is f_{n}\stackrel{w*}{\to}f_{0}.

Strong convergence

If \{f_{n}\} converse to f_{0} by the norm on X^{*}, or ||f_{n}-f_{0}||\to0(n\to\infty)

It is the same as uniformly convergence for a series of operators.

References:

1. http://en.wikipedia.org

2. Optimization by Vector Space Methods, Luenberger, David G., 1998

Posted in convergence, space | Leave a comment

Understand the Maximum Likelihood Estimation

Let X_{1},X_{2}\cdots, be iid random vector with density or probability function f(x,\theta), where \theta is the unknown parameter and suppose the true value of \theta is \theta_{0}.

The likelihood function: L(x,\theta)=\prod_{i=1}^{n}f(X_{i},\theta)
The logarithm likelihood function: l(x,\theta)=\Sigma_{i=1}^{n}logf(X_{i},\theta)

Note that these two functions are actually joint probability functions of the iid data X_{1},X_{2},\cdots; we call them “likelihood” functions instead of “probability” functions because we now consider them as functions of the unknown parameter \theta.

The maximum likelihood estimation \hat{\theta}_{MLE} is found by maximize the likelihood function, and we will show the idea behind this procedure.

Jessen inequality states that for a concave function g, Eg(X)\leq g(EX) for any random variable X. g(x)=logx is concave, so under \theta_{0}, E_{\theta_{0}}logf(X,\theta)-E_{\theta_{0}}logf(X,\theta_{0})=E_{\theta_{0}}log\frac{f(X,\theta)}{f(X,\theta_{0})}\leq log[E_{\theta_{0}}\frac{f(X,\theta)}{f(X,\theta_{0})}]=log[\int\frac{f(X,\theta)}{f(X,\theta_{0})}f(x,\theta_{0})dx]=log1=0.

That is to say, max_{\theta}E_{\theta_{0}}logf(X,\theta)=E_{\theta_{0}}logf(X,\theta_{0}), since E_{\theta_{0}}logf(X,\theta)\leq E_{\theta_{0}}logf(X,\theta_{0}).

Now a natural criteria for finding \theta_{0} is to find the parameter which maximizes h(\theta)\equiv E_{\theta_{0}}logf(X,\theta).

However, the function we want to maximize h(\theta) is unknown because h(\theta) has something to do with the unknown \theta_{0}.

But note that \theta_{0} only appears in the mean operator E_{\theta_{0}}, we can overcome this problem by using sample mean operator \frac{1}{n}\Sigma instead.

Finally, we reach that we can find a estimator by maximize l(x,\theta)=\Sigma_{i=1}^{n}logf(X_{i},\theta).

There will be a loss when using sample mean to replace the real one, and the quality of this estimator depends on the law of large numbers.

(Update: 2012/Feb/17) We want to maximize logP(X;\theta), or equivalently, letting

(logP(X;\theta))'=\frac{P'(X;\theta)}{P(X;\theta)}\to0.

The above formula can be interpreted as we want

1. P(X;\theta)\to1, or as large as possible,

2. P'(X;\theta)\to0, or as small as possible.

These two points implies that we hope our estimation \hat{\theta} can make the possibility of the observed data X as large as possible, and at the same time, we want some kind of stability of that possibility, i.e., we hope the rate of change in P(X;\theta) – that is P'(X;\theta), as small as possible.

References:
A Course in Large Sample Theory(Lecture notes), Xianyi Wu
A Course in Large Sample Theory, Thomas S. Ferguson

Posted in statistics | Leave a comment

Convergence in Probability and Convergence with Probability 1

Definations:

For a random vector X and a sequence of random vectors X_{1},\cdots X_{n},\cdots defined on a probability space (\Omega,\mathcal{F},P).

1. Convergence in Probability or Convergence of probability measures, is kind of convergence in measure, in concept of the measure theory.
X_{n}\stackrel{P}{\to}X: P\{||X_{n}-X||>\epsilon\}\to0 for any fixed \epsilon>0.

2. Convergence with Probability 1, is kind of pointwise convergence in real analysis, it is also called almost surely convergence in the measure theory.
X_{n}\stackrel{a.s.}{\to}X: if P\{X_{n}\to X\}=1 or P\{X_{n}\nrightarrow X\}=0.

Convergence in Probability is weaker than Convergence with Probability 1, as stated below:

Theorem: X_{n}\stackrel{a.s.}{\to}X\Rightarrow X_{n}\stackrel{P}{\to}X

Proof: \{X_{n}\to X\}={\omega: for each \epsilon>0 there exists an N(\omega)>0 such that ||X_{k}(\omega)-X(\omega)||\leq\epsilon for all k>N(\omega)}

={\cap_{\epsilon>0}{\omega: there exists an N(\omega)>0 such that ||X_{k}(\omega)-X(\omega)||\leq\epsilon for all k>N(\omega)}}

={\cap_{\epsilon>0}\cup_{n=1}^{\infty}{\omega: ||X_{k}(\omega)-X(\omega)||\leq\epsilon for all k>N(\omega)}}

={\cap_{\epsilon>0}\cup_{n=1}^{\infty}\cap_{k>n}{\omega: ||X_{k}(\omega)-X(\omega)||\leq\epsilon}}

So X_{n}\stackrel{a.s.}{\to}X\Longleftrightarrow P\{X_{n}\to X\}=1

\Longleftrightarrow P\{\cap_{\epsilon>0}\cup_{n=1}^{\infty}\cap_{k>n}\{\omega:||X_{k}(\omega)-X(\omega)||\leq\epsilon\}\}=1

\Longleftrightarrow P\{\cup_{n=1}^{\infty}\cap_{k>n}\{\omega:||X_{k}(\omega)-X(\omega)||\leq\epsilon\}\}=1 for all \epsilon>0

\Longleftrightarrow\lim_{n\to\infty}P\{\cap_{k>n}\{\omega:||X_{k}(\omega)-X(\omega)||\leq\epsilon\}\}=1 for all \epsilon>0

\Longleftrightarrow\lim_{n\to\infty}P\{\cup_{k>n}\{\omega:||X_{k}(\omega)-X(\omega)||>\epsilon\}\}=0 for all \epsilon>0

\Rightarrow\lim_{n\to\infty}P\{||X_{n}-X||>\epsilon\}=0 for all \epsilon>0, or P\{||X_{n}-X||>\epsilon\}\to0 for any fixed \epsilon>0, that is X_{n}\stackrel{P}{\to}X.

if X_{n}\stackrel{a.s.}{\to}X, let the set of unconvergence points as U_{a.s.}=\{\omega:X_{n}(\omega)\nrightarrow X(\omega)\} then P\{U_{a.s.}\}=0.
if X_{n}\stackrel{P}{\to}X, let the set of unconvergence points as U_{P}=\{\omega:X_{n}(\omega)\nrightarrow X(\omega)\} then we donot have P\{U_{P}\}=0.

Convergence in Probability do not require P\{U_{P}\}=0 but release the condition to \lim_{n\to\infty}P\{U_{P}(n)\}=0, where U_{P}(n) is a set which will go smaller as n\to\infty.

From the proof above we can see that U_{P}=\{\cup_{\epsilon}\cap_{n=1}^{\infty}\cup_{k}\{\omega:||X_{k}(\omega)-X(\omega)||>\epsilon\}\} and U_{P}(n)=\{\omega:||X_{n}-X||>\epsilon\}, that is to say U_{P} is bigger than U_{a.s.}

References:
A Course in Large Sample Theory(Lecture notes), Xianyi Wu
A Course in Large Sample Theory, Thomas S. Ferguson

Posted in analysis, convergence, probability | Leave a comment