johnramey

Don't think. Compute.

Wait til the Weekend

without comments

Python for Statistical Computing

MetaOptimize - A Machine Learning Q&A Community (Similar to StackOverflow)

Using bootstrap in cluster analysis

Git Magic – A Git Tutorial as a Video Game Analogy

Bayesian Reasoning and Machine Learning – Textbook with free online textbook (in beta) by David Barber.

Bioconductor Experiment Data Packages – A list of packages with experiment data (a lot of microarray)

Bioconductor One-day Overview Course – From Harvard Biostatistics Department (PDF)

Clustering and Visualization of Microarray Data – This is the best presentation I have seen of the topics, including clustering evaluation (PDF)

Statistical Microarray Data Analysis – This excellent presentation from the same guy includes the one above and discusses a much broader scope. (PDF)

Han-Ming Wu’s Site – This is the professor that released the above two presentations. He has more information on his site. (Only some English)

Very slick poster with ggplot2 graphics – Note the github project at the bottom.

Concentrations of Measure – This is Prof. Tyrone Vincent’s great presentation on probability inequalities from PASI

Machine Learning Video Lectures and Notes – Professor Tom Mitchell at Carnegie Mellon

Bayesian Statistics – Scholarpedia Entry (Recommended by Prof. Andrew Gelman)

A Large Collection of Notes on Stochastic Processes

Written by ramhiser

December 7th, 2011 at 5:52 pm

Steve Jobs’ 2005 Stanford Commencement Address

without comments

Given that there are almost 13 million views of Steve Jobs’ commencement address, I am certain that I missed this video when it went viral. I am glad that I did not see it until now because I may not have appreciated his words of wisdom. And although there are numerous quotes that I could list, I think my favorites are closely related to a few words of wisdom that my grandmother told me when I was younger.

Don’t settle.

Stay hungry. Stay foolish.

You’ve got to find what you love, and that is as true for work as is for your lovers. Your work is gonna fill a large part of your life, and the only way to truly be satisfied is to do what you believe is great work, and the only way to do great work is to love what you do.

It’s good to be reminded of things like this once in a while.

Written by ramhiser

December 4th, 2011 at 1:57 pm

Pseudo-Random vs. Random Numbers in R

with 2 comments

Happy Thanksgiving, everyone. Earlier today, I found an interesting post from Bo Allen on pseudo-random vs random numbers, where the author uses a simple bitmap (heat map) to show that the rand function in PHP has a systematic pattern and compares these to truly random numbers obtained from random.org. The post’s results suggest that pseudo-randomness in PHP is faulty and, in general, should not be underestimated in practice. Of course, the findings should not be too surprising, as there is a large body of literature on the subtleties, philosophies, and implications of the pseudo aspect of the most common approaches to random number generation. However, it is silly that PHP‘s random number generator (RNG) displays such an obvious pattern nowadays because there are several decent, well-studied pseudo-RNG algorithms available as well as numerous tests for randomness.  For a good introduction to RNG, I recommend John D. Cook’s discussion on testing a random number generator.

Now, I would never use PHP for any (serious) statistical analysis, partly due to my fondness for R, nor do I doubt the practicality of the RNG in R. But I was curious to see what would happen. So, created equivalent plots in R to see if a rand equivalent would exhibit a systematic pattern like in PHP, even if less severe. Also, for comparison, I chose to use the random package, from Dirk Eddelbuettel, to draw truly random numbers from random.org. Until today, I had only heard of the random package but had never used it.

I have provided the function rand_bit_matrix, which requires the number of rows and columns to display in the plotted bitmap. To create the bitmaps, I used the pixmap package rather than the much-loved ggplot2 package, simply because of how easy it was for me to create the plots. (If you are concerned that I have lost the faith, please note that I am aware of the awesomeness of ggplot2 and its ability to create heat maps.)

It is important to note that there were two challenges that I encountered when using drawing truly random numbers.

  1. Only 10,000 numbers can be drawn at once from random.org. (This is denoted as max_n_random.org in the function below.)
  2. There is a daily limit to the number of times the random.org service will provide numbers.

To overcome the first challenge, I split the total number of bits into separate calls, if necessary. This approach, however, increases our number of requests, and after too many requests, you will see the error: random.org suggests to wait until tomorrow. Currently, I do not know the exact number of allowed requests or if the amount of requested random numbers is a factor, but looking back, I would guess about 20ish large requests is too much.

Below, I have plotted 500 x 500 bitmaps based on the random bits from both of R and random.org. As far as I can tell, no apparent patterns are visible in either plot, but from the graphics alone, our conclusions are limited to ruling out obvious systematic patterns, which were exhibited from the PHP code. I am unsure if the PHP folks formally tested their RNG algorithms for randomness, but even if they did, the code in both R and PHP is straightforward and provides a quick eyeball test. Armed with similar plots alone, the PHP devs could have sought for better RNG algorithms — perhaps, borrowed those from R.

Click the images for a larger view.

 

 

Finally, I have provided my R code, which you may tinker with, copy, manipulate, utilize, etc. without legal action and without fear of death by snoo snoo. Mmmm. Snoo snoo.

 

Written by ramhiser

November 25th, 2011 at 3:08 am

Posted in r

Tagged with , ,

Wait til the Weekend

without comments

My Question on Stack Exchange about Lagrange Multipliers with Inequality Constraints

One-liners which make me love R: Make your data dance (Hans Rosling style) with googleVis

Response Surface Plot Example in R with rgl

Excellent Set of ‘Data Mining’ Notes from Professor Shalizi at Carnegie Mellon

Annotated Computer Vision Bibliography – A HUGE list of links from various disciplines related to pattern recognition, machine learning, facial recognition, etc. Highly recommended for exploration.

A paper from Strimmer Lab that discusses James-Stein  shrinkage estimators related to one of my papers

Fast SVD for Large-Scale Matrices

Spectral Variation, Normal Matrices, and Finsler Geometry - Provides a great discussion on the development of the Hoffman-Wielandt theorem and describes several inequalities related to the Frobenius norm of the difference of two matrices

A Note on the Hoffman-Wielandt Theorem for Generalized Eigenvalue Problem - An interesting development of diagonalizable pairs of Hermitian matrices.

Seminar Materials for Bayesian Reinforcement Learning

 The Shame of College Sports – An article that has been highly recommended to me about corruption in college sports

UCSDs Computational Mass Spec Blog – I like how they compile papers and comment on them in blog form with various details about each. I am tempted to adopt their method.

Written by ramhiser

September 16th, 2011 at 7:08 pm

Posted in Wait til the Weekend

Tagged with

Wait til the Weekend

without comments

Written by ramhiser

June 25th, 2011 at 3:06 pm

Posted in Wait til the Weekend

Tagged with

Listing of Statistics and Machine Learning Conferences

without comments

Occasionally, I will query Google with “statistics conferences”, “machine learning conferences” or “pattern recognition conferences” and the like. But often, it is difficult to obtain anything meaningful other than the conferences of which I’m already aware (such as JSM, ICML, some IEEE conferences). Today, I found WikiCFP, which is a “A Wiki for Calls For Papers.” This seems to be what I needed. In particular, the following are very useful to me:

It seems limited for statistics though, as JSM is not even listed.

Written by ramhiser

June 12th, 2011 at 5:20 pm

Conway’s Game of Life in R with ggplot2 and animation

with 3 comments

In undergrad I had a computer science professor that piqued my interest in applied mathematics, beginning with Conway’s Game of Life. At first, the Game of Life (not the board game) appears to be quite simple — perhaps, too simple — but it has been widely explored and is useful for modeling systems over time. It has been forever since I wrote my first version of this in C++, and I happily report that there will be no nonsense here.

The basic idea is to start with a grid of cells, where each cell is either a zero (dead) or a one (alive). We are interested in watching the population behavior over time to see if the population dies off, has some sort of equilibrium, etc. John Conway studied many possible ways to examine population behaviors and ultimately decided on the following rules, which we apply to each cell for the current tick (or generation).

  1. Any live cell with fewer than two live neighbours dies, as if caused by under-population.
  2. Any live cell with two or three live neighbours lives on to the next generation.
  3. Any live cell with more than three live neighbours dies, as if by overcrowding.
  4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction

Although there are other versions of this in R, I decided to give it a shot myself. I am not going to provide a walkthrough of the code as I may normally do, but the code should be simple enough to understand for one proficient in R. It may have been unnecessary to implement this with the foreach package, but I wanted to get some more familiarity with foreach, so I did.

The set of grids is stored as a list, where each element is a matrix of zeros and ones. Each matrix is then converted to an image with ggplot2, and the sequence of images is exported as a GIF with the animation package.

Let me know if you improve on my code any. I’m always interested in learning how to do things better.

library('foreach')
library('ggplot2')
library('animation')
 
# Determines how many neighboring cells around the (j,k)th cell have living organisms.
# The conditionals are used to check if we are at a boundary of the grid.
how_many_neighbors <- function(grid, j, k) {
  size <- nrow(grid)
  count <- 0
  if(j > 1) {
    count <- count + grid[j-1, k]
    if (k > 1) count <- count + grid[j-1, k-1]
    if (k < size) count <- count + grid[j-1, k+1]
  }
  if(j < size) {
    count <- count + grid[j+1,k]
    if (k > 1) count <- count + grid[j+1, k-1]
    if (k < size) count <- count + grid[j+1, k+1]
  }
  if(k > 1) count <- count + grid[j, k-1]
  if(k < size) count <- count + grid[j, k+1]
  count
}
 
# Creates a list of matrices, each of which is an iteration of the Game of Life.
# Arguments
# size: the edge length of the square
# prob: a vector (of length 2) that gives probability of death and life respectively for initial config
# returns a list of grids (matrices)
game_of_life <- function(size = 10, num_reps = 50, prob = c(0.5, 0.5)) {
  grid <- list()
  grid[[1]] <- replicate(size, sample(c(0,1), size, replace = TRUE, prob = prob))
  dev_null <- foreach(i = seq_len(num_reps) + 1) %do% {
    grid[[i]] <- grid[[i-1]]
    foreach(j = seq_len(size)) %:%
      foreach(k = seq_len(size)) %do% {
 
        # Apply game rules.
        num_neighbors <- how_many_neighbors(grid[[i]], j, k)
        alive <- grid[[i]][j,k] == 1
        if(alive && num_neighbors <= 1) grid[[i]][j,k] <- 0
        if(alive && num_neighbors >= 4) grid[[i]][j,k] <- 0
        if(!alive && num_neighbors == 3) grid[[i]][j,k] <- 1
      }
  }
  grid
}
 
# Converts the current grid (matrix) to a ggplot2 image
grid_to_ggplot <- function(grid) {
  # Permutes the matrix so that melt labels this correctly.
  grid <- grid[seq.int(nrow(grid), 1), ]
  grid <- melt(grid)
  grid$value <- factor(ifelse(grid$value, "Alive", "Dead"))
  p <- ggplot(grid, aes(x=X1, y=X2, z = value, color = value))
  p <- p + geom_tile(aes(fill = value))
  p  + scale_fill_manual(values = c("Dead" = "white", "Alive" = "black"))
}

As an example, I have created a 20-by-20 grid with a 10% chance that its initial values will be alive. The simulation has 250 iterations. You may add more, but this takes long enough already.

game_grids <- game_of_life(size = 20, num_reps = 250, prob = c(0.1, 0.9))
grid_ggplot <- lapply(game_grids, grid_to_ggplot)
saveGIF(lapply(grid_ggplot, print), clean = TRUE)

Written by ramhiser

June 5th, 2011 at 6:04 pm

Posted in r

Tagged with , , , ,

Wait til the Weekend

without comments

Written by ramhiser

June 3rd, 2011 at 12:01 pm

Posted in Wait til the Weekend

Tagged with

Wait til the Weekend

without comments

Written by ramhiser

May 25th, 2011 at 2:50 pm

Posted in Wait til the Weekend

Tagged with

Getting Started with Some Baseball Data

without comments

With all of the discussions (hype?) regarding applied statistics, machine learning, and data science, I have been looking for a go-to source of work-unrelated data. I loved baseball as a kid. I love baseball now. I love baseball stats. Why not do a grown-up version of what I used to do when I spent hours staring at and memorizing baseball stats on the back of a few pieces of cardboard on which I spent my allowance?

To get started, I purchased a copy of Baseball Hacks. The author suggests the usage of MySQL, so I will oblige. First, I downloaded some baseball data in MySQL format on my web server (Ubuntu 10.04) and decompressed it; when I downloaded the data, it was timestamped as 28 March 2011, so double-check if there is an updated version.

mkdir baseball
cd baseball
wget http://www.baseball-databank.org/files/BDB-sql-2011-03-28.sql.zip
unzip BDB-sql-2011-03-28.sql.zip

Next, in MySQL I created a user named “baseball”, a database entitled “bbdatabank” and granted all privileges on this database to the user “baseball.” To do this, first open MySQL as root (mysql -u root -p)

CREATE USER 'baseball'@'localhost' IDENTIFIED BY 'YourPassword';
CREATE databas bbdatabank;
GRANT ALL PRIVILEGES ON `bbdatabank`.* TO 'baseball'@'localhost';
FLUSH PRIVILEGES;
quit

Note the tick marks (`) around bbdatabank when privileges are granted. Also, notice the deliberate misspelling when I constructed the db. WordPress freaks out on me because mod_security steps in and says, “Umm, no.” For more info about this, go here and here (see the comments as well).

Finally, we read the data into the database we just created by:

mysql -u baseball -p -s bbdatabank < BDB-sql-2011-03-28.sql

That’s it! Most of this code has been adapted from the Baseball Hacks book, although I’ve tweaked a couple of things. As I progress through the book, I will continue to add interesting finds and code as posts. Eventually, I will move away from the book’s code as it focuses too much on the “Intro to Data Exploration” reader with constant mentions of MS Access/Excel. The author means well though as he urges the reader to use *nix/Mac OS X.

Written by ramhiser

May 24th, 2011 at 9:34 pm

Posted in statistics

Tagged with , ,