Monday, June 23, 2014

Build python virtual environment.

You don't need root previledge at all. This tutorial will help you understand how to install software on linux without root.

What is virtualen: 

This article is very helpful.

Install virtualenv +pip : 

follow this stackoverflow post, answer by Walker-Hale IV.
after you install the virtualenv, your prompt should look like:
and you "which python" should be the one in your virtualenv, because it made a copy of it.

NOTE, don't move the folder of the virtualenv ( py-en0 ) around, some setting are using ABSOLUTE path.


Install openblas: 

NUMPY and SCIPY are optimized using blas, lapack package, you can install openblas to get everything, and openblas has many other favorable features. 

I need to install a local library of openblas, so first thing is to make a .local directory:
"
cd
mkdir .local
"
then in any folder, download openblas from github ( to do this you need to have git)
"
git clone git://github.com/xianyi/OpenBLAS
"
then cd into openblas folder: 
make FC=gfortran
sudo make PREFIX=$HOME/.local install 
after this your openblas library is install in $HOME/.local/lib

install numpy: 

From this point on, I follow this post:

for the site.cfg: 
[default] 
library_dirs=$HOME/.local/openblas-base

[atlas]
atlas_libs=openblas
"
and you can aslo install scipy following it. 

To test, you can do: 
"
python -c "import numpy.core._dotblas"
"
if the import is successful, your numpy is using the right openblas .




Tuesday, June 17, 2014

Set up ubuntu on VAIO pro 13.

old macbook has some issues, install Ubuntu 14.04 to VAIO pro 13.

This blog is the first post from my VAIO.

these are the posts the helped so much:
first make a bootable usb of ubuntu installation, following ubuntu online tutorial.
http://www.ubuntu.com/download/desktop/create-a-usb-stick-on-mac-osx

follow this post to backup and for overall installation. 
http://steffankarger.nl/2013/12/10/ubuntu-13-10-on-the-sony-vaio-pro-13/
NOTE:
  1. in 'trying ubuntu', use disks tool to erase everything on the disk. make they all 'free space'. 
  2. 'sudo parted -l' will also detect your usb as /dev/sdb, this might cause some problem about MBR or GPT, but you can just ignore them, because they are unrelated to your ubuntu system. 

after everything in the previous post, the booting is still problematic, sometimes it 'gave up waiting the root device' and lead me to initramsf, where I have to try 'reboot' many times until a successful reboot.

So, don't reboot after the previous post, there are more work to do. If you rebooted and not able to boot into ubuntu, you can use your usb again and 'try ubuntu'

then you follow this post:
http://askubuntu.com/questions/88384/how-can-i-repair-grub-how-to-get-ubuntu-back-after-installing-windows
 I actually didn't use step 7


Now, the booting takes seconds fast to finish, I am very happy with it.

The touchscreen is such a productivity boost!!!!

Saturday, June 14, 2014

Use Pandas and Decision Tree.

The most powerful decision tree module I know is DecisionTree: DecisionTree . 
Decision Tree needs a specific input format:


The question here is how to use Pandas to prepare such a .csv input. 

Keywords: 
  • DataFrame.one_column.astype( str ). 
  • to_csv( filename, quoting = 2) . 
First you need to load the csv with read_csv
You need to save the csv with quoting = QUOTE_NONMERIC. See the following explanation copied from https://docs.python.org/2/library/csv.html . 

You also need to convert the index from numeric to alphabetic, by : 

df.index = df.index.astype(str) 

thanks. 



Buffer Overflow Example.

This post shows an example of the danger of BUFFER OVERFLOW error for breaking password.
Consider the following C code:

The danger is caused by using gets(). If the user input a string that is longer than 15, you will see the problem:
NOTE: you can compile this code by gcc -o bo bo.c 
and execute it by : ./bo

 
GCC gives a warning, I can ignore it, and I will pay the price.





Here are the execution examples:
What it supposed to do:

      




This is why BUFFEROVERFLOW is dangerous, when the input is large enough that overwrites the variable passed.   





This is what happen when the input is too large that it overwrite outside the stack: 

Wednesday, June 11, 2014

Install vim + python for non-root user.

First dowenlod vim74 from here:
vim74 link

unzip the file use the insturction in that link.

use this git tutorial for configure.
git tutorial: compile vim from source

change --prefix=$HOME
then
make
make install