Friday, December 26, 2008

Why I have switch to Python?

In 2005, by the time I was working as a "Machine Learning Specialist" at Coradiant, I had to do lot of preprocessing. At that time, I was doing more data-mining then machine learning. The fundamental difference is that machine learning use model to do prediction and data-mining use mostly model to understand links between variables (no prediction is required). In order to do my work, I chose Matlab because I love their visualization functionality and it contains a machine learning package that includes a Decision Tree module that was perfectly suits my need to do root causes analysis. Unfortunately, Matlab is a disaster to do preprocessing and this task was representing the biggest part of my work on that project and C++ isn't better. Doug Eck, a Music and Machine Research consultant, introduced me to Python to help me doing the preprocesssing work. What a discovery! A year later, I released the first alpha version of mlboost.
Later, I worked on a real machine learning task, building a bot detector that give a bot probability base on sessions information. Without python and mlboost, our real-time bot detector prototype would had never be a reality within a so tight schedule.

Later, with the discovery of numpy, scipy, matplotlib (with pylab interface, you get the same api as matlab), python-mysql and ipython, I don't need anymore matlab. With pydev eclipse plugging, I have a powerful python development environment.

So basically, I have switch to python because:
  • I can do extreme prototyping at a speed that I have never seen
  • I have an amazing community (matplotlib, pydev, scipy etc.)
  • The language amazing syntax allows me to write code with minimal number of lines in an elegantly and readable way (ex: list comprehension, *argv, **args, mocking etc.)
  • The simplicity I can do wrapper with non python libraries (ex: swig)

Here is interesting links:
Matematica vs Matlab vs Python (same fct comparison code)
How to Install Python as a Replacement to Matlab
pythonxy package Python(x,y)
I like this figure to represents to community over python.

No comments:

Post a Comment