Monday, March 20, 2006

Python, not reptilian (anymore)

Python, a hacker's tool and nothing more till a couple of years ago has suddenly burst into the limelight. I know hardcore Python fans are gonna flame me for calling Python this, but the language has matured and is usable by normal programmers only in the last couple of years. My definition of a normal programmer is someone who programs (writes code, designs stuff, graffiti and the works) day-to-day but is not addicted to it so much so that sleeping and bathing become a tiresome ritual. Let me leave out the hacker demi-gods who are happy with even LISP from the discussion on language usability for time being. They could take LISP and Perl, bang their heads and make it work, so I guess language usability/elegance might not matter much to them.

So what is so good about Python that makes it usable to the programmer community in general? I guess I would seek to answer that during the course of this post.

1. Python promotes adherence to coding standards, indentation for example. Indentation, IMHO should not be treated as an add-on feature (that is optional) but is something absolutely necessary to produce readable code. Python enforces this by making sure that blocks of code should follow the same level of indentation. For e.g. a "for" loop will work only if all the statements in it have the same level of indentation, though you are allowed to throw in braces for that good 'ol feeling. Braces don't make a difference here to the Python interpreter, the indentation surely does. (Braces signify a dictionary in python, a datatype akin to a hash table in other programming languages)

2. Python allows/encourages easy plugging in of modules. This might not be something unique to Python, but it's still wonderfully implemented. Python modules to get the most common programming tasks done are readily available and are a few key strokes away from your pet project. Python might not boast of something equivalent to CPAN (for PERL), but it’s just that re-usable Python code is all around the place and not centralized like CPAN.

3. Python is cross platform across OSes (so is PERL and C++ and Java). I believe its the best cross platform implementation (albeit with heavy favoritism for Unix like systems). It doesn't need a third party Virtual Machine (VM) to run it (it however needs python to run), nor does it rely heavily on system libraries to make it powerful. PERL is good, but I haven't seen too many PERL programmers on Windows, not yet.

4. Python is interpreted. This might be construed as a disadvantage by many because compilers are supposed to be faster and can dole out better assembly. But Python breaks that notion and is very fast for an interpreted language. Don't believe me? Try it out, it’s only a bit slower than comparable C code and C is the very definition of speed amongst high level languages. -- Python source code is automatically compiled into Python byte code by the CPython interpreter. Compiled code is usually stored in PYC (or PYO) files, and is regenerated when the source is updated, or when otherwise necessary. This bytecode is executed by the interpreter at runtime. There are options to compile a binary instead of bytecode using tools like py2exe.

5. Python is actually compiled into an intermediate form called bytecode which can then be run in a traditional interpreter. The advantage of this bytecode is its platform independence. Bytecode generated on Windows would run without any (major) modifications on UNIX flavors. However, it is to be noted that bytecode is not portable across versions of Python, so bytecode generated by Python ver 2.2 might not be compatible with Python ver 2.5.

6. Python is excellent when you want to try out some small piece of code (for say a proof of concept or an experiment). You don't need to worry about traditional boiler plate stuff here, most of that is taken care already. So you can start hacking immediately and almost all the code that you write would be actual, executable code and very little fluff.

7. To extend a little bit on point 6 (because it’s very important), Python is the best language for prototyping. When you want to hand out a prototype/model to a client for review, Python allows you to do just that, no frills attached. Its famous Rapid Application Development (RAD) paradigm is not restricted to words and you can see the difference when you start coding. I cannot emphasize enough on this point, because it could save man hours of work. A prototype which I develop over a week (without much effort) is easily throwable than something developed by a dozen guys in a month. So I would not stick with code which is not usable just because I took time to write it. What if the client changes the entire requirement/design after the prototype is ready (they are famous for it, aren't they). If I have a cute little Python prototype which does everything that the 'real' thing needs to do and it was developed in a short span of time, I can use it or throw it and not much is lost (except a few broken Python fans' hearts maybe).

8. Python is well integrated with some of the most famous libraries/toolkits out there. Python can integrate with COM, .NET, and CORBA objects. For Java libraries, use Jython, an implementation of Python for the Java Virtual Machine. For .NET, try IronPython, Microsoft's new implementation of Python for .NET. X, Tk, Motif and a gamut of other graphical toolkits are also readily available for use within your Python project, so GUI development is a snap, like never before.

9. Python is extensible. If any portion of your Python module needs speeding up, you have the option of writing it in C/C++ and integrating it with the rest of the stuff using Swig or Boost Python (there might be more, I could think of only Swig and Boost now). A brief note here, Python is good enough or quick enough for most requirements and embedding C code might really not be necessary, but you have the option nevertheless.

10. Python looks good. Isn't that silly, you would say. Why should a programming language 'look' good. I guess code needs to look good and promote readability/reviewability even if its months since you wrote it and not obscure to the point of losing some hair to understand what you wrote a couple of months ago. Most programmers spend more than half of their time reading code written by others. So its of utmost importance that code written by everyone is readable. PERL is notoriously famed for its obscure syntax which allows nearly un-decipherable code (at least for humans) to be written. If your code looks ugly in Python, it most probably is not going to work either. How do you find out if your code is ugly? I guess its more of gut instinct to start with and becomes an art with experience. So is it impossible to write code in Python that looks ugly, lacks clarity and yet works? Well, I guess someone can actually try it out as an exercise but be rest assured that Python will make it at least a dozen times tougher than PERL to write obscure code.

11. Python promotes concise code and takes obscure code out of the picture by providing easy and beautiful ways to write working code for pretty much everything. You don't need to write obscure code just because your application is really complex, you would still have an elegant way out in Python and that elegant way would be easy and obvious for most programmers. C is elegant alright, but the most elegant solutions in C might not be the most obvious.

What else in Python deserves mention, well its a long list. Visit some links I have given below and they should help you get a better idea and even a head-start with the language, if you are not already initiated.

Quotes about Python:

Python success stories:

Getting started:

Do a Google on Python and you should get a wealth of information to aid you in your quest.

Welcome to the world of Python and Happy Hacking!


  1. I think you got a lot of things wrong about python.

    1. {} Braces are not ignored in python. They are the dictionary literal, and you do stuff like a = {'foo':'bar'} with it.

    2. Python has the cheeseshop, setuptools and easy_install, which when everything works fine is a lot better then CPAN, ohyeah, and it's distributed too.

    3. Python needs python to run. Like Java needs jre to run, and C# needs the .Net runtime to run.

    4. Python is not interpreted. Any python is first compiled to bytecode and then executed. The bytecode is cached on disk so unless a module changed it's not compiled again. cpython is about 100-200x slower then C. rpython (a PyPy compiled subset flavor of python) is about 2-5x slower then C.

    5. It's not optional to compile python to bytecode. See remark 4.

    6. Python does scale down or up, it's a good tool that is not tied to a specific usage scenario. You can write small stuff with it or the very largest stuff. It will be a joy all along.

    7. I will never get this attitude of writing a good prototype that does everything the customer needs, and then trow it away. I do understand throwing it away, but the next rewrite will be done in python too... so I really don't get it.

    9. Don't forget ctypes, which lets you write plain C and then dynamically load it into python. Has a few advantages.

    I should mention that:
    - games (like Battlefield, civilization and countless others) use python
    - Python is used for web programming (django, pylons, wsgi, turbogears etc.)
    - Python is used for P2P desktop applications (like Bittorrent)
    - Python is embedded in programms like OpenOffice, Inkscape and Gimp
    - Python is used for a zigillion of tasks in popular Linux distributions, like package managers
    - Python is used to run large-scale distributed computing hardware (at CERN, big particle accelerator) to process real-time data streams of several gigabytes per second (during experiments)
    - Games are written in python, see
    - A lot of software at google runs on python, both the very largest scale applications as well as very small scale stuff.

  2. Anonymous4:21 PM

    Florian : Thanks for the comments. When I said "Braces don't make a difference here", I meant braces are not used for scoping. I am aware that braces are used for dictionaries. You can put a brace around your loops and conditionals on their own line if they add readability to the code, they wont be interpreted by Python.

    CPAN is a centralized one stop shop and comprehensive, am not sure if Cheeseshop is anywhere close.

    Python needs a VM like Java and .NET, I don't see which statement of mine contradicted this.

    I guess I got the bits abt interpretation/compilation all wrong, terribly sorry about that.

  3. The part where you say it doesn't need a 3rd party vm. I wouldn't see python as a third party vm, but then I wouldn't see the jre as a 3rd party vm either, or the .Net runtime.
    The three big vm based popular things around all have a 1st party vm that is required to make them work.

    cpython (the executable) happens to be both, the compiler, vm and the whole development environment of python, maybe that is unique (given that it's only about 10mb in size).
    Last I remember a JDK is some 100mb upwards, and a .Net development environment is gigabytes of stuff.

    py2exe creates a simple executable that invokes execute('import main') or something similar on a C level. It sets an environment variable that lets ./ be on the sys.path (PYTHON_PATH usually).
    in this you've got your main module that gets executed. The rest of your software is stuffed as *.pyc's into this
    In short, py2exe pretends that you have a standalone executable.

    No Cheeseshop isn't anywhere close CPAN. But this is not because we failed to get there, but that we don't want to go there. If you've personally tried to use CPAN you know that as soon as you've some fancy dependencies or some C code strewn in together with the perl, things get "challenging" at best :)

    No instead we have (at least in my opinion) something potentially more interesting. Let me outline why I think that.

    Distutils does a good job of beeing able to take a properly prepared tar.gz of your software, and install it as a package on your system.
    Setuptools improves a little on that, and makes it possible to additionally manage the location of the installation a bit more flexible, and also allows you to define a version requirement which your software wants to have fullfilled before it can start. Setuptools also introduces a wonderful packaged format for package distributions called *.egg, which is a zip that contains everything the package needs to run.
    easy_install uses parts of setuptools to be able to obtain a tar.gz or an egg online, and execute the whole process of compiling and installing. It also has the ability to search (at different locations) for a given package should you not pass in a specific URL or location of the thing to install.

    This means that in an ideal world, things like "easy_install turbogears"

    1. obtains the turbogears package
    2. analyzes it's dependencies
    3. gets all the dependencies and their dependencies
    4. goes from the bottom up trough each package and if need be also compiles any C source automatically using the compiler available (it even abstracts the details of windows specific compilation out)
    5. puts every finished package as a single zip file into your desired site-packages directory.

    And because that can be cumbersome for windows users, you do that usually once on windows, then tell distutils to create a bdist_wininst (which creates a windows installer, button and stuff), and with some fidgeting you can package all other already finished packages into that installer, so you've got a one-file installer having all dependencies for windows users.

    In my book, that far surpasses CPAN, by lightyears :)

    Of course it breaks down every now and then when an author isn't setuptools concious or willing to conform. But that's a social rather then a technical thing.