Why is parallel programming difficult?

Why is parallel programming so hard? At least it’s supposed to be hard — that’s the myth. Of course there’s also a myth (among non-programmers) that all programming is hard, for everyone except 12 year-olds in movies.

One reason parallel programming is hard is we don’t get any practice. Programmers aren’t used to thinking parallel. Introductory programming is linear programming, and most real-world bread-and-butter programming tasks are don’t require a parallel approach. And frankly, it’s easier to understand and express a linear program, and it’s certainly easier to debug.

And besides, parallel hardware has always been exotic, at least up until now. So there hasn’t been much demand for parallelism, and the tools reflect that. Programming languages are (mostly) linear, described as linear text that becomes a linear token stream. Years of parser and compiler research has focused on grammars to translate that linear token stream into linear machine code.

Programs were linear even before text files. Fortran programs used to be stored on stacks of punch cards.

So our tools and habits encourage linear thinking. We introduce unnecessary constraints and dependencies that break parallelism without even thinking about it.

And that’s one reason parallel programming is hard. A big reason.

Comments

103 Responses to “Why is parallel programming difficult?”

  1. reverse phone lookup on August 12th, 2012 5:02 am

    The thing i for example about your blog is that you always post direct to that the point information.
    ,.

  2. reverse cell phone lookup on August 16th, 2012 4:26 pm

    Thank you for this incomparable article. I especially enjoyed reviewing it and ought to discuss it with everyone.

  3. reverse phone lookup on August 23rd, 2012 2:42 pm

    That can happen if your computer clock has the incorrect date and/or
    time. You possibly can have to reset it. Double click on that the clock in your notification area or means
    tray and check that the settings