Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

(R) evolutionary changes

When developing software you are dealing with changes. Changes in what your creations have to do; changes in the way you create your software. The process of change sometimes seems to be a goal on itself. We are almost beaten to death by both the enormous amount of new tools and techniques to create software. To some of our customers changing specifications has become a form of survival on itself. All too often the words revolutionary or evolutionary come by to stress the importance of change. In this post I want to take at change from the perspective of my original studies as an evolutionary biologist. I do know biological evolution can be laden topic, not just in the US. I do take evolution as a given part of biological science. Developing software is a science as well; which deals with observation and repeatable experiments. Developing software can even be a form of art but it does not answer any questions on the meaning of life. I’m quite happy discussing the non scientific aspects of evolution with you, but not here on CodeBetter.

On revolutionary changes I can be short. In the days I went to university kicking against anything available was all part of social life. But I don’t think revolution taken too seriously leads to much good. Looking at history shows so many well intended fresh ideas which ended in a situation worse than the one they tried to overthrow. CodeBetter used to be the basis of a revolutionary blogger who considered the state of software development to be in status quo which required an uprising of the coding masses. In his struggle he alienated quite a few people from a cause which was on itself not bad at all.

Let’s now take a look at some aspects of evolution and see what happens when you try to apply them to software development.

At the basis of evolution lies variation. Not all offspring has the same qualities, those whose qualities are best fit in their environment are selected as parents for the next generation. There is also a huge variation in the tools to develop software. Those tools which produce the most working software are most likely to be selected for the next projects. It is a survival of the fittest. Note that this selection happens at time of reproduction. A strong parent which carries a lethal disease which does not expose itself until a later age is not selected against. A software tool which produces a shiny project which was built with a lot of wizardry is likely to be selected for the next project. At a later age of the project, when it has turned out to be un-maintainable, the tool has already spread.

So this selection which happens at the time of reproduction is like taking a snapshot. In the next generation, under circumstances which are different, selection can favor something completely different and push change into an opposite direction. In many an old textbook on evolution you will see progressive evolutionary lines. Step by step the small rabbit-like mammal evolves in a swift racing horse. Far more famous is the line from a chimpanzee to a modern human being. The lines suggest a line of progress. A less biased study of the fossil record and the already mentioned observation of selection as a snapshot do show that evolution is in fact a wide spreading tree. With many branches which end in extinction. When you follow one specific path in this tree, jumping from branch to branch, you can construct a classic progressive evolutionary line; but it is still one of the many paths possible. The same applies to software tools. Does anybody remember languages like Algol-68 or Logo? At the time they looked like the starts of very promising lines. But they have vanished into oblivion and given place to languages with far older ancestors. Ancestors who only served a niche market for quite some time. This zig-zag behavior can also be observed within a line. Notorious are the many, many versions of Delphi. Version 5 is still alive in many a location, but version 6 is almost extinct. After that the .NET environment has lead to an adaptive radiation of Delphi versions. Some are far more successful than others. Yet another analogy is living fossils like Cobol. That language has never changed that much over the years and is still fit enough to survive.

(Image nicked here)

So far we have only observed that there are changes in life forms and in software tools. A very interesting aspect of evolution is the time scale over which these changes happen. Up till recently it was believed that evolutionary changes happen gradually over time. In every generation there is a variation in the offspring and selection will cause the generations to differ from their ancestors. When scientist actually started to track and measure this gradual change they found out that the species studied actually did not change at all over a long period of time and space. Except for small isolated populations which rapidly changed after which these changes spread over a large area. This was formulated in the theory of punctuated equilibria by Stephen Jay Gould and Niles Eldredge. (You may know Gould as the great essay writer who has reached a large audience; his major profession was that of an evolutionary biologist) This pattern of punctuated equilibria can also be observed in software. I remember myself trying to convince the Delphi crowd of the merits of .NET. Not long after that I had to explain what Delphi was to the .NET people. TDD has been the hobby horse of a minority of evangelists, but the way it is now integrated in asp.mvc looks like a rapid main stream acceptance.

So far I have named some parallels in biological and software evolution. To me the major lessons are that there is no such thing as a constant linear progress. Many a path taken can lead to a dead end; there are periods of fast changes and there are periods of standstill. There is also no such thing as a plan or a direction. There is no intelligent design; there is no direction things will go in the future. All we can do is observe what has happened up till this very moment.

But there is one very big difference between biological and software evolution. It is human culture. Biological variation is dictated by the genes. Changing these is something which only happens in Science Fiction. But many more complicated life forms, like human beings, also pass information to next generations using other coding mechanism. Like stories, books or the internet. Any attempt to capture cultural evolution using the theory of biological evolution so far has lead to stories which might look appealing but have a very weak scientific base.

So take these words with some grains of salt and do concentrate on your human skills to learn and teach to keep the process of change under control.

This entry was posted in Chatter. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://petersgekko.codebetter.com pvanooijen

    Latimeria (aka the Coelencath) lives. A living Algol68 compiler with a runtime ecosystem to live in !

    Thanks !

  • http://sourceforge.net/projects/algol68 NevilleDNZ – http://sourceforge.net/projects/algol68

    Algol 68? Logo? Zig Zag!

    Check out: http://rosettacode.org/wiki/Dragon_curve#ALGOL_68

    What happened to ALGOL 68? Basically C was free and already installed on all the best computers.

    PROC dragon = (REAL in step, in length, PROC(REAL)VOID zig, zag)VOID: (
    IF in step <= 0 THEN
    turtle forward(in length)
    ELSE
    REAL step = in step – 1;
    REAL length = in length / sqrt 2;

    zig(45*degrees);
    dragon(step, length, turtle right, turtle left);
    zag(90*degrees);
    dragon(step, length, turtle left, turtle right);
    zig(45*degrees)
    FI
    );

  • http://codebetter.com/blogs/peter.van.ooijen/ pvanooijen

    Jurgen,

    Both Gould and Dawkins are both respected biologist and do agree on a lot. They differ on their agenda. Gould’s “hidden” agenda was that there is no main agenda point. He has always stressed contingency; which is indeed a main point in self organization. And Gould always opposed to ideas being stretched outside of their context. A comment on Wim Kayzer (as a fellow Dutchman you’ll know probably him) once was “You’re so hopeless romantic”. Dawkins likes to stretch ideas. But what works for evolutionary biology does not have to work for cultural progress. And don’t get Gould started on progress.

    This chaotic pattern of contingency imho very well describes the changes in software. To handle this change we better use our human capabilities to control the process of change. Imho with techniques like TDD and separation of concerns we are now developing ways to keep change manageable.
    The word evolutionary, which started my ramblings, is used a lot. It does describes the past, but to look ahead there should be better terms.

  • http://www.noop.nl Jurgen Appelo

    Peter,

    Of course there is more to evolution than just gene selection. I don’t think Dawkins has ever claimed that it gene selection is the only thing that matters. He has just shown, without a doubt, that it is a very important (if not the major) contributor to evolution. I think he has also shown us, quite convincingly, that group selection does not exist. Still, there are other contributors to evolution, of which self-organization is another big one. In fact Dawkins has actively taken part in workshops at the Santa Fe institute, where complexity and self-organization are being studied. And I think I read that Gould also looked favorably upon self-organization.

    BTW, I know of the Gould vs. Dawkins debates. And of those two Gould was more often accused of having a hidden agenda. I also read that Dawkins said that he and Gould agreed on far more things than those few details that they disagreed on.

  • http://codebetter.com/blogs/peter.van.ooijen pvanooijen

    Jurgen,

    Yes that’s the one. I know Dawkins’s work is considered very important by a lot of people. But not by all evolutionary biologist’s. The best review I’ve read was a one-liner : “The selfish chemical bound”. Which should say it all.
    My main problem with the work of Dawkins is that he considers selection to happen on the level of the gene. Which fits perfectly in the line of population genetics. A good model for chicken breeders or Drosophila populations in the lab. But what people with a paleontological (the history of evolution), like Gould, or embryological (the translation between gene and actual characteristics) background have demonstrated is that there is far more to evolution than just selection of favorable genes.
    Besides that I don’t like Dawkins style. It’s to much “British conspiracy theory science”. But that’s personal.

    I don’t know a broad understood term for cultural evolution. Heritage ?

  • http://www.noop.nl Jurgen Appelo

    Peter,

    I don’t understand what you are referring to when you talk about a weak scientific base in relation to Richard Dawkins. Are you talking about The Selfish Gene? It is considered to be one of the most important works of the 20th century in its field. I’ve read it and I can only agree.

    Maybe the term evolution is indeed misleading. I don’t know. Fact is, we adapt ideas and technologies to match our changing environments. For want of a better term, I’ll stick to cultural evolution for the time being.

  • http://codebetter.com/blogs/peter.van.ooijen/ pvanooijen

    Jurgen,

    I’m somewhat skeptical about the idea of cultural evolution. The book of Richard Dawkins, who suggested the term meme, is one of the works I had in mind when talking about a weak scientific base. I do agree, culture plays a very important role in the change of software through time. But when it comes to culture I would prefer not to use the term evolution at all. Saying all that undermines my whole story as yet another misuse of the term evolution :)

    Chuggle
    Biologist were early adapters when it came to IT. They work with large sets of complex data, IT is just a great help. The job market for biologist is not very good, many of them (like me) ended up in doing IT as a full time job.

  • http://www.noop.nl Jurgen Appelo

    Peter,

    Why not just compare software evolution to cultural evolution, instead of biological evolution? I agree that the biological analogy breaks beyond the gene level. But can’t we just compare all evolution of software tools and technologies with the evolution and spreading of cultural ideas (or memes)? After all, any technology is nothing more than a coded version of a collection of ideas.

  • Chuggle

    On a biological theme I believe Alan Kay got his ideas about Object Orientation from looking at how bacterial cells communicated with each other (he originally trained as a microbiologist)

  • http://codebetter.com/blogs/peter.van.ooijen pvanooijen

    Projecting into the future when it comes to economics has proven to be wrong far to often. Just take a look at stock exchanges to get the most extreme examples. Applying existing trends will miss crucial new factors. In the example of the stock exchanges it missed the effect of computer generated biddings. Which went so fast it completely destabilized the market.
    When it comes to computer hardware we do know it will be so powerfull that it will not be that critical on performance of the software. But our imagination just falls totaly short on what forms it will take. The most famous example here is how IBM mistook the PC for a souped up type writer and gave mr. Gates the opportunity to run away with it.

    You cannot control change itself, but you can be prepared to handle it. And thrive on it :)

  • John

    Controlling change and thriving on it are two very different things.. I strive to do the latter..

    And, I think it is possible to project into the future of software development. Why? Software development is greatly shaped by things such as economics and computer hardware. We can identify historical trends in both. Projecting into the future is merely applying the trends out into the future.