A vision on future software development environments

Recently I contributed to a discussion (on a restricted discussion group) about the (lack of) user-friendliness and user-required functionality of a discussion forum website. It is the website of a tool vendor that sells software development tools. Apparently, this company is more interested in selling software tools than in applying them to give their user community a good user experience with them. You probably guess which company I am talking about, or at least consider two candidates.

One of the things I was wondering was that if they are not willing to invest in providing a strong, flexible and properly equiped website facilities, but the users do want a strong, flexible and properly equiped website facilities, why not let the users themselves develop it. This would perfectly fit in the open-source approach that this company (claims to) support.
Even more, if this company would provide all their development tools (requirement tool, modelling tool, design tool, configuration management tool, defect management tool, test tools, deployment tools, project management tools, etc.), in an integrated software development environment, fully operational with licenses and databases, then they only have to find trusted users who are willing to invest their free time in it (or their bosses time).

This is one side of the story that is about to come. The other side is that there are a lot of companies in need of good (software) developers and there are a lot of companies that hire good software developers on contract basis to those companies. The problem is that with the globalisation (of which outsourcing to India and China is an exponent) the "good" people are available at the wrong place. So either they have to move or travel, or they have to be able to work remote (or by multisite replication and data interchange). This brings an extra burden on the software development environment (and the management) and since it is not a core business process the senior management is unlikely to invest a lot of time, effort and money on it.

But... if we look at the open-source development community, they have a different approach which I envision as the future for (commercial) software development too. For open-source development, the "foundation" either provides a downloadable software development environment or a website (or conglomerate of websites) that provide a remote software development environment. This way, the foundation provides the means for software development and brings it to the people working in software development, rather than bringing the people to the software development environment. This way, the development community can be (and usually) is distributed globally, yet forming a single development group.

Given that it is becoming more and more difficult to setup, maintain and evolve a mature software development environment, with state-of-the-art development methodologies, state-of-the-art tools, perfect integration and interaction between the various parts of the facilities, streamlined information exchange between different data bases, I expect that the future of software development lies in software development hosting companies. These companies are similar to email hosting, website hosting, blogger hosting, search hosting, portal hostings, etc.

• SDE hosting companies provide a complete software development environment, consisting of tools and database for business modelling, requirement management, architecture and design modelling, coding, testing, deployment, configuration management, project management, continous integration/builds, etc. These tools are licensed, integrated and able to exchange and share information and upgrade with the pace of the tool vendors.
  
• SDE hosting companies provide a (probably limited) set of development paradigms, from RUP to Prince2, Agile, SCRUM, XP, Waterfall, Spiral, Incremental and other "standardized" methodologies. They watch the trends and upgrade with the pace of knowledge/competence development in those areas.
• SDE hosting companies work for a (probably limited) set of application domains or market, such as web-development, financial products, health, security, automotive/aeronautics, etc. so they can tailor the methodologies and tools towards it. They will probably also specialise in small, medium, large or extremely large (and complex) products.
• SDE hosting companies provide (restricted and secure) access to the software development environment from all over the world, through web-clients, VPN or other secure means, possibly with multisite replication behind the scenes to improve local performance. This way, multinational companies, teleworkers and travellers and open-source works can make use of it.
  

In particular technology companies often require quite fundamental hardware development (new chips, dedicated equipment, etc.), mechanical, optical and physics developments (miniturising in semiconductors, radiology science in health and treatment, encryption and recognition in security). So the SDE hosting companies must also provide hooks to (automatically) transfer, install and test on in-house equipment outside the SDE host environment and feedback information to the SDE host environment.

Client customers, i.e. companies that actually do the software development to make their products, will hire the SDE from a SDE hosting company and will be relieved of setting up all the different tools, databases, exchanges between database, etc. and don't have to "invent" and maintain the methodology based on "some guy who has read the book about it" or a consultant who is more interested in making managers happy, avoid company politics and get his money than in achieve the best solution and overcoming the resistence and politics do achieve that (and risk his reputation and the reputation of the company he is representing).

So the SDE hosting companies will only provide the means in terms of tools, databases and processes, but do not do the actual software development with it. Because they are specialised (or will become more and more specialised) in tools, databases and processes, and since they service various different customers, they can pay explicit attention to optimizing the software development environment by using specially configured servers, performance tuning, load-sharing facilities, fault-tolerance and fall-back, etc. while spreading the costs over a multitude of customers. The customers will save the overhead of "all those servers doing almost nothing for 70% of the time and being overloaded for 30% of the time", outage due to maintenance, upgrading, troubleshooting, etc.

Using this approach, we will separate how to develop software from what software to develop. For decades, many large and complex companies have been so arrogant to say that they are "different" because the software they make is so much different, and thus refusing to comply with the "same" way of software development. For decades, the tool vendors have been so arrogant to say that their tools and the methodologies that lay behind them are the solution to make software development more productive, more reliable and predictable, cheaper and faster, without actually being able to help when even with the tools the customer is not able to get it out properly ("You are using it in an inefficient way, because of ... your architecture ... your processes ... your company culture ..."). I think the time has come to accept that how to develop and what to develop are two different fields of expertise.