Building Software in-House – Why?

I came across a survey by Cadence Design Systems, the electronic design automation (EDA) company, in which they asked 256 members of their on-line community whether they used internally-developed EDA tools in their design workflow.

The answer was that 36% of them did, and the majority (76%) of those planned to continue to do so at the same level or to increase their usage.

That reminded me of a Gartner study in late 2008 which looked at the wider software market and concluded that currently-deployed software breaks down as follows.

Again, they saw no impending decline in the internally-developed software. They did find a shift from proprietary to open source, but that's another topic!

The Gartner study may have a larger sample, but it leaves a considerable amount of confusion as to what constitutes "internally developed" - what do I call it if I build an in-house application but using an open-source MySQL database?

The interesting aspect about the Cadence study is they drill down into the reasons why those 92 or so respondents chose to use internally-developed EDA tools.

The top reasons were:

1. We need capabilities not available commercially (26%)
2. Need tools specific to company/application/process (23%)
3. Less expensive than buying commercial tools (16%)
4. Problems with tools can be fixed quickly (16%)

They found that these tools were being developed by the design teams themselves, and that they were in use all the way across the silicon design workflow.

They were asked if they saw any limitations in using internal tools. The popular answers were:

1. People who designed tool(s) are no longer around (24%)
2. Lack of support (19%)
3. Internal tool development is costly (17%)
4. Lack of integration with commercial tools (16%)
5. Tools are not updated for current design challenges (15%)

I'm going beyond the Cadence study from here on, but I recognize a familiar picture of a design team that has to do its own thing because the tool it needs is not available, or is too expensive. But then it finds itself locked-in to one or two team members (always the busiest ones!) and it can neither afford the time to transfer their skills or swap out the tool even when they can afford it.

It was a nightmare when the company acquired, or was acquired by, another company. Turf wars broke out whether the ‘common tool chain' was "theirs or ours". It wasn't a petty squabble either - hundreds of hours of re-learning and re-drafting of design processes were at stake. I'm sorry to say that not every Executive team fully grasped the significance, as they focused on how the deal was perceived in the stock market, etc.

EDA companies have, to their credit, spotted the problem of tool cost and introduced programs for startup companies that make it possible to defer the cost of tool acquisition. That begins to address the problem for the EDA tools.

For the wider problem of design-critical and business-critical software, the best hope comes from Open Source Software. But it isn't much of a solution if one or two key members of the team are ‘seconded' into a sub-project, even if it is using OSS. Hiring freelance contractors to do it is very expensive, and that just highlights the problem that people take knowledge with them when they leave.

The compromise is to buy in enough of a solution that gets the team started, but at the same time also delivers them the source code so they are not locked-in. Then they have a choice - maintain it themselves or pay for bespoke development as they need it.

As OSS becomes more mature and more modular, it is becoming more feasible to configure a very sophisticated business platform using plug and play components. No two companies are identical, so no two companies need use the exact same components.