https://orcid.org/0000-0001-6895-9312
Figures
Abstract
Hardware-based startups risk having longer times-to-market, deterring investment in the clean technologies that are critical to a sustainable future. We interviewed 55 leaders at hardware startups, 20 of which are cleantech, mapped their development timelines, and found prototyping to be the longest development step (median of 19 weeks per prototype) regardless of prototype complexity or iteration. Qualitative interview analysis reveals the prototyping team’s choice of development style is a major factor affecting timeline. We define two development styles: natural and structured, typified by free-form exploration and rule-based execution, respectively. On average, natural development takes 35% less time than structured, and is thus preferred for early iterations, but adopting structure at strategic points is needed for timely commercialization. Critical points of transition to a structured style include adding new team members or engaging external partners, which demand clear communication and expectations. When pivoting to a new product or market, returning to a natural style is beneficial.
Author summary
Hardware-based startups risk having longer times-to-market, deterring investment in the clean technologies that are critical to a sustainable future. We interviewed 55 leaders at hardware startups, 20 of which are cleantech, mapped their development timelines, and found prototyping to be the longest development step regardless of prototype complexity or iteration. Interview analysis reveals the prototyping team’s choice of development style is a major factor affecting timeline. We define two development styles: natural and structured. On average, natural development takes 35% less time than structured, and is thus preferred for early iterations, but adopting structure at strategic points is needed for timely commercialization. Critical points of transition to a structured style include adding new team members or engaging external partners, which demand clear communication and expectations. When pivoting to a new product or market, returning to a natural style is beneficial.
Citation: Looney E, Buscariolli A, Yang MC, Raymond G, Buonassisi T, Peters IM (2024) Strategic styles of hardware product development could accelerate commercialization in cleantech startups. PLOS Sustain Transform 3(3): e0000101. https://doi.org/10.1371/journal.pstr.0000101
Editor: Ana Delicado, Universidade de Lisboa Instituto de Ciencias Sociais, PORTUGAL
Received: March 7, 2022; Accepted: January 31, 2024; Published: March 20, 2024
Copyright: © 2024 Looney et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All interview questions and those answers that are anonymized are available in the manuscript and Supporting Information files. Some data cannot be shared publicly due to privacy of the interview participants.
Funding: TB, EEL acknowledge support from U.S. Department of Energy’s Office of Energy Efficiency and Renewable Energy (EERE) under Solar Energy Technologies Office (SETO) Agreement Number DE-EE0007535. TB EEL acknowledges support from Singapore’s National Research Foundation through the Singapore-MIT Alliance for Research and Technology’s Low-Energy Electronic Systems (LEES) IRG. All authors acknowledge support from the U.S. Department of Energy and National Science Foundation joint Quantum Energy and Sustainable Solar Technologies (QESST) grant. IMP acknowledges support Bavarian State Government (project "PV-Tera – Reliable and cost-efficient photovoltaic power generation on the Terawatt scale", No. 44- 6521a/20/5). EEL acknowledges support by the National Science Foundation Graduate Research Fellowship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Successful commercialization of hardware products is essential to a sustainable future by combating climate change with renewable energy technologies, screening for disease with medical diagnostics and devices, and increasing production efficiency with robotics, among many other critical applications. Unfortunately, hardware product development (PD) is often less attractive to investors due to the barriers of long commercialization timelines and large upfront investment [1,2]. Commercialization timelines are shown in Fig 1, with the years to exit (merger, acquisition, or initial public offering) for cleantech, medical, and software sectors are compared for the years 2006 to 2011 [1].
Number of startups that successfully exit in a merger and acquisition or initial public offering plotted against how many years from incorporation it took to exit. The dotted line indicates the envelope around software with which hardware startups compete. Raw data provided by Benjamin Gaddy from [1].
Software companies with three-year modal average exit times are the most attractive to investors who start receiving a return on investment (ROI) at that time. Medical device startups, often slowed by hardware timelines and medical regulations, have longer exit times than software. Sustainable hardware technologies, called cleantech in this paper, are the slowest to exit and have the fewest total companies overall with a bi-modal distribution and peaks at four and eight years. The slower time-to-market for cleantech is especially problematic due to the urgent need for sustainable solutions today. Furthermore, long timelines lead to less investment overall and therefore less technology development to help with the goal of sustainability long term.
The overarching goal of this work was to find the PD steps that most need acceleration and determine levers that practitioners can use to accelerate PD timelines, hastening time-to-market and attracting more investment for technologies critical to a sustainable future. We focus on hardware product development with an emphasis on sustainable, or clean technologies, which is shortened throughout the work to ‘PD’ for simplicity. Questions we intended to answer include: How long do PD steps take for early-stage hardware startups? What are the parameters determining the lengths of the longest PD steps? What levers are available to accelerate PD? What are some best practices or strategies to overcome limiting parameters in PD?
This work sits in the intersection of engineering and social science, as we aim to understand how to accelerate PD by gathering actionable data from startups using social science methods. Within engineering, there is ample research on innovation and new product development methods and strategies (i.e., the application of the agile product development movement to hardware) [3,4,5,6,7,8]. Within policy, business management, and finance, there is also a large body of social science and technology research that focuses on improving innovation and development outcomes with solutions and frameworks for either policymakers or institutional investors [9,10,11,12,13,14,15,16]. There are few papers in the literature that address early-stage hardware startup product development from the viewpoint of the practitioners themselves [17]. This paper focuses on individual companies and their PD timelines to address practitioners within early-stage hardware startups directly. Through a series of semi-structured interviews with practitioners, company workflows are mapped, bottlenecks identified, and a strategic framework for acceleration in hardware startups developed.
Analyzing 55 interviews with hardware startups, 20 of whom were cleantech, lasting on average over one hour, this work aims for a unique combination of breadth and depth. Identifying the root-causes for long development timelines in hardware startups requires a broader database than typical case studies can offer. Developing a framework for accelerated development demands a deeper analysis than encompassed in the scope of typical short surveys. With the findings of this work, we hope to inform early-stage cleantech hardware startups and facilitate accelerated hardware product development for technologies that can enable a sustainable and prosperous future.
Results
Interview data collection for qualitative and quantitative analysis
We developed and implemented a facilitated survey (interview) combining engineering domain expertise and sociological research methods, both quantitative and qualitative. The interview included sections encompassing product planning, requirements, modeling, concept generation and selection, complexity, prototyping, manufacturing, and more. The final interview contained approximately 170 questions, lasted between 45 minutes and two hours, and the majority were conducted in person. Over 80% of the interviews (45) were recorded for transcription and an online survey form [18] was filled out by the interviewer in real time.
A geographically diverse set of startup companies participated with 35 (63%) from North America, 7 (13%) from Europe, and 13 (24%) from Asia. All of the companies interviewed create hardware products, and the focus of the interview was on the development timeline leading up to the first production quality product.
Cleantech hardware companies were originally the sole target for this study including startups working on battery, wind, and solar technologies as well as energy efficiency, greenhouse gas sensing, EV charging, and more. However, the set of available companies matching this description that had completed at least one prototype was a prohibitively small sample size. Therefore, the facilitated interview was opened to any early-stage hardware startup creating physical products including scientific equipment, medical devices, quantum computing, consumer electronics, food technologies, microfluidics, and more. The inherent similarities in early-stage hardware product development independent of sector meant that the other hardware startups brought valuable data to the study. They also provide a point of reference to see what differences cleantech startups may have. In the end, almost half of the companies interviewed identified as cleantech with the remaining half including companies from medical, robotic, aerospace, and other sectors.
In this sample, the majority of companies had core intellectual property (IP) in a novel application or service provided. The remaining companies in the sample had IP in novel materials, processes, assemblies, or business models. Furthermore, the competitive strategy of most companies was in the technology itself rather than customer focus or cost leadership. Over 93% of the interviewees (51) were technically trained as engineers or scientists and have an education level beyond a bachelor’s degree. The position of over 90% of the interviewees (50) at the time of the interview was as an executive and/or founder of their company. The remainder of the interviewees were lead engineers or scientists.
In this work, a hybrid of quantitative and qualitative analysis often referred to as mixed methods was used to explore what the bottlenecks in the PD process are and how they can be overcome. The quantitative parts of the survey included workflow mapping and product complexity metrics. An example of a company workflow mapping and its components are shown in Fig 2. We developed a workflow for each startup interviewed and used the timing needed for each prototype as well as development ‘milestones’ such as incorporation, modeling start date, etc. to calculate times for PD tasks seen below in Fig 3. To our knowledge, the mapping of PD timelines and calculation of task timing for over 50 hardware startups has never been done before.
An example of a workflow timeline developed through interviews. Incorporation (orange), prototyping (blue), modeling (red), and operational milestones (green) are key components.
Left: product development process times in years, Right: Number of weeks per prototype for the first five prototypes.
To complement the quantitative data gathered, open-ended questions about each PD step were asked. To analyze these results and find conclusive insights, qualitative research methods from the field of sociology were employed. The main method used in this work is qualitative coding on all transcribed interviews, which is "a way of indexing or categorizing the text in order to establish a framework of thematic ideas about it" [19]. More information about this methodology is provided in the methods section.
Using this mixed methods approach, we first endeavor to answer our initial research question of how long PD steps take for early-stage hardware startups.
Prototyping is the longest PD activity with an average time per prototype of 19 weeks
By aggregating timelines from the interviews in Fig 3, we find the amount of time in years each PD step takes to complete. Total prototyping time is the longest PD step at around 2.5 years for 5 prototypes while the remaining PD activities have a median of less than one year. Here prototypes are defined as preliminary examples of a product used to evaluate design and performance [20]. Gathering user preferences and targeting a market are next in length with medians of over half a year. In the early-stage PD studied herein companies typically create multiple, iterative prototypes before arriving at a final design ready for mass production. There were no significant differences found in the timelines of cleantech vs other hardware startups.
To understand prototyping time from a different angle, we plot the amount of time taken per prototype and find a median time between 17.5 and 20.5 weeks. For the first three prototypes, the time to completion grows slightly with each iteration. However, overall, the amount of time it takes to prototype stays close to consistent for each iteration, suggesting a "characteristic prototyping time" of about 19 weeks per prototype. Taking this characteristic prototyping time and multiplying it by 5 prototypes total results in 1.8 years. This number is below the 2.5 years median obtained from the workflow analysis for the overall prototyping period. The discrepancy is due to downtime between iterations, which was not included in the time for single prototypes. This downtime accounts for an additional six weeks on average per prototype. This time is often spent using that prototype for demonstrations and marketing as well as determining what to do for the next iteration.
To understand how the interviewees perceived the length of each PD step, we asked what PD step slowed them down the most and what they thought could realistically be accelerated. Most of the interviewees believe that prototyping is the slowest PD step, matching reality. Also, they believe uniformly that prototyping could realistically be made faster. There is less confidence by the interviewees in speeding up any PD activities other than prototyping.
Given this outcome, we now focus our next research question on prototyping as the longest PD step: What are the parameters determining the lengths of the longest PD steps?
Product complexity is not correlated with time to prototype
One hypothesis we pursued while designing the interview was that complexity is positively correlated with development timeline, meaning the more complex the product, the longer it takes to create. In literature, product or engineering complexity is usually defined by breaking the concept into key dimensions or metrics, for example, complexity based on size, component count, interfaces, control, assembly, etc. [21]. In this work, we designed a set of questions for interviewees to gather key complexity parameters adopted from literature and supplemented to target our specific hypothesis. The complexity parameters and literature used are described in detail within S1 Text and S1 Fig. In Fig 4, we plot complexity parameters against prototyping times with ordinary least squares (OLS) regression trend lines.
Plotted with ordinary least squares regression trend lines. Green is for cleantech and blue is for other sectors. In the top left is the complexity metric that takes into account several complexity parameters: part count, number of custom parts, percentage outsourced of build and design, and design for outdoors.
The major finding from Fig 4 is that we found no significant correlation between how complex a product is and the time it takes to prototype was found. This is a surprising result as intuition would suggest that a more complex product would take longer to prototype. In Fig 4, the cleantech companies are in green and the other companies are in blue. There are no significant differences in the complexity parameters between these two categories. The only visible trend is possibly even more counterintuitive: prototyping time decreases with the number of functions the product performs. The cause of this trend is unclear. We speculate that it could be due to several things; one is that for products the interviewee conceived of as simpler, there is a tendency to enumerate more non-essential functions, whereas for complex products, the interviewee focused on the one or two main functions. It could be that of feature creep or overengineering occur more frequently in simpler products during development. Overengineering describes the process of designing a product to be overly complex, having more features or greater robustness than is necessary for its functions, which leads to inefficiencies. Feature creep is the expansion of product requirements during prototyping by adding features beyond the original scope of the product [20]. Another reason for this could be that product complexity is managed by modularity, and that the more modular the product has, the less time it takes to prototype [21]. To validate this trend a more rigorous definition of essential functions vs. other functions would be needed. This is an area for future research.
Due to the lack of correlation between complexity and time to prototype within these companies, we continue to search for drivers of prototyping time. To do this, we use qualitative codes to investigate the impact of PD processes on prototyping.
Development styles correlate with time to prototype
One qualitative grounded code that emerged from the analysis deals with the interviewees placing their development process on a spectrum from organic and natural PD to highly structured PD. The section of the interview on defining requirements prompted many responses pertinent to this spectrum. While responses varied between companies, a division between a more natural approach and a more structured approach was apparent.
For example, some companies describe their design decisions mentioning the lack of structure:
Interviewer: “So, did you use traditional engineering requirements?”
C.6 (consumer electronics company): “No. We didn’t do anything traditional. It was pretty organic.”
Interviewer: “Okay. So, would you say official, loosely worded, or no requirements were discussed?”
C.6: “I would say the middle one. Less official, loosely worded. Just like, ‘We kind of need this. Let’s try it.’ We built this hacky solution and fixed it.”
This exchange demonstrates one approach to PD that we propose to define as natural and organic. No formal requirements were used and there was a description of things “just happening” or evolving naturally as the team worked toward designing and building prototypes. These descriptions were common among the startups interviewed, with descriptions such as “going with the flow,” letting the design “evolve,” and undertaking “organic” or “natural” processes.
On the other end of the spectrum were companies that approached PD in a rigorous, structured manner. There were a few categories of these companies: those with an investor or grant that imposed structure, those with experience in industry using structured PD, and those that had recently practiced structured PD in a university setting. For example, one company working in medical device design explained:
Interviewer: “Were traditional engineering requirements used?”
C.5 (consumer electronics company): “It was imposed, right? It’s required because it’s an FDA regulated product. So, there’s no optionality there. If you want to bring something to the market, you have to because the FDA requires it.”
This enforced structure can be beneficial to some companies. For example, another company expressed that the structure mandated by their government grant in the form of milestones was a useful strategy:
Interviewer: “Looking back, would you spend more or less time on requirements creation?”
C.8 (cleantech company): “On the requirements creation. No, I don’t think so. I mean it definitely seems like a necessary part of the process and it’s good to have those [grant] milestones to try and hit. So, I mean, I think it’s a pretty good strategy.”
In the above interview excerpts, the division between natural and structured PD models is evident. Before detailing more analysis on how prototyping times are affected by these two development styles, both natural and structured, we define the two sides of the spectrum.
We define natural innovation as a flexible process in which prototyping iterations evolve following the intuition of participating engineers generally without rigid planning or requirements. Natural innovation allows for free-form ideation, rapid switching between designs, and exploration of a comprehensive design parameter space.
We define structured innovation as an ordered process in which prototyping iterations follow prescribed and codified plans including defined engineering requirements with procedures in place for altering plans. Structured innovation creates order that ensures the design meets functional and cost targets, ensures the product can be manufactured, and facilitates communication about the design and the fabrication process.
Each interviewed company falls somewhere on the spectrum from natural approaches to more structured approaches. Using analysis from the interviews, we developed a method for placing companies on this spectrum. Each time an interviewee mentions in the interview one of the steps on the spectrum in Fig 5, they are assigned one point at that position on the spectrum. We outlined eight steps ranging from “directly mentioning a lack of structure, organic, or natural style” on the natural side, to “following traditional engineering requirements” on the structured side. Once all the transcripts were processed, the overall totals were summed, normalized, and a number between 0 and 1 was calculated placing the company in one of the three categories: natural, hybrid, and structured.
Top: Spectrum from natural to structured process with criteria for categorization of companies along that spectrum. Bottom: Startup companies on the spectrum vs. how many weeks it takes to complete one prototype on average. The orange error bars are 25% of median to 75% of median values of the data.
We find that companies using highly structured processes for prototyping had an average of 31 weeks per prototype including the downtime before the next prototype. Using the average of six weeks downtime, this was 25 weeks per prototype. Companies using natural processes averaged 19 weeks for prototyping time plus downtime, or 13 weeks prototyping time, about half the time of structured processes. Hybrid companies were at 28 weeks prototyping time plus downtime. This result points to natural processes being the better choice if prototyping quickly is the only goal. Several excerpts from the interviews back up this finding, including:
Interviewer: “If you could go back in time and do things differently, would you spend more or less time on requirements? Could you speed up the process?”
C.1 (cleantech company): “I probably would spend a little bit less time arguing about whether it should be this value or that value cause really at this stage, it doesn’t matter. You just have to pick something and go with it. You’re going to learn how to adjust it… One of our early angel investors gave me a piece of advice that is a soundbite motto, but I found it very useful. He basically said, ‘There’s no good decisions and bad decisions. There’s just decisions and then you work to make them good.’ So, it’s basically saying don’t waste time analyzing it to all hell.”
In this quote, the interviewee is pointing out that early in the PD cycle, flexibility and speed are key to iterating quickly. However, we learn from the interviews that structured processes also have a role in efficient PD.
EL: “If you could go back in time and do things differently, would you spend more or less time on requirements? Could you speed up the process?”
C.9 (cleantech company): “Actually, I would spend more time on requirements. In the process, we made a number of different prototypes, which I think looking back is not necessary, if we had proper detailed requirements already done. That could also speed up the process.”
In this excerpt, we see that the company may have lost time by not having requirements defined and understood early enough leading them down some wrong paths. At some point, once the technology, implementation, and market have aligned, structure in the form of rigorous requirements can enhance PD efficiency rather than slow it down.
The crux in implementing a development style for startups is the question of when to use a natural approach versus when to deploy a structured approach. From Fig 5 in conjunction with overall analysis of the interviews, it seems that for the first few prototypes natural PD processes is likely the best approach, as structure can bring paperwork, extra tasks, and careful evaluation which explains the much longer prototyping time. However, at a certain point in the life cycle of a company, structure can actually accelerate PD by avoiding timely and expensive distractions. Therefore, there is a balancing act that startups must perform to get through prototyping efficiently, having a natural process long enough to move quickly and creatively at first, but bringing in structure soon enough to not waste time in the long run. One CEO explained this tension and his compromise between these two extremes in a hybrid approach:
Interviewee: “How important was defining requirements for you?”
C.26 (electronic equipment company): “It didn’t happen like I sat down and I came up with the requirement. While making the thing it evolved… It’s a compromise because you can perfect one design and make sure that it’s going to work in [the] first shot. But the other approach is just to do it fast. Okay, let’s get it, try it, fix it, fix it.”
From this qualitative analysis, we have found that a balance between natural and structured PD is one way to potentially accelerate PD for many hardware companies.
A case study about the pitfalls of deploying the wrong PD style
In this section, we highlight a specific case study of a cleantech company that was efficiently prototyping in a natural regime, but experienced large slowdowns in product development later in the process. For this company, the first five prototypes had a median prototyping time similar to that found for all companies in the study. After the fifth prototype however, there was a total redesign for prototype 6 that fell victim to both overengineering and feature creep, taking a total of 1.5 years to finish. As the company was running out of money and was close to collapse, a new CEO, former investor stepped in to overcome these issues. He explained:
C.44 (cleantech company): “Unbeknownst to the salespeople that were trying to sell [the product] and to the board of directors, [the company] undertook a top to bottom redesign of the product. It took over a year before another unit was in the field.”
Interviewee: “Did it fix the problem?”
C.44: “No. Actually, nothing was better. Everything was worse. It was more complicated, more expensive, more difficult to make, more sources of failure.”
After the new CEO was brought on, there was a year-long reversing of the over-complicated prototype, resulting in an extra 2.5 years of development time that could have been shortened or completely avoided (Fig 6):
Timeline for company that struggled with overengineering and feature creep which slowed the PD down by over 2.5 years.
In this example, the first five prototypes were accomplished relatively quickly beginning with a natural development regime and gaining more structure over time. Then for the sixth prototype the company pivoted to a substantially new, untested design that did not emphasize design simplicity, maintainability, or other key requirements. The team was also by this point using a more structured development style, and the product became overly complicated lacking rapid testing and user feedback. This demonstrates the need for reverting to a natural development style for each major design pivot as the new concept must be rapidly built, tested, and verified in the hands of the customer. If the company needed to pivot, then using a natural development style may have prevented such a lengthy slowdown. And, if the company had not pivoted and was using a structured style for the long-tested design, then the slowdown may also have been avoided. This example shows the perils of using structure on brand new designs before the technology, implementation of that technology, and market type have coalesced into one product goal [22].
This example could also shed light on why product complexity is not correlated with prototyping time. One hypothesis is that there is a tendency toward perfectionism and overengineering with simpler products as compared to more complex products. In the example above, the design pivot made the product overly complicated potentially due to the designers’ perception of the product’s straightforwardness. If a product is known by the team to be complex, there may be a greater prerogative to do “quick and dirty” natural prototyping and decision making due to the constant pressure to get something working. However, for products seen as less complex or “simpler” by the team, there may be a tendency to try to make a prototype perfect or add unnecessary features rather than just getting something working. These competing effects might cause an averaging out of how long it takes to prototype leading to no trends with complexity. Some of the interviewees working on less complex products mentioned without prompting that there was a risk of “feature creep” for their products due to simplicity of core functions. The example above with Company 44 demonstrates the harms of complicating a design with new features especially when working in a structured development regime. This hypothesis and its potential toward revealing why less complex products take the same time to prototype as more complex products is a promising future research direction.
Strategically deploying natural and structured styles to accelerate PD
We see from the interviews that bringing structure in at the right time could potentially accelerate overall PD but could also slow down PD if brought in at the wrong time. Therefore, an important question becomes: how do startups know when to migrate from natural to structured PD and back again? Several best practices were distilled from the interviews for when to deploy different PD styles.
One point in PD at which bringing in structure is advisable is when the team expands beyond the first founding members. For example, one company CEO, who first approached design naturally said:
Interviewer: “Do you use traditional design requirements?”
C.15 (cleantech company): "So we kind of go through the prototypes quickly instead of having a really rigorous computation of all the requirements required… [We] made some decent progress but then when I brought in other people… I noticed that that process just doesn’t work for certain people… And then I realize that I need to bring in the more formalized project management structure. It’s through that that we will write down the different design requirements."
The need for structure in this case involves the need for clear communication and expectations. With new team members who must be brought up to speed and understand company expectations, structure is crucial.
Another point at which structure should be brought in is when communication with partners becomes necessary. Several companies expressed that a lack of documentation and structure made initial interactions with manufacturers or other outsourcing firms inefficient and a time sink. For example, one company explained that an entire prototype was wasted, as their outsourcing partner did not understand what they verbally described and produced something unworkable. After structure was introduced with formal documentation and review, this problem was overcome.
Another example of the need for structure when dealing with outside partners comes from Company 28 in this exchange:
Interviewer: "If you could design or build anything differently, or outsource more or less would you have done something differently?"
C.28 (consumer electronics company): "We would have worked with our contract manufacturer and brought him into the design sooner… Just documentation. It’s an investment that pays off. The more you do the better. And it’s never too early to give the design to your contract manufacturer once the industrial design is done."
Without structure, communication was ineffective between the stakeholders in these examples. Structure is shown to be incredibly important when bringing in outside partners who must understand the intricacies of the product and agree to the conditions of PD.
Discussion
Innovation is complex. Hardware innovation is even more so, because the long timelines between “investment” and “return” mean that more factors can influence outcomes. Various parameters in the innovation process change with time (e.g., customer needs & willingness to pay, competing tech, and more), thus long innovation cycles face difficulties converging to a marketable product. We see a clear need to accelerate hardware-development timelines. We see evidence in our data, that hardware-development timelines can be compressed by adopting natural and structured development styles strategically throughout PD.
What levers are available to accelerate PD? What are some best practices or strategies to overcome limiting parameters in PD?
In this work we have focused on prototyping as the main lever to shortening PD timelines, as it is the longest PD activity and the one most practitioners believe can be sped up. We find that technical complexity of the product does not necessary correlate with prototyping times. Changes in complexity cannot then easily be used as a lever to accelerate development. Therefore, we focus instead on the environment external to the product itself such as processes, funding, team makeup, etc. for levers with which to accelerate prototyping. We find several potential levers through qualitative coding including a division between natural and structured development processes. We find that a natural style of PD is about twice as fast as structured when working with early prototypes, but that a structured style must be adopted at some point for optimal PD.
The natural and structured modes of development can help lend depth to the strategies and theories of innovation and product development found in literature. In The Lean Startup, Eric Reis defines a process by which a startup can find success through iterative build-measure-learn cycles with continual customer feedback [23]. At each iteration, during the learn part of the cycle, the team decides whether to pivot to a new design/idea/prototype or to preserve and move forward with the current. Reis suggests that a successful startup would work to accelerate this feedback loop. What the framework of natural and structured development styles brings to this picture is that a prototype’s development cycle needs to be accompanied by the appropriate development style. When working on a prototype’s development cycle at the beginning of creating a product, a natural process of rapid prototyping with little to no documentation and an emphasis on speed is preferred. However, as the design matures during later development cycles, if this natural style persists, the prototype will become stuck in a premature stage unable to graduate to a commercializeable product. However, if structure in the form of documentation, requirements, and clear deliverables, is brought in after 2–3 prototypes the product can evolve toward market readiness.
A key point that is revealed by this framework is that if the team determines that a pivot is necessary, a reversion to a natural style of prototyping is needed for rapid development of the new idea. The transition from natural to structured development in some ways is organic as more team members are brought on, outsourcing begins, and the design of the product grows more precise. However, a reversion to a natural style of working from a structured one is not as easy. In the Innovator’s Dilemma, Christensen describes some of this difficulty within the context of large incumbent companies being poor innovators or new product developers [24]. Within the natural and structured framework of development, this can be explained by the fact that large companies have entrenched structure and processes for development, making it difficult to work with a natural style within these organizations.
Some large companies recognized this and work to create an insulated team or internal division that is not beholden to the structure imposed on the larger organization to foster innovation and new PD. A prime example of the success of such an effort is Lockheed Martin’s Advanced Development Programs, also known as Skunk Works [25]. This internal organization is given autonomy and freedom to perform research and development activities outside the usual bureaucratic regime of the larger organization. This program was widely successful and the term “skunk works” has been used by many other corporations or institutions trying to create such an agile internal team.
These issues are often not thought of as problems that startups face as they are by definition early-stage and should have more flexibility to be in the natural PD regime. However, what we find in this study is that startups must walk a fine line between natural and structured PD to get a product to market with the limited resources available to them. The natural to structured development spectrum outlined herein is a framework that we hope startups can use to inform strategy as they move through prototyping.
Within this study we highlight some key milestones that indicate readiness to move from natural to structured development and back depending on the stage of prototype readiness and team makeup. For example, we find that a company is poised to move toward a structured process if the company has completed a design, verified market fit, and started to hire new employees or begun to work with multiple outside partners or vendors. On the other hand, as demonstrated with the case study of C.44 (Fig 6), if a company is pivoting toward a new market or completely new design, a reversion to natural development is recommended to avoid costly slowdowns.
We find in the interviews, that many times transitions between natural and structured development are executed by startups without clear strategic intent as the transition itself happens organically. Conversely, pivots back to a natural process once structure has been introduced can be harder to accomplish. We encourage startups to use the natural and structured PD framework to change development styles with strategic intent. This means, during times of pivoting toward new markets or designs, startups would intentionally move back into a natural development style. Furthermore, this means startups would bring structure in strategically when pushing a fully designed and verified product to market, bringing on more team members, and engaging with manufacturers. With these strategies intentionally deployed, we expect this framework can help accelerate PD for hardware startups.
Furthermore, there are emergent productivity tools that can aid these transitions including sentiment analysis for understanding user preferences, generative design tools to explore the parameter space of design while enforcing design for manufacturing principles, and machine learning based tools to accelerate hardware development. We envision these emerging tools making transitions between natural and structured styles of product development easier for startups in the future.
Materials and methods
Sample selection and data collection
Our goal in choosing the subjects was to find a sample that best represents the larger population of companies generally. For this work, a purposeful sampling technique was used meaning that the sample was selected based on characteristics of the population and our goal of answering the research questions outlined in the introduction [26]. To answer these questions, we needed to sample from hardware startups in general with a focus on the subset of participating cleantech companies. These hardware startups needed to have at least two prototyping iterations complete and preferably more to provide the data we needed. With several market segments involved, the data can be compared between sectors including cleantech, medical, consumer electronics, IoT, and others. We were most focused on sustainable, or cleantech companies, and 20 of the final 55 companies identified as cleantech. The companies that agreed to participate came from around the world with 63% from North America, 13% from Europe, and 24% from Asia. Over 70% of the interviewees were engineers or scientists with qualifications beyond a bachelor’s degree, either a Master’s of Science, Doctorate, or Masters of Business Administration. Over 90% of the interviewees were executives (CEO, CTO, etc.) and/or founders of their company. The remainder of the interviewees were lead engineers or scientists.
Participants were contacted through online forms, LinkedIn connections, and publicly available email addresses. Of the over 500 cold contacts made, around 40 agreed to the interview, an acceptance rate of less than 10%. The companies chosen for an interview were therefore based on willingness to participate, making it a non-probability sampling technique. This brings some amount of bias to the sample companies as they self-selected to be interviewed. However, there were some factors combating this including a countervailing snowball sampling effect. For example, one company CEO would introduce us in person or by email to other company leaders for the study. The likelihood of these subjects participating was much higher than if contacted cold. In the end, 55 companies from around the world agreed to participate. Out of the 55 interviews, 50 agreed to be recorded, and were subsequently analyzed via audio file and through transcription. This resulted in 1175 pages of transcription to process. In addition, each answer was documented by the interviewer in Qualtrics survey software which was then exported into spreadsheets for analysis. The interview questionnaire can be found in S1 Questionnaire.
This work was submitted to the Committee on the Use of Humans as Experimental Subjects (COUHES) at MIT for approval as it includes human participants. The study, protocol #1812621468, was determined to be exempt after review by the COUHES pursuant to Federal regulations, 45 CFR Part 46.101(b)(2). Informed consent was received verbally by all participants in the study.
Interview methods
We chose a facilitated survey format for this interview combining engineering domain expertise with sociology research methods both quantitative and qualitative. The sample size, n, was between 30 and 55 for individual questions depending on the company. The sample size and interview length were targeted for a mix of breadth and depth allowing for a broader understanding than what typical case studies allow as well as a deeper analysis than the scope of typical surveys. This unique study provides a new "bottom-up", interdisciplinary, and mixed-methods perspective on the hardware PD process.
The content of the interview is outlined by the Product Design and Development textbook by Ulrich, Eppinger, and Yang [27], used in the MIT product engineering class to guide student through the PD process as it might happen within a company setting. The generic PD process was adapted from this text for the interview as can be seen below in Table 1.
Quantitative and qualitative analysis methods
The quantitative parts of the survey include the workflow mapping, complexity metrics, and jobs to be done opportunity landscape [28]. These quantitative metrics are gathered to generate numerical findings and work toward generalizing any conclusions from the timelines. An ordinary least squares (OLS) regression analysis was done on the complexity metrics enumerated in Fig 4 to understand if complexity is a driver of prototyping times. We wanted to understand, for example, if the total number of parts in the prototype had any correlation with time to prototype. Standard continuous error bars were plotted for the standard deviation of the trendlines.
To complement the quantitative data gathered, open ended questions about each PD step are asked. To analyze the results and find conclusive insights, qualitative research methods from the field of sociology are employed. The main method used in this work is qualitative coding which is "a way of indexing or categorizing the text in order to establish a framework of thematic ideas about it."[19] There are two main types of coding. One is data-driven (grounded) which is when the researcher allows themes to emerge from the documents with no pre-existing framework. The other is concept-driven a priori which is when the researcher applies a pre-existing framework when analyzing the documents. With either of these approaches, coding is an iterative process of code refinement and comparison to the source text [29].
In this work, we used a mixed approach using two kinds of codes, concept-driven and data- driven. Data-driven or grounded coding allows themes to emerge from the documents, while concept-driven a priori coding applies pre-existing theoretical frameworks to analyze the documents [19]. Trained engineer and the interviewer for the study, Erin Looney employed concept-driven coding with pre-existing hypotheses and framework. Andre Buscariolli, a trained sociologist, employed data-driven coding with no pre-existing framework. Through a series of meetings, the codes found in common or mutually agreed upon through compelling evidence by both coders are pursued. Through this work, over a dozen codes were found for the 55 interviews and were iterated upon until several specific coded themes developed, and the team focused on the theme that emerged around natural vs. structured development styles.
Conclusions
Hardware startups generally, and cleantech startups specifically risk having longer times-to-market, deterring investment. Prototyping is the longest development step regardless of prototype complexity or iteration. To be competitive and therefore promote a sustainable future, the cleantech startup community must find and adopt levers to shorten prototyping times. We find that the startup’s choice of development style is one lever that can affect timeline. On average, the natural PD style defined in this paper takes 35% less time than the structured style, and is thus preferred for early iterations, but adopting structure at strategic points is needed for timely commercialization. We find that startups should transition to a structured style when adding new team members or engaging external partners, which demand clear communication and expectations. We further find that when a startup pivots to a new product or market, returning to a natural style is beneficial. Using these levers and more, we hope cleantech startups can get to market faster, creating the changes we need to realize a sustainable future.
Supporting information
S1 Text. Complexity Metrics and Literature: Supporting information on the how the complexity metrics for this study were determined and informed by literature.
https://doi.org/10.1371/journal.pstr.0000101.s001
(DOCX)
S1 Fig. Quantifiable parameters on product complexity.
https://doi.org/10.1371/journal.pstr.0000101.s002
(TIF)
S1 Questionnaire. Interview Questionnaire: Document detailing the facilitated interview questions used in this study.
https://doi.org/10.1371/journal.pstr.0000101.s003
(PDF)
Acknowledgments
General: The authors deeply thank all the participants in the interviews for engaging in this research. We thank Benjamin Gaddy for providing the raw data used to create Fig 1, and Eugene Fitzgerald for helpful discussions.
References
- 1. Gaddy B, Sivaram V, O’Sullivan F. Gaddy B., Sivaram V., & O’Sullivan F. (2016). Venture Capital and Cleantech: The Wrong Model for Clean Energy Innovation, (July). 2016;(July). Available from: https://energy.mit.edu/wp-content/uploads/2016/07/MITEI-WP-2016-06.pdf
- 2. Ghosh S, Nanda R. Venture Capital Investment in the Clean Energy Sector. SSRN Electron J [Internet]. 2012; Available from: https://www.hbs.edu/ris/PublicationFiles/11-020_0a1b5d16-c966-4403-888f-96d03bbab461.pdf
- 3. Gunasekaran A. Agile manufacturing: A framework for research and development. Int J Prod Econ. 1999;62(1):87–105.
- 4. Palsodkar M, Yadav G, Nagare M. Recent trends in agile new product development: a systematic review and agenda for future research. Benchmarking An Int J [Internet]. 2022; Available from:
- 5. Pearson RJ, Costley AE, Phaal R, Nuttall WJ. Technology Roadmapping for mission-led agile hardware development: a case study of a commercial fusion energy start-up. Technol Forecast Soc Change [Internet]. 2020;158(March):120064. Available from:
- 6.
Lima GLB, Ferreira GAL, Saotome O, Da Cunha AM, Dias LAV. Hardware Development: Agile and Co-Design. In: 2015 12th International Conference on Information Technology—New Generations. 2015.
- 7. Weichbroth P. A Case Study on Implementing Agile Techniques and Practices: Rationale, Benefits, Barriers and Business Implications for Hardware Development. Appl Sci. 2022;12(17).
- 8. Atzberger A, Paetzold K. Current challenges of agile hardware development: What are still the pain points nowadays? Proc Int Conf Eng Des ICED. 2019;2019-Augus(August):2209–18.
- 9. Mulcahy D, Weeks B, Bradley HS. We Have Met the Enemy…and He is Us: Lessons from Twenty Years of the Kauffman Foundation’s Investments in Venture Capital Funds and the Triumph of Hope Over Experience. SSRN Electron J [Internet]. 2012; Available from: https://ssrn.com/abstract=2053258
- 10. Watson J, Byrne R, Ockwell D, Stua M. Lessons from China: building technological capabilities for low carbon technology transfer and development. Clim Change. 2015;131(3):387–99.
- 11. Delina LL. Accelerating Sustainable Energy Transition(s) in Developing Countries: The Challenges of Climate Change and Sustainable Development. 2018.
- 12. Henderson RM, Newell RG. Accelerating energy innovation: insights from multiple sectors. 2011.
- 13. Clark KB. Project Scope and Project Performance: The Effect of Parts Strategy and Supplier Involvement on Product Development. Manage Sci. 1989;35(10):1247–63.
- 14. Cusumano MA, Mylonadis Y, Rosenbloom RS. Strategic Maneuvering and Mass-Market Dynamics: The Triumph of VHS over Beta. Bus Hist Rev. 1992;66(1):51–94.
- 15. Henderson RM, Clark KB. Architectural Innovation: The Reconfiguration of Existing Product Technologies and the Failure of Established Firms. Adminstrative Sci Quarterly, Spec Issue Technol Organ Innov. 1990;35(1):9–30.
- 16. Ulrich KT. The role of product architecture in the manufacturing firm. Res Policy1. 1995;24(3):419–40.
- 17. Berg V, Birkeland J, Nguyen-Duc A, Pappas IO, Jaccheri L. Achieving agility and quality in product development—an empirical study of hardware startups. J Syst Softw. 2020;167.
- 18. Qualtrics. Qualtrics. 2019.
- 19.
Gibbs GR. Chapter 4: Thematic Coding and Categorizing. In: Analyzing Qualitative Data. 2007.
- 20.
Escudier M, Atkins T. A Dictionary of Mechanical Engineering (2 ed.). Oxford University Press; 2019.
- 21. Crespo-Varela JR, Kremer GEO, Tucker CS, Medina LA. An analysis of complexity measures for product design and development. Proc ASME Des Eng Tech Conf. 2012;3(PARTS A AND B):523–32.
- 22.
Elliot B. Anything is possible: Managing feature creep in an innovation rich environment. In: IEEE International Engineering Management Conference. Piscataway, NJ; 2007. p. 304–7.
- 23. Baldwin CY, Clark KB. Design Rules: The Power of Modularity. 2000.
- 24. Fitzgerald E, Wankerl A, Schramm C. Inside Real Innovation. 2011.
- 25. Reis E. The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. J Prod Innov Manag. 2012;29(3):508–9.
- 26.
Christensen CM. The innovator’s dilemma: When new technologies cause great firms to fail. Harvard Business School Press. 1997.
- 27. Bommer M, DeLaPorte R, Higgins J. Skunkworks Approach to Project Management. J Manag Eng. 2002;18(1):21–8.
- 28. Palinkas L. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Heal. 2015;42(5):533–44. pmid:24193818
- 29. Ulrich K, Eppinger S. Product Design and Development. 2015. 1–425 p.