An Engineering Approach to the Scientific Method

Editorial

Part of the primary motto of this journal is to provide engineering approaches to enhance the power of the scientific method. The engineering approach is essentially the engineering design process. The scientific method is similar but is hypothesis driven vs. design driven. In the scientific method a hypothesis is generated as a potential answer to an important question. An experiment is tested to prove or disprove the hypotheses. The results of the experiment are discussed in the context of the original question and whether the hypothesis was proven or not. The results are also compared to other studies looking at the same or similar questions. Therefore the study is looked at whether it follows from previous studies, what are the ramifications of the results related to the original question, and what needs to be examined next; further test the hypothesis or test a different hypothesis.

Again, the design process is similar, but different in some important ways. Instead of a question there is a problem or need. Instead of a hypothesis there is a design constraint(s) of what the solution has to be able to do. Experiments are done to determine if the solution meets the design constraint(s), which is similar to testing a hypothesis; in fact a design constraint can be written as a hypothesis. The big difference is that design constraints are quantitative and hypotheses are typically not. Hypotheses are typically proven or not proven based on statistics and whether something leads to a statistically significant difference. However, a statistical difference does not mean the difference is significant enough to matter; with hypotheses not normally quantifying how big a difference is required.

In the Bioengineering world, the problem is typically a clinical medicine one. It could be with current treatment the healing is too slow, the device or new tissue are not strong enough, or not something else enough. The new treatment or device either has to augment the tissue or stimulate the tissue to meet a clinical performance requirement (e.g. handle a certain load--which will be used as the example here to help better illustrate a point). A design constraint would be quantitative and specify how much load it needs to handle (plus normally some factor of safety). A hypothesis would be: Does this new treatment lead to a significant increase in load carrying ability compared to current treatment; without assuring it meets the necessary clinical performance requirement.

Implicit in the problem is that it is a significant problem. Further, implicit in the design constraint is that meeting it will have a significant impact on the problem. This clinical significance is the major difference between the scientific method and the design process, as well as implied by the first word in the journal title (significances). Currently medical schools have determined one of the most important skills a clinician should have is the ability to apply math and science to physiological systems (to clinical medicine once they graduate); which is the definition of bioengineering. If asked, it would be an understanding of the scientific method in order to choose the best treatment for a given situation. The skill they are actually looking for is to be able to use the design process (at least the first few steps) in order to choose a treatment that meets the required clinical performance constraints (or as close as possible). They want clinicians to “think like engineers” but would never admit that (I have tried).

Clinicians should not choose one procedure over another just because it produces statistically significant improvements. First, a significant problem has to be established. Speeding up the healing or increasing the strength of something that is already adequate clinically is not beneficial (unless it significantly reduces cost or gives a significant additional benefit). Then procedures are chosen if they can meet the clinical performance design constraint(s) that current treatments cannot, or at least make a clinically significant improvement; getting closer to the clinical performance desired.

So a research paper that uses bioengineering approaches to enhance the scientific method requires discussion of clinical impact (the significance of the bioengineering advancement). There are multiple ways this can be looked at: the desired clinical performance improvement, the potential effect on clinical performance of a solution, or the benefit (cost, time, resources) to critical stakeholders (patient or health-care providers). All three ways are potential design constraints; with the third also a commercializability concern. It is of course not necessary to discuss the commercializability of the studied treatment, however, it will not have any clinical impact without making it to the marketplace. At least a paper should place the research in the continuum of steps toward the development of a marketable product.

This is important for justification; justification of the need for the study, the approach used, and the significance of the results. It does depend, however, to a degree on the type of study and where in the design process it fits. An applied paper should be design driven, even if it is written as hypothesis driven. There should therefore be design constraints. The study should explain where it fits in meeting these design constraints. Design constraints can be broken down into different types. There are “have to(s)” and “would like to(s)”, which can be clinical performance design constraints as well as pre-clinical design constraints.

Each study should also have its own design constraint(s); what it is trying to show relative to the design constraint(s) with emphasis on the clinical performance design constraint(s). Then the significance of the study becomes how the limitations of the methodology effect the ability to meet these study design constraints and how this relates to the clinical performance design constraints necessary to solve the problem (or at least make a significant clinical impact).

Too often, even in applied journals, papers fall short related to the engineering design process, which effects its ability to justify the study, the approach, and/or the significance of the study; because they are hypothesis driven. First is in establishing a problem; both how far short of the needed clinical parameters are current treatments and how significant a problem this is. Most papers will describe the clinical problem, but not what is lacking with current treatments and what significant clinical issue this deficit causes.

The next place is in specifying the design constraint(s), so any proposed solution can be judged on whether it meets all the “have to” design constraints as well as any “would like to” design constraints. The comparison of solutions should be on the clinical significance of meeting each of their “would like to” design constraints as well as their associated costs and risks, since any solution not meeting a “have to” design constraint can be eliminated. Many of the “would like to” design constraints are meeting the “have to” design constraints above the minimal level. Assessing these improvement “would like to” design constraints again requires determining how big an additional difference in clinical performance would actually make a difference as well as how big an improvement on the “have to” design constraint would lead to that level of difference in clinical performance. For example, the desired clinical performance may not be a “have to” design constraint, but be broken into a “have to” design constraint plus one or more “would like to” design constraints. The first “have to” could be (going back to the load carrying ability example) the minimum improvement in load carrying ability that makes a significant clinical impact; with the “would like to” design constraints one or more improvements in load carrying ability that also make a significant clinical impact. Part of the engineering design process is optimization; making improvements to meet more “would like to” design constraints.

Determining and quantifying design constraints is normally an iterative process. The pre-clinical constraints are what we believe the design needs to be or do in order to meet the clinical performance design constraints, which we are most likely not testing, unless this is a clinical study. So the desired pre-clinical performance design constraints cannot actually be determined until the relationship between pre-clinical performance and clinical performance is known. A study may look at just feasibility of meeting the pre-clinical design constraint(s), which is believed to be necessary to meet the “have to” clinical performance design constraint(s).

So although it is unlikely that the design process is complete; where the study fits into the process has to be justified. As a minimum the specific improvement in clinical performance should be specified (as quantitatively as possible) as well as the believed relationship between the pre-clinical performance design constraint(s) the study is focusing on and the clinical performance design constraints (as quantitatively as possible). In the discussion, what the study showed relative to the design process should be explained as well as, at least in general, what future studies are needed to determine if the proposed solution could meet the clinical performance design constraint(s). Too often a paper will claim it showed the potential of the solution to be used in a clinical situation without identifying the problem with current solutions, the improvement in clinical performance desired, or what additional studies would be needed to show the solution could actually meet the clinical performance design constraint(s).