In Reply:
In the reply to our article,1Prof. Kerssens makes the central assertion that “Foremost, a model that generates discrepant parameters depending on its assumptions or underlying structure should not invalidate the behavioral findings it attempts to model.” This statement contains two common misconceptions.
The first misconception is that “behavioral findings” should be counted as somehow more direct and thus less problematic evidence of something compared with model-based analyses. However, this view fails to take into account the fact that there is no such thing as a model-free measurement. Every analysis makes assumptions about data. The set of these assumptions forms what is called a measurement model. Thus, every analysis is necessarily model based, and the measurement models may be more or less adequate. For instance, the analysis mentioned explicitly by Kerssens on the difference between responses to old material presented under anesthesia versus new material not presented before implies a measurement model comprising the assumptions (1) that responses to new words are exclusively based on guessing, (2) that responses to “old” words are exclusively based on memory and guessing, (3) that in the latter case, memory and guessing processes are strictly additive, and which implies (4) that the assumed underlying distribution of the evidence variable is rectangular.2All these assumptions may be inadequate. For instance, assumption (1) precludes strategic processing such as generating unusual words, which, however, has been observed before.3Assumptions (3) and (4) imply deviations from signal detection theory that have been criticized.2Another problem of this model is that “memory” is assumed to be a single homogeneous process that cannot be decomposed further, an assumption that obviously need not be adequate, and one that would not even allow for the simple distinction between automatic and controlled memory processes.
Second, it is not correct to state that we used one model, which generates parameters that depend on the assumptions of the model. Rather, we applied two different measurement models for the process-dissociation procedure, one of which has been shown to be more adequate than the other.4Obviously, the better of the two models should be used for data analysis, which is what our analyses clearly confirm. The use of an inadequate model has led researchers (and would have led us) to conclude that there was memory for intraoperative events, which in fact was not there.
A further point is that Kerssens wonders why we did not include studies that used the extended measurement model for the process-dissociation procedure and found evidence of automatic memory processes.3,5This point is well taken. We did not include these studies for several reasons. First, these studies were not instances of the point we wished to make, that is, using inadequate measurement models may lead to inadequate conclusions. Second, although we do mention studies that found evidence of memory for intraoperative events (see p. 301 in our target article1), it must be realized that our article was not meant to be a meta-analysis in which every single study on this issue was to be included. For instance, we did not include a study by Kerssens et al. 6in which no evidence of memory for intraoperative events was found using the process-dissociation procedure. Third, an adequate treatment of these studies and their results would have added complexity to an already complex article. Fourth, this is because the data from one of these studies3are difficult to evaluate. Although the authors reported probabilities for controlled and automatic processes of approximately 0.04 to be significantly different from zero, our model-based reanalysis suggests that the latter was not the case. The reason for discrepancy is unknown. This leaves one study5in which the parameter estimate for automatic memory processes was 0.10, and in this case, the difference from zero was statistically significant in the original study and in our reanalysis. However, note that the size of the parameter estimate is still small. Given this, including this study would not have added information beyond that already conveyed by another study, which we did include and in which there was evidence for a small (0.04) contribution of automatic processes to postoperative memory performance.7
In summary, it must be realized that data analyses—even those of raw observable responses—always imply measurement models that need not be adequate in the assumptions they imply about the data. What is more, we believe that we can confidently maintain our original conclusions that the choice of the measurement model is critical when using the process-dissociation procedure. Using an inadequate model may easily lead to spurious “evidence” of memory formation during anesthesia. The extended measurement model should be used instead. By using this model, evidence of memory for intraoperative events seems to be rare at least at “deep” levels of anesthetic depth, and where it seems to be present, the probability of memory processes contributing to postoperative memory performance seems to be small.
*Campus Virchow-Klinikum and Campus Charité Mitte, Charité—Universitaetsmedizin Berlin, Berlin, Germany. daniel.hadzidiakos@charite.de
