I hope your semester is off to a good start.
Please read the Crandall (article 1) and Schmidt (article 2) articles on the importance of replication in the sciences and write a discussion board post by 9PM Monday, August 29th. It can be one post that speaks to both articles (no need for separate posts).
Please do so independently (before reading others' posts). Afterwards, feel free to reply to others' posts. There are plenty of interesting ideas in these articles and I am interested to see what you think of them.
We will discuss the articles on Wednesday, September 7th.
Hope you are all well!
Best,
Dr. Braasch
This comment has been removed by the author.
ReplyDelete
ReplyDeleteTo begin, I felt that both articles were fantastic articles that set the tone for the rest of the semester as both defined and broke down piece by piece what “replication” really means (i.e. stability of knowledge) and what it entails. While I felt the Schmidt article did a better job of deconstructing replication than the Crandall article—though this could have been an artifact of how the papers were structured in that Schmidt had a wider lens in discussing both exact and conceptual replication were Crandall focused more on conceptual replication—both provided readers with a clear cut definition of replication, even though it appears a clear cut definition is not agreed upon in the literature. To learn that something as crucial to science as replication does not even have an agreed upon definition boggled my mind. The feeling that I was getting was that this disagreement stemmed more from how one operationally defines replication. I believe that it is in this hedging that researchers that fail to have their work replicated often build their argument for why their work did not replicate by putting forth arguments such as “Well, what does replication even really mean in this context?”. The literature appears to have easily agreed upon both the literal and operational definition of exact/direct replication, but it appears that within the realm of conceptual replication things are far more murky.
Of the two types of replication, I am of the belief that conceptual replications are far more valuable to social science as a whole. With that being said, conceptual replications appear to be far easier to dismiss if a study fails to replicate by the simple argument of “the study was not an exact replication and any of the changes that were made could account for the failure to replicate”. Crandall admits that this is obviously a major shortcoming within conceptual replications and that conceptual replication has become looked upon as nothing more than a vehicle for confirmation bias. Crandall also tries to make the argument that failure to replicate is vital to science as a whole as it allows us to avoid “Conceptual Type I Error” —the adoption of theories that are false.
In order to best avoid such shortcomings as Conceptual Type I Error, Schmidt discusses science providing a “recipe” for replication. The recipe that Schmidt is referring to is in essence a Methods section. When cooking, the more detailed the recipe the less likely you are to make a mistake and the more likely the meal will turn out as it is “supposed” to. Speaking from my own experience, Italians rarely use a stringent recipe when cooking and the best way to learn to cook is by physically cooking with whomever you may be trying to learn a recipe from. This is exactly what Schmidt is getting at during the discussion of tacit knowledge (i.e. knowledge that cannot be simply written down and is often procedural in nature and therefore escapes the researcher’s conscious awareness). To me, it appears that tactic knowledge holds the key to replication but the problem is that it is nearly impossible to physically meet with and a run an experiment with all the researchers whose work you wish to replicate or whose work you are citing and believing to be true. However, I do feel that there should be more open communication between researchers (I mean we are all glued to our phones and emails already right?) and that if we truly want to address the issues with replication in science and continue to move the field forward then we need stop looking at science like an enterprise and start looking at it more like a think tank were ideas are openly shared and not treated as secret as the clues one may find in a Dan Brown novel.
Both of these articles, especially taken together, were sufficient for establishing the types of replication and the problems that arise in the social sciences when it comes to replicating studies. The importance of conceptual replication, in particular, stood out to me the most. Both articles agree that successful conceptual replication is more beneficial than direct replication to the scientific community, as direct replication of studies does not add much to the knowledge-base of whatever is being tested. Where they disagree, however, is the importance of unsuccessful conceptual replication. While Schmidt views unsuccessful attempts at conceptual replication as basically useless, Crandall and Sherman defend the importance of these failures to replicate. Here I stand with Crandall and Sherman. Schmidt appears to be viewing the value of these failed attempts to replicate conceptually from a small lens, or on an individual scale, while Crandall and Sherman see a bigger picture. If taken together as a whole, multiple failed attempts at conceptually replicated studies can reveal to us information about a theory that we may not otherwise know. For this reason, I too see how both successful and unsuccessful attempts at conceptual replication can provide important information to the scientific community.
ReplyDeleteThat being said, I definitely appreciate the systematic way Schmidt lays out the foundations of replication in his article. There were several points he made that resonated with me, one being how the community of natural sciences functions so much differently than that of the social science. It made me think, wouldn’t including replicated studies in journals be more of a concern for social sciences, especially given it is often seen as lesser than “hard” sciences? Also, while replication is a concept that quietly lies beneath the surface of social sciences that is generally understood, why isn’t it being more explicitly taught? I, myself, believe in ways that if I were taught what this article emphasizes (replication in a systematic manner) when I first got involved in psychological research as an undergraduate, it would have made getting started easier.
To start, I thoroughly enjoyed the detail in which both of these articles discussed replication. I appreciated the way Schmidt went into a deeper explanation regarding replication in many forms, while Crandall focused more on conceptual replication. These readings provide a framework that has helped me to gain a more coherent understanding of why this is such a valuable topic within research. There is such a vast importance for replication in many forms in terms of continuity within research, which is why is comes as no surprise how various elements can often lead to overlap in opinion. Personally, I tend to lean in the favor or conceptual replication, but am curious to see if there are others who feel differently and why they feel this way.
ReplyDeleteIn Crandall’s article, I found the direct comparison for how the two types of research can contribute to knowledge extremely eye opening. Crandall makes it clear that direct replication does not contribute to existing findings, it more or less adds new information in a very unambiguous manner. Crandall explains that ideas are out unit of measure in conceptual replication, therefore leading to the accumulation of knowledge in multiple facets regarding the original study/studies. To me it is obvious which of these makes the larger contribution to the field, but poses the question, why are we not using the information from failed and successful conceptual replication more frequently? There is obviously a place in research for failed direct replication, so why do we put such little value on failed conceptual research? This is one issue I found with Schmidt’s article, as there appears to be a lack of support for failed conceptual research.
Schmidt’s article discusses irreversible units, something I had never given much thought to until reading this article. As elementary as it may sound, I had not considered the difficulty in replication when dealing with behavior and units of measure that are ever changing. It’s obvious, measuring things like behavior, emotion, and thought is extremely difficult, but it also has a direct effect on the degree in which the study can be replicated. It is discussed in Crandall’s that in conceptual research, ideas are our unit of analysis, making the involved constructs and variables much more difficult to define, measure, and manipulate. This all goes back to the idea it is almost impossible to conduct an exact or conceptual replication. However, I personally find the scientific process of operationalizing and defining these types of variables as providing some serious weight and value to their claims.
Regarding the Schmidt article, I did find the example on direct replication very enlightened in terms of why direct replication is very scarcely published. The idea that directly replicating previous work was a reason for a research proposal to be rejected is very confusing to me. I don’t understand why direct replications and their findings, especially when successful, are disregarded. Why do we constantly choose new information over solidifying and increasing detailed knowledge about something else?
The two articles this week dealt with the two different types of replication in research. There is quite a bit of debate concerning which type is “better.” Both have their shortcomings, and while a direct replication may be easier to conduct, an indirect (or conceptual) replication seems to be the more ideal of the two. A conceptual replication is more difficult to achieve, as unknown variables can skew the data, but research that elaborates on a theory or phenomena is valuable to the scientific community.
ReplyDeleteA major concern that I have is that even after a study has been replicated, either directly or indirectly, those false claims can still be believed to be true by the general public. An example would be that boys are better spatially while girls are better with language. The differences between sexes are actually insignificant, but teachers swear it is true. Another example is that MMR vaccines cause autism. Every study conducted since the original one was published has debunked this theory, yet it still impacts the public’s view of vaccines in general and the MMR vaccine specifically. I’m interested in how much a false claim impacts scientific research-- does incorrect public knowledge drive scientists to do more research to set the record straight?
One line that jumped out at me in the Schmidt article was on page 92, which read, “… replication is important simply because there is agreement among scientists that replication is important.” What other aspects of scientific research are absolutely essential like replication, but might only be considered essential because there is agreement in the scientific community? I’m having trouble thinking of one, but I’m sure there are other ones out there. (I’d love to hear everyone’s opinion on this one!)
Another question I have about the Schmidt article pertains to tacit knowledge. Does conducting research with an experienced scientist who has already run the experiment before bias the younger researcher, and does that bias impact the rate of successful replications? I would think there would be a bias present in direct replications, as the primary researcher has an idea of what the results should be. For indirect replications, however, I don’t think there would be a bias. I enjoyed reading both of these articles because they gave me a good refresher on the nuanced differences between the two different types of replication.
As a natural skeptic, my initial feeling upon reading the two pieces was quite well received. From the standpoint of a scientist, there are a few presuppositions that one walks into the field with—and one of the more prominent philosophical sentiments is the idea that we can glean causality from the experimental manipulation of variables. In science, our observations serve as proof of our concepts, but really all that we can say about our predictions are that they are (or aren’t) statistically significant. Conceptual replications seem to be a fascinating rebuttal, in that they propose different methods of operationalizing the same variables in which we can provide greater support for overarching theories.
ReplyDeleteWith that said, I worry that the articles made the assumption that all researchers have these grand and “all-explaining” theories in mind when constructing their initial study, when it’s my experience that science typically takes a more gradual and progressive route of formulating complex explanatory models. If we’re defining the conceptual replication as demonstrating the same theoretical concepts using different techniques (and the articles outline myriad methods of doing so), does that mean that any body of research relating to a similar topic is really a “replication?” I guess that my primary question would be: “How do we differentiate between an elaboration (offering new findings that fill the gaps in prior research) and a conceptual replication?”
As the articles both point out, the issue of confirmatory bias also presents itself as quite challenging when doing any form of replication. After reading them, I was left wanting a better safe-guard and solution to this very troubling stumbling block. Our capacity to rationalize any inconsistencies with methodological errors is only further confounded with the acknowledged “tacit” nonverbal language of methodology. How can we objectively evaluate these facets of replication that are so necessary in demonstrating the validity of theoretical models?
Furthermore, while these articles both highlighted the importance of replication (directly and arguably more importantly, conceptually), I think that some of the responses put forth by the editorial committees have valid points as well. As much as we’d all like to think that gigantic grants can never-endingly fund all of our projects, there needs to be some accountability in regards to how resources are allocated. Especially when considering the blatant deficiency in psychological research in informing policy-making decisions, I think that a reiteration of theoretical models that require immense time, money, and effort can understandably come across as frivolous to institutions that (quite frankly) have an agenda. Too often in psychological research, it seems that information is obtained “for its own sake,” rather than to influence policy and the avenues required in making impactful changes. There are certainly studies worth conceptually replicating, but it should be closely examined if the study proposed for replication actually undergirds theoretical values that can be evaluated for practical purposes.
Perhaps a better angle at addressing this problem of garnering support from publications and editing boards would be letting the policy be proof of the replication. For research that is able to inform policy, let the policy serve as the sort of “conceptual” replication, because it should also yield similar conclusions. I acknowledge, of course, that not all research has practical application; but if the research is not readily translatable to a pragmatic form, I feel that the topic likely needs more consideration and elaboration. Because what is the point of replicating research if its meant to be relegated to theoretical mumbo-jumbo that doesn't affect peoples' lives in meaningful ways?
Both articles make clear the benefits of doing conceptual replications, but I think that Schmidt’s article brings up an interesting point that conceptual replications are in a way risky to conduct. If the results of the original experiment do not match the results of the attempted replication, you have no way of knowing which experiment has the “correct” results.
ReplyDeleteAn additional risk is that it can be challenging to get a replication study published, especially if the study is a direct replication or if the replication itself fails. With the possibility of dedicating months to a project only to get nothing (or at least not enough) out of it, there is a lack of incentive to conduct replication studies. The riskiness involved in replication research in general may be part of the reason that it is not as common as perhaps it should be.
After reading through nearly all of the Crandall and Sherman article, it started to feel like they were pushing for conceptual replications at the expense of direct replications, which I did not agree with. At the end of the article though, I was glad to see that they included a section (although it was quite a small section) on why direct replications are still useful and important, especially for research that is more applied.
Tying into that, one thing that stood out to me from the Crandall and Sherman article is how the replication process can be so different for different disciplines and even sub disciplines. For example, direct replication is very challenging for social psychology research because of how people vary so much across time and places. Whereas direct replication is much easier for disciplines such as geology or chemistry because their subjects do not vary in the same way that people do.
I think understanding these differences is especially important for us in this class because we come from so many different departments. So what may be the “best” or “easiest” type of replication for one of us may not be the same for others in our class. Similarly, our speakers throughout the semester come from a wide variety of disciplines and sub disciplines, so it will be interesting to see how these differences may play into how they talk about replication.
Both articles introduce the basic definitions of replication and two kinds of replication. Crandall’s article focuses on the characteristics of exact replication and conceptual replication, and highlights the importance of conceptual replication. This makes sense to me because I personally think it is the idea or theoretical hypothesis that inspires people to discover different ways to operationalize and replicate across operationalizations. And when the idea or theoretical hypothesis is confirmed, it will help to build, fulfill, and expand the paradigm, and further advance scientific knowledge. As far as the kinds of replication, I prefer to use the term direct replication rather than exact replication because I totally agree with the points that Crandall made in the article that we have to admit that the process of conducting social science is subjective, the replication cannot be truly “exact”. Crandall’s article did a good job in comparing the outcome of when replication succeeds or fails for both exact/direct replication and conceptual replication.
ReplyDeleteIn comparison, Schmidt’s article not only explains the differences between direct replication and conceptual replication, but also does a good job in organizing functional approaches to replication. Schmidt introduces and evaluates different replication functions using different classes of variables, which presents a clear picture about what researchers should consider when it comes to conducting a replication study. This is very helpful for me to understand direct and conceptual replication when it comes to how to conduct a replication study and why to conduct a replication study.
In addition, both articles mentioned the issue of publication pressure of replication. Crandall points out that failed conceptual replications are much less likely to be published than failed direct replication. Schmidt suggests combining replication and new elements in a study to get published. I used to think that replication study may be easier than other study, but I changed my mind after reading these two articles. Because if I plan to conduct a replication study, I not only need to pay close attention to how to replicate since there are many factors can influence the replication process, I also need to put a lot of effort on justify the results especially when it is a failed replication.
In the pair of articles for this week, the proposed preference for conceptual—as opposed to direct—replications in psychological research was supported fairly soundly. In psychology, cognitive science, and science, in general, the underlying goal is to continue advancing toward a greater understanding of objective reality and the rules which govern the world we live in. As such, these articles provide a very compelling argument for the need to focus on scientific advancement, rather than verifying isolated findings (though, this has its place, as well).
ReplyDeleteScience is intended to be a self-correcting process which allows us to understand the laws which govern the natural world; however, these large-scale concepts are investigated through careful evaluation of underlying theoretical constructs. It is unlikely that any single publication will change the way we view the world or even our sub-discipline. In spite of this, however, each study does have value: it supports or refutes a claim. Each one of these claims may be small when compared to the larger body of knowledge, but they do allow for evaluation of one area of inquiry. It is for this reason that the argument for conceptual replication is so powerful: conceptual replication allows researchers to investigate the principle underlying a given finding, rather than imperfectly trying to duplicate the results of previous work.
To be clear, direct replication does have its place and purpose, as expressed by both articles. One area where direct replication is of critical importance is in evaluating methodology. This can take the form of the physical equipment being used (e.g. the recent critique of fMRI), the validity of a construct (e.g. IQ as a measure of intelligence), or the ability of a finding to generalize across populations. Additionally, science is made up of individuals and, regrettably, some have deceived the scientific community for various reasons. As a result, sometimes it is necessary to ensure a claim is not fraudulent (e.g. Piltdown Man). In these domains, certainly direct replication is not only the best method, but may be the only truly viable method, for evaluating the legitimacy of a finding.
Although direct replication does provide a valuable service, conceptual replication allows for scientific progress. More specifically, Schmidt’s proposal for how to carry out conceptual replication provides a fairly straightforward way of ensuring that conceptual replications are done in the most effective way possible (2009, pp. 95-98). It is important to acknowledge, however, that this does come at a cost. As both articles discuss, it is important to not only be correct, but to be the first person to present an idea. To illustrate, virtually everybody knows who Charles Darwin was, yet Alfred Wallace is a name generally recalled only vaguely. For this reason, the act of conducting an exact replication, followed by a series of single-variable manipulations is unattractive to researchers, especially if the study requires a large amount of resources already (e.g. time, money, participants). In spite of the demands it places on the researchers, this does seem like a great idealistic goal, though it does not seem very feasible, barring a revolutionary change of the editorial process of journals, the tenure system at institutions, and motivation, itself.
In spite of the perceived replication crisis in psychology, Schmidt does raise the point that while explicit replications are rare in the social sciences, replication does occur under the disguise of follow-up studies. It is in this area that it seems the most progress can be made. Although there are idealistic goals which scientists ought to strive to achieve, a more pragmatic solution may simply be to be more explicit about these procedures and to be more aware of the intended purpose of a replication, to ensure the right method is chosen.
I found both articles to be enlightening and thought-provoking. The Schmidt article in particular was very helpful to me in conceptually breaking down the general idea of replication. It is however, surprising that replication as a scientific concept is not clearly defined (i.e. not agreed upon) by the larger scientific community. The overwhelming scientific support for replication in conjunction with the lacking literature in the area contributes to my newly found fascination with the topic. The operational definitions of direct replication and conceptual replication outlined in the text seem very straightforward at first. However, I found them more difficult to view as discrete as I continued reading. Viewing these concepts as falling on a continuum (as stated in the Crandall text) made the difficulty of distinguishing them much more difficult. In the Schmidt article, this was compounded with the rhetorical solution and at what point does one call a replication a replication? In this case, is replication based on a combination of results and intent?
ReplyDeleteI had never considered the rhetorical solution until reading the Schmidt text. In the example given, it seems as though there is a formula for interpretation given replication:
Experiment X replicated Experiment Y:
• Outright presentation of experiment X as a replication of Experiment Y
Experiment X failed to replicate Experiment Y:
• List of ways Experiment X differed from Experiment Y. Therefore, X is not a replication, but a completely different study.
Another thing in particular that struck me repeatedly while going through both articles was the idea of sexy research. Journals and other publication outlets reveal a bias toward flashy findings. This makes sense in terms of readership, but does not necessarily make sense when it comes to simply adding to existing replication literature. There is knowledge to be gained from both successful and failed replications, given the research was well-conducted (“good”) research. While I do think that in general, conceptual replication is more valuable to the scientific community, there is value in direct replication. Particularly when it comes to findings that are novel or innovative. Replication is such an important aspect of empirical science and I look forward to delving deeper its many facets.
I thought that both of the articles were insightful for understanding the importance of replication and the distinction between the different types. It was clear that both articles favored conceptual replication, which is understandable given the constraints and demands of social science. In a way, the description provided in the Crandall article, of conceptual replication as promoting theory and driving scientific progress, reflects my conception of psychology’s strength in general: to construct malleable, informative, and functional means of understanding complex, ever-changing, and socially-influenced phenomena. I believe that this broader meaning making, regardless of the inherent instability of many psychological constructs, is the true appeal of psychology; therefore, it makes sense for conceptual replication to be a larger cornerstone in the field. Further, as both authors mention, social sciences do not lend themselves to direct replication as well as hard sciences do, particularly when social sciences deal in changing constructs rather than invariant physical components and laws. In the Crandall article, the mentioned current spike of interest in direct replication appears to be, in part, an instance of the long-standing desire to make "soft" science appear "hard”, to examine constructs as if they were tangible and always capable of simple and direct manipulation. While many aspects of this endeavor are warranted and improve the overall quality of the research conducted, I believe one must also acknowledge the role many seek to fill as psychological/social science researchers and use replication to support the field's strengths rather than merely as an attempt to seem more like a natural/hard science. If not, researchers in psychology may run the risk of becoming overly focused on isolated variables without devoting enough thought to developing ever-more informative theoretical frameworks that drive comprehensive understanding. In effect, psychological and other social science researchers appear to face the burden of balancing art and science in a way that hard science researchers do not always experience, and this undoubtedly influences how researchers in these fields address topics such as replication.
ReplyDeleteThough I support the importance of conceptual replication posed in the articles, it is clear that one must be careful to construct research in ways that provide helpful insight. As mentioned in the Schmidt article, one always runs the risk of introducing such an array of changes that disparities with past findings are virtually impossible to interpret. Further, the importance of direct replication for appropriate purposes cannot be undervalued, and I believe there are many instances in which direct replication needs to work hand-in-hand with conceptual replication to further understanding. In particular, there is great need for direct replication, as the Schmidt article detailed, in applied areas of research, particularly when specific discrepancies in results could have adverse effects on individuals. Aside from this, it would also seem helpful to utilize direct replication to better analyze intriguing findings, for instance by examining whether certain differences in outcomes happen repeatedly before assuming that theoretical relationships operate differently for different groups or in different contexts. This idea meshes well with the discussion provided in the Schmidt article about follow-up studies. Looking at psychological research realistically, I agree with the Schmidt article’s conclusion that one of the most efficient ways to incorporate replication into research would be to have both a direct replication piece to support one’s methods and quality of research design, while also incorporating unique aspects that increase the breadth of understanding.
The ability to replicate experiments provides science its greatest power to inform consistency and generalizability, but the limits of replication can often be misunderstood. Perhaps the most interesting takeaway from the articles concerns the nature of social science research. In both discussions of conceptual replication, the necessity of strong theoretical approaches to research become clear. Understanding complex phenomenon that are socially bound cannot be achieved by simply performing direct replication as noted by Crandall and Sherman. Faults in theoretical models can only be discerned by testing those models. With direct replications focus on replicating effect and not investigating the formal model assumed, undue focus on direct replication can blind and limit scientific progress.
ReplyDeleteThe direct replications of Brehm’s and Stoner’s work reveals the pragmatic nature of the institutions and processes involved in science. Scientific achievement is limited by a number of factors: funding, the publish or perish environment, an undue focus on presenting entirely ‘novel’ findings, and much more. Therefore, it is not surprising to see a history of scientists falling into a pragmatic research program. As Schmidt mentions, it can be entirely impossible to learn how to replicate experiments without learning directly from this source, and this passing of knowledge has historically led researchers astray. Maintaining a strong tradition of questioning and testing the theoretical models advanced for a phenomenon is paramount.
There is obviously still great importance for direct replication in science. Schmidt covers the goals and forms concisely in his discussion of his functional approach. What concerns me the most is the limits of reach for direct replication studies. Complexity in formation and and enactment aside, direct replication can inform us. The hesitancy of social science journals to publish direct replication studies limits precise investigations into effects. Again, we can readily understand this as an attempt at pragmatism. Direct replications do not offer any novel information. Pages in journals are a scarce good, and research funding can be even scarcer. Replicating effects is surely not flashy, and journals want to demonstrate an impact by publishing findings that generate discussion.
This discussion on replication effectively conveys that science is not solely an informative and problem solving endeavor. It is also a social process. It occurs within a system of shared norms and resources that often require practical approaches.
Crandall (2016) emphasized how important conceptual replication is in science research, especially social science field. It was interesting that depending on characteristic of research field, importance of the type of replications shows different. Unlike other science fields such as physics, chemistry, and biology which deal with micro-scaled or smaller specific world, so that same experimental setting, occasionally even room temperature and humidity are critical to result, similar method, and similar subject group, in social science field just replicating same experimental setting and similar result could has no meaning. In social research, whereas, showing consistency in conceptual idea based on complicated variables affecting experimental results. It seems plausible that since variables of social study are usually society itself, culture, language, or history, it is impossible to control these variables. Therefore, I agree with the authors’ argument that replicating theoretical idea regardless of experimental setting such as subject group, or method in social research is crucial.
ReplyDeleteSchmidt (2009) also highlighted the concept of replication and how important is especially empirical study. It covers both importance and limitation of replications, and how actually we can deal with replication in our research in real setting, therefore it gives me a more practical way of thinking about replication issue. In terms of irreversible unit, as the author addressed in this paper, I once had this kind of curiosity when I was working with patient group. We invited patients having particular disease and examined how their disease affect their semantic processing. We invited same people once per week and did 3 month follow-up. Regardless of time interval, it couldn’t help, but their performance got better because they got used to task. But, although limitations, replication is still important issue in science research. Author suggested for some functional approaches which seems to be helpful.
These two articles helped me to have a clear point of view of different importance between direct replications and conceptual replications. I thought that I am aware of the importance of replication, but after reading these articles, I realized that the replication in my mind was mostly direct replication. It seems that I had a very vague and undistinguished notion about two different replications. I thought if I can replicate one experiment perfectly in terms of both methodology and result, that would mean the experiment is success and result is reliable. But according to this articles, it could be not if the replication comes out of experimental skill, without firm and correct conceptual foundation.
In both of these papers the author discussed the types of replication and its uses. Schmidt (2009) discussed more specifically on conceptual replication and its function but the other paper Crandall (2016) discussed more specifically on both exact and conceptual replications. As a student of Educational Psychology I studied about tacit knowledge and epistemology.
ReplyDeleteSo in my mind I am thinking all these works to create new knowledge.
In the Crandall article, besides saying about both exact and conceptual replications. The author also states that, how we should use these two kinds of replication in theoretical hypothesis. The conceptual replication can contribute more then direct replication in both theoretical development and scientific advancement. But this article does not specified what type of scientific advancement?
In terms of explaining the value of exact and conceptual replication. The author did really a nice job. As a new reader in this topic I was kind of lost for a while. But the nice and clean explanation of value of replication was very helpful for me to grasp this idea. It is really interesting how he showed the conceptual replications succeeds or fails in different theoretical frame work of the study.
In my mind I am thinking does replication has any relationship with epistemology? It might be a stupid question. But to me it looks like both epistemology and conceptual replication deals with higher order of thinking. And also helps to create new ideas in the scientific world.
In the Schmidt’s article I have found the examples of direct replication very useful then Crandall’s article. He showed very good point how researchers are scared of publishing direct replication. It is also interesting that in social science very few replication studies have been published so far. Schmidt’s also explain nicely why it is hard to find direct replication study published, is because something already been published is worthless to publish again and they are not being acknowledge by the scientist in the field of social science.
But above all, I enjoyed reading both of the article. One thing that I wonder about both of this studies that they are not well defined in terms of methodological point of view like other experimental research article. In this point of view, if someone is interested in researching in this area they might use these two article as a theoretical framework. But not for methodological reference.
Our readings this week were both survey articles regarding replication in psychology. Both papers defined and described various aspects of replications, including the pros/cons and motivation for them. In particular, they focused on differentiating between exact and conceptual replications.
ReplyDeleteThe Crandall paper was a fun read with a light tone. Many of the pros/cons of replications are straight-forward and well known, but I did learn about the implications of a failed replication. This is also the first time I've seen the terminology laid out for me.
Reading this paper comes at perfect timing for me, since I am currently writing up a replication study on information foraging. I had just received some emails from a collaborator about not knowing exactly how to phrase the replication aspect of the study, but now I'm (overly?) confident on how to write it!
A somewhat surprisingly thing is their stance on exact replications, which seems to be against them, unless you *really* have a motivation to do so. But in my field (human-computer interaction), the community has been criticized for their immense desire for novelty over soundness (and with it, replication), and as a result there has been a push for replication. This paper provides some nice food for thought that just doing replication for replication's sake may not be the best use of our time... But on the other hand, replication studies are so rare in HCI (except for a few big ideas, such as Fit's law and GOMS) that I think *any* replications would be beneficial.
The Schmidt paper started off very dry but it picked up when describing the functional approach. Both of these papers provide great examples/metaphors that anyone can understand and slowly walks through them.
This paper dived more in-depth by focusing on the goals of replication studies in a different, more fundamental (and also neutral?) way than the Crandall paper. Though, after reading the Crandall paper I feel like I already learned most of this and again, that one was a more enjoyable read.
In the end, I thought replication was simple and straight-forward but there are a lot of nuances that I had never thought of. These papers should serve as great primers for the rest of the semester!