Hello Everyone,
Thanks so much for your thoughtful posts! I hope you feel the two articles were a nice introduction to considerations of replication, why they-re important for cognitive science and this course specifically.
Please read the Elman (1995) article on language as a dynamical system and write a discussion board post by 9PM Monday, September 5th. This article was chosen by Andrew Olney. At least from my perspective, it may be one of the more difficult articles we will read this semester. Please focus on the "big ideas" an think about them in relation to theme of replication; do not get lost in the minutiae of it.
In case it is helpful, I here reproduce (no pun intended) some possible discussion points from the syllabus.
- the theoretical grounding presented in the articles
- the rigor of the experimental methods
- the appropriateness of the statistical analyses
- the clarity/novelty/theoretical import of the results
- important connections to other theories, experimental manipulations, results we may have discussed
- the study’s ecological validity (do experiences represent the “real world?”)
- other questions/discussion points that may arise from your analysis of the
article(s)
Please post your discussion as a response to this article. Afterwards, feel free to reply to others' posts. There are plenty of interesting ideas in this article; I am interested to discuss them with you on Wednesday, September 7th.
Also, as a reminder, we will meet at 2:20, first discussing the replication articles and then saving time to discuss the Elman.
Hope you are all well!
Best,
Dr. Braasch
I will start off by saying much of this article was over my head in the beginning. I have very little background in linguistics or language systems, or the advanced mathematical processes that go into dynamic systems theory, and this will likely show in my response. However, I did find a handy description of hidden units as though described to a five-year-old when I googled to get more background knowledge of the subject (because I was totally lost), which was useful!
ReplyDeleteGiven all of this, I’ll approach my response from a “big picture” standpoint. Overall, I believe this article is a good example of how questioning mainstream theory—which many are afraid to do—and viewing processes using a different approach can be worthwhile and build on the existing body of knowledge, as well as open doors to future research. Elman does a proper job at laying out the problems and conflicts within the traditional approach to language processing, and showing how a dynamical system can address those problems. His logic is sound, and given how the brain is an extremely complex bundle of neurons with many different networks functioning within it, it makes sense to view language processing from a dynamic perspective. However, and I believe Elman makes note of this toward the end, this system is hardly able to be applied in a natural setting, and is inherently mechanistic. This, of course, brings up issues when trying to test the theory in natural speech. Could it be accomplished? Though, these are issues with the traditional theory as well, and I believe that instinctively, the dynamic approach brings a lot to the table that the traditional theory does not.
Given that this article was written in 1995, and given how little I know on the topic, I assume this has worked as a springboard for further research in this area, especially regarding artificial intelligence systems for language processing. Studies using the previous approach could be replicated using the new model, and possibly shown to be more efficient. If anyone has more knowledge about this area, please feel free to share!
I think this will be a common theme, but this article was definitely difficult to work my way through. I have had no exposure to language systems or research in that area. That being said, looking at this article from a larger viewpoint made it much simpler.
ReplyDeleteIn this article, Elman begins by presenting us with the more traditional view of language processing that involves a lexicon similar to a dictionary that is full of words governed by set rules. After laying out this traditional view of language, Elman goes on to present us with his dynamical viewpoint. While both constructs were pretty confusing to me, I can see the value in both. I think that is an important point to note, both in science and in life. Breaking away from traditional viewpoints has value. This value exists in science even when new ideas or concepts cannot be statistically supported. There is, and will always be, value in trying something new or looking at a tried and true concept from a different point of view. Elman notes again and again that his dynamical system cannot account for language processing in its entirety, but points out that this truth is not some deficiency in his model. I think it is important to take different viewpoints for what they are worth and to use them to be best of your ability. Elian's dynamical system, even though not perfect, was able to present new ideas about the capacity of language processing that had long been questioned.
I think these ideas play nicely into the topic of replication. I do not think that Elman was attempting to replicate the more traditional viewpoint of language processing, nor replace it, but instead to add to it. We can view replications of studies in this way also. A failed replication, similar to a new view on language that does not quite explain everything, is not a useless endeavor. Neither mean that anything is wrong, but instead that there are areas in that particular research that can be improved upon. Going back to last week's articles, I wonder how difficult it was for Elman to have this work published. At the time this was published, I'm sure he was looked at like a fool for attempting to come up with a new model of memory. Again, there is some intrinsic value in seeking out a new way to grapple with heavily researched concepts. Regardless of whether I know much about language processing, I think Elman was successful at pushing the envelope and introducing fellow researchers to a new way to think about language.
This week’s article was beyond challenging, and I feel those of us without a background in linguistics may not have been able to appreciate the overall contribution of Elman’s work in his specific field. However, I think it is safe to say we can all recognize the importance in his way of thinking and conducting his research to produce such important results. I was thoroughly impressed with the way the background and introductory information was easily understood and applicable. There were elements I considered to be more difficult to understand than others, however the overall subject matter was made relatively clear. The thing I found most impressive about Elman’s work is the way in which he completely reworked a pre-existing idea into something entirely new. This is the type of thinking and planning, I believe, we all aspire to achieve as researchers. After reading last week’s articles, one can really see how unparalleled Elman’s work is and the importance of his achievements.
ReplyDeleteDespite how unique and challenging his work was, he was able to produce results that laid a solid foundation for future research. I found it refreshing to see how Elman never once claimed his work was without flaw or that he had answered a question in its entirety, he constantly acknowledged the ways in which his work could be improved. I am still trying to grasp the idea of understanding mental representation as a trajectory in our mental space rather than something our mind constructs as a reaction to stimuli, but I do appreciate how this understanding produces many new ways of adding to and modifying existing information regarding language processing. It does raise many questions in terms of future replication, perhaps his findings are far more difficult to transfer to an academic or learning environment rather than a lab? Maybe future researchers can find ways to take this mechanical aspect of his work and translate it to a more natural setting?
Because of my unfamiliarity with linguistics, I am very eager to hear any responses, issues, or comments about Elman’s findings from those with deeper knowledge on the topic. While I don’t feel like any of the findings were a stretch or unclear, I am curious to see if those who understand and appreciate linguistics see flaws with his research. I would also be interested to hear the views and opinions of those who better understand this mechanism application or potential for application in terms of prediction.
In regard to the article as a whole—I had the advantage of reading this for the 3rd time (as I had to read this previously for a class on Neural Networks and it required me to read it 2x because I was lost the 1st time!)— so I more or less understand what Elman was trying to get at this time around. I feel that language is the perfect example of a dynamic system, a system constantly in flux and whose output (speech) is constantly changing depending on the parameters that exist (environment). From a processing standpoint, we rely on prior exposures (i.e. learning phases) to understand the rules (i.e. parameters) and how these rules have an influence on our outputs—the hidden layer is responsible for these rules and transformations that have an effect on the final language product. I felt that Elman did a fine job of making his argument for language processing as a dynamical system—he just maybe could have done a finer job if he explained things a wee bit more clearly so it didn’t take 3x times to understand.
ReplyDeleteThe main argument that Elman makes—and in the experiments that he presents—revolves around being able to teach a neural network to predict the next word in a sentence. Now though a simple recurrent network cannot do this with 100% accuracy—as it first needs to learn through trial and error and varies depending on the corpus that is trained with—the fact that it “learns” to categorize possibilities based on sentence position is quite remarkable. Now, in regard to replicating a network such as this, I know from experience that it can be done and can generate similar if not the exact same output if the corpus, inputs, and weights are exactly the same. I also know that when trying to replicate these networks, a comma in the right or wrong place within the code or algorithm can completely change how your network processes the information you present it.
The main advantage to building a neural network such as these is that a network itself is a replication of human thought and processing. It allows us to predict and observe processing as it would unfold over time and understand how a human learner may learn given a certain input and how that input may be affected by a given set of parameters. So I feel that the main question does not center around how we can go about replicating a neural network, but rather how a neural network can go about replicating us.
The article we read this week tackled dynamical systems and how they might be used to predict the next word in a sentence after hearing a selection of language of a certain length. Dynamical systems are complex, and similar models have been applied to other branches of psychology. In developmental psychology, for example, the dynamic systems theory describes how complex systems become organized, keeping in mind the multiple influences over many levels of complexity (ex. peer relationships, cognitive development, attachment) over immediate and long periods of time. Language development and processing is just one facet of the all-encompassing developmental approach.
ReplyDeleteIf I were to replicate this study, I would alter the sentences used to model human speech for the computers. The examples used in this study do not represent typical speech patterns, in my opinion. Consider sentence 7(b) on page 218: “The man the boy the woman sat heard left.” Another way I would expand the study would be to do as the author suggested and expand to processing paragraphs. This has better generalizability, although it would introduce many variables not dealt with in the present study. Any ideas as to what some of those might be?
The author seems right to focus on temporality and context. The phenomenon of word-prediction (finishing sentences) is one that I engaged with crucially while working on my master's thesis. Predicting the end of sentences is something we do all the time, and take for granted, but it is actually rather complex: it entails memory of previous sentences, of what is currently being said and its perceived similarity to the past, as well as our direction to the future. The model developed in the article does a fairly decent job of simulating these factors - the notion of "state space" seems a bit abstract but may be a rather appropriate analogy - but, as the author points out, language, including this specific phenomenon as well as others, is far more complicated than accounted for by the model. For instance, knowing a person informs what we expect them to say. If a particular friend of mine says "let's go for a..." I assume the next word will be "walk," but if my roommate utters the same sentence, I am confident that the final word will be "beer." The author points out embodiment as an additional factor to consider, with an emphasis on interface with a world. It is important to point out, I think, that "others" are a significant part of the world, especially in community defined activities such as language. In short, not only is context important, but context is also a rich notion itself.
ReplyDeleteThe author also points out, I believe, that "time is the context in which we understand the world." This is remarkably similar to the Phenomenologists' claim that "temporality is the horizon of the understanding of Being," especially since they tie "Being" to the notion of the world. This author may be aware of this claim in Phenomenology, and certainly seems to make near explicit use of David Hume (who was an important forerunner of the Phenomenologists) when claiming that we know about cause and effect by perceiving one thing and then another; Hume hashes this out in terms of "constant conjunction."
There is also an interesting claim early on about object permanence as a factor of time which, I believe, is related to various notions of "apperception."
Methodologically speaking, I am wondering about the significance of inputting only positive instances to the network, i.e., grammatically correct sentences. Language use, for humans, is not - at least developmentally speaking, but perhaps in other ways as well - a matter of grammatical correctness so much as pragmatic trial and error with "correctness" being defined (pragmatically) in terms of what gets accomplished by language. Obviously the system, not being connected to a world except in a reactive and informational sense, would have some difficulty accounting for this kind of worldly pragmatism, although in a sense the systems grammar is its own kind of practicality...
This article is really difficult for me because I know very little about linguistics and computational approach to cognition. I got lost when I read first part of the article. And it finally started to make sense after I read 8.3 the problem of time. Based on my understanding (if I understand correctly), Elman proposed a dynamic approach to understand human natural language processing, specifically the ability to predict the next word in a sentence after being presented with the first word in a sequence.
ReplyDeleteElman started introducing and questioning some controversial assumptions from the traditional approach, and then introduced his suggestion of using a dynamic system when considering computational approach to language processing. When Elman thinking about the shortcomings of traditional theories of language processing, instead of using direct replication or theoretical replication, he proposed another theory to test whether there is a better way to understand language processing in the real world. Because if the existing theories or models are based on the flawed understandings or do not make sense when applied in the real world, it is worthless to replicate them. Therefore, Elman presented trajectories through hidden unite state space to zoom in what is going on when making predictions. In addition, the trajectories also showed how the syntactic constraints, semantic and pragmatic goals, discourse considerations, and processing constraints work together in a more “natural” way before the final output.
Even though Elman did a proper job in emphasizing the importance of time and context in language processing, he also admitted that his work cannot fully explain this process because there are still some limitations to capture the natural semantic relationship with the world. I think it happens to all scientific research, especially social science, because the findings, results, or theories are always not perfect, but every attempt to discover a new theory or replicate previous studies will help people to understand more about the real world.
The goal of this work has been a common one in NLP research: to build a system that takes some partial sentence as input and predicts the next word in the sentence. Having taken classes in NLP, AI, and neural networks, none of this work is really surprising, although this may have been big at the time. It is however the first time I have seen it framed as a way to learn more about how people understand language.
ReplyDeleteI'm curious how this would compare to a more straight forward approach, like building an n-gram probabilistic model. The only advantage I could see to the NN is that it could potentially find higher level patterns (similar to regular expressions) and weigh words/n-grams differently. For example, the sentence "A * B" where * could be replaced with X, Y, Z, or some recursive structure, would each have its own probability using n-grams. But if the nn can find the pattern, then it may just use the 'A' and 'B'. I'd also expect the n-gram approach to fail in a humanistic way as well since it can't predict a combination of words that it hasn't been trained on.
Again, this was probably quite the accomplishment at the time but with the huge advancements in NLP in the 2000s and computers along with researchers releasing corpora consisting of millions of sentences each, I'm a bit underwhelmed. Having read about more modern approaches though, it is entertaining to go back and read something like this!
Although this reading was fairly dense, there were some general ideas which could be extracted and generalized regarding both language and the critical, but sometimes non-explicit, role of theory in cognitive science. Beyond a possible history of psychology course, theory is not discussed very frequently (at least, in my anecdotal undergraduate experience), yet it is crucial to understanding what questions are asked, why they are being asked, and what the underlying assumptions are in the quest for these answers. In spite of the sometimes sparse coverage of the topic, it is crucial in understanding psychology as a science.
ReplyDeleteIn the middle of the 20th century, behaviorism was on its way out after various challenges could not be shaken. While behaviorism was the prominent theoretical view, there was little discussion of cognition, instead, the focus of research was on behavior. This likely was influenced by the power of conditioning in explaining behavior, and the idea that cognition was sloppy and difficult to measure. However, several key experiments and theoretical challenges were able to ultimately shake the foundation of behaviorism by demonstrating that learning could occur without responding and that it could occur without reinforcement. These two, seemingly small, discoveries from rat mazes contributed to the birth of cognitive psychology.
The reading for this week was a presentation of a smaller-scale version of this same idea, and underscores the importance of theory in aiding scientists in determining what questions to ask. The explanations at the time were inadequate in conclusively answering the questions language researchers had hoped to answer, and the incorporation of a series of new ideas (e.g. connectionism, time) led to new hypotheses regarding the processes underlying language comprehension. It is worth noting, however, that this chapter was not the mark of an end of a paradigm, or a complete abandonment of previous concepts; rather, it asked some additional questions, provided initial evidence to support a hypothesis, and laid out the foundations for how language could be better understood, as well as better examined experimentally. By laying out the previous assumptions and empirical evidence and explicitly evaluating them, new ideas could be introduced and tested.
One important caveat needs to be made regarding the importance of theory, however. A great deal of psychological work today is not intended to answer these larger-scale questions. Instead, much work is focused on applied solutions to problems (e.g. improving learning in the classroom, novel treatments for disorders). For this reason, it is understandable that theory is not as emphasized as other areas, such as methods. Indeed, some work may not require much discussion of theory, at all (depending on how one defines it), and it is not my intention to suggest everyone ought to study dense history of psychology texts. I do believe, however, that the conversation should be made explicit, from time to time, between the application and underlying conceptualization of psychological ideas and this text demonstrates that (in spite of the ironically dense and jargon-rich writing in this paper on language processing).
I will begin by saying that this article is incredibly dense. I found it difficult to parse through but also informative. One aspect that I appreciated was that Elman takes a widely-accepted theory and comes at the problem from a different perspective. I think this is critical to the advancement of human knowledge in general to not only approach a problem from a different perspective but to do it in innovative ways. While now this approach is considered old news, it was hugely cutting edge at the time. I also appreciated the fact that at the end of the article it is stated that this approach to natural language processing is not a full model. This section of text acts as a kind of jumping off point for other researchers to take the methodology proposed and tweak it in order to answer some of the glaring questions or criticisms raised. This is a common-place practice, but I thought Elman was ver straightforward about the challenges and criticisms associated with this technique.
ReplyDeleteI think that embodiment is incredibly important in language. I would think the biggest problem with automatizing embodiment is the massive variability in possibilities. This variability is pervasive on multiple levels including cultural and individual levels. One could predict very different things given the contextual circumstances. Prediction in this sense I think would be very difficult because the models do not connect language with the physical world. Therefore, it can generate statements that are implausible or impossible given worldly constraints.
However, in case anyone reading this wants to take a peek at something funny, go to this link: https://twitter.com/deepdrumpf it is a neural network that was trained using Donald Trump transcripts. Enjoy!
This comment has been removed by the author.
ReplyDeleteFrom the outset of this article, I found myself having to re-read sentences several times in order to grasp the meaning, and even then, I felt like I didn’t really have the background to properly understand all of the individual components of the piece. I actually was somewhat familiar with the lexical model of language processing, and it seemed very intuitive, as Elman explained the details of its mechanisms. I think that it’s for this reason that it had remained largely unchallenged and is aptly called the “traditional” perspective—because it seemed to have a cogency and logicality that we probably wouldn’t question.
ReplyDeleteWith regards to word order and word prediction, I thought of a few inquiries that I didn’t feel the article did a good job of putting to rest. Obviously in the English language, things like word order and subject/verb agreement matter—But the degree to which it matters varies widely across languages. Having taken Latin in high school, I think that it would be a good candidate as a language whose words (not necessarily contextually) connote a grammatical usage that is extremely precise. Latin is also a language where word order is much less important, due to this precision of grammatical outlining being attached to the words themselves. This, I would presume, would hugely complicate a measure of predictability, given that the next word could be a constellation of different words and word types.
Another thought that occurred to me would be the analysis of something like bilingualism. Depending on the stage in which you learn the language in life, the “gut feeling” of grammatical correctness that people rely on has a varying intensity. I’m curious how the dynamic systems approach would react and explain language processing with relation to a non-native speaker. The traditional lexicon perspective would probably just suggest that it is an expansion of the terms, but I’m sure such an issue would require a much more thoughtful response from those who subscribe to that model.
I consider the biggest limitation in methodology to be that it focused only on the English language, and also used examples that would be confounded potentially by other languages. For tonal languages (such as Mandarin, for example), I think that the dynamic systems approach probably seems more intuitive—Because context is hugely important, especially for non-native speakers who can’t differentiate the tones as well. I’m also wondering how one might assess practitioners of a nonverbal language, such as American Sign Language-- Which have deletions of several words for concision’s sake?
My final thought on the topic is probably the most troubling, but I guess that I failed at grasping this study’s relevance to replications in cognitive science. It’s likely that I just didn’t conceptually understand the article well enough to see its relationship to replication, but I certainly was trying to keep that in the backburner of my mind as I progressed through the piece.
As a reader with little to no background in the subject, the subject matter was quite complex. While difficult to really assess without much context, the theoretical model of language presented was quite fascinating. I cannot speak much to specifics of testing or conclusions drawn from such (as I have absolutely no background in using networks), but the theory was presented in a straightforward manner. There is a certain feeling of intuitiveness to describing language as a dynamic system.
ReplyDeleteI am left with a number of questions about the appropriateness and generalizability of this theory and methodology. Perhaps this comes from having little experience in the topic, but how would this method of testing hold up in other languages? Also, the training set of data appears very formal. How much can this model be expected to hold up when analyzing language that occurs in more natural settings? While Elman readily admits that the networks do not account for embodiment and that other models face the same criticism, I wonder what could be learned from further analysis in such environments.
I had a lot of trouble reading this paper, but that may be because I know next to nothing about language theory. This reading was a good introduction to language theory, despite my complete lack of prior knowledge on the subject (a sink or swim situation, I think). The figures and trees were over my head, but I think I grasped the gist of his argument, which is that the human brain’s capacity for language is fluid, not a lexicon of words and a list of rules to apply to them. I like this theory, because it speaks to some research I have done myself, which relies on the belief that brain activity is a finite resource.
ReplyDeleteI do wonder if the assumptions he mentions and tries to refute are assumptions I will continue to run into today, over a decade after he wrote this. If/when I take a course studying language theory, should I expect to be taught more in the lexical access and recognition view, with its rules for retrieval and understanding? Or am I more likely to have the dynamic viewpoint taught to me as the most commonly accepted model? I expect that more up to date theories may be a combination of the two.
The theoretical explanation and complexity of the quantitative analysis of the article was much harder for me. Obliviously the topic itself was new for me as well.In this article, Elman focuses on natural language process and it is dynamic system. And according to Elman’s model dynamic does indeed provide a fruitful framework for the research with language cognition.
ReplyDeleteI liked the idea of grammar and lexicon, which is basically the repository of facts that or rules which is can be combined to form sentences. In some model lexicon has been described as passive model. In the active model of lexicon, the internal arrangement of lexicon is less than an issue and because it is content addressable. In some theories lexicon has been described as the recognition has been occurs at the point where a spoken word become uniquely distinguish from others. In lexicon the sentence structure itself has been debated.
On the other side of discussing the Elman’s article, I did not see any clear methodological explanation. I do not know why. This might one of the reason that I can not make enough justification in terms of experimental method.
The other point of this article is it is written in 1995. It could be more interesting if we can have another article which is more recent in this topic.
Overall, I enjoyed reading the article. I would like to see more explanation of lexicon and how really it works in terms of sentence formation. In addition some explanation on more recent study in this field could be wonderful.
In this article, the author suggested dynamical system to process human language, different from traditional computational view which focused on the features of language as discrete and passive unit. The author focused on the ability of predict next word in sentence which involving memory of prior word and understanding of complexity of language. As an attempt to explain language processing with dynamic system, the author adopted Simple Recurrent Network(SRN) which describes processing of sequential input and output. As we all know, language is temporal behavior, so processing of it should regard time pass. Under this fact, it seems plausible that adopt dynamic network to process language since dynamic network successfully provide a field of processing temporal feature combined with language processing. To put dynamical network system under language processing, the author brought new interpretation about lexicon and grammar (language processing rule). In this article, lexicon, different from traditional view that described passive words structure, was described structural state space which allows active performances of words in sentence, which drive certain processor. With this concept of lexicon, processing rule, grammar, flowed as dynamic systems to suggest right pathway to certain direction for correct processing.
ReplyDeleteThis article’s perspective toward language processing, dynamic network, seems make sense and modern to me. Of course, as the author mentioned in this article, still there are many problems to solve with this network model for language process, in my opinion, this dynamic network seems more suitable to language processing. Since human brain is not a Turing machine, processing human language as simplified and symbolized system may miss many complex and elaborated aspects of language processing. In that sense, adopting dynamic network, having more complexities and providing more active and direct space for processing language seems plausible. Moreover, as author mentioned in this article, dynamic system has more wide space to explain polysemy and accommodation of language which makes language processing more complex and rich, and it seems more matched with semantic network than traditional view.
To me, the Elman article presents a couple of unique issues regarding replication. First, I believe the article provides an example of an important point about continued replication in instances where there is no general consensus regarding the foundational elements of a specific topic. In a case such as this, the possibility of perpetuating erroneous claims may be accentuated, particularly when dealing with direct replication. In these instances, conceptual replication provides a far better chance of revealing alternative explanations and underlying mechanisms, though one may still be constrained by overarching assumptions regardless of other differences in theoretical variables and methods. Applied to this week’s article, Elman asserts that the traditional approach to language processing may contain some fundamental flaws, and therefore he presents an entirely re-conceptualized approach to language processing. In this instance, he presents how results discovered through the standard approach may be achieved through his considerably different dynamic systems approach, as well as how some of the gaps in previous approaches may be addressed. To me, this provides a good example of the importance of conceptual replication. It would seem that direct replication would then be particularly helpful in evaluating whether others might use the same systems-based approaches to achieve similar outcomes, though I wonder to what extent direct replication should be sought. Is it a good idea to focus too heavily on direct replication in an area where foundational elements are still prone to substantially change? Further, what is the value of previous direct replication findings when the basic assumptions underlying the results are discovered to be, at least partially, erroneous? A final question that emerged was to what extent the theoretical underpinnings of a research question might be changed while still being considered replication. For instance, if the theoretical perspective changes significantly enough, one may be considering certain outcomes in inherently different ways, possibly moving a certain variable from a focal point to something merely tangential or an artifact of some error in previous thinking.
ReplyDeleteSecond, the article provoked much thought about ecological validity. Throughout the article, I found myself wondering how well the findings obtained through these distinctly non-human processes, operating devoid of real-world context, would relate to real-time, human processes and outcomes. For instance, I found that using only grammatically correct sentences might raise some problems when applied to real human interactions, as virtually all real-time interactions are riddled with grammatical and other such inconsistencies, and yet people may still be able to accurately process and adequately predict following words and possibly even phrases. Further, even the author addresses that the dynamic systems approach does not provide a full model of language use and that one of the most limiting qualities is that the networks presented are merely reactive and do not think. In this regard, I wonder to what extent non-human aspects of research might inform knowledge that may be applied to human functioning. Further, I often wondered how the theory might function in relation to languages that follow drastically different rules of language. In this regard, one might only find basic elements that might be replicated across languages, while the broader systems may need considerable alteration when taking a wide range of nuances and contextual factors into account.
This article was quite a dense read. Even though I have taken a neural networks course, I still had a difficult time understanding a lot of the things that Elman was trying to convey in this article (though neural networks was admittedly not my strong suit!).
ReplyDeleteAs suggested, I tried to only focus on the big picture of the article, which was still challenging for me because grasping the big picture often involved understanding several different details. One of the things that I liked about this article and that stood out to me was seeing how Elman challenged some of the traditional ways of thinking about computational linguistics. He instead laid out his dynamical systems approach (although at times perhaps not as clearly as he possibly could) as a new framework for understanding linguistics, and he proceeded to provide evidenced explanations for why he believed this was a better framework than the framework that most had been following up until that point.
I started trying to think about how this article fits in to this course’s topic of replication. As far as this article goes, it does not seem to focus too heavily on replication of the neural networks studies themselves; however, it instead focuses more on the attempt of the neural networks studies to in a way replicate expected outcomes based on specific theories or to replicate experiments done using human participants. Through training the networks based on certain parameters, one can try to use neural networks to model what a researcher expects occurs during our cognitive processing of language. This serves as a way to test theories about human language processing using the completely different, new method of dynamical systems modeling on a computer. In this way, the process of using neural networks to examine hypotheses about linguistic processing is essentially like a type of conceptual replication.