Let's stop mistaking 'thought experiments' for science

Trends in Cognitive Sciences recently published a provocative letter by a pair of MIT researchers, Ted Gibson and Ev Fedorenko, which has been causing a bit of a stir in the language camps.  The letter - "Weak Quantitative Standards in Linguistic Research" and its companion article - have incited controversy for asserting that much of linguistic research into syntax is little more than - to borrow Dan Jurafsky's unmistakable phrase - a bit of "bathtub theorizing."  (You know, you soak in your bathtub for a couple of hours, reinventing the wheel).  It's a (gently) defiant piece of work: Gibson and Fedorenko are asserting that the methods typically employed in much of linguistic research are not scientific, and that if certain camps of linguists want to be taken seriously, they need to adopt more rigorous methods.

I found the response, by Ray Jackendoff  and Peter Culicover, a little underwhelming, to say the least.  One of the more amusing lines cites William James:

"Subjective judgments," they claim, "are often sufficient for theory development. The great psychologist William James offered few experimental results."

Yes, but so did "the great psychologist" Sigmund Freud, and it's not clear whether he was doing literary theory or "science"...  More trivially, James was one of the pioneers of the fields and didn't have access to the methods we now have at our disposal.  That was his handicap - not ours.

We can contrast that (rather lame) response with what computational linguist Mark Liberman said about corpus research last week in the New York Times:

"The vast and growing archives of digital text and speech, along with new analysis techniques and inexpensive computation, are a modern equivalent of the 17th-century invention of the telescope and microscope."

Here, here, Mr. Liberman.  I couldn't agree more.

Last month, Michael Ramscar and I published a seven-experiment Cognitive Psychology article, which uses careful experimentation and extensive corpus research to make something of a mockery of one piece of "intuitive" linguistic theorizing that has frequently been cited as evidence for innate constraints.  Near the end of the piece, we take up a famous Steve Pinker quote and show how a simple Google search contradicts him.  After roundly (and amusingly) trouncing him, Michael writes - in what must be my favorite line in the whole paper -

"Thought-experiments, by their very nature, run into serious problems when it comes to making hypothesis blind observations, and because of this, their results should be afforded less credence in considering [linguistic] phenomena.”

No doubt, this one-liner owes some credit to a brilliant P.M.S Hacker quote (actually a footnote to one of his papers!):

"Philosophers sometimes engage in what they misleadingly call 'thought-experiments.'  But a thought experiment is no more an experiment than monopoly money is money."

Let's stop mistaking 'thought experiments' for science.

88 responses so far

  • In physics, thought experiments are actually very useful.... Just sayin'. :)

    • melodye says:

      Just to clarify: by thought experiments, I mean an experiment in which the evidence for the claim is furnished only by a theoretical description of how something might work. The 'thought experiment' suggests it 'might logically work in this way' (or, in linguistics, 'this might be more acceptable than that').

      In physics, presumably, you can test the 'results' of your thought experiments - rendering them a source of inspiration, perhaps, but not a source of experimental data. In linguistics, the result of the 'thought experiment' itself is often taken as hard evidence. That's a serious problem, and it's the one that Gibson and Fedorenko take up in their papers.

      Obviously, everyone spends time thinking about how something logically might work - but it's highly misleading to call that an 'experiment,' in the scientific sense. There is a distinction between logical argumentation and empirical testing, and it's one that separates philosophy from science. Of course, this is not to say that philosophy doesn't (or shouldn't) inform science - this is precisely the subject of Hacker's paper. But thought experiments do not and cannot provide us with "scientific" evidence.

      • Rob Knop says:

        I was going to post that, but Jen beat me to it. But, yes, we're talking about two different things. In Physics, a "thought experiment" is a way of working out (possibly surprising) results of a theory or a postulate. The thought experiments themselves may not be tested against experiment, but the theories must be if we are to believe them.

        E.g. one classic thought experiment in Special Relativity shows that if there were contraction _perpendicular_ to the direction of motion, it would not lead to a consistent reality. This doesn't prove or disprove SR, but it does help us figure out the consequences of the two postulates of SR.

    • I haven't gotten all the way through the comment stream yet, but it should be pointed out that linguistic judgments are *not* thought experiments: they are *actual* experiments. That is, you ask a question (are such-and-such constructions grammatical?) and then collect data to answer it. The sample size may be small (as small as 1), but that isn't necessarily a disqualification. A lot of the very best psychophysics employs sample sizes in the range of 1-3, with no harm done.

      One can have a reasonable argument about whether these sample sizes are sufficient for the things we language researchers care about (probably sometimes yes and sometimes no), and one can have a reasonable argument about whether judgments can be contaminated by one's priors (probably!). One can certainly ask whether providing a detailed method section wouldn't be a good idea in linguistics (I think so!). Sprouse & Almeida ask these questions directly, which is very much to their credit.

      But linguist papers are *not* thought experiments. Not that there's anything wrong with a thought experiment, anyway.

  • Alex Clark says:

    What do you think of the recent paper on lingbuzz by Jon Sprouse and Diogo Almeida?

    http://ling.auf.net/lingBuzz/001195

    I think there's more to say about the validity of the classic linguistic introspective judgement.

    • melodye says:

      Thanks for sharing this, Alex. I haven't had a chance to read it in any detail yet, but I gave it a quick first pass. Do you know if they share their experimental stimuli anywhere? It's somewhat difficult to understand exactly what they did without it, and (surprisingly) their methods section has no examples.

      I would encourage you to take a look at our paper, as well. Although the topic is not directly related, the paper does call into question how much insight we can really gain from acceptability judgments, without first taking into consideration word (or phrase) frequency. We propose that acceptability ratings reflect our sensitivity to linguistic conventions as a result of learning, and give evidence for this in noun compounding.

      To put it differently - it may be that linguists' intuitions about acceptability judgments will turn out to be well supported experimentally (where the extent of the experiment is - as in Sprouse & Almeida - simply a question of "how acceptable is this relative to baseline?"). However, that's still not telling us much about what those 'acceptability judgments' are reflecting in terms of language processing. Generative linguists typically want to make mechanistic claims - saying e.g., that these judgments reflect a kind of native constraint, or are evidence for a universal grammar. As we illustrate in our paper, that leap is absolutely unwarranted.

      • Colin Phillips says:

        There's nothing mysterious about Sprouse & Almeida's materials. They tested (roughly) all of the example sentences from an easily available syntax textbook. This was done in order to test the allegation that linguists' theories are built upon fake data.

        The "debate" over the reliability of intuitive judgments is a distraction from the real problems that we all need to solve about language. It seems to be doing a fine job of reinforcing disciplinary prejudices, while boosting citation counts for a few participants. Large scale judgment studies generally show us that obvious contrast are obvious, and subtle contrasts are subtle. Some phenomena turn out to be highly lexically dependent, and other are not. All very well, but not exactly news. The real action lies in making sense of the systematicity. The mud-slinging is not new, and it is not getting us very far.

        • melodye says:

          Colin, it's not that I think they're mysterious; I just want to see what they are, exactly. I want to know which ones replicated and which didn't, and what I can learn from it. I wouldn't presume to debate a paper when I wasn't absolutely sure about what they were testing.

          After you posted, I took a look at your 2007 paper on this topic: "Should we impeach armchair linguists?" Your claim is that -

          In order for there to be a crisis... it would need to be the case that (i) Intuitive judgments have led to generalizations that are widely accepted yet bogus. (ii) Misleading judgments form the basis of important theoretical claims or debates. (iii) Carefully controlled judgment studies would solve these problems. Although I sympathize with the complaint that one can find many cases of poor data collection in the linguistics literature, I am not sure that any of (i)-(iii) is correct.

          I think that our paper on 'level ordering' is illustrative of a case in which (i) and (ii) are true. However, I don't think that (iii) needs to be the case, because I don't think judgment studies are necessarily all that helpful, particularly in comparison to careful corpus work.

          One of the surprising findings we present in our recent paper (see Exp. 7) is that regular plurals that end in sibilants behave like irregular plurals distributionally. This finding is not something we could have discovered via intuition; indeed, it goes against the grain of most of our intuitions about how regulars and irregulars behave. However, it is something that makes sense of our (and others) experimental data, and it comes straight out of a corpus study. The idea that, at times, our intuitions may coincide well with our findings is fine; the problem, as Ted and Ev point out, is that without doing the quantitative legwork, we have no idea when they've led us astray. How else are we to 'make sense of the systematicity'?

          See also: Baayen and Moscoso del Prado Martin (2005) for a similar unraveling of intuition.

  • Marcus says:

    I am not a language researcher, but find all of this stuff fascinating, as similar discussions go on in my area. Isn't the core of the issue here a mismatch between the types of methods used and the types of questions that can be answered with them. Seems like the issue with linguistics is that they overreach what types of questions their methods can answer?

    I have had some interactions with philosophers who did honestly think that thought experiments were just as good as real experiments. That a very eloquent idea of how something might work was the end, and data confirming that was an unnecessary add on. It's very strange to listen to someone essentially describe what would be a great experiment and then dismiss actually doing it as a waste of time.

  • becca says:

    Thought experiments have a critical place in science- on the edges where we don't yet have the technology to test things experimentally.

    • melodye says:

      Here, again, I would suggest the word 'philosophy' or even 'speculation,' but not the word 'experiment.' I like how Hacker distinguishes between philosophy and science, and I think the distinction is relevant here:

      "Scientists seek to understand why the phenomena they investigate are as they are and behave as they behave. They do so by way of empirical explanation, which may take various forms, e.g. hypothetico-deductive, inference to the best explanation, or explanation by reference to intervening mechanisms. All these are subject to empirical confirmation or refutation. To that extent it is
      misleading to suggest that philosophy seeks not for knowledge of new facts but for an understanding of familiar facts – as if science did not satisfy that need. Philosophy cannot explain phenomena in that sense at all. So whatever its quest for understanding is, it is not akin to the understanding achieved by the empirical sciences.

      Nevertheless, philosophy can contribute in a unique and distinctive way to understanding in he natural sciences and mathematics. It can clarify their conceptual features, and restrain their tendency to transgress the bounds of sense. ...[Philosophical] questions are answered by conceptual investigations, not by observation and experiment."

  • James Davis says:

    Yes, I totally agree. Linguistics is a bit shaky from time to time.

    So, are you willing to part with a large chunk of your physics/biology/chemistry/math funding to allow linguists to expand our corpora more effectively, to go to the places in the world we need to document better, and to set up the kind of careful and rigorous experiments we need in order to really get to the heart of the matter? It's very hard to isolate things in language. You do an experiment and then, while looking at all the data, realize it's a mere tangent to what you needed to know.

    See, this is the problem I see with most of this. Linguistics is trying hard to understand something scientifically while being constantly viewed at many universities as one of the humanities and getting humanities funding. Trust me, if you think experimental research is hard, try doing it while the English department is the one deciding your tenure… they don't exactly encourage---or even like---the scientific method.

    • melodye says:

      I know that not all languages have adequate corpora yet, but the Contemporary Corpus of American English and the British National Corpus are pretty fantastic...

      • American Indian Languages says:

        Two corpora of English do almost nothing for a theory of language in general. That amounts to about 1/6000th of what we need.

    • Jeremy Kahn says:

      It seems a little combative -- not to say unfair -- to point the finger at Melodye to say "*your* physics/biology... funding", as if Melodye (or even Gibson & Federenko) were playing for some other team.

      Linguistics is underfunded; there's no doubt. But its funding is about at par with the departments that also use this sort of argumentation -- departments of English literature or Philosophy don't have much funding. Neither do departments of mathematics -- so it's not a question of "hardness" or physics envy -- it's that linguists, well, many of us, have been making the case since Chomsky that we're amazing *because* we don't need data (I can "consult my native-speaker's intuition" comfortably from my cramped, underventilated office). University and research institutions have responded accordingly ("you'll work for peanuts? AWESOME. Peanuts it is!").

      Those of us who say "empiricism -- you keep using that word, but I do not think it means what you think it means" end up jumping ship just to find people who will fight for funding for data and resources elsewhere. Field linguists go to anthropology (or missionaries), phonetics and psycholinguists to psychology and neuroscience departments, corpus & mathematical linguists to computer science, or electrical engineering (or dot-coms, like myself).

      *Part* of the answer might be for linguistics departments to more firmly identify themselves as allied with those departments that *have* made the case for more experimental and (small-e) empirical research: biology, neuropsychology, and engineering, for example.

      If we (as linguists) can explain better why we need the funding for data, we're doing just that -- identifying ourselves as scientists, not black (white?)board philosophers. But if we keep claiming we're so awesome at spinning blackboard straw into gold, we're going to keep getting blackboards and straw.

  • Marc Hersch says:

    All experiments are thought experiments and because all thought is bounded by the structure and process of language, including those seemingly objective concepts that we operationalize in terms of measurement, an explanation of language and the construction of meaning, makes the problems of physics trivial by comparison.

    As physicist S. Hawking noted in the concluding chapter of The Grand Design,

    “We form mental concepts of our home, trees, other people, the electricity that flows from wall sockets, atoms, molecules, and other universes. These mental concepts are the only reality we can know. There is no model-independent test of reality.”

    AND

    “We seem to be at a critical point in the history of science, in which we must alter our conception of goals of what makes a physical theory acceptable. It appears that the fundamental numbers, and even the form, of the apparent laws of nature are not demanded by logic or physical principle. The parameters are free to take on many values and the laws to take on any form that leads to a self-consistent mathematical theory, and they do take on different values and different forms in different universes.”

    A model is a theory. A theory is narrative. Narrative is language.

    The "critical point in the history of science" is that it must cease putting the cart before the horse. The numbers have no intrinsic meaning. The horse is our intention that pulls the our scientific inquiry forward. Understanding the construction of meaning through narrative is the first step toward the viable theory of the process of language. The numbers are always embedded in narrative.

    To give up thought experiments is to give up thought and if you think about it, that is exactly what has been happening among those who think that numbers hold the answers.

  • Marc Hersch says:

    Allow me to add a note regarding model dependent reality and the study of language.

    If our thought experiment (taken for granted) is that language carries meaning, we will categorize, count, measure, and plot the parts of language in order to determine how meaning is conveyed. When we do this we will obtain findings and those findings will be interpreted as meaningful foreground (assignable cause) against the background field of our model.

    If, on other hand, we adopt a model that asserts that language produces meaning rather than coneys it, the reduction of language to its constituent parts will be of only passing interest. Instead we will ask how the process of language behavior gives rise to meaning.

    If we take for granted, the thought experiment that underlies our inquiry we are stuck in that model, and with diligent effort we will find supporting evidence. Another model will also produce supporting evidence, but first we must engage in the intention driven thought experiments that can brings us other useful models.

  • daedalus2u says:

    I think that neither of those models is correct. Language neither generates meaning nor conveys meaning.

    Meaning is only generated when neural networks instantiate a mental concept. The meaning is in the instantiation of that mental concept. The only things that can be communicated are mental concepts.

    Language is a data stream that humans use to try and convey a mental concept by translating the mental concept into the data stream of language. That data stream is transmitted and then up-converted to a mental concept (or not) in the neural network of the person being communicated to.

    For meaning to be conveyed, the mental concept instantiated in one person's neural network needs to have a one-to-one mapping to the mental concept instantiated in the second person's neural network.

    The system that converts the mental concept into the data stream of language and back, is what I call a “theory of mind”. That “theory of mind” has to be shared between the two individuals that are trying to communicate. They must also share neural networks that can instantiate the same mental concept that they are trying to communicate. If the neural network of one can't instantiate the mental concept, then that concept cannot be conveyed or even thought about. Appreciating that one is unable to think about a certain concept is quite difficult.

    I have blogged about this.

    http://daedalus2u.blogspot.com/2010/03/physiology-behind-xenophobia.html

    The “theory of mind” that people use are not all identical. They can be pretty similar between people who know each other very well. Identical twins have very similar “theories of mind”. People who have never met and don't speak the same language don't.

    The “theory of mind” is used for data compression. The bandwidth of speech is insufficient to transmit the “data” needed to instantiate the mental concept being communicated without gigantic amounts of data compression. That is what a shared “theory of mind” accomplishes, it allows for the communication of very complex ideas with a puny data stream. The “theory of mind” is the communication protocols that are shared (or not) that allows for that communication.

  • Diogo Almeida says:

    I think you are completely missing the mark here when you characterize what linguists do as "thought experiments". This is simply false, even by the very definition that you seem to be using:

    "by thought experiments, I mean an experiment in which the evidence for the claim is furnished only by a theoretical description of how something might work. The ‘thought experiment’ suggests it ‘might logically work in this way’ (or, in linguistics, ‘this might be more acceptable than that’)."

    What linguists do is not a "thought experiment" in this sense at all. It is, by any reasonable definition of the experimental method, a real experiment. The linguist starts out with a hypothesis about a specific grammar (or sets of grammars), deduces testable predictions from the hypothesis and then proceeds to test these predictions on real language data. When a linguist presents something like (a-b) below, she is presenting a summary of real data, collected from a real experiment:

    (a) What do you think that John bought?
    (b) *What do you wonder whether John bought?

    This basically means that the linguist, after testing some tokens of the relevant sentence-types, either solely on herself or on some informants, found out that the sentence-type exemplified by (a) is more acceptable than the sentence-type exemplified by (b). This is not a claim about how "it might logically work", it is a claim that "for the population under consideration, it is how it works".

    Whether this means that the initial theory, the one that motivated the experiment, is the only correct one is, of course, entirely up for grabs, but that will be the case in any kind of empirical science.

    Ted Gibson, Ev Fedorenko, Jon Sprouse and myself have debate this and other topics related to G&F's TICS and LCP articles on the TalkingBrains.org blog, if you are interested: http://www.talkingbrains.org/2010/06/weak-quantitative-standards-in.html

    • melodye says:

      Thanks for the link to the debate Diogo. I think my position on this is well-summarized by Ted & Ev:

      "We think it is misleading to refer to quantitative evaluations of claims from the syntax literature as "replications" or "replication failures." A replication presupposes the existence of a prior quantitative evaluation (an experiment, a corpus result, etc., i.e. data evaluating the theoretical claim). The claims from the syntax/semantics literature have mostly not been evaluated in a rigorous way. So it's misleading to talk about a "replication failure" when there was only an observation to start with."

      I simply do not agree with your statement that -

      "There is no "extra" rigour that comes from being able to run inferential statistics beyond what you get from thoughtfully evaluating theories, and systematically investigating the data that bear upon them (in the case of linguistics, through repeated single subject informal experiments that any native speaker can run)."

      What do you mean by 'systematic investigation of the data,' in any case? Absent corpus data, I'm not sure how your investigations can be 'systematic,' when what you're doing is asking a single native speaker whether something is acceptable or not. Furthermore, in any instance where you ask yourself or another linguist, the 'experiment' is not conducted blind. The 'population under consideration' is thus not a valid test population, at least by the normal standards of what is meant by a scientific test. As Ted & Ev say, this is an observation that might lead to a plausible experiment, not an experiment in itself.

      In the lab I worked in at Stanford, I worked with Michael (a British speaker), Justine (a native of Hong Kong), and Ariel (a native of Ohio) and all of us had different judgments about certain turns of phrase. Moreover, Justine did research actually showing how readers diverge in their sensitivities to the distributional patterns in language, based on their exposure. Joan Bresnan and Marilyn Ford published like-minded work last year in Language. Work in this vein is illustrative of why individual judgment does not (necessarily) furnish us with good evidence about what we're testing.

      Corpus research is also a powerful tool for testing our intuitions against a large tract of linguistic data. As I mention in my comment to Colin above, both Ramscar & Dye (2011) and Baayen & Moscoso del Prado Martin (2005) include systematic corpus analyses that inveigh against intuition. In our paper we find that regular plurals act like irregulars when they end in sibilants; in B&M's paper they present a similarly counterintuitive finding for denominalization.

      • Diogo Almeida says:

        Hi Melody,

        Here's a little illustration of what I mean by "systematic investigation of the data". Assume that Hypothesis H predicts contrasts X, Y and Z, and you test a number of sentence-types that illustrate these contrasts. Contrasts X and Z were really easy to get, but assume that contrast Y didn't pan out as you expected. This forces you to take a closer look at contrast Y and you realize that there could be a confound influencing the results. Then you test another set of sentence types illustrating contrast Y', where the putative confound was eliminated and you see whether the results changed, and they do. Now that all the predictions of Hypothesis H were experimentally confirmed, you decide to show these results to your colleagues, and one of them suggests that Hypothesis H' actually provides a more parsimonious account for the data. You think about it for a while and you deduce the predictions made by Hypothesis H', and then you go and test them. And this goes on and on and on.

        What part of any of this fails to conform with the scientific method?

        • Jeremy Kahn says:

          The procedural mechanism you describe actually *does not* conform to the scientific process.

          "contrast Y didn't pan out... could be a confound influencing the results": Why is "confounding factor" only considered post hoc, and only for contrast Y? Contrast X and Z should also be revisited if a confound is discovered, but this sort of consideration is often (in linguistics) not considered. Data-driven methodologies tend to be much more robust, if only because (for example) computers can revisit ALL the data (contrasts X and Z as well!) with the confound included in the model.

          Failing to revisit X and Z is not merely an oversight in explanation -- it actually IS the way much linguistics argumentation (post-Chomsky) is done. The usually-accepted argument for binary branching of syntactic trees, for example, has to do with computational efficiency arguments, but those arguments really should be thrown out in light of our (post-1959) neurobiological understanding of the neocortex's capacity for massively-parallel computation (not to mention, paraleiptically, that we now have MUCH more sophisticated understanding of computational requirements for semi-supervised learning, post-1998).

          • Diogo Almeida says:

            Hi Jeremy,

            under the scenario that I proposed, the confound was supposed to affect only contrast Y, that is why only contrast Y was revisited. If the confound affected also X and Z, then sure, one needs to revisit those as well.

            So, now that this is clarified, what in this scenario fails to conform to the scientific method?

          • Jeremy Kahn says:

            Diogo: [replying to myself to avoid exceeding max comment depth]

            If you're only *looking* for confounds that affect things that don't fit your hypothesis, you're a priori biased only to find such confounds.

            One should look for confounds (even new ones ones!) on X and Z *as well*. Otherwise your method has confirmation bias: you only look harder at results that *don't* fit your hypotheses. The occasionally-obnoxious, but scientifically valid, approach is to ask "well, what *else* could explain my data *besides* hypothesis H".

            In practice, confirmatory data should get an even *closer* look than disconfirmatory data, to combat our own understandable desire to have our preferred hypothesis be the right answer.

            Engineers seem to get this -- they dogmatically and religiously separate training from test data in order to make sure that the systems they're building generalize properly (though they say "works well in the field" or "avoids overfitting" or occasionally even "doesn't cheat").

          • Diogo Almeida says:

            Hi Jeremy,

            And what you just mentioned (ie, asking the question "what else could explain the data?"), isn't it exactly in my scenario what happens here:

            "Now that all the predictions of Hypothesis H were experimentally confirmed, you decide to show these results to your colleagues, and one of them suggests that Hypothesis H’ actually provides a more parsimonious account for the data. You think about it for a while and you deduce the predictions made by Hypothesis H’, and then you go and test them."

          • Jeremy Kahn says:

            Diogo:

            The confirmation bias problem still holds. Revisiting previous results in light of a change to the theoretical paradigm is called "regression testing" in engineering, and is closely related to "replication" in scientific methodology.

            Counting on other researchers to propose alternate explanations (rather than being accountable to changes in your own theory's implications when changing your theory) tends to push the problem to social assertiveness (or reputational clout), which is not science but Bonobo-style monkey-troop status games.

            By playing reputation games instead of being clear about the terms of falsifiability (not to mention testing the implications of theory revisions against available evidence), we're really stomping our own toes in linguistics, especially in the effort to get taken seriously as a science.

          • Diogo Almeida says:

            Hi Jeremy,

            I think we are talking completely past each other here. I frankly don't see what you are objecting to. For instance, you seem to object to the fact that in my toy scenario, it was another researcher that proposed a different account for the same data that I was investigating. But that is completely inconsequential; it could have been me after giving the matter more thought. The point I was trying to make was simply that at some point alternative accounts will be proposed and then explored, and it really doesn't matter by whom. I still don't see what exactly is unscientific about this. If I understand your objection (and I don't think I do), you seem to be suggesting that every time you come up with a theory, you are also responsible for coming up with *every possible alternative theory* that would account for the same data. Your mention of "regression testing" is also a little puzzling: theory construction and product design are fundamentally different things, but you seem to be talking about the latter as a good model for the former.

            So since the example has not really helped in clarifying your position to me, I guess I will just come out and ask it: What exactly is your definition of science, such that it excludes generative linguistics?

  • Ty-bo says:

    Funny, I've never heard (to my memory) the phrase 'mice-infested', although often hear 'mouse-infested'. Likewise, it seems from memory that I've heard the related phrase 'mouse infestation' more frequently, as I'm wondering if I've ever heard the phrase 'mice infestation'.

    I might add that this isn't a totally infrequent topic of speech, either, since a friend and neighbor works in the pest control business.

    Pinker really did take the sloppy/lazy way out and assume that that his preference in speech was standard, it would seem here.

  • Karthik Durvasula says:

    Reinventing the wheel...part 1:
    This discusion about "acceptability" and grammaticality is really old hat for linguists. It was discussed and acknowledged at least about 46 years back. That one needs to be very careful in interpreting acceptability judgments is known to any respectable linguist.

    Here are the relevant quotes in print:
    (1) On the idea that acceptability and grammaticality/grammaticalness aren't the same thing:
    "The notion 'acceptable' is not to be confused with 'grammatical'. Acceptability is a concept that belongs to the study of performance, whereas grammaticalness belongs to the study of competence...but the scales of grammaticalness and acceptability do not coincide. Grammaticalness is only one of many factors that interact to determine acceptability." (Chomsky 1965, pg. 11)

    (2) On Performance issues:
    "The more acceptable sentences are those that are more likely to be produced, more easily understood, less clumsy, and in some sense more natural....The acceptable grammatical sentences often cannot be used, for reasons having to do, not with grammar, but rather with memory limitations, ... stylistic factors, ...and so on." (Chomsky 1965, pg. 11)

    In short, no respectable linguist is going to immediately infer ungrammaticality from a sentences that are unacceptable. It has to be shown that there is a consistent pattern to the "madness", and that the consistent pattern can be grammatically described. It is true that sometimes this will turn out to be a mischaracterisation, and the pattern is better accounted for by other (non-grammatical) factors. Syntactic work in the last few decades has argued for at least some such cases.

    In the interest of full-disclosure, it is also true that there are some bad linguistics papers out there, that don't actually do this. But, this is to be expected of any science...there will be at least a few papers, which have notions that are inconsistent with the majority opinion. If this is to be the basis of a criticism from outside the field, then I am sorry, but it is pretty irresponsible of the said critics.

    Reinventing the wheel...part 2:
    (3) "...to make something of a mockery of one piece of “intuitive” linguistic theorizing that has frequently been cited as evidence for innate constraints..."

    Pinker bases his rather strong but ungainly claims based on discussions between morphologists in the 1980s. A simple google search will also tell you that many linguists have shown really problems with the generalisation that Pinker adopts, by the end of the 80s. (Your claims about the accepted innateness of any idea are based not on the primary literature on the matter, but from an outsider. This is irresponsible.)

    Finally:
    And perhaps most importantly, this rather amusing straw-man of a linguist that many outside the field have constructed is as hilarious to linguists as he is to the outsiders because nothing could be further from the truth.

    Also, all everyone's talked about till now is "syntax"...hopefully those that are going to argue that "linguistic methods" are ridiculous will also pay attention to the work done in phonology, phonetics, semantics, pragmatics...

    All I see in these "arguments" against linguistic methodology is rhetoric. The arguments don't seem to be informed about the little fact that the field just doesn't work the way that it's been characterised to do so.

    I do want to say, constructing ridiculous strawmen is something that only people not familiar with an idea or people with a preset agenda do. It might be an excellent rhetorical strategy and might win debates, but I will let you decide if that is really a scientific basis for exploring anything.

    • melodye says:

      Sigh. This is a blog post, for goodness sakes, not a dissertation.

      First, the paper we just published is not primarily about Pinker; it's about level ordering, which was first proposed by Kiparsky in the 1980's and has gone through iterative enumerations, through the late 1990's (see our literature review, pp. 2-6). The major thrust of our critique is leveled at Gordon (1985) and Alegre & Gordon (1996). You, I might add, could have deduced as much from a simple reading of our paper... ;)

      Second, if you really want some adorable Chomsky quotes, I have a "top of the pops" listing here. I have always thought the competence / performance distinction was a crafty way for Chomsky to deny the validity of corpus data, thereby making his own claims largely unfalsifiable.

      The deeper point one might make, is that the Chomskian approach to language is not obviously scientific. Historically, it consisted largely in showing why certain (e.g., probabilistic or predictive) approaches to language 'logically' couldn't work, using strawmen models, and then proposed a framework for how it might otherwise work: an innate generative grammar. To prop up this system, Chomsky then laid the ground-rules for fifty years of pseudoscientific 'experimentation,' which is not - according to usual scientific standards, actually experimentation at all - while craftily disbarring the use of other, more obviously scientific methods (e.g., corpora).

      I'm not saying that Chomsky's proposal was wrong, or that nothing good has come of the last fifty years of linguistic research into, e.g., generative grammar. What I am saying is that there's something highly suspect about the foundations of the enterprise, and that perhaps we should rethink that going forwards.

      Is this: mudslinging or deep-seated frustration? Clever rhetorical strategy or political suicide? Hard to say, really.

      You do get points for calling Pinker a "rather amusing strawman of a linguist." I've also edited the post to make it clearly about syntax/semantics, in general, and generative grammar, in particular. I think there's amazing work being done today in, e.g., computational linguistics, natural language processing, corpus linguistics, historical linguistics, sociolinguistics, comparative linguistics, and so on and so forth. It's critical to understand that I am not trying to dismiss linguistics, as such, and neither are any of the people whose papers I cite. The message is: it's high time to adopt new methods, so we can all start speaking the same language.

      • Karthik Durvasula says:

        It is true that occasionally you are going to find new results that find fault with the previous findings, and science depends on this. This can be shown even for what you imply to be rigorous experimental techniques. If one were to use the same argument that you are using, then it would lead to what would be considered a rather ridiculous position that rigorous experimental techniques are to be abandoned.

        No one here is arguing that all the data that floats around is accurate. All the above comments have said as much. And you are going to find cases where it is later found to be incorrect (connection to p-values should be obvious, here). But, this doesn't invalidate the approach.

        Your fundamental argument boiled down to its essence, as I see it, is that you don't "believe" the standard linguistic methods are good enough.

        As for your psychologising on Chomsky's deeper reasons for doing what he did, it is sort of silly, and is obviously unscientific.

        And the "competence / performance distinction" is an idealisation, and has been claimed to be so repeatedly. Is it unfalsifiable? Perhaps, but no more than the strong belief that the universe is systematic in the natural sciences. Falsifiability is a condition on scientific experimentation, not on scientific approaches.

        If you were distinctly aware of the Chomsky quotes, then I don't see how you can make the claims you did in the main text. Clearly, knowing a quote is quite different from understanding them.

        The claims that follow from the actual paper (primarily questioning the generalisation) are far more narrow than the rhetoric that is being used to discuss it.

      • William Idsardi says:

        Kiparsky 1982 (http://phonology.cogsci.udel.edu/~heinz/classes/607-F08/readings/kiparsky1982.pdf) was not the first to propose level-ordering, as the first paragraph of Kiparsky's chapter clearly states. It was Dorothy (Muffy) Siegel who first proposed this, in her 1974 MIT dissertation (http://dspace.mit.edu/handle/1721.1/13022), which was a reinterpretation of the affix typology (+, #, =; see p.26 of Siegel's dissertation) in Chomsky & Halle 1968 (http://www.amazon.com/Sound-Pattern-English-Noam-Chomsky/dp/026253097X). See Fabb 1988 (http://www.springerlink.com/content/x30615527pk78652/) for criticism and an approach based on selectional restrictions between affixes. I'm not sure why Siegel and Fabb aren't included in your literature review, nor why the indispensable Marchand 1969 (http://www.worldcat.org/title/categories-and-types-of-present-day-english-word-formation-a-synchronic-diachronic-approach/oclc/000032957) was apparently not consulted.

      • Jeremy Kahn says:

        Historically, it consisted largely in showing why certain (e.g., probabilistic or predictive) approaches to language ‘logically’ couldn’t work, using strawmen models, and then proposed a framework for how it might otherwise work: an innate generative grammar.

        a lovely summary, Melodye. And most of the "logically couldn't work" argumentation is just the sort of thing at which I'm vigorously waving my tiny little fists elsewhere in this thread.

        Chomsky's proposals were radical and groundbreaking (in linguistics -- let's talk socio-media-politics some other time). Some pieces of it -- e.g. the Chomsky hierarchy -- broke tremendous ground in the ability to mathematically reason about the descriptive power of symbol strings, which revolutionized compiler design and parts of computer science.

        But those arguments for innate language faculties are mostly arguments from ignorance -- and we are no longer so ignorant. It turns out that general purpose computing can do *all sorts* of things we -- the community of linguists -- didn't think were possible in the 1960s, that are actually possible now.

    • Aviad says:

      I'm a little bit late into the game, and so perhaps will go unnoticed, but I felt the need to point out how far from reality your characterization of some aspects of linguistic research is. For example, I have no idea on what you base your claim that “no respectable linguist is going to immediately infer ungrammaticality from a sentences that are unacceptable”. Chomsky's quotes are the reflection of some ideal which even he himself does not follow. I'd be happy to be corrected, but from my years of experience in the field I have never witnessed a syntactician do anything but assume that whatever pattern he finds reflects ungrammaticality (i.e. a syntactic problem). In fact, having pointed out to syntacticians that some pattern or other depends on contextual factors (and perhaps is not syntactic at all), I've gotten more than once a response along the lines of "but I'm a syntactician". Frankly, this kind of a statement is a testament to the sorry state of the field.

      • Diogo Almeida says:

        Since, for whatever reason, this supposed "debate" about linguistic methodology is such an emotionally charged issue, here's a suggestion to make it more productive and less vitriolic: Why don't we avoid talking solely in anecdotal evidence (eg., "I have never witnessed a syntactician do anything but assume that whatever pattern he finds reflects ungrammaticality") and start talking in observations we can all independently evaluate and discuss?

        For instance, when you claim that the field of syntax is in a "sorry state", what exactly do you mean by it, and, in case you have any data to substantiate your claim, what would it be?

  • AK says:

    I started reading the paper you linked, then switched to scanning due to time constraints. I have two major comments:

    The entire discussion of inflection revolves around plurals, presumably because plurals are just about the only sort of inflection English retains. However, German evidently retains nominative, genitive, dative, and accusative. I saw no mention of work on this language (or any other than English, unless the Scots children in experiment 6 were native Gaelic speakers who had learned English as a second language, which I doubt). It would be interesting to discover whether the same results apply to a more noun-inflected language, and especially if case behaves the same way as numbers.

    The other item involves neurology. If there are different mechanisms involved in processing regular vs. irregular forms, there's a good chance different regions of the brain would show up in PET scans or fMRI. Obviously if there were no differences it wouldn't prove anything (absence of evidence isn't evidence of absence), but certainly if there were differences it would constitute a starting point for whole new rounds of theorizing.

    An example of the sort of thing I'm talking about is "Dissociating neural subsystems for grammar by contrasting word order and inflection" by Aaron J. Newman, Ted Supalla, Peter Hauser, Elissa L. Newport, and Daphne Bavelier, Proceedings of the National Academy of Sciences, 107 (16), 7539-7544 DOI: 10.1073/pnas.1003174107. (A non-paywall version appears to be at http://www.bcs.rochester.edu/people/newport/pdf/Newman-etal_PNAS10.pdf as of this writing.)

    • melodye says:

      AK - thanks for commenting and sorry your comment didn't appear immediately. The spam filter for Scientopia often incorrectly flags comments that contain multiple links.

      As you noted, we tested English here, but German would also be interesting to look at. The lead author on the paper, Michael Ramscar, spent some of his childhood in Berlin and is proficient in German - he's currently working on an extensive comparative corpus study of English, German and Spanish (although, in this case, looking at entropy management over nouns).

      I haven't yet had a chance to read Newman et al., but I would encourage you to also take a look at Haarald and Fermin's paper, which I reference above - it seems relevant. Here is a brief synopsis of it, which I wrote up as part of a literature review:

      First, it is worth touching on some work done by Baayen and Moscoso del Prado Martin (2005), which thoroughly and meticulously examined the merits of Kim et al’s claim, from the vantage point of lexical statistics. The pair set out to determine whether there were systematic semantic differences between regular and irregular verbs, and concluded that there is “a conspiracy of subtle probabilistic (graded) semantic distributional properties that lead to irregulars having somewhat different semantic properties compared to regulars” (p. 669). Specifically, they found that irregular verbs tend to have more meanings than regulars (i.e., greater ‘semantic density’) and that irregular verbs tend to cluster in semantic neighborhoods (i.e., a higher proportion of their closest synonyms tend to also be irregular). They also found that this occurs on a graded scale, with large subclasses of irregulars behaving more like regulars, as compared to smaller, more idiosyncratic subclasses.

      With these results to hand, Baayen and Moscoso del Prado Martin found that a wide range of empirical findings suggesting ‘dissociations’ between regulars and irregulars could be accounted for in terms of their respective distributional and semantic properties, without recourse to any putative grammatical rule. (Their analysis ranged from studies involving association norms and word-naming latencies, to those involving familiarity ratings and neuroimaging). The force of their work is clear: contrary to Kim et al.’s claim, meaning and form are interrelated, if perhaps not in any simple, collapsible, or deterministic manner. As Huang and Pinker acknowledge, these findings are “consistent with the [general] idea that people tend to generalize inflectional patterns among verbs with similar meanings.”

      In other words, when we find dissociations, they don't always mean what we think they do.

      • AK says:

        Thanks for your reply, Melodye.

        I've scanned the paper you mentioned, with special attention to the meta-analyses of the PET and fMRI studies (Jaeger et al. (1996) and Beretta et al. (2003)). My immediate reaction is that, while this may be a necessary lower-budget alternative to original research, it's far inferior to new studies that use word lists matched for inflectional entropy and semantic density.

        A study duplicating Jaeger et al. (1996) and/or Beretta et al. (2003) but with word lists (and other conditions if necessary) modified for equal inflectional entropy and semantic density might well be feasible. It would probably involve a smaller word lists (so as to match characteristics), and should therefore probably be done with a larger number of subjects. (I'm guessing here, as I'm not really familiar with the subject.) If both latency times and brain region activation turned out to be pretty much the same, I'd guess it would be enough of a paradigm breaker to get published pretty much anywhere (although you'd probably have to make sure all the i's were crossed and t's dotted before submitting it to hostile reviewers' scalpels). And if there turned out to be significant differences, perhaps you'd have to modify your own opposition to the idea of innate UG.

        The major problem I see with both models is the assumption that there is only one "processing route" used in a specific language operation.

        From my own (admittedly amateur) digging into the neurological literature (as well as general complexity theory), the model I find most persuasive is one of a large number of more or less specialized modules, with some number (usually >1) working in parallel in any operation. I would expect a hierarchy of circuits with different levels of detail and processing speed, with a single most-detailed circuit at the top of any process, with progressively "faster and sloppier" circuits offering progressively lower-probability predictions of the outcome of the top circuit. Different down-stream processes would be hooked to the different predictor circuits in ways depending on experience (both childhood and recent) as well as genetics and perhaps random differences in prenatal (or even postnatal) development.

        In this case, I would guess that the longest latency time would occur when the predictors "fail"; that is they predict, and set the stage in downstream processing for, a different outcome of the top-level detailed circuit than actually occurs. In this case, downstream processes would have to "back up and start over".

        WRT the idea of two processors, I've been playing around with the idea of two co-evolving systems, one based on memory and one on agglutinative/inflective rules. The language paradigm within any small band would be expanded through rule-based productivity by individuals with better rules modules, while people with less effective rules modules would have to depend on superior memory, or else suffer from strong competitive disadvantages, leading to selective improvement in both modules.

        In that regard, let me point out that mouse/mice and, indeed, most "irregular" forms are actually samples of alternative regular genders with small samples (e.g. louse/lice). I can't think off-hand of any English word whose plural isn't some rule-based modification of the root. (Baayen and Moscoso del Prado Martin don't list any in their examples on page 3.) I suppose person/people might do, although there is a perfectly good regular "persons" and people is also a singular.

        BTW, during searching (unsuccessfully) for non-paywall versions of Jaeger et al. (1996) I found "Dual-Mechanism Morphology" by H Clahsen, University of Essex, Colchester, UK, which makes some of the points I did.

        Beretta, A., Campbell, C., Carr, T., Huang, J., Schmitt, L. M., Christianson, K., Cao, Y., 2003. An ER-fMRI investigation of morphological inflection in German reveals that the brain makes a distinction between regular and irregular forms. Brain and Language 85, 67–92.

        Jaeger, J. J., Lockwood, A. H., Kemmerrer, D. L., Van Valin, R. D., Murphy, B. W., 1996. A positron emission tomographic study of regular and irregular verb morphology in English. Language 72, 451–497.

        • melodye says:

          AK - you wrote:

          "The major problem I see with both models is the assumption that there is only one “processing route” used in a specific language operation."

          I understand the impulse, but extant dual route accounts suffer major shortcomings. Seemingly the most severe problem is that it's not clear that there actually are "rules" at all (the linguistic data is far too nuanced and complex). Instead, it seems that there are broadly generalized patterns, from which more idiosyncratic patterns have to be discriminated. (For example, with regards noun pluralization, the +/s/ ending is initially generalized over both regulars and irregulars, before discrimination learning separates out irregular forms from regulars).

          If you assume that what gets learned is probabilistic, rather than rule-based, you can give a much neater accounting of the data. It's also quite possible to show how what is learned gets learned. If you're interested in our lab's work on this topic, I would recommend:

          Ramscar (2002) "The role of meaning in inflection: Why the past tense does not require a rule."

          Ramscar & Dye (2011) "Expectation and error distribution in language learning: the curious absence of mouses in adult speech."

          Ramscar & Hubner (under review) "One more stop before the green-light: how semantics can make sense of inflection."

          Finally, there's no reason to think that there couldn't be several systems (a functional circuit, perhaps) that is implicated in language learning and processing. What I'm saying is that I don't think a symbolic processor (some neural instantiation of 'rules,' say) plays any part in that.

          • daedalus2u says:

            I think that is correct, the neural networks that instantiate language do not do so by implementing an algorithm. There is work that shows that savants who can do calender calculations do not do so by implementing an algorithm.

            http://www.ncbi.nlm.nih.gov/pubmed/16453069

            I presume that language is similarly non-algorithmic (and is the savant ability of non-autistic individuals).

            I think this is a difficulty of Chomsky's approach, he is looking for an algorithmic solution. When an algorithmic solution can't work, he invokes “it is innate” rather than looking for non-algorithmic solutions that could work.

          • AK says:

            Hi Melodye...

            I haven't had a chance to go over the papers you linked yet, but I will. A couple of quick comments, though:

            WRT "mouses", to me the word feels much more correct than "mice" for the plural of the thingee I use to manipulate my PC screen. Just mentioning.

            I agree that the "extant dual route accounts suffer major shortcomings". So, IMO, do extant connectionist accounts. So do the efforts (in the neurological literature) to localize specific operations to specific regions of the brain. And I'm not just talking about language here.

            In fact, all representations of scientific paradigms are simplistic, it's just a matter of which neglected complexity you're willing to put up with for the sake of research. (IMO this is implicit in Kuhn's work.) Attacking a rule-based paradigm because "a symbolic processor (some neural instantiation of ‘rules,’ say)" isn't completely necessary to explain the observed evidence won't have any effect, because in rule-based paradigms the existence of rule based processors is part of the default hypothesis.

            Default hypotheses are part of the paradigm, and you will no more convince somebody working in the rule-based paradigm with connectionist explanations than they will convince you with rule-based explanations.

            My prejudice (and I admit that's what it is) is based on evolutionary logic as well as digging into neurology, including the argument(s) made by Eva Jablonka and Marion J. Lamb (CF Evolution in Four Dimensions and peer-reviewed work cited therein) for the selective advantage of innate pre-knowledge as a redundant improvement to a more generalized learning system. I'm going to assume the existence of "symbolic processor[s]" as part of the brain's toolkit until shown real neurological evidence that they don't exist. I'll grant that my reasons are partly intuitive, but so are those by which connectionists reject rule-based explanations.

    • There's actually quite a lot of such work. Check out papers by Michael Ullman, Steve Pinker and others. The results are, it should be said, controversial, with some authors finding divergent findings.

  • Karthik Durvasula says:

    And finally, the level ordering hypothesis has been questioned from the very beginning in the linguistics community that discussed lexical phonology - search for "bracketing paradoxes" and related phenomena.

    So, to claim that your experiment has shown something linguists haven't known thru the traditional methods is just false.

    It is true that the experimental data showed that there is more nuance to the story than previously thought. That's a good result. But, this doesn't invalidate the previous approaches because they found out that the level-ordering hypothesis was wrong too. If anything, your results vindicate the position that traditional approaches have done their job.

    So, here's the big question - Why waste a tonne of money and time running an experiment, whose basic results we already knew thru far simpler methods?

    No one is questioning the need for proper scientific standards. What's being questioned is the appropriateness of imposing a whole array of scientific standards (as opposed to only the relevant ones) given that the added constraints haven't really given us anything new. Rigour is of course important, but rigour comes at a cost (both temporal and financial), and is to be used judiciously. Otherwise, we will be stuck asking silly questions for an extremely long time.

    • melodye says:

      I regularly post provocative pieces to my blog - the idea is both to provide a forum for anyone who is interested in thinking about these issues, and to provide me with a workspace in which to test out ideas. One of my most consistent commenters works in generative grammar, and I very much appreciate that - while we frequently disagree - he routinely sends me papers and keeps discussions lively. The idea is for this to be amusing and hopefully, educational, for all parties. That isn't always the case, but it's certainly the aim.

      I'm going to keep my reply as brief as possible:

      1. What you call "added constraints" (e.g., using corpora) will revolutionize the field. The claim that they haven't given us anything new is demonstrably wrong; there are new postings to Language Log on this subject every week. Further, there is very little cost (temporal or otherwise) in utilizing corpora - they are wonderfully easy to use and provide instant access to vast quantities of linguistic data. In the age of Mechanical Turk, experimentation is also becoming easier and faster. Obviously, these techniques cannot be applied to all languages or all problems in linguistics; nevertheless, their potential import is enormous. To my mind, continuing on with the same old modes of linguistic experimentation when there are newer, far better (and more obviously scientific) ones, is a bit like saying "Why use a telescope when I can see the stars with my naked eyes?"

      2. Chomsky's work is ripe for interpretation - by a literary theorist. While I would not purport to be a Chomsky scholar, I have read several of Chomsky's books and many of his articles, and if you want a really good example of someone not understanding what they're reading, you might look to MacCorquodale's critique of Chomsky's critique of Skinner (1970; see also Palmer, 2006).

      3. Please be careful to distinguish what I attribute to the papers I cite (above) and what I have said in the discussion and in my own personal remarks. My opinions, and the stated arguments of the various scholars, are certainly not identical. I would not want what I've written in the comments to be attributed to Ted and Ev, to Mark, to Michael, or to Dan. My post was also written for non-experts, so that's something to consider in evaluating it.

      4. Finally, your statement that "falsifiability is a condition on scientific experimentation, not on scientific approaches" is, I think, disingenuous. The competence / performance distinction is one which suggests that we should be highly suspect about the only data we have access to (namely, what people actually do say). This leads us right back into "intuition" - what seems right? what should people be saying? what might we imagine they are capable of saying? As opposed to "What do they observably say, and how often?" This takes us dangerously far afield from scientific observation, given that a) intuition is not hypothesis blind and b) we are not constrained by actual linguistic data.

      One of the fantastic things about corpus data is that we can actually start to build a mechanistic account of why children and adults use the constructions they do; why they make the 'errors' they do; and how their acceptability judgments vary, depending on exposure. Moreover, all of this can be done by examining the normal structure of English (i.e., the 'input'), without the bias of our intuitions, or the prejudice of our hypotheses. This is not so much 'silly,' as science as it is normally practiced.

      My question is: if linguists in generative grammar really believe that language works the way they think it does, then why fear corpora? If anything, corpus data should confirm what generativists have long suspected to be true. The problem, as I see it, is that generativists have been out describing epicycles - and corpora are about to introduce the ellipse.

      • Karthik Durvasula says:

        1) You misunderstood my claim.

        "Why waste a tonne of money and time running an experiment, whose basic results we already knew thru far simpler methods?"

        This was in response to your paper which you had suggested I read. The fundamental "result" of the paper that the level-ordering hypothesis is questionable was already known thru standard linguistic techniques.

        As far as arguing linguistics issues, there is absolutely no references to bracketing paradoxes in the paper (taught standardly in intro morphology classes)...which would have made the whole basis for the experiment rather moot. How is it that one is alleging to argue against a whole field of expertise, without actually paying attention to what is considered rather basic in the field?

        I am not against appropriately advanced techniques, but you are simply reinventing the wheel, and are caricaturing linguistics, when there can be a far more healthy relationship in cross-fertilisation.

        2) I am not against corpus studies or more elaborate techniques - I myself have done such. The question is does this mean the traditional results are wrong. This is the crux of your claim.

        If your claim is there are more sophisticated techniques, so let's use them, I will say, sure (but as is appropriate). But ur claim is different, that the older techniques of data collection have misled us. This is extremely hyperbolic (they have perhaps misled us in a few cases, but this is by logic true of ANY method one chooses to use). But, whatever your opinion is, you can't play bait and switch.

        There is a tonne of modern syntactic/linguistic work done my generative grammarians using more nuanced techniques. But the insights from such experiments have added nuance to our theories, not falsified old 'established' results, at least largely (the modest differences are always to be expected of any techniques given they are subject to standard statistical considerations, of course). This is the heart of the point Almeida and Sprouse make in their paper.

        Almost all the places where 'modern' techniques have been claimed to have "proven" or "shown" (as funny as they sounds) traditional theories to be "wrong", were already cases that there were sufficient arguments from traditional data collection techniques.

        This is the fundamental problem everyone's been pointing out. What you claim to show is simply a non-sequitur from what you actually show. I am not sure I can make this point any clearer.

        • melodye says:

          There is no point in me defending a paper you haven't actually read. This point has become abundantly clear.

          “It is very difficult to get a man to understand something when his tribal identity depends on his not understanding it.”

          • Karthik Durvasula says:

            Clarification:
            Point 1 was in reference to your paper.

            Point 2 was in reference to your blog post.

            (I posted as much last night...but for some reason, it didn't show up. lol, the vicissitudes of the internet)

      • Avery Andrews says:

        One issue is that a lot of syntactic research needs not just corpora, but parsed corpora (e.g. treebanks) which are much harder to produce. We certainly need to know more about the kinds of relationships and discrepancies that exist between informally collected intuitions (of the 'can you say this' variety, and 'if so, what does it mean'), more carefully collected ones as in surveys, large or small, and the distribution of things in corpora.

        Chris Manning in one of his statistical linguistics introductions for example notes that subcategorization frames that are very rare in corpora tend to be unacceptable, while the common ones are not, establishing a positive relationship. But then Barddal and Eythorsson found that 'Equi-Deletion' of dative experiencers in German, which has been claimed by pretty much everybody to be bad(but OK in Icelandic), actually shows up in texts (including Emmanual Kant).

        The one thing that annoys me about Sprouse and Alameida is that they don't say what 2% of the textbook 'data' is wrong. - there's really no excuse for any of it to be. I think there's also an effect whereby experienced syntax teachers drop topics where the students don't accept the judgements - this happened to me with parasitic gaps - my Aussie students have always found all the examples hopeless, with no difference between the supposedly good ones and the bad ones. So claimed data from textbooks might be better than that in the literature at large, due to undocumented further testing.

        Another factor is that intuition in psychology has some different history from what it has in linguistics, such as the Titchener-style 'Structuralism' that Behaviorism replaced, wherein people were supposed to be able to intuit that forex the visual appearance of an apple was supposed to be intuitive resolvable into little patches of definite color (or something like that, it's been a long time since I looked at that stuffl, and, lacking adult supervision by a psychologist, I might have not understood it quite right). There's intuitions and intuitions. The linguistic ones *ought* to be like JNDs in sensory psychophysics, or the standard perceptual illusions, I think. But the consequence is that people need to be very clear about exactly what kinds of intuitions they're talking about.

        • Diogo Almeida says:

          Hi Avery,

          if you are interested in the 2% of the contrasts from Adger's book that did not replicate in formal experiments, you can check out appendix B of the following manuscript:

          http://www.socsci.uci.edu/~jsprouse/papers/empirical_foundation.pdf

          And just to clarify, we made no claims that these examples from the book were *wrong*; in fact, at least one of the tests [the first two-way ME in appendix B] has been extensively replicated by several different people using formal experiments.

          • Avery Andrews says:

            Thanks! I'm relieved not to have found it in the lingbuzz paper, on a recheck of that. I suspect, however, that the conclusion on pg 29 of that might be somewhat overstated, on the basis that there might be more irreplicable judgements in the full literature than in intro textbooks. Professionals doubtless learn there way around the problem areas, but it's not reasonable to expect visiting psychologists to do so, so I suspect that a bit of a cleanup is called for.

  • Karthik Durvasula says:

    And this is exactly the right question “Why use a telescope when I can see the stars with my naked eyes?” - though you are seeing it in completely the wrong light.

    We don't need electron microscopes to tells us tables are different from the floors they abut. In fact, if you used a high-power electron microscope, and depended purely on that, you would probably waste a LOT of time before you came to that conclusion.

    The generative linguist's claim is not "let's not used more advanced techniques". The question is "Is it appropriate/useful for the level of generalisation we are trying to make?"

      • Karthik Durvasula says:

        lol. you haven't read what I (or anyone else has) said carefully enough. This is interesting, but in no way refutes the results of classical approaches.

        In the simplest English: Fabb (1988) showed almost exactly what you did with the classical approach. The very same approaches that you call unscientific. So, how was the 7 experiment approach an improvement? You deal in insults and ad hominems, but haven't answered the basic question. Period.

        The crux of the argument lies in the first two paras - which is what I encourage you to think about.

        Maybe, you should take heed of “It is very difficult to get a person to understand something when their tribal identity depends on their not understanding it.” why not try other ad hominems or even veiled insults "put the cart before the horse"...? Clearly, in your opinion, that is good science.

        There is a sense in which you are calling an approach unscientific without actually looking at (without bias) the actual results of the approach. In short, let our opinions about the scientific method themselves be hypotheses, and I leave it to you to see, if either approach has been falsified.

        If you still refuse to see it, there is not much any of the other commenters and myself can do. There is clearly a bias, and at least acknowledge it, instead of calling the other viewpoint unscientific (for which, you have provided no evidence, other than empty claims; and for each of which, a counter claim could just as well be made).

        • Alex Clark says:

          "There is a sense in which you are calling an approach unscientific without actually looking at (without bias) the actual results of the approach."

          This is highly questionable -- are you really claiming that the results of a method justify the methodological short cuts?
          In the case of generative linguistics which has almost no firm results at all, this seems completely untenable.

          • Alex Clark says:

            Actually, let me retract the final sentence of that post which is unhelpful and inflammatory.

          • Karthik Durvasula says:

            In the context of a disagreement over the methods, where (logical) arguments aren't convincing to either side given the bias that is inherent to all of us (which is clearly the current case), I don't see the harm in being result-oriented. Ultimately, (hopefully) we are all looking for insight into a process.

            Clearly right now there are (at least) two camps of researchers with very distinct (and even contradictory) approaches to science - what's worse is that each side thinks that their position has been severely misunderstood by the other side. In which case, treating scientific methods themselves as hypotheses and looking at the results (insights) they have generated is rather constructive, to avoid being entrenched in methodological turf warfare :).

            It makes me uncomfortable too, that I said it (cos clearly, it is a provocative). But, at the same time, I don't see another way around the stalemate that leads eventually leads to unproductive muck-raking.

            One might call this the "end justifies the means approach" to scientific methodology...or even Scientific pragmatism. haha.

            Alex, I invite you to have me for lunch after this comment.

        • Avery Andrews says:

          I can't help noting that the lastest upload to lingbuzz, by Hornstein and Rodriguez, is a report on a small survey they did to establish some data that their theory depends on. So, evidently, there are people in hard-core current Chomskian generativism who agree that the empirical aspect of the subject needs a bit of work.

          • Alex Clark says:

            I feel that the main methodological shortcoming of "hard-core current Chomskian generativism " is not the standards of data collection, but the standards of formalisation of the generative grammars themselves.

          • Diogo Almeida says:

            Hi Avery,

            I think your conclusion on the Rodrigues & Hornstein paper is a little too strong. Their paper, to the extent that I undertand it, is about trying to solve a theoretical discussion in which the crux of each side's argument lies on the status of the data. And there, frankly, I don't think anyone in the generative camp is against the use of formal experiments.

            In fact, I don't know anyone in the generative camp who is against the use of formal experiments, period. I think everyone agrees that formally conducted experiments, corpora and whatnot are all useful things, and have their place. The reaction you are seeing from people in the generative camp is *not* about them being against these tools, it is rather about the strong claim from some corners that the field is *unscientific* without them, and that basically we know nothing, or very little, because the standard methodology is so bad and unreliable. That is the strong claim made by papers like Gibson & Fedorenko (2010) and by people like Melody here, who says things like:
            "The problem, as I see it, is that generativists have been out describing epicycles".

            I for one think this is a major red herring. I sincerely doubt that people like Gibson & Fedorenko and Melody will all of a sudden become minimalists (or at least GBists) if they are shown that the data underlying the theory is actually not bad at all. Their disagreement is much more profound, and involves, I think, a very different idea of what a scientific theory of language should look like.

            To put it very bluntly: if the "problem" with linguistics were just the data, it would have been solved years ago. Take Magnitude Estimation, for instance. It has been available in a neatly packaged format to linguists and psycholinguists for 15 years (since at least 1996). This is roughly the same amount of time separating Aspects from LGB. Did a revolution happened in linguistic theory (or psychology of language, for that matter) in the last 15 years?

            The problem is in fact much worse. Simple forced-choice tasks have been available since basically the inception of modern cognitive psychology and incidentally, they are much more powerful than Magnitude Estimation for detecting differences between sentences. If skeptics of traditional linguistic methodology were really concerned with the empirical foundation of the theory, they cannot possibly claim that they couldn't do anything about it before 2010. So what has stopped them?

            The fact of the matter is that the mistrust of linguistic theory runs much deeper than a basic mistrust of the data, and the most strident critics very seldom engage with the theory in any meaningful way to actually know what the theory is, much less what data is supposed to support it. For instance, Gibson & Fedorenko (2010) use an example they claim came from Barriers (Chomsky, 1986), but they quite literally miss the point of Chomsky's discussion, and test for something that no one ever argued for. Another example is the point I think Karthik was trying to make here, the paper by Ramscar & Dye (2011). If I understand Karthik's point correctly, Ramscar & Dye (2011) are basically beating a dead horse, that has been dead since at least 1988.

            I think there are real issues facing linguistic theory (for instance, what Alex hinted at in his comment), especially when it comes the relationship between linguistics and psycholinguistics, but this idea that it will all be solved by "reruning every paper ever published using a sample of 20 or 30 undergrads" is just wishful thinking.

          • Avery Andrews says:

            The blog seems to have a limit on recursion in replies so:

            @Alex: I suspect that the underlying problem may not be so much absence of formalism as absence of adequate understanding of what it means - we have all of these frameworks floating around at levels of formalization ranging for highly rigorous (type-logical or pregroup grammar) to very much less so, but working out what any of them really really implies about the nature is not easy at all. So minimalists often don't go very far in formalization because they don't see the point. I think LFG provides a nice balance between formal rigor and intuitive clarity, but, obviously, plenty of people don't see it that way.

            @Diogo: I'd prefer not to speculate about what people 'really think', but just try to figure out what aspects of the how things appear to be done might make their criticisms look plausible, and either change the procedures or refute the criticisms. Gibson & Fedorenko 's view would have been much closer to the reality of the field 40 years ago, and it does take an amazingly long time for news of new developments to get around, especially when it's not trumpeted in introductory textbooks, I suspect.

            Some subjective evidence that there has been a change might be that when I attended Howard Lasnik's course at the 2005 LSA, if I hadn't had a laptop in front of me, I would have worried that I'd been transplanted to a Life on Mars episode, because he really was going on in the old way with delicate judgements of very complex sentences, and stood out in this way from the other presenters. So Lasnik's data could probably use a bit of more formal replication (he might possibly have already done it).

          • Avery Andrews says:

            A bit more: I notice that ed. 2 of Carnie's textbook contains a small discussion about data (while Culicover's does not), but nothing about surveys large or small; a paragraph citing some literature about these would be enough, I think. Perhaps in the 3rd edition?

            Another little experiment I might do, but not today, is seem how long it takes me, from my current smattering of superficial and half-remembered information, to find a book or article that would tell me how I should have done the small surveys I reported on in my 1990 Icelandic Case paper (in the Modern Icelandic Case volume), and why. Small surveys, because they are the most reasonable thing for a theoretician who is nervous about the status of some of their data to undertake.

          • Alex Clark says:

            Another recent lingbuzz paper is "Medial position adjunct PPs in English and the encoding of sentential negation" which criticizes a fairly classic example of bad data in the generative tradition. It's difficult to read that paper and be completely happy with the empirical standards in the field.

            @Avery

            What I meant was that given the examples in Jon Sprouse and Diogo's paper, and some grammar G how can we tell which examples it correctly predicts?

            More trivially, if I look at some paper talking about phases in the MP, how can I tell whether their model predicts the grammaticality of "John died" or "died John"?

            Will people regularly take the data from the Sprouse paper and say -- we correctly predict grammaticality for 97% of these examples .. here are the places where we go wrong?

            The bigger problem is that the theories are insufficiently specific to make detailed predictions about the grammaticality of individual sentences. I may be being too pessimistic here.

  • Avery Andrews says:

    I little experiment I just did was to look at the first article in the latest Linguistic Inquiry, which is about using PF erasure to rescue that-trace violations; the first claimed ungrammatical sentence in a language other than English is

    *Gianni_i sembra a Maria [t_i essere stanco].

    (_i as referential index subscript, t for trace)

    Sources given are a $140 book by Cedric Boeckx, and another one by Nunes, neither in the library (I ordered the Boeckx one), so that would be the end of the evidential trail for the average psychologist, I would imagine.

    If the reference had been to a journal article by, say, a Rizzi student, I very likely would have been able to find out more about how the claim was established - how many people were consulted, whether texts were looked at for possible examples, etc. etc. (contrary to widepsread impressions, many GB/MP workers are quite conscientious about this kind of thing). So part of the problem is the prevalence in linguistics of books that you can't look at quickly unless your uni library happens to have them. Plus the habit of sourcing putative facts from the most recent piece of work in your school of thought that used them, rather than the original work where they were proclaimed (or subsequent literature in which they have been reviewed).

    A linguist who wanted work on this wouldn't take very long to track it down and verify it, but what is an outsider to do? This kind of problem could be greatly ameliorated by some relatively simple steps, such as making sure that the empirical data portions of books such as Boeckx's were available online, and correcting the referencing habit I just whinged about above.

    *****

    As it happens, I consider the claim highly credible - I think I've seen it before, and exactly the same constraint is active in Icelandic, as I discovered to my frustration in the early 70s, when I wanted there to be examples like:

    Mary(N) seems John(D) intelligent(N)

    so that I could stick them into the subject-to-object raising construction and find:

    Sigga(N) says Mary(A) to seem John(D) intelligent(A)

    but I couldn't get any Icelanders to accept the first one (where was Clever Hans?? somewhere out in the more distant reaches of the back paddock, it would appear), making it pointless to inquire after the second.

  • Avery Andrews says:

    & this discussion shouldn't go on without some music, Charles Ulrich's classic 'Please Mr. Postal (from some time in the 70s. Linked to with another classic, 'Colorless Green Blues', here.

  • Asad Sayeed says:

    (Hi Diogo!)

    I wrote a slightly tangential post on the, uh, sociology of this kind of discussion over at my blog, so I hope no one minds if I self-promote and leave a manual trackback on this here. Because it's long and not directly about the issue I didn't leave it as a comment here.

    I wanted to jump into this discussion myself but time constraints... I guess I'll write a more comprehensive response to this some time later when this is all stale as a blog post too.

    • Diogo Almeida says:

      Hi Asad! nice post on your blog! whenever you have the time, please do chime in here :-) !

  • Aviad says:

    I'd say the burden of proof lies with the person who made the original claim based on quotes from Chomsky; this is not evidence for what syntacticians actually do.

    I'd like to know, then, of phenomena that haven't been attributed to the syntax at some point or other in the generative literature. Assuming that language isn't just syntax, we should find plenty examples of this sort, no? What I am familiar with are cases in which syntactic explanations were originally proposed but have been proven inadequate, such as negative islands or focus intervention effects.

    The field in general is in a sorry state if the best linguists can do in explaining data is assuming a priori that it falls under whatever subfield it is that they work on; that's what the individual saying "but I'm a syntactician" meant. Imagine a cardiologist treating everything he finds as a heart problem; luckily, linguists are not responsible for anybody's life.

  • Aviad says:

    My last post is in response to Diogo Almeida's from March 18, 2011 at 7:38 pm

    • Diogo Almeida says:

      Hi Aviad,

      re: "burden of proof"

      The problem with anedoctal evidence is that others have to take it at face value. I have no way of independently verifying your claims about what linguists do. Maybe in your experience (which might or might not be extensive, or might or might not include interactions with reputable linguists), syntacticians do reflexively jump from unnacceptability data to theory. Or maybe you just completely misunderstood what linguists do in your interactions with them. I have no way of knowing. All I can say is that your version of "what linguists do" does not accord with my own training in grad school and my own personal experience in the field. I can't speak for Karthik, of course, but I assume he bases his assertion on similar grounds. The problem is that you can't verify our assertions either, you have to take them on our word alone. That is why I actually think that Karthik's use of quotes from Chomsky is a slightly better way of doing things: You can independently verify them. Also, Chomsky 1965 is a foundational text in linguistics, and presumably anyone working in the generative tradition would have been exposed to it. Of course it does not "prove" what syntacticians actually do, but it does provide a degree of objectivity regarding what syntacticians would strive to do, or would profess to be their goals.

      This is a real serious issue, because I do think linguists have a horrible PR problem when it comes to their methods, to the point of some outside the field having really strongly held beliefs about them that are, in my own experience, uncharitable charicatures or just plain false. A recent published example is Gibson & Fedorenko (2010)'s claim that generative syntacticians (and semanticists) test their theories only on themselves and using only one single trial. All I can say is that I have never seen a syntactician (or semanticist) do that, ever. But apparently G&F have. So who are you going to believe? There is no obvious solution here, and it does get more serious when article reviews are involved. Jon Sprouse and I had a problem in one of our recent submissions, where we tried to rebut G&F's description of linguistic methodology, and some reviewers sided with G&F's version over ours, depite the fact that Jon and I are professional linguists. Things start getting a little problematic when the perceptions of people outside the field trump the perceptions of people within the field, and the former get to be published and the latter don't.

      • Alex Clark says:

        One of the causes of this is that when you look at articles in syntax; for example the most recent issue of Linguistic Inquiry:
        e.g. this paper which seems fairly typical, you do not see any evidence of careful control for possible performance factors, or testing on multiple subjects etc.
        If it's not in the scientific record, then you can't blame people for thinking it doesn't happen.

        • Avery Andrews says:

          As I discussed above in this thread, this paper seems to me to be at the lower end of actual practice. It might be good if people in general had a more solid idea of the reliability is of different standards of investigation. So in my 1982 paper on Icelandic (Bresnan (ed) _The Mental Representation of Grammatical Relations_) I claimed, consistently with what I had found with a rather disorganized collection of 'can you say this'-type informant sessions, that sentence like below were ungrammatical with either choice of case (p475):

          *hún/hana segist vanta peninga
          she(N/A) says-herself to-lack money
          (the verb vanta 'lack' takes an accusative subject, while
          segjast 'say oneself to be' takes a nominative, and, under
          the theory presented in the paper, this contradiction in
          requirements is supposed to rule the sentence out).

          But, when I did some small questionnaire-based surveys in 1983, reported in my paper in the Maling&Zaenen (ed) 1990 _Modern Icelandic Syntax_ volume (p 206), I found that the actual responses were a lot more variable; 6/17 people found the nominative 'fully acceptable' as subject of segjast when its complement calls for an accuative subject, only 2/17 finding it 'thoroughly unacceptable', and the rest putting it at various levels of 'dubious'. With a downstairs verb calling for a dative subject, the results were more like what I said in the original paper, but still variable.

          Slightly more than a third of the respondents judging a supposedly ungrammatical sentence as 'fully acceptable' is something that people ought to want to know about, I think, so we see the limitations of interviewing a small number of people (about 7-8). Maybe it would have come out interviewing more people in Iceland, and one would also want to know what kinds of examples show up in corpora (it will probably be easy to find out in a few years when the Icelandic treebanking projects are more advanced). Otoh I don't think the results of the questionnaire showed that the original theory was just wrong.

          In general, how much work you need to do in order to avoid various kinds of errors is something I think we ought to want to know more about. Plus how the acceptability judgements garnered by these techniques correlate or not with facts of usage.

          I rather doubt that the state of data in linguistics is really worse than it is in other places where it matters a lot more (eg biomedical), but it's ours, so we ought to try to clean it up.

    • Diogo Almeida says:

      re: "field in a sorry state"

      I am really at a loss as to what your objection to the field is. You seem to object to the fact that sometimes linguists propose a "syntactic" account that is later on abandoned in favor of a "non-syntactic" account. But why is that an issue? Any account of an acceptability pattern, grammatical or non-grammatical, is always going to be a hypothesis, that will stand or fall according to their ability to accomodate new facts, or provide insight into other previously seemingly unrelated issues.

      Furthermore, it is not even entirely clear that linguists always "a priori" favor "grammatical" over "non-grammatical" accounts as you seem to believe. For instance, just on the other thread, I mentioned how Jackendoff & Culicover (1971) favored a parsing strategy account for the perceived unacceptability of questions on the first object of a double object construction. Hofmeister & Sag (2010) would be another example of linguists favoring a non-grammatical account, this time for some island effects.

      As for the islands effects you mentioned, the issue seems to be that there is a dispute about the proper locus of the effect, if it is on the syntax or on the semantics of the constructions. Apparently, the semantics locus has been more favored recently. But why is that an issue? In fact, how could you even think things could work differently? Data does not come labelled as to what caused it, so what exactly is the problem in trying different kinds of explanations for it, and see which ones gives "the best" general account?

  • Aviad says:

    Diogo,
    I wasn't talking about the methodology in terms of data collection, as I noted in another thread here, but rather data interpretation. With respect to data collection, I suspect the truth is somewhere between your perhaps idealized view of work in generative syntax and G&F's overly negative conception.

    As for data interpretation, there might not be any problem if indeed it were just a matter of adjudicating between different explanations. However, the reality is that the generative framework is syntacticocentric in various senses, and so often there is no discussion of non-syntactic explanations, even when syntacticians acknowledge that their explanations fall short. To a large degree, the sociology of the field is dictating the results we currently have. If you don't believe me, I recommend reading through the literature on weak crossover, my pet peeve. To make a very long story short, Chomsky argued against Wasow's non-syntactic explanation in the late '70s, which, unsurprisingly, led to its abandonment. Wasow's data was also forgotten, and people have gone off in odd directions without solving basic problems with the syntactic analysis. Thus, 20 years later, you have people like Reinhart admitting that "As is well known, none of the existing accounts for weak crossover gets close enough to capture the facts." (Reinhart 1998). The same thing has happened with inverse scope, where only recently have people noticed that May's purely syntactic analysis won't work.

    True, in the end we might reach the "best" account. It seems to me, however, that all of the garden paths along the way would have been avoided had syntacticians recognized the possibility of non-syntactic explanations and taken systematic interspeaker variation more seriously.

    • Avery Andrews says:

      So, what do you think of the treatment of WCO in Joan Bresnan's 2001 LFG book? This could be described as superficially syntacticocentric, in that discrete *'s are pasted onto various sentence by discrete grammatical principles, but I think this is superficial, because nothing bad would happen to the theoretical architecture if these principles were replaced by gradient and variable ones, or disappeared entirely (by the philosophy of PPA, this is a bug, not a feature).

      By contrast, in Jacobson (1999) `Variable Free Semantics', the WCO phenomenon is quite important for the architecture, and removing it would seem to make a mess of her account of how antecedents bind pronouns, so this account is deeply syntacticocentric w.r.t. WCO.

      • Aviad says:

        I think WCO is not a syntactic, but rather an information structural, phenomenon. Roughly, the idea is that an operator has to be a topic to take inverse scope, and scope is a necessary (and sufficient) condition for variable binding. This kind of idea was around in the 1970's, in Wasow's work for example, though because no one had a good understanding of information structure, the terms were imprecise (e.g. Wasow's "determinateness") and no one could connect their non-syntactic accounts to the structural notion of scope. I think I can do this, and thus account for a boatload of unexplained data which has been lying around since the 1970's.

        I'd be happy to provide further details; in any case, my point isn't about WCO per se. It's about the way in which syntacticians handle variation in their data, and the glibness with which they attribute it to "dialects" or something of the sort. Diogo seems to think that the way they work is just fine, and that if they happen to be wrong in interpreting data we'll find out through further research. I believe the problem is much deeper and ideological in some sense.

  • Diogo Almeida says:

    Re: "It's about the way in which syntacticians handle variation in their data, and the glibness with which they attribute it to "dialects" or something of the sort."

    Hi Aviad, sorry I missed your last posts. I am still having a hard time understanding your dissatisfaction with the field. What is surprising about syntacticians trying to give a syntactic account for the WCO effects? To the extent that the accounts work, they are doing a good job. To the extent that the accounts fail to work, then maybe it could be the case that a better account is to be found elsewhere. If that is the case, however, the problem wold be one that syntacticians would be ill-equipped to investigate to begin with. You seem to be objecting to the fact that no good non-syntactic account was available to compete with the syntactic one in the early days, but how is that a fault of generative syntacticians?

    And how does this phenomenon relate to interspeaker variation to begin with? I am afraid don't think I see the connection there. In other words, how could the interpretation of interspeaker variability constrain the choice that we as linguists *always* have in trying to account for phenomena at different levels of analysis (syntax, semantics, pragmatics, etc)?

    • Avery Andrews says:

      Another point in the same general direction as Diogo's is that syntax vs non-syntax is disjoint from variational vs. non-varational. In non-standardized languages, there can be huge variation in grammar, as a colleague of mine notices with respect to a large collection of Dutch letters that she studies from before the written language was standardized. Involving information-structure is also not a badge of non-syntacticity, since there are many syntactic structures whose main or sole function is to code information structure. Joan Maling's work on the 'New Passive' (or 'New Impersonal') in Icelandic, and followup studies by many other people, is a nice example of syntactic non-uniformity.

      I think the real problem is that is often very tempting to put some variable, weak phenomenon in a central position in the justification of some grammatical theory, in a way that requires assuming that it is solid. It is annoying that this so often seems to lead to 'elegance', but I think it's just a siren song that we need to be wary of, and resist when appropriate.

  • Aviad says:

    Hi Diogo (and Avery),
    Thanks for the response. I'm glad we can keep this to a serious, cordial, and non-personal tone; much of the discussion around these issues is sidetracked to ad hominem attacks.

    I'll try to clarify. I'm dissatisfied with the ideological glasses which syntacticians use when approaching data. The default assumption, it seems to me, is that a phenomenon is syntactic; evidence for this is in the huge amount of syntactic analyses of phenomena discovered in language. I'd say that the non-syntactic accounts you mention in another thread are the exceptions that prove the rule.

    This ideology isn't surprising; it originates in Chomsky and his first students. I'll give you another example where it appears to have influenced the direction of the field: inverse scope. May proposed a purely syntactic account for this phenomenon, and everyone jumped on the bandwagon, because it was syntactic and Chomsky endorsed it. However, already in the 70's it was clear that a syntactic account is inadequate, and people like Kroch, Kuno, and Reinhart pointed this out. You have many quantifiers which should give inverse readings and yet they don't, in a systematic way. What's worse for a syntactic account is that you have cases where an inverse reading should be ruled out yet it's possible.

    How does this connect to interspeaker variation? Of course we can't know in advance what the source of variation is, but we at least have the obligation to try and explain it, particularly if it's so pervasive as in WCO or inverse scope. And here's an option that was raised already by Newmeyer (1983): perhaps this variation reflects differences in how speakers contextualize examples we give them. Schutze (1996) and others have also emphasized the need to test examples in contexts. Have syntacticians ever tried to do so? Have they tried to explain the variation they find?

    Here's my modest suggestion: we should be linguists first and syntacticians second. The field would be in a much better place if syntacticians cared more about explaining data than holding on to a syntactic explanation for the data.

    • "You have many quantifiers which should give inverse readings and yet they don't, in a systematic way. What's worse for a syntactic account is that you have cases where an inverse reading should be ruled out yet it's possible."

      I can see how this could be evidence against a particular syntactic account. How is it evidence against syntactic accounts in general? Even among linguists, different people mean wildly different things by "syntax". Given that, what kind of evidence could -- even in principle -- completely rule out a role for syntax?

      • Aviad says:

        It's evidence against all the existing syntactic accounts of inverse scope that I know of: they're predicated on the idea that if the syntax rules out a given reading and the structure remains constant, there shouldn't be particular lexical items that allow a reading. But this is exactly what you find, arguably because it's not the syntax blocking certain readings to begin with.

        • Avery Andrews says:

          I don't get this, because most current syntactic theories have a lot of stuff in the lexical items, one way or another. Maybe more details about which syntactic theories can't account for which inverse scope facts would be helpful. For example, the scheme on pg 334-335 of my paper in the Bresnan Festschrift is supposed to allow scope restrictions to be stated in the LFG+glue semantics framework in a pretty flexible way, but might of course fail on the particular things you're thinking about.

  • Jay says:

    Semantics. This article is a waste.

  • Jay says:

    You simply don't care for the misuse of the word experiment. Words are just boxes around objects and concepts and I believe that strict use restrains cognitive flow and imagination. A "chair", or whatever, exists simply as it is and is no more or less a "chair" or anything else by being "mistakenly" labeled a "table." A "chair" may function equally as a "table" or "weapon" or "wood" or whatever else and however you label it in no way changes it's superposition anywhere but in your mind. All objects and concepts are in superposition, and while rules must exist for a language to function it is important to remember that absolute definitions are false and restraining. "Thought experiment" is a valid term, IMHO, because it conveys the meaning. There are many things worth bitching about but this isn't one of them.

  • Wow, amazing blog layout! How lengthy have
    you been blogging for? you make running a blog look easy.
    The entire glance of your website is magnificent, as smartly as the
    content!

  • Genuinely when someone doesn't know after that its up to other users
    that they will help, so here it occurs.

  • […] you have been following the debate on acceptability judgments and other linguistic methods, you may want to check out computational […]

Bad Behavior has blocked 373 access attempts in the last 7 days.