Monthly Weather Review Editors' Blog

Regarding Prof. Jim Steenburgh's posting "about submitting a paper"

2009-09-23T15:26:00.001+03:00

I agree wholeheartedly with Professor Steenburgh's comments. As a reviewer, I have often felt that a manuscript that I was being asked to review was not nearly in as "ready" a state as possible. Not uncommon are manuscripts of this type with multiple authors; it seems doubtful that all of the "authors" have participated in the preparation of, or even proofread, the final submitted version.

The submission of poorly edited manuscripts or the premature submission of manuscripts can be viewed as an abuse of the peer review process. Asking reviewers - whose time is valuable - to review these manuscripts is disrespectful and ultimately destructive to the peer review process - reviewers become unenthusiastic and wary about agreeing to do reviews.

It would be desirable for MWR and the AMS to institute policies to discourage the submission of ill-prepared and poorly edited manuscripts.
An issue is that editors and reviewers are currently often unwilling to reject a manuscript, which has some scientific merit, solely on the grounds that it is poorly edited and inadequately prepared.
One possibility would be to ask editors and reviewers to identify such manuscripts. An editor, assigned such a manuscript, could return it to the authors without sending it out for review; and a reviewer, who received such a manuscript, could decline to review it on the grounds that it was not in a suitable state for submission in the first place. Such a decision or recommendation would be different from a rejection - the author(s) would be asked to submit an adequately prepared revised manuscript.

Herschel Mitchell
Editor, MWR

How to respond to reviewers

2009-09-08T15:33:00.003+03:00

Anonymous raised some interesting questions about responding to reviewers in a comment on a previous post. My response was getting quite rich in content, so I thought I would submit a full blog entry instead. Bottom line to Anonymous: I think you are approaching your response to reviews correctly.

As a reviewer and generally as an editor, I would be thrilled if every author responded positively to all the reviewer concerns and the manuscript was improved as a result. We could have one round of reviews, and the paper would be published. That situation would be easiest for everyone (reviewers, editor, authors). :-)

Although reviewers generally provide more useful comments than not, reviewers are not always 100% right in their reviews. Therefore, I tell authors that they should usually accept the majority of reviewer comments---especially the major comments---if they want to have smooth sailing through the review process on the way to eventual publication. I call this knowing how to play the game. A positive response to 70-80% of the comments, including intelligent responses to the major issues, usually makes me feel pretty happy about the way the review process goes. No one gets everything they want, but the result is a dramatically improved manuscript that should be acceptable for publication. When authors and reviewers know how to play the game, they submit decent manuscripts that are improved during the peer-review process and are published after one or two rounds of reviews. As an editor, I don't have to write long decision letters for such manuscripts---I trust that the authors will know how to take the reviews and revise the manuscript accordingly.

If an author does not meet these standards with their revision (isn't playing the game), then I as editor am put in an awkward and potentially compromised position. I am trying to balance competing effects: publish good, if not the best, manuscripts and publish them quickly. In most cases, I respect the reviewers (I selected them!) and their opinions, so I want to see that the authors have listened to what the reviewers said and have made positive changes to their manuscript. But I also want to give the authors some license to write the manuscript as they envision it and to publish their work quickly. I am handling 20-30 manuscripts at any given time, and the more peer-reviewed manuscripts that I can send to the publisher, the fewer are in my in-box, so to speak.

Like most conflicts in life, you have to know when to pick your fights. If you are going to battle the reviewer on a major comment or two, then accepting most of the other comments will give me as editor some relief that you are taking the review process seriously. That is why I say 70-80% as an estimate. I don't expect reviewers to always be right or provide exceptional comments all the time, and I feel authors should be able to defend some of their material against possible reviewer misunderstandings. But, if an author blows off half or more of the reviewer comments, then I think the author is being too resistive to change and I get annoyed. I am particularly troubled if the author blows off too many minor comments, which should be relatively straightforward to implement in a revised manuscript.

Good editors know when to bring this back and forth between reviewers and authors to a close and quickly. For manuscripts for which major revisions are required (about 50% of manuscripts submitted to MWR require major revisions), I try not to go more than two rounds. If I have to have a third round of reviews, someone didn't do a good enough job (usually the author).

Quote about submitting a paper

2009-08-31T10:33:00.003+03:00

As a reviewer, I see a lot of papers that are sent in with the idea that they will do the final editing after the reviews (or perhaps that the reviewers will provide what they need to edit to final form). My personal view is that when you submit a paper it should be in final form and that you should be comfortable with the paper going directly to press as is. It is a waste of time for all of us to review anything less.
-Prof. Jim Steenburgh, Chair, Department of Atmospheric Sciences, University of Utah

Welcome a new Editor-Herschel Mitchell

2009-04-25T11:10:00.003+03:00

To accommodate the growing number of manuscripts on data assimilation, we have added a new editor, Herschel Mitchell of Environment Canada. Herschel will be handling manuscripts on the topics of data assimilation, Kalman filtering, ensemble data assimilation, the ensemble Kalman filter, and operational global and regional NWP.

Welcome aboard Hersch!

Dave

The Unacceptable Practice of Shopping Manuscripts Around

2009-04-11T08:28:00.004+03:00

Lately, I have seen situations at MWR and other journals where a manuscript was rejected, only to be resubmitted to the same journal or a different journal with only minor changes, if any, being made. In the cases I am familiar with, the reviews recommending rejection contained reasonable requests, including rerunning model simulations, fixing improper English language, and replotting figures.

Let me be perfectly clear.

Submitting your manuscript to the journal is a privilege, not a right. It is a privilege that can be revoked by the editor or publisher of the journal.

You are imposing upon an editor and several reviewers, all volunteers, to improve your manuscript for publication. Even the most critical reviews offer advice that can make your manuscript better. To ignore their efforts and resubmit the manuscript with only minor changes is a blatant disregard for the time of others. Most editors do not tolerate such behavior, and your manuscript will be rejected.

A similar infraction occurs when authors are found to be “shopping around” rejected manuscripts between journals. Because atmospheric science is a relatively small discipline (compared to physics or chemistry, for example) and your area of specialty may be even smaller still, chances are that some of the same people that knew about your original manuscript at the first journal will see it again at the next one. Not making major revisions to a rejected manuscript, whether or not it was submitted to the same journal, is simply unacceptable.

(Thanks to Andrea and Russ Schumacher for identifying an error in this post, now fixed.)

December 2008 issue of MWR

2009-02-12T09:47:00.002+02:00

As the AMS processes the backlog of manuscripts, speeding up the time to publication (now about 200 days from the time of acceptance, down from 240 days last year), you'll notice the issues of MWR getting thicker. The December 2008 issue probably sets the record for length of a single issue among AMS journals: 727 pages.

NOAA OAR Outstanding Scientific Paper Awards

2008-10-21T20:55:00.002+03:00

NOAA OAR awards their yearly Outstanding Scientific Paper Awards. This year, two papers were published in MWR, and another was won by one of our Editors and one of our future Associate Editors.

From Richard Spinrad's email:

The Outstanding Scientific Paper Awards were established to recognize the NOAA Office of Oceanic and Atmospheric Research (OAR) Federal employees, and Cooperative Institute (CI) scientists associated with OAR who published outstanding scientific peer-reviewed research papers, review papers, books, monographs, and chapters of books that have contributed to or contain the results of research sponsored by OAR.

I would like to congratulate the following winners of the 2008 OAR Outstanding Scientific Paper Awards:

GFDL
Shaoqing Zhang, Matthew J. Harrison, Anthony Rosati, and Andrew Wittenberg. System design and evaluation of coupled ensemble data assimilation for global oceanic climate studies Monthly Weather Review, 135(12: 2007), 3541-3564.

ESRL – PSD
Thomas M. Hamill, Jeffrey S. Whitaker, and Steven L. Mullen. REFORECASTS: An Important Dataset for Improving Weather Predictions. Bulletin of the American Meteorological Society, 87(1: 33-46) doi:10.1175/BAMS-87-1-33

ESRL – GSD
Tracy Lorraine Smith, Stanley G. Benjamin, Seth I. Gutman, and Susan Sahm. Short-range forecast impact from assimilation of GPS-IPW observations into the Rapid Update Cycle. Monthly Weather Review 135(2007), 2914-2930, doi: 10.1175/MWR3436.1

(Image from treehugger.com)

Highlights in the September issue of MWR

2008-10-01T17:37:00.002+03:00

There are lots of papers on tropical meteorology in this month's issue.

Papers of note:

Sasaki’s Pivotal Contribution: Calculus of Variations Applied to Weather Map Analysis--John Lewis and S. Lakshmivarahan

The Impact of Analysis Error on Medium-Range Weather Forecasts--Kyle L. Swanson and Paul J. Roebber

Synoptic Control of Mesoscale Precipitating Systems in the Pacific Northwest--Paul J. Roebber, Kyle L. Swanson, and Jugal K. Ghorai

Vortex Lines within Low-Level Mesocyclones Obtained from Pseudo-Dual-Doppler Radar Observations--Paul Markowski, Erik Rasmussen, Jerry Straka, Robert Davies-Jones, Yvette Richardson, and Robert J. Trapp

Climatology of High Wind Events in the Owens Valley, California--Shiyuan Zhong, Ju Li, C. David Whiteman, Xindi Bian, and Wenqing Yao

Outgoing and Incoming MWR Editors

2008-09-21T11:19:00.003+03:00

At the end of 2008, we will retire two editors from Monthly Weather Review. I am very sad to see Jim Doyle and Nolan Atkins go. Both have been remarkable editors during their years of service. Nolan has served for three years, and Jim has served for five years.

In return, MWR will get two new editors.

Todd Ringler is a great match for the increasing numbers of papers on climate models and weather that we get at MWR. In addition, he has expertise in numerical methods for numerical weather and climate prediction models. He works for the Climate, Ocean, and Sea Ice Modeling Group at Los Alamos National Laboratory. His web page is http://public.lanl.gov/ringler/ringler.html.

To accommodate the growing number of tropical and large-scale circulation submissions, I am pleased to announce that Matt Wheeler will be joining us. Matt is from the Centre for Australian Weather and Climate Research, a partnership between the Bureau of Meteorology and CSIRO in Melbourne. To learn more about Matt and his scientific expertise, his web page is http://www.bom.gov.au/bmrc/clfor/cfstaff/matw.htm.

Please welcome these new editors to our editorial board.

Journals online are leading to more recent and fewer citations

2008-08-13T14:12:00.004+03:00

When The Economist is covering a story about science publishing, you know it has got to be interesting!

James Evans published an article in Science that studied 34 million articles published in journals that made their archives available online. He found that "as more journal issues came online, the articles referenced tended to be more recent, fewer journals and articles were cited, and more of those citations were to fewer journals and articles."

My observation is consistent with Evans's results that more and more authors are forgetting where we came from, not citing older literature, and limiting the number of research citations. Also, more specialized fields, like data assimilation and ensembles, don't have as rich a history (despite recent historical reviews in MWR by John Lewis on the scientific origins of these fields).

Evans's article reminds us all that part of being a scientist is conducting the scholarship of where our science came from. Being aware of and reading articles, not only the most recent ones, but the historical ones, as well. Without those shoulders to stand upon, we risk not seeing farther.

REFERENCES:

Evans, J. E., 2008: Electronic publication and the narrowing of science and scholarship. Science, 321, 395, DOI: 10.1126/science.1150473.

Lewis, J.M., 2005: Roots of Ensemble Forecasting. Mon. Wea. Rev., 133, 1865–1885.

Lewis J, Lakshmivarahan S (2008) Sasaki's Pivotal Contribution: Calculus of Variations Applied to Weather Map Analysis. Monthly Weather Review: In Press

(Thanks to Roger Edwards and Steve Weiss for alerting me to this article.)

(Photo from Jupiter Images, as appearing in The Economist)

What if your manuscript gets rejected?

2008-08-11T12:28:00.003+03:00

If the editor has encouraged you to revise and resubmit to the same journal, you should do so. That means the editor thinks that your paper has the potential to get published, if you follow the advice of the reviewers and the editor. Consider the recommendations of the reviewers and editor seriously, revise the paper, decide on a strategy (same journal or different journal), and resubmit the revised manuscript if you think the work is good enough. If you resubmit to the same journal, state in your cover letter that this manuscript was previously rejected and include what revisions you have made to address the reviewers' concerns.

This last point is important. Even if it is not required, I encourage authors to write a response to the original reviewers and send that response to the editor when they resubmit, as if they were resubmitting following major revisions. There are several advantages of this approach. First, like writing the manuscript helps clarify your argument, I believe that writing a formal response benefits the clarity of the revisions to the manuscript. Second, the authors get their say against hostile reviewers. Third, the response allows the editor to weigh the relative views of the author versus the reviewer. Fourth, based on how effectively the editor thinks the author rebutted the reviewers, the editor can choose to send the revised manuscript back to the original reviewers (with the response) or choose entirely different reviewers who are unfamiliar with the struggles of the manuscript.

(Image from http://www.webdesign-guru.co.uk)

How Editors Make Decisions

2008-06-16T16:22:00.002+03:00

How do editors take the two or three reviews and make a decision?

Decisions made by editors are not strictly “majority rules” or the average recommendation.

Sometimes the most severe criticism is what the editor relies on; sometimes it is the most respected reviewer (e.g., associate editor, particular expert, senior scientist).

In some cases, previous submissions by the authors have not resulted in publishable manuscripts. Given this history and marginal reviews, the editor may opt to reject the manuscript for fear of not receiving an acceptable revised manuscript.

Sometimes the reviewers may have indicated certain concerns---the sum of all the concerns from all the reviewers may be such that the manuscript cannot be revised and resubmitted in a reasonable time, so the editor rejects the manuscript.

Other times, it is a combination of many of these reasons.

Finally, the manuscript may simply not be appropriate for the journal. In these cases, the editor may recommend transferring the manuscript to another journal.

Photo by Getty

AMS Publications Commission meeting

2008-06-06T12:15:00.003+03:00

I have recently returned from the AMS Publications Commission meeting. Nearly all Chief Editors of all the AMS journals were present, as were the Publications staff, Chief Editorial Assistants, and the director of Publications Commission Dave Jorgensen.

The main issues at the Publications Commission meeting were the high costs of color figures in AMS journals, the long time between when a manuscript is accepted by the editor and when it appears in print, and the declining number of submissions in most of the journals. Some evidence suggests that these concerns are related. The Publications Commission is resolved to reducing the cost of color figures to authors, reducing the time to publication to a goal of 120 days (the Director of Publications bet the Publications Commissioner that we would be averaging less than 160 days by June 2009), all while maintaining the high editorial standards and publication quality that the AMS offers.

More from the Publication Commission meeting in future posts.

May MWR Highlights

2008-06-06T12:11:00.002+03:00

The May 2008 issue of MWR has the following paper on an upper-air dataset for North America that extends back to 1922.

A Monthly Upper-Air Dataset for North America Back to 1922 from the Monthly Weather Review by Tracy Ewen, Andrea Grant, and Stefan Brönnimann

There is also a four-part series on the potential vorticity diagnosis of the severe convective regime by Gold and Nielsen-Gammon.

April MWR Highlights

2008-06-06T12:07:00.002+03:00

In the April issue of MWR, I would like to highlight the following two papers.

Climatology of Tropical Cyclogenesis in the North Atlantic (1948–2004) by Ron McTaggart-Cowan, Glenn D. Deane, Lance F. Bosart, Christopher A. Davis, and Thomas J. Galarneau Jr.

This paper presents a climatology of tropical cyclones showing that baroclinic effects (not the latent heat release) are important during 60% of tropical cyclogenesis events.

High-Resolution Observations and Model Simulations of the Life Cycle of an Intense Mesoscale Snowband over the Northeastern United States by David R. Novak, Brian A. Colle, and Sandra E. Yuter

The Novak et al. paper is especially noteworthy because they created dual-Doppler analyses from the operational WSR-88D network for the first time.

Negative results

2008-04-18T16:53:00.002+03:00

James Correia sent this interesting article on publishing negative results. This articles makes the case that you shouldn't need to publish negative results. I have thought that we didn't make enough effort in atmospheric sciences to publish negative results. We always publish the big storms, but null cases that were forecast to be big storms are rarely ever published. Here are two counterexamples, plus a third study showing that a number of factors do not affect storm longevity.

Richter, H., and L.F. Bosart, 2002: The Suppression of Deep Moist Convection near the Southern Great Plains Dryline. Mon. Wea. Rev., 130, 1665–1691.

Doswell, C.A., D.V. Baker, and C.A. Liles, 2002: Recognition of Negative Mesoscale Factors for Severe-Weather Potential: A Case Study. Wea. Forecasting, 17, 937–954.

MacKeen, P.L., H.E. Brooks, and K.L. Elmore, 1999: Radar Reflectivity–Derived Thunderstorm Parameters Applied to Storm Longevity Forecasting. Wea. Forecasting, 14, 289–295.

What are your thoughts?

What happens when reviewers have different opinions?

2008-04-14T18:28:00.002+03:00

Recently, I have handled several manuscripts as editor where reviewers offer widely different opinions. For example, Reviewer A might like the paper, recommending only minor revisions. Reviewer B might request major revisions, and Reviewer C might recommend rejection. What is going on? How does an editor make sense of this?

Several things could be going on.

1) The paper is controversial with diverging viewpoints. The number of papers that are truly like this are less common that one would think.

2) One or more of the reviewers is biased for or against the paper, giving a review that would be atypical. In these cases, an editor can generally see through the problem person and make a reasonable judgment.

3) The reviewers do not take time with the manuscript or may be inexperienced, and so don't give a fully adequate assessment of the paper (usually minor revisions, on a paper that requires much more work, as noted by other reviewers).

4) The author has failed to adequately state the purpose and utility of the work. Without doing so, reviewer opinions become diverse, if not plain confused. Some may want one improvement, whereas another reviewer wants the opposite. In these cases, the authors need to more clearly present their work and defend it. Then, reviewers can take a stand for or against the work.

In all cases, good editors can sort through the different opinions and make a reasonable decision.

Scare tactics by publishing companies

2008-04-14T18:11:00.002+03:00

As I have been describing in this blog, publishing companies are afraid of open-access journals and other open-access bills being considered by Congress. One of their arguments is that open access will harm the peer review process.

Such undue government intervention in scholarly publishing poses inherent risks and problems, including:

Threats to the economic viability of journals and the independent system of peer review
The potential for introducing selective bias into the scientific record

"Open access" does not equal "no peer review." That is one of the distortions that the publishing companies use to scare Congress into killing reasonable bills that allow taxpayer-funded research to be accessed for free.

In fact, the highest ranked journal in the atmospheric sciences in terms of impact factor is Atmospheric Chemistry and Physics, published online only by the European Geosciences Union. The journal has a unique peer review process. You submit the paper, and after an initial assessment of quality by an editor, gets posted online for all to read and comment upon nonanonymously. At the same time, anonymous peer reviewers assess the manuscript. After a specified time, the author then must respond to all comments, anonymous and nonanonymous. The manuscripts at ACP eventually get published or rejected, as we do. Page charges are smaller than the AMS (covering web hosting, etc.), rejection rates are only 16%, and there is no cost to access the article. Therefore, the author pays all charges. The readers pay nothing to read the article. This is why open access is so popular among scientists. How many articles would you be able to read online if you didn't have to pay? I know I would read a lot more.

The AMS sits somewhere in between big publishers and open access. The articles are restricted to subscribers for five years, after that, they are free to everyone. That, in my opinion, isn't a bad option. Some journals will allow their otherwise restricted articles to be open access. For an extra $1000 or so in page charges, authors can make their published article free to everyone, not just journal subscribers.

Although I am not a total fan of e-publishing for some contexts, open access is one thing I cheer loudly for. As a fan of science, so should you.

Looking for Review articles for MWR

2008-02-18T10:52:00.003+02:00

When I became Chief Editor this year, one of my goals is to increase the number of Review articles we publish. Few may know, but one of the departments in MWR is entitled "Review." Few papers have been published over the years in that department. In my mind, two good examples include:

Keyser, D., and M. Shapiro, 1986: A Review of the Structure and Dynamics of Upper-Level Frontal Zones. Mon. Wea. Rev., 114, 452–499.

Schultz, D.M., and P.N. Schumacher, 1999: The Use and Misuse of Conditional Symmetric Instability. Mon. Wea. Rev., 127, 2709–2732; Corrigendum, 128, 1573.

If you are interested in contributing a Review article, please let me know. Also, if there is a topic of a Review article that you would like to see and that would be appropriate for MWR, please let us know and I will try to identify potential authors.

Harvard professors to vote on open-access

2008-02-13T11:29:00.001+02:00

Publish or perish has long been the burden of every aspiring university professor. But the question the Harvard faculty will decide on Tuesday is whether to publish — on the Web, at least — free.

Faculty members are scheduled to vote on a measure that would permit Harvard to distribute their scholarship online, instead of signing exclusive agreements with scholarly journals that often have tiny readerships and high subscription costs.

The full article from the New York Times is here.

Why early-career scientists should review papers

2008-02-04T11:21:00.000+02:00

Sometimes when I ask prospective reviewers to review a paper, I hear that they are too inexperienced to review the paper (or some other similar reason). Here is what I told an early-career scientist once.

1) Reviewing papers is something that I believe people should start early in their career. I wouldn't have asked you if I didn't think you could do this.

2) Reviewing papers is good practice for writing and revising your own research papers.

3) I have several other reviewers in mind to review this paper, so your opinion isn't the only one to be considered.

4) Reviewing a paper means that you get to see the paper before it is published. Otherwise, you can't see the paper.

5) Reviewing this paper might give you some ideas for your own research.

6) Reviewing this paper will force you to accelerate your learning on some related topics.

How important is proper use of English grammar?

2008-01-02T15:14:00.000+02:00

This is the body of a rejection letter that I wrote and occasionally gets sent to authors whose papers get rejected. Unfortunately...

I serve as the Editor of your recently submitted manuscript. I am sorry to inform you that I am rejecting it for publication. Your manuscript has numerous grammatical mistakes and nonexistent words that inhibit the ability of a reader to understand your arguments. According to page 23 of the AMS Authors' Guide at http://www.ametsoc.org/PUBS/Authorsguide/pdf_vs/authguide.pdf:

"All manuscripts must be written in the English language. Neither AMS editors nor staff have the time available to edit manuscripts that require extensive grammatical changes, as can sometimes be the case with authors from non-English-speaking countries. While the AMS wishes to encourage the international exchange of scientific results through its journals, it requests that such authors make their own arrangements to ensure that submitted manuscripts are already in correct English. If not, their submissions may be returned unreviewed."

The magnitude and extent to which correct English is not employed in your paper is such that I am returning your manuscript unreviewed, as per the AMS guidelines. Only if your manuscript undergoes significant revision may it be resubmitted as a new article. Should you wish to revise and resubmit your manuscript to any journal, I recommend that you hire a technical editor who is proficient in the English language to improve the manuscript or seek out the advice of a native English speaker who will take the time to provide feedback to you in revising your manuscript. Without either one of these approaches, I am afraid you will find your manuscript rejected by most journals.

Where do reviewers come from?

2008-01-02T12:28:00.000+02:00

Some authors have wondered whether we get our reviewers from other planets. In fact, we Editors get our ideas for who to select as reviewers from our friends and colleagues, the Associate Editors of the journal, the reference list of the manuscript, web or publication searches, or a list of recommended reviewers provided by the author.

Thoughts on reviewers

2008-01-02T12:16:00.001+02:00

One of the principal tasks of Editors is obtaining peer reviews of manuscripts submitted to the journal. These reviews are used as guidance in our decision-making process. Note that the reviews do not mandate to us Editors what the decision should be. (More on decision making in a later post.)

I like to ask early-career scientists to provide reviews. There are many good reasons to include an early-career scientist in the peer-review process. Some of the people I ask are flattered and politely decline because they don't feel prepared to do the review. Here are reasons why I think more early-career scientists should participate in the peer-review process.

1) Reviewing papers is something that I believe people should start early in their career. I wouldn't have asked if I didn't think that the early-career scientist could give a quality review.

2) Reviewing papers is good practice for writing and revising your own research papers.

3) I almost always have several other reviewers in mind to review this paper, so the opinion of the early-career scientist isn't the only one to be considered.

4) Reviewing a paper means that you get to see the paper before it is published. Otherwise, you can't see the paper unless you directly ask the author for a copy.

5) Reviewing the paper might give you some ideas for your own research.

6) Early career scientists usually have more time and typically deliver longer, higher-quality reviews than some senior scientists who don't have the time or take the peer-review process as seriously as they should.

So, if you get a request for a review, you should consider accepting it.

Disclaimer

2007-11-21T15:37:00.000+02:00

I am the 2008 Chief Editor of the Monthly Weather Review, an American Meteorological Society Journal. All posts represent my personal thoughts on the subjects of science, scientific publishing, and meteorology. They do not necessarily represent positions of the American Meteorological Society (AMS), the AMS Council, AMS members, funding agencies, or staff.