Navigation

Minutes

  • Meeting Minutes 
  • Please enter minutes on this page, with the newest at the top. Put a line between entries. Highlight action items in RED with a name/initials indicating who's responsible for the task and a due date, if there is one

__________________________________________________________________________________________________________________________

Assessment Meeting Minutes for 6/2/2014

Present: Linda (chair), Brendan (helped with minutes), Konstantin, Lisa, Dan, Russell, Steve, Sarah, David (minutes)

Next meeting: Thursday, 6/19/2014 at 2pm

Action Items:

  • Dan: Transfer surveys
  • Lisa: Talk to Ken about Ithaka Q15.2 and 16.1 and Q4(?)
  • Constatine: Looking at categories for each question. Talk to colleagues on medical campus about survey needs, and data management
  • Brendan: Working on Google Analytics
  • David: Help with display logic, ask JD and Jenna for input on data management question
  • Sarah: Ithaka Q 22.2: Sarah will compare to our faculty Q35 to see if there is anything we want from Ithaka. Look at reports to assess importance of disciplines
  • Russ: Examples of where WA survey wording better than ours

Brendan: MIT Survey:

  • Some questions are much more specifically oriented. We’ve been trying to stay general with our questions but perhaps now we should start considering using more targeted questions.
  • Linda: “We should start keeping track of the “follow-up questions” like that one.
  • MIT Q13 (where to start looking for a specific ebook) could be useful as a simple statistic/longitudinal stat. Ithaka has a similar question.
  • Q24 (“How would you allocate $100 to improve library system”) Brendan likes a lot. “It’s fun!” Pretty easy to analyze too. Problems: many people can’t add up to 100, and the options may not be open ended enough.
  • Q25’s second question (“Are there other library services you think MIT should be offering”) is good in Linda’s eyes. It focuses on what the users want, not what we think they want.

Dan: Transferring Surveys

  • Skip Logic causing problems.
  • Display Logic ALSO causing problems.
  • Dan will continue to tinker with help of David/Brendan.

Russ: Comparing UW grad & undergrad surveys

  • “Is it typical for us to combine graduate and professional students?”
  • Sarah brings up concerns about using qualtrics for this survey if we’re having so much difficulty with skip/display logic. (“Well we have SurveyMonkey for another year if we need it”).
  • We really need to focus on the wording.
  • Possible wording: “Have you used the library’s services for an internship or sponsorship?” , or something to that effect.
    • “We followed UW’s model.”
    • We should include Marlene if we’re going to continue to discuss wording for Law users.

Lisa: Ithaka Survey

  • Ithaka Q 7.3: possibly useful for follow-up to survey
  • Q 13.4: Possible springboard for question regarding publishing Steve: compare faculty attitudes towards publishers vs. libraries for publishing advice, etc. Russ: should library be responsible for some of these issues?
  • Q 15.2: Question about whose responsibility it is to teach information literacy
  • Q 16.1: Bring Ken into discussion about whether we want something like 15.2 and 16.1
  • Q 22.2: Sarah will compare to our faculty Q35 to see if there is anything we want from Ithaka.
  • Q 6.5 and 8.3: David will ask Jenna and JD to look at data management questions and give advice

Assessment Meeting Minutes for 05/19/14

Present: Linda (chair), Brendan (minutes), Russell, Dan, Sarah, Konstantin, Steve, David, 
Next meeting: Monday, June 2nd, 1:00-2:30; Administrative Conference Room Mugar

Action Items:
 - Russ: Compare UW grad & undergraduate surveys.
 - David: Compare undergrad & grad BU reports.
 - Dan: Transfer all other BU surveys
 - Konstantin: Compare the other BU surveys to each other.
 - Brendan: Compare the other BU surveys to each other & troubleshoot Qualtrics issues.
 - Sarah: Determine any problems with ceasing to collect discipline information & survey keeper


Announcements:
 - Brendan received a promotion to Assessment Data Specialist (or something like that).  Now we can offload all our projects onto him!


Plans for New Survey:
 Linda brings up two important questions:
  1. How will we go about keeping track of our ideas/things to update or edit?
    • If we use the minutes, we'll need to be really organized.
    • Survey Keepers (see Wiki's side bar) are used to keep notes about conventions we've agreed upon, past-encountered problems, and survey decisions to avoid in the future.
    • General consensus to use minutes.
  2. Should we do one per year, or a three-in-one survey every three years?
    • Steve: Our response rates may change if we don't actively target a single user group.
    • David: If we put out a single report at the end for all three, it won't be targeted enough.  We should put out individual reports for separate user groups and perhaps an executivesummary of all three.
    • Sarah: We can filter the survey by user group.
    • Longitudinal statistics casually discussed.  If we want to start recording longitudinal stats, we need to make the changes with this report.  They have been used by other libraries and yielded interesting results.
    • If we do one survey per year, we're putting out one report per year as well.  Worries about survey and analysis burnout float around.
    • David: We could also have the home page link separate surveys for other user groups (everyone likes this idea a lot)
    • We could try using display logic in place of skip logic to bypass the skip issues.
    • General agreement on 3 surveys in one year, each tailored and targeted toward the specific user group, linked from a main survey page, and individual reports instead of a comprehensive report.
Dan's update - Uploading Old Survey to Qualtrics:
  • Differences between the two:
    • The introduction transferred over into Q1.  Not noticeable from user's screen.
    •  Problem where the respondent cannot use the back button over any logically skipped survey content.
David's update - Reviewing Survey Reports:
  • The reports referenced every question, but there wasn't a lot of deep analysis.
  • "Part-time" faculty wasn't a particularly useful demographic.  We should exclude it next time.
  • Question: are adjuncts considered part-time?
  • Q3 wasn't particularly useful either (whether faculty taught or used federal aid)
  • Perhaps we can simplify by not collecting department information.  Sometimes departments are fuzzy with self reporting anyway.
  • Brendan: "Can we extract info from email?  MIT collected no demographics and used email addresses to create demographics."  IRB would have a conniption fit if we started collecting personal data like that.  The anonymity would not exist anymore.  Not worth the time.
  • We can use conditional questioning to work around the parts people find confusing (e.g. with Med & BMC/school related fields)
  • David: "I want to ask about data management in the open access section"
  • Ranking importance is fine, but what we really could use is frequency data. 
  • We should add "in the last year" and drop "academic year" in place of "last year" so the data is current and not related to past experiences.
Russ - Analyzing UW Survey:
  • "UW's survey is about the survey taker.  Ours is about what we think is important to them."
  • We need to package questions from their vantage point.
  • UW's survey is clean and minuses.  Ours can be wordy at times.  It has its plusses & minuses.
  • They included phrasings like "Thinking back to your last..." kind of questions.  May be more useful for infrequent services.
Out of time. Next meeting decided for June 2nd.

__________________________________________________________________________________________________________________________
Assessment Meeting Minutes for 04/03/14

Present: Linda (chair), Dan (minutes), Russell, Brendan, Lisa, David, Konstantin
Next meeting: Thursday, May 1st 1-2:30; Administrative Conference Room Mugar

Action Items:.
Linda will renew Survey Monkey.
Brendan will create a google doc in case groups wish to post comparison info before meetings.

Dan will transfer the prior BU faculty survey from Survey Monkey to Qualtrics (2 copies).

All can begin looking at other survey instruments (UW, MIT, Ithaka) for helpful comparisons.

Announcements:
- Dan announced that Feb 27th marks the 1st full day of external link tracking through Google Analytics (GA).  One can view stats on internal and external links for any page on our site, though those numbers exist in two separate places within GA.
- Linda announced that she and Tom C., Steve, Stephanie, and possibly David and Brendan will be attending a session at the Hyatt Regency on Academic Analytics.  The session will specifically be aimed at librarians.

OCLC Assessment event update
 
Linda will serve on a panel at an OCLC meeting at Brandeis on April 22nd called "Getting the Right Fit: Tailoring Assessment Strategies for your Library."  More info is available here.

MINES
update
MINES is proceeding very smoothly now.  We have had a 0% drop out rate.  We need to think about how to back up the data periodically.  David believes we can download the data at any time, and we agreed that perhaps a schedule should be followed to download and backup the data periodically, although IS&T seems to trust the cloud.

Faculty Survey planning
We began planning for a faculty survey, not in exact order of the agenda, but the info is here below.
Tasks
i.    
Game plan- general, but not unanimous, consensus to perform surveys individually instead of grouped. We also decided to renew Survey Monkey (Linda will do this) once more as the deadline is fast approaching and it may come in handy
ii.   
Developing the survey - will follow these steps
1. 
comparing to other surveys - Small groups will do initial comparison work and then bring suggestions to an AC meeting for all to discuss.  Brendan will create a google doc in case groups wish to post something before meetings.
a.  Dan will transfer the prior BU faculty survey from Survey Monkey to Qualtrics and look for any differences (particularly in skip logic).  Dan will create 2 copies on Qualtrics - one to edit and one for posterity.
b. Linda and Konstantin will then look over comparisons from our own grad and undergrad surveys
c. Next, Lisa and Brendan will look at comparisons to the MIT survey
d. Lastly, Russ and possibly others will compare to UW and Ithaka instruments
2. final pass thoughts - examine possibilities of skip logic, questions or other items to eliminate to keep the survey short, and look through our survey keepers list
iii.     Timeline - We're thinking about Spring 2015.  Russ expressed that we should be mindful of the timing, as Law faculty will be relocated during Spring 2015. Survey fatigue
iv.    
IRB - Linda will accomplish
v.     Publicity/Recruitment - poster from Anita Greene group and emails
vi.    
Launch - nothing noted
vii.    
Analysis, quantitative - nothing noted
viii.    
Analysis, qualitative - have no one currently for this. there is the possibility of going back to tag previous surveys.
ix.    
Report writing - will likely rely on in-house talent.
x.    
Staff involvement in changes - we need to examine this more.

__________________________________________________________________________________________________________________________

Assessment Meeting Minutes for 01/29/14

 

Present: Linda (chair), Brendan (minutes), Russell, Lisa, David, Steve, Konstantin, Sarah, Dan

Next meeting: Friday, Feb. 14th (Valentine's Day) 2-3:30; Administrative Conference room Mugar


Action Items:
- Notify Tim that Pubmed (and other resources?) should be placed behind EZproxy
- LINDA: Call Amy about list being proxied. (DONE 2/3/14- LP)


Announcements:

- Russell will be joining us henceforth as a new member of the Assessment Committee. Welcome!


Tuesday, 1/21 Phone Call w/ UMass Amherst’s Rachel Lewellen

Present were David, Steve, and Sarah and Linda.  Overall, a very positive, informative call.  UMA started out with a year long implementation and collected data from all users who started Ezproxy sessions during two, 2 hr. periods per month. In UMA's second year-long implementation of MINES, they switched to n = 140, and then reduced finally to n = 120. It’s important not to change n too far along so that data won’t have to be excluded, although Terry Plum implied that collection data can be adjusted if a change to N is needed during an implementation. Over 70% of surveyed users completed the survey at UMass (!!!), no complaints registered, not even about frequency. UMA did not have a comment box, but did have an email link. They only received one email. UMA included a question about why survey taker is using the resource.  Shared graphs are promising; of note the URL collection stats of those surveyed generally reflects the URL collection stats of the entire user-base (ie statistically relevant data).  Very promising overall!


Let’s apply this to our MINES! 

- We can use the URL data that will be gathered, but it will involve more involved analysis than other sources of data.

- Konstantin: “We should use ‘BMC Resident’ for primary status question”.  Everyone agrees with the tactful word choice.

- Google Chrome is being problematic for everyone with scrolling issues (only for most recent update).  It may interact with survey if someone tries to get to bottom of primary status list. “Maybe we can change the number of visible items on the dropdown list” suggests David.  Brilliance!  Later, “Alas, we cannot do that” informs David.  Foiled by technology again.  We shall investigate during test period whether Chrome will be problematic

- “High school student shouldn’t be at top of list. It’s not ordered by age (Brendan grumbles), so we should order it by the groups most likely to use it, ie put High school students below staff on the drop down list.  General consensus, Brendan dissents vehemently.   Too bad for him.

- David’s suggestion to get the sponsored question to drop down works.  An excellent solution to the problem we were having.

- “ask@bu.edu” to be changed to “assess-l@bu.edu” so assessment committee members can personally field survey questions.

- On criteria for assigning survey-testing: “Perfect for those who have miffed you!”
- Linda: “We can’t go live until we’re sure the Med campus and Pubmed work.  We need to get Pubmed behind EZproxy.  And guide writers will need to update PubMed links.  Also many Law resources won’t be EZproxied.

- Thoughts on n:

   * Larger student base and greater usage means n can be larger (or smaller? No, larger… we think).

   * We can change n within the first month if our number doesn’t suffice.

   * Linda: “Let’s go around and see what everyone thinks is reasonable.” Crickets

   * After some deliberation, Brendan: “100 or whatever is reasonable”.

   * Number-crunching Steve: “100 gets us just enough data with 70% expectation rate.”

 

MINES Testing

Directions will be on the site.  Mugar to have two demo sessions on Friday and Monday, one in the morning, and one in the evening.  Everyone else to schedule demos for the other branches. Remember, we want to test on as many platforms as possible, even mobile ones.  Also we need to test for blank survey results.  Proposal: Go live on 11th @ 12:00? … eh, maybe not.  We’ll see.

 

Miscelany

-       Steve, on n: “Yes means no, and no means yes”

-       Russ, on difference between Law Library and other libraries: “We actually provide service” (excellent smack talk, if I may say so)

 

NEXT MEETING?

-       Linda: “How does Valentines day work for everyone?”  All concur. There will probably be Valentine’s goodies, and Brendan will probably bring something (he didn’t mention this though). 

Meeting adjourned.

_________________________________________________________________________________________________________________________
Assessment Meeting Minutes for 01/16/14

Present: Linda Plunket (chair), Dan Benedetti (minutes), Russell Sweet, Brendan DeRoo, Lisa Philpotts, David Fristrom, Tim Lewontin
Next meeting: Wednesday, Jan. 29th 2-3:30; Administrative Conference room Mugar

Action Items:.
Linda will work out what staff should test which databases (refnews/alphabetical?)
Linda will change wording on MINES survey as outlined in 'B' below
Sarah will be writing up the instructions for staff testing
Konstantin will answer this question: If asked to give their primary status, would faculty on the Med campus be confused between a 'faculty' option and an 'affiliate' option?
David will look at Qualtrics closely to see if we can easily get the URLs patrons will be forwarded to on completion of the MINES survey.
All who can should attend a call with Rachel Lewellen, Assessment Librarian from Umass Amherst, Tuesday (01.21, 10am)

Announcements:
- Linda reported the library wide meeting was a good success.
- We've been invited to an undergrad faculty council, and a presentation will be given by Tom C. and David.
- Brendan was asked by Dan to help with the Google Analytics initiative.
- Today's meeting will be about MINES, so read this data visualization article in preparation for the next meeting:
Azzam, T., Evergreen, S., Germuth, A.A., & Kistler, S.J. (2013). Data visualization and evaluation. In T. Azzam & S. Evergreen (Eds.), Data Visualization, part 1. New Directions for Evaluation, 139, 7-32.

MINES Initiative updates
Nearly ready to implement, but need to set up staff testing on a test website.  We will be looking for any minor issues, probably with 'refnews' staff, with Linda working out which staff should test what databases.  Sarah will be writing up the instructions for staff testing, and all should review them.  Russ also notes we should start the surveying early enough before finals, or alternatively wait until early June.
A) Some institutions are putting open access databases behind ezproxy in order to track through MINES.  David reminded us that many patrons go straight thru google to resources, so we won't be able to get them.  Tim pointed out that we don't send pubmed thru ezproxy because of the way their website used to work, but we can now.  Because of possible complications with redirects, we should carefully test the performance of very important databases like pubmed. 
B) We worked on the wording of the MINES questions.  Terry Plum had two suggestions on wording: 1-that we don't need an 'other' on q2.CORRECTION BY LINDA: TERRY'S COMMENT WAS NOT THAT WE DIDN'T NEED IT BUT ASKED IF RESPONDENTS WOULD KNOW THAT OTHER REFERRED TO OTHER-AT-BU.  2- on the 'purpose' question, to include 'research unfunded (faculty)', because it is primarily faculty that do research and she wanted to distinguish this from scholarship.   We decided to change this to "unfunded research and scholarship" and not include '(faculty)' or '(students)' in answer options.  We also decided to add an answer option for 'library staff'.  Brendan caught an incorrect plural of "affiliates'.  Finally, a question for Konstantin came up: If asked to give their primary status, would faculty on the Med campus be confused between a 'faculty' option and an 'affiliate' option?
C) Two places are trying to get answers on what resources patrons are using, thru MINES: Umass Amherst and Univ of Toronto.  There will be a call with Rachel Lewellen, Assessment Librarian from Umass Tuesday (01.21, 10am) that all who can should attend.  Umass is also increasing the number of times the survey will be filled out, by having the survey pop up more often.  We decided that we should also track resources, and David will look at Qualtrics closely to see if we can get this easily, as it seems should be possible.

__________________________________________________________________________________________________________________________


Assessment Meeting Minutes for 10/7/13

Present: Linda (chair), Sarah (minutes), Steve, Brendan, Dan, Lisa, David, Konstantin

Next meeting: Thursday October 31st at 2pm.

Action Items:

Dan- open a ticket re: google analytics Primo problem

Linda will work on process of staff feedback as part of analytics cycle

David demo quantitative data presentation

Konstantin will demo qualitative data presentation

Dan will demo GA

Announcements:

university council mentioned David's article that appeared in BU Today

Jack asked us to do some Primo usability work- requires followup.

Dan- tested if primo search terms were being captured in Primo analytics account - no. He has an email in to Brendan at IT, will open as a ticket

All lib mtg for Undergrad Survey scheduled for Nov 14th


Initiatives updates 

Steve- ARL stats mostly done. UStats made the e-collections numbers easier than in past years, probably also more complete.
A number of differences from past years' numbers mostly reflecting changes in how we were gathering the stats
Circ numbers are very tricky in Alma, and vastly different from III approach, so extra challenging this year

Dan- Website analytics will start counting outgoing links and searches from top page. GA not working in Primo currently.

Lisa- adobe suite finally installed

Librarywide Meeting 3-4pm Nov 14th 
Terrace Lounge in GSU 

Discussion
Primo usability- wat that would look like, is it all end-user, or also

Assessment cycle- solutions that need to be passed up the chain


Assessment Meeting Minutes for 9/16/13

Present: Linda (chair), Sarah, Steve, Brendan, Dan, Lisa, David, Konstantin (minutes), David

Next meeting: October 3rd, 1pm.

Action Items:

-review final report by Wednesday 4pm.

- plan library wide meeting  (around October 17)


  • Undergraduate survey

-          David posted it - the final version is ready!

-          Everyone should read it by the end of today 9/17 (4pm) or Thursday morning the latest.

-          Bob Hudson will send it  to the University Provost and the Associate Provost, Undergraduate Affairs.  The report will then be shared with the deans of the undergraduate schools, as well as other key administrators on campus, such as the Dean of Students and Vice President for Enrollment and Student Affairs. The report will also be sent to library directors.

-          Sarah will work on the News Letter for the Mugar websites

 

  • Survey Comments:

The deadline for returning the survey comments were August 15. We will give a new due date after our library meeting. Only changes that were made will be posted on the web site. (Linda will send an email to Dan)

 

  • MINES

Linda reported: We don’t need an IRB approval-  so we don’t have to include the question about exclusion of students under 18 years old.

Konstantin presented the draft MINES survey to the Assessment Committee for review. Lots of good discussion and a few suggestions for changes ensued. Significant decisions made by the group included:

·         The survey will be optional rather than mandatory for users.

·         No comments box.

·         There will not be any links included on the survey.

·         The survey will be as short as possible both in terms of number of questions and choices within questions.

 

Questions for Scott Macomber, IS&T were raised.

 

·         The Assessment Committee would like this survey to be optional rather than mandatory. We would like the survey to be presented to  every Nth user, and then for the user to have an easy way to opt out from the survey page. How is an optional choice implemented?

·         What happens when a user hits the "back" button either from the survey page or from the resource if they've already been passed through to the resource?

·         If a user hits close (or submit or finished) and they haven't filled out the survey, what gets reported? Is it blanks? We are concerned since it is possible to design the survey such that the first answer option for each question is visible in its text box.

·         Approximately how many times is EZproxy called on an  average day? This will help the Assessment Committee decide whether N (as in Nth user) should equal 250 or 500.

·         When will the test survey be ready?

·         Is the survey a webpage or a pop up?

·         Will we be able to change the survey after we give it to you for testing?

·         Can we change the Nth number between now and January? What about changing the Nth number after we go live in January?

·         How will you re-direct the user to the correct URL once the user has completed, or opted out, of the survey?

·         Will survey be accessible on mobile devices (the Qualtrics survey looks fine on a Google smart phone.)

 

  •  ARL Statistics

October 7th – due date

 

  • USAGE STATS

Steve reported on the use of journal statistics.  Steve, Tim and Brian are developing a plan for cancelations for this fiscal year. The deadline for making cancelation decision on September 1.  

 

  • DATA VISUALIZATION:

Lisa submitted a request for a cloud based Adobe Suite.

 

Usability issues -  Lisa brought up the question of usability testing for library search – we will discuss it next time.

 


Assessment Meeting Minutes for 9/6/13

Present: Linda (chair), Sarah, Steve, Brendan, Dan (minutes), Lisa, David

Next meeting:  September 16th 11:00AM

Action Items:
  • David will revise the report to create a new version, with suggested edits.  Thank you, David!
  • Dan will send Yelena's suggested edits to David in an email.
  • Dan will add three survey keepers indicated below on the survey keepers page on this wiki.
  • Brendan will check over the quotes in the survey report to make sure they are consistent with what was written in the survey, and that none are duplicated.
  • Brendan will ensure that figures in the report use consistent phrases and capitalization.
  • Sarah will check footnotes and captions on figures to ensure consistent phrasing and capitalization of BU Libraries.
  • Sarah will check through the report looking for where percentages or numbers begin sentences.
  • Linda will share the video Shirley created with the committee through email.
  • Linda will perform a last review, and coordinate sending out the finished report to Bob, the Provost, etc.  Dan will then help post it to the website.
  • Steve will finish up serials work with Maggie, and then introduce her to David for work on the data dashboard he has created.
Announcements:
    Dan announced that Yelena reviewed the draft, and had only four or five comments.  Dan will send these to David in an email for consideration.  Some were the same as points raised in today's discussion, and he will note those for David in the email.

    Discussion:

    1> Review of David's draft began and many edits were made on the fly.  Reported below are items that rose to action items, or that were of some significance.
    a) Review began with David and Brendan speaking about denominators on Figure 8.  The issue was whether to ONLY tally those students who selected Mugar, for example, as a most frequently visited library (Brendan's query), versus tallying those students who selected Mugar as a
    most frequently visited library AND those students who selected Mugar as a next most frequently visited library (as it is in David's draft).  It was decided that as long as the footnote is very clear as to what is being tallied (and it is in this case), we are satisfied it can stand as is.
    b) Sarah suggested we make sure quotes are accurate as to what students actually wrote in the report.  Brendan will check over the quotes in the survey report to make sure they are consistent with what was written in the survey, and that none are duplicated.
    c) Sarah noted that some sentences in the draft began with percentages or numbers.  The committee decided that percentages were ok, but that other numbers should be written out as words.  survey keeper 1Sarah will check through the report looking for where percentages or numbers begin sentences.
    d) Lisa asked about the capitalization and usage of BU Libraries and all its wonderful variations.  Sarah will check footnotes and captions on figures to ensure consistent phrasing and capitalization of BU Libraries.
    e) In going through the report, we noted some figures used inconsistent capitalization of labels.  Brendan will ensure that figures in the report use consistent phrases and capitalization.
    f) It was suggested that we consider making the URLs in the report live links.  The committee decided against this, in part because of style considerations, and in part because we would like to keep folks focused in the report rather than leading them elsewhere.  survey keeper 2.
    g) Brendan pointed out that where a percentage appears in the text of the report, the quoted number may not exactly match the corresponding numbers in a chart.  This slight difference is due to the way that the numbers have been rounded.  The commitee decided this is ok, as it is a common statistical practice.  survey keeper 3.

    2> There was a discussion about what tasks might be best suited for our student worker Maggie.  Her work on the Serials Review with Steve is nearly complete.  Various projects were considered, and then it was decided that David's data dashboard project may be a good fit.  Steve will finish up serials work with Maggie, and then introduce her to David for work on the data dashboard he has created.

    3> Next steps for the undergraduate report were considered.  David will revise the report to create a new version, with suggested edits.  Thank you, David!  Linda will share the video Shirley created with the committee through email.  Linda will perform a last review, and coordinate sending out the finished report to Bob, the Provost, etc.  Dan will then help post it to the website. Steve will finish up serials work with Maggie, and then introduce her to David for work on the data dashboard he has created.


    Assessment Meeting Minutes for 8/14/13
    Present:  Linda (chair), Sarah(minutes), Konstantin, Steve, Brendan, Dan, Lisa
    Next Mtg: Aug 22 Thursday 1-2:30

    Survey keepers
    • ask if they live on/off campus (impact w/ lib as space, hours, etc)
    • entire cycle (all 3 surveys) should use same style in reports for ease of camparison

    We noted that after we finish writing the report, we should review the comments for actionable themes for library admin to act on (eg broken outlets) that might otherwise be missed.

    We went over 9 of the remaining 11 comments areas for best quotes for the survey:

    Hours (SRS) 

    , outlets



    DB-renovation, overcrowding
    KS- positive services, positive subj libs
    Linda- group study, light, noise

    a couple executive summary comments

    Report
    steve 
    suggests flipping default graph layout from low-high to high-low
    BU Libs consistency
    89-87%- statistically significant percentage difference for "most important"

    ss and ks -we tagged them as we detected patterns- comments themes frequency//table p.3 reword the headers in table on p.3? 

    heated debate about qualitative data





    Assessment Meeting Minutes for 8/12/13

    Present: Linda (chair), Sarah, Konstantin, Steve, Brendan, Dan, Lisa

    Next meeting:  August 14th 1:30PM

    Action Items:

    • Finish reviewing comments for representative ones to include in David's report for the top 15 commented areas:
      • Konstantin 4,8
      • Sarah and Linda: 1,2,5,7,9, 11, 12,13,  & 14
    • Discuss a rough draft of David's report
    Announcements:

    • Kelly accepted a job in sunny San Diego! The committee will make a decision if and when to invite a new member to represent BU Theology Library.
    • Training sessions
      • Alma Analytics - Wednesday 8/14 11am @  Mugar Library (Administrative Offices)
      • Qualtrix -  register here: http://www.bu.edu/phpbin/training/register/index.php?admingroup_id=55   Tip: even if the class is full there's still a chance to attend if there's a "now show"  The class this Thursday 8/15 at the Medical campus from 2:30-4 is full but there are other classes at the CRC.
      • NVivo - next session is in October. 10/4 10am


    Minutes:

    Reminder: the target deadline for submitting the Undergraduate Library Survey report to the Provost is the first week of September

    Our method of working with the comments: through the committee's "crowdsourcing" we've identified the most representative comments (i,e, our focus is on the comments' tone and message)

    To see comments in NVivo: \Dropbox\Storage\Shirley\Comments\2013 Undergraduate Student Library Survey Comments ALL\NVivo Files

    Important: "sitting  and furniture" category were separated into two categories but in NVivo the nodes are nested, i.e. "sitting" is the child of "furniture" node. 57 items are broadly about the furniture and 66 are specifically about the sitting. There's no overlap. We should relabel them or conglomerate into "Sitting & Furniture."

    Important: the comments are presented in the order of importance based on today's discussion.

    Dan: #10  comments about citation managers

    The prevailing sentiment in comments was that EasyBib rocks (RefWorks not so much).  EasyBib was by far the most mentioned tool.  There were just a few positive comments on RefWorks. 

    The committee liked this quote best (particularly the 2nd sentence).
    "EasyBib is okay, but no good if you are doing APA. I really like RefWorks, we just started using it this semester in my WR150 course."

    Here are two others I chose that are more representative of the overall sentiments of undergrads.
    "Easybib is my go to site for citing research papers."
    RefWorks is an extremely disappointing tool. EasyBib is much more effective and user-friendly."

    Finally, here's a good one if we wish to present "the information need" in the report:
    "I would like to learn more about EasyBib or other bibliographic managers as I think it could be very helpful and let you work more on your research paper than the bibliography."


    Steve #3   Library Resources

           "  The research capability that BU Libraries offer is amazing. The amount of databases that I have access to is very helpful when I need to find information, whether it be critical essays for my writing class or scientific papers for my chemistry or biology classes."

            "  I really appreciate the library's selection on a wide variety of topics. I know that I'll always be able to find a book specifically pertaining to any research paper or extracurricular interest I might have."

          I think the library is a wonderful and vital part of our university. This year I have taken primarily science courses, which is the main reason why I don't use the resources offered by the libraries (Interlibrary borrowing, databases)."

          "Online resource articles are great and have more books/articles online for students to access would be a great help."

           "The amount of material at these libraries is fabulous. I have always been able to find a large amount of information and books on whatever topic I was looking for."

           "I use the databases/eJournal access all the time but never go to the library to do it."

           Some short & pithy comments:

            "Wonderful and extensive."

           "Electronic access is paramount!"


    Lisa # 6 Mugar Library General Positive Remarks

    Most of these statements were very short and general- good, great, that type of thing. The prevailing theme was that Mugar is a good place to study. Here are two quotes along those lines:

    "Mugar is great! I love knowing I can go there and study or find a good print source for a paper. Everything is easy to use, and if you're stuck, there's always someone available to help."

    "Mugar seems to have the best set up and is very easy for me to study in. This library in particular has become a very comforting place for me to study on those late nights."

    Lisa # 15 BU Library Search   

    The themes in this section included difficulty finding known items and frustration over irrelevant search results.

    "It’s very difficult to figure out if the library actually has a certain book. Also, many of the links to online journal articles or books that come up as results in the Mugar search engine do not work."

    "The overall library search engine can be very frustrating to deal with. Oftentimes, the words that I search are not related to what the search engine produces."


    Discussion:

    Dan suggested that we can choose a comment to represent the usefulness of EasyBib for the undergraduates and/or choose a comment that compares the use of EasyBib and Refworks.  The group wanted to know if it's possible to identify comments by the status - freshman, sophomore, etc.

    Steve shared his observation that the representative comments were positive about textbooks. Linda suggested to take a look at OpenEducation at UMass (Amherst) - this is a good resource for the survey follow up. Textbooks are important for the undergrads and we can do something about it with these comments. (see http://guides.library.umass.edu/content.php?pid=87648&sid=1714807 )  The OpenEducation initiative offers incentives to the faculty at UMass Amherst to create an openly available "course pack" in lieu of textbooks.

    We discussed the stylistic "quality" of comments. We agreed that comments cannot be 'cleaned up' so not to distort the meaning of the comments we can correct grammatical mistakes only so that we could use these comments in the final report.

    There was some concern about the misgivings of using qualitative data and quantitative data in the mixed method since our goal is to identify representative comments and not to claim that the comments reflect the overall trends in collected quantitative answers. 

     
    -------------------------------------------------------------------------------------------------------------------------------------

    Assessment Meeting Minutes for 7/29/13

    Present: David, Sarah, Dan, Konstantin, Brendan, Lisa, Steve (Minutes)

    Next meetings:

    • August 12th 1:00PM
    • August 16th:  Time TBD on August 12th.

    Action items:

    • Review all comments for representative ones to include in David’s report for the top 15 commented areas:
      • Dan: 10
      • Konstantin: 4, 8
      • Lisa: 6, 15
      • Sarah: 1, 2, 5, 7, 9, 11, 12, 13, & 14
      • Steve: 3
      • Note: Dan & Sarah will look into the difference between commented areas 9 & 13
    • Plan to share examples with each other before we meet on the 12th

    • Review and comment on David's draft report (http://personal.crocodoc.com/sc19LS - to make comments, add yourself as a  collaborator, can select text or area (e.g., on graphs); particularly, take a close look at the Next Steps section.) Plan to discuss at our next meeting.

    Announcements:

    Top 15 comments areas:

    Here is a table of the final Comments totals/hot spots.


    TOP 15 Most Commented Upon Issues

    1. Extended Library Hours Requests - 24hr (111) and longer (164) 275

    2. Complaints about Mugar Environment (Renovation requests) 174

    3. Positive Remarks on Library Resources 137

    4. Positive Remarks on Library Services 123

    5. More Outlets in Mugar Requests 119

    6. Mugar Library General Positive Remarks 96

    7. Overcrowding in Mugar 74

    8. Subject Librarians Positive Remarks 71

    9. Seating (Comfortable Seating requested/complaints about hard chairs) 66

    10. Bibliographic Management Tools (Students were very interested in these) 64

    11.More Group Study Space Requested 59

    12. More Light in Mugar Requested 58

    13. Furniture in Mugar (hard chairs, want comfortable seating) 66

    14. Noise in Mugar 55

    15. BU Library Search - Problems with access or use 49




    Assessment Meeting Minutes for 7/15/13

    Present: Linda (Chair), David, Sarah, Dan, Konstantin, Steve, Lisa (Minutes)

    Next meetings:

    • July 29th 10:30AM: Determine how to choose illustrative comment examples for David’s report. These should come from Shirley’s 15 categories of most common themes.
    • August 12th 1:00PM
    • August 16th:  Time TBD on August 12th.

    Action items:

    • David will follow up with Sara G. & Brendan DeRoo re: inviting Brendan to serve on committee.
    • Dan will clarify with Brendan G about what Google Analytics’ numbers mean, specifically the 58% “reloading” or library/index.php stat.
    • Dan & Kelly will suggest to Tim/web committee that the javascript be initiated to track outgoing external link data. 

    Announcements:

    • Now that Shirley is gone, we have a vacant space on the committee, preferably to be filled by a parapro. Brendan DeRoo is suggested by Linda & David. Everyone approves.
    • N-Vivo training will be offered on 7-24-13 in the CAS computer lab. 4 of us have signed up so far.
    • Qualtrics training is offered a few times this summer from IS&T. Using Qualtrics for the surveys instead of SurveyMonkey would save the library money. Linda encourages us to sign up.

    Discussion (Updates on AC initiatives)

    Dan: Google Analytics

    • Met with Brendan Gannon from IS&T last Thursday.
    • A problem with Google Analytics is that it is not tracking outgoing external links (ex: WorldCat Local). July 2012-June 2013 shows that Primo page views are higher than library homepage views in Google Analytics. Could be due to med, law users who access Primo from from different library websites.
    • Goal: to see what links are being clicked on our homepage. 13% use databases a-z: by far the most popular link. Low usage: news, scan & photocopy, suggest a purchase.
    • There’s confusion about what the percentages mean. The percentage is really the number of hits a link gets, not clicks? What does it mean when 58% of hits are of the library/index.php page?
    • We do not was to exclude our own use from this number.
    • Steve:  What does >100 more pages mean? We don’t link to 100 pages from our homepage. Sarah: Thinks we could have 100 hotlinks on our homepage.
    • Based on the stats, Dan thinks our homepage is not very well designed. Linda thinks this is a premature conclusion.
    • What could we do based on this data?
      • Think of better ways to highlight links/services. Are some of the links not obviously hyperlinks
      • Push frequently used links to top. Ex: hours.
      • There is Javascript we can use to track outgoing links. Everyone thinks it would be good if we could get this data.

    Linda: ARL Stats

    • Russ, Steve, Tim, Sarah H and Linda will meet on the 22nd.
    • Alma analytics can help us answer collections questions. But there are issues with circulation data through Alma and we don’t know why numbers are wrong.
    •  Will include Joe from MED after first meeting.
    • Managed sets in Alma help with stats. Sarah H is good person for questions about managed sets.
    •  We should include a cautionary note that we have a new system this year so numbers are rough.
    • Steve: We can also double check the stats from Alma against Victor’s numbers.
    •  Konstantin will join meeting on 22nd.

    Sarah & Konstantin: MINES

    • How do we implement MINES? The little publicly available information on this involves SFX, which we have moved away from. Most likely possibility: EZProxy.
    •  Will meet with IS&T including Scott Macomber and Peggy Watts Gup to discuss how to implement MINES with EZProxy. Marie Kelley is likely to be project manager for this.
    • After that, next step is getting in IS&T’s schedule.
    • What will we get out of this? Broad level demographic data(grad, undergrad, faculty, researcher, on, off campus, school) and purpose (funded researcher, unfunded research, teaching, service).
    • Next steps: determine what have other schools have done with this data? How have they used it to get funding, communicate value, etc?

    David: Undergrad Survey Report

    • Has data in SurveyMonkey & JMP.
    •  Modeling off of Graduate Report.
    • Is looking for trends and is breaking things down by library: Mugar, Pardee, Pickering, SEL. Example: Compared expected library to actual library. Steve points out that Mugar has most seats and longest hours.
    • David has an outline that we can review.
    • More comment analysis this year. David needs illustrative examples of comments, we can help with that and editing.

     Steve: Serials review.

    • Working w/ Maggie to determine cost & use data for individually subscribed journals List of journals, once we finalize will include about 400 titles. How we get the usage data for these journals is the next step. 
    • Print journals: Search BULS for all print journals and check to see what we have e access to.
    • Next steps: Develop criteria for evaluating journals. Examples: For cost per use data, what would the cost cutoff be? For print journals, would the criteria be a marginal subject area & and access via certain platforms? 

     


    Assessment Meeting 6/24/2013

    Present:
    Linda, Lisa, Steve, David, Shirley, Dan, Kelly Sarah

    Next meeting: 10:30am on July 15th (Dan on Google Analytics, Linda on ARL stats)

    Action items:

    • All: Get comments on assessment plan to Linda by Thursday (she is sending plan to directors on Friday)
    • All: Get any suggestions on open positions to Linda by 1pm tomorrow (25th).
    • Linda and Shirley: Talk about what comments should go to Bob. 
    • Shirley: Send out comments to staff, with form to collect what people are doing about them. 

    Announcements:

    Linda made several announcements:
    • Reported that Jack's comments on plan are that we need assessment of instruction (we will wait for new position), and usability (not his word) for BULS.
    • Mentioned that cross-library Primo group is being organized by Tom 
    • There will be an AULS meeting tomorrow with Bob to talk about open positions. Shirley brought up need for internal training; possibly part of communication position? Sarah said emphasis should be on internal communication; would help with staff morale. 

    Shirley's Presentation on NVivo for Analyzing Survey Comments

    [This is my “stream of consciousness” transcript of high-points of her presentation.]

    Just talking about one part of large program. Created project, imported data from data set (excel file from SurveyMonkey). Will put latest version in DropBox "Undergraduate Survey 2013 last.nvp".

    Can choose fields to be tagged, in this case all comment fields (shown in white).

    Create "nodes" (equivalent to tags, with hierarchy) Took quick and dirty approach, there are many others possible approaches.

    To tag, highlight all or part of comment, right click, "Code Selection" at existing or new node.

    Hierarchy can be edited (drag and drop, or cut and paste). Aggregate node to include children.

    View->Coding Stripes lets you see what codes are applied to given record.

    When you create new node, you need to look back to see what else should go there. Started with tags from previous survey.

    Faster than Excel process, but not as happy with end product.

    For resources, did breakdown by subject manually while coding. May be possible to do programmatically, but would be more work up-front.

    Need not consistently tag positive vs negative.

    Can combine data sources (though this project had a single source).

    For Mugar, a lot of positive general comments. Negative comments tend to be specific.

    We should not edit Shirley's file, archive it somewhere, possibly in IR with permanent embargo.

    For research librarians, should send all of them all comments.

    Unused "Respondent" nodes are for matrix stuff which wasn't done.

    Queries: Word Frequency, with stop list

    Reports: Canned or create you own. Into Word document.

    Report Coding by Node summary, use to create top 10 list

    Has book on NVivo, Linda will keep it. Paula is ordering another one.

    Group will talk at next meeting how to carry work forward.


    Goodbye to Shirley

    Everyone said nice things about Shirley, which can be summed up as “Irreplaceable.”




    Assessment Meeting 6/10/13 

    Next mtg Monday 6/24/13  9:30-11 
    Present: Benedetti (minutes), Riddle, Starikov, Plunket (Chair), Fristrom, Leslie-Smith, Philpotts 

    Action items
    • All- look at and comment on the Assessment Plan (2013-2016); make sure the Assessment Plan meshes with the Goals
    • To update the Plan with any changes, please send an email to Linda requesting the change by 6/19
    • All- rewrite goals to eliminate the use of "we"
    • David will be working on the draft of the report
    • Shirley will continue working on the comments
    • Dan will investigate frivolous responses within the survey
    Announcements
    1. The newly revised Assessment Plan is up in Google Drive, in the Goals Folder 
    2. We will spend the next meeting discussing the Comments from the Undergrad Survey 

    Status Update on Goal Statements and the Assessment Plan 
    Assessment Committee (AC) members should look at and possibly comment on the Assessment Plan (2013-2016).  The Assessment Plan is in the Goals folder on Google Drive, and will be submitted to the four library directors for review when completed.  The current time frame for the plan is 3 years, but could stretch to 2017 - it will be an evolving document. 
    If there are items within the Assessment Plan that don't mesh with your Goal Statement, one or the other should be updated.  To update the Plan with any changes, please send an email to Linda requesting the change.  AC members should revise Goal Statements with any suggestions from the last meeting.
    As we move forward with our plan and goals, we should think about how to best structure our meetings to best support work in our areas.
    Later in the AC meeting, we did some live editing of the MINES goal document.  AC members should investigate the use of "we" in their goal documents and eliminate those references. 

    David's Overview of Survey Monkey Results from the Undergraduate Survey 
    The Undergraduate Survey Report is largely expected to follow the general outline from the Grad Survey Report.  Some areas of possible further investigation highlighted at meeting were:
    • breakdown of CAS majors by class year, to look at just where the undecided folks are, or other factors
    • check out folks who chose no primary library and then commented on that section
    • interesting that undergrads use journals as well
    • want help finding scholarly info but have not had a class with librarians or viewed a guide
    • indicate Search is more important than physical space, yet use physical space much more often
    • tie question 19 (would library assistance be helpful...) back to faculty
    Other considerations came up in our discussion.  How do we implement and market instructive objects?  Students want to know what we have to offer.  We need to consider surveying all three patron groups at the same time, using skip logic. 
    Dan noticed at least one facetious survey respondent via an inappropriate comment.  There could be others who simply have clicked through the survey to get into the iPad mini drawing, but how do we know?  A possible answer would be to include a question like: "If you're still reading this click 2".  But the general feeling among the AC was that we likely have a statistically insignificant number of these problem respondents in our results - Dan will investigate.
    David will likely have a draft by August and we would like to produce the Report for submission to the Provost in September.

    Shirley's Preliminary Discussion of Work on Comments from the Undergrad Survey 
    Shirley is using NVivo to work on the comments this time around, which is a slightly different approach.  She is not using queries but setting up nodes.  Areas of the libraries that seem to have garnered the most comments:
    • 242 comments on resources
    • hours - desire for being open 24 hours a day
    • environment (natural sunlight, outlets, new furniture, complaints about noise)

    __________________________________________________________________________

    Agenda for AC meeting on 5/24

    1.       Review of re-worked goal statements

    2.       Discussion about the suggestions from the Data Management Committee

            3.       Preliminary look at Steve and Maggie’s organization of use/cost data for journal subscriptions


    Present: Struble , Riddle (minutes), Starikov, Plunket (Chair), Fristrom, S. Smith, Philpotts, Benedetti


            Next meeting: 9:30 on June 11

            Announcements
            Shirley leaving at the end of June
            
            Action Items
            David will delete prize files from Dropbox and Google and send a note to the group on how to lose those files from our computers
            Linda and David: Decide which files we are keeping--this is specifically for the survey
            Linda will write a formal thank-you to the data management group
    David and Linda and Shirley will document naming conventions; document software data for data management and survey workflow documentation
            Linda will add "Next Steps" category to everyone's templates
            Think about who will take on responsibility for survey comments
            
            Goal Statements
            Linda and Konstantin will massage old assessment plan into new plan--will be much shorter and link to goal statements; should be finalized by mid-June
            Everyone will post final version of goals statements by next meeting; think in terms of quarters for next steps; think about what help you need on your projects
            Change 'Leaders' to 'Project Leads'
            Look at Lisa's next steps and do that for our goals; under 'Timeline'
            
            Agenda
           
            E-resources data discussion: Steve and Maggie
           Goal is to evaluate all subscriptions; three or four broad classes of material--journals in electronic format (solely or print and e)--subset of things we get in packages; print only; databases     
              First step is cost per use data (for e journals first)--need use data and cost data--none of the data is in the same place
              1500-1000 titles
              Looking at CPU over three years--going back to 2010 where possible; showing article accesses--PDF download, e.g.
              Cost data is in Millennium
              Deleting stuff we don't have cost data for; Maggie is going through Millennium right now to gather cost data
              Putting things in UStat from ExLibris--cloud-based; generate usage reports for all journals entered so far; annual trends, most frequently used; cost-usage statistics by title--for example, Nature is 2 cents per use
              Right now looking at use for individually-subscribed journals only
              Still need to: add SUSHI accounts--protocol that allows ExLibris to go to journal platform and harvest use data automatically; want to make it talk to Alma to pull subscription costs
              need to figure out how to get out 2013 cost data--some in Millennium, some in Alma
              Need to export Ustats and compare to list of serials
              There is a general login, so people can view.
              Helping us identify stuff that we may want to review
              Any cancellation decisions need to go to vendors by September 1 at the latest
              Cost per use is only one factor--this will help us identify what to evaluate to get rid of

              This presentation is a model for how to update us on goals. Hoping someone will be presenting at each meeting for goal updates.

              Data Management group report
              Three suggestions
              Consolidate files
              Have consistent filenaming convention
              Should have dropbox or googledocs for everything

             Concerns about dropbox: is it reliable enough? Dropbox  Not FERPA compliant; all of prize files should be deleted--this is the only sensitive information; not for long-term archiving, but desktop mirroring service

              David will delete prize files from Dropbox and Google and send a note to the group on how to lose those files from our computers
              Going forward, the list of participants should not go to everyone 
              Use IR for long-term data management and storage
              Need to decide which files we are keeping--this is specifically for the survey; Linda and David will take this on
          
              Evaluating E-Resources
              The outcome is that this is cyclical, becomes a routine activity
              
              MINES
              By itself, MINES doesn't give very useful data, but combined with what Steve's doing, user surveys, is helpful; for U administration, what's being used for research and what's being used for teaching is helpful; gives enough information to inform assessment activities in other areas; recognizing the holes in our PR and education; will be an evaluation
              There is a good bibliography on MINES--if people get a chance, look at articles; also 1 hour webinar; need to think about technical implementation
              3 implementation factors: where to put it? ezproxy server? somewhere else? don't even know what options are
                   other two are--whether we make it mandatory or not? whether we do it two hours a month for twelve months or nth number of hits or 3 month periods with frequent pop-ups and mandatory

              Visual Reports
              David asked good question about who these reports are for and how they will be using them
              Short list of what you would recommend as the next reports
              Under purpose--add secondary audience; first place is administration, next place is to users; can't expect undergrads to read twelve-page report; will clarify that that's in the future
              One of next steps is to reassess potential audiences, and how to make this an ongoing process--treat this as a pilot

              User Surveys
              
              ARL Stats
              All libraries are pulling data from Millennium
              Should be more about next steps for the future; uniquely situated because we have a fresh slate now; 
              will take ARL form and fill in data according to form--this needs to be articulated
              
              Google analytics
              Goal is website improvements, not google analytics
              News items; website design is very static--looking at news items
              Outgoing links tracking is problematic
              May never get off the top page
              Worth talking to web committee about and/or Jack about changing the website
              Initial steps--evaluate at some point to see if it's still worth doing
              Re-write in light of this is a true evaluation 

    assessment mtg 5/10/13


    Next mtg Friday 5/24 9:30-11


    Present: Struble (minutes), Riddle, Starikov, Plunket (Chair), Fristrom, S. Smith, Philpotts


    Action items

    • DF- document template for goals by 5/15

    • Linda and Konstantin- address communication in assessment plan

    • All- rewrite goals in template form and read over all goals by 5/24 mtg


    Announcements:

    1.Linda discussed ARL stats annual mtg

    • web discovery as a factor in counts not clarified this year

    • space: want square footage and seats comparatives b/c of crunch. May become a new question, or a separate site, but not this year

    • One voice asked to think about outcomes rather than just expenditures, as a better match for what our campuses are asking us for

      • Libval w/ UTennessee and Carol Tennopir is one tool to start looking at Return on Investment

      • U Minnesota’s work to track by individuals

    • 3 takeaways from Linda

      • Look at libval

      • Look at U Minnesota’s work

      • Ithika’s (countrywide) faculty survey- compare with our local data (Maybe to present at conf in Chicago in 15 months)


    2.Hired a student-Maggie. She’s available through summer until she graduates in Dec 2013. Steve working with her currently on use data for journals. Ustats is an ex libris use-data program for counter compliant data, that was part of SFX (sushi client), that we are now looking at taking advantage of. Can also add in cost data from Alma. Will be good for cost per use and evaluation, especially of journals.


    Review Goal Statements:

    rewrite and target specific areas

    map goals and timeline more closely to revised assessment plan (2-3 year)

    template w/ uniform goal statements

    • purpose (including context)

    • identified leaders

    • methods - (including cost effectiveness and needed tools/support [e.g., data crunching student labor])

    • timeline - specifically next steps

    • anticipated outcomes

    • followup


    Other goals or things the Assessment committee should be doing, or does do but has not documented


    • leadership of assessment-driven work librarywide

    • better communication w/ staff about how assessment informs planning




    4/18/13 Assessment meeting

    Minutes for 2013-04-018


    Present: David, Sarah, Dan, Steve, Linda (chair), Shirley, Lisa, Konstantin (minutes)


    Next meeting: 10:30-12, Thursday April 18, Mugar conference room


    Agenda items for next meeting: Discuss Graduate Survey results, report on selection of winners, discuss comments for F14 goals


    Action Items: upload shared documents (re: FY14 leadership group plan, i.e. Grad survey/Impact Response, ARL, Google Analytics, MINES) to Google Docs (5/25) and comment by 5/8


    David and Dan - communicate to Shirley 3 names (iPad winners) for a photo-op, normalization of survey data (dupes, 18+, undergrads)


    Shirley - follow up: Mary McGowan and students (library video)



    1. Research Assistant position

    Linda, Steve and David will interview Mia (Meggy), a graduate student, for the Research Assistant position (the Assessment Committee needs help with usage stats analysis, comments, perhaps Google Analytics.) We also hope to set up an infrastructure so that we’ll be able to have a permanent position in the future.


    2. Update to Survey Results and Library Staff Responses

    Mary McGowan will be working with her students on creating a library video in which library staff will share their ideas and report on actions taken in response to the graduate student survey. Shirley will follow up with Mary.


    3.Undergraduate Survey

    We reached 21% response rate!

    Planning our distribution of prizes:

    -We need to verify the following:  1) undergrad status 2) 18+ 3) remove duplicates

    Our strategy: 2 volunteers (DF and DB)  - 

    David and Dan will vet  dups, undergrads, 18+

    Shirley will work on the photo-op (David and Dan will provide 3 names to Shirley)

    The list of all winners will be deleted after the prizes have been given out.


    4. Graduate Survey

    Quantitative analysis – David encouraged everyone to check the analytics in Survey Monkey and report to him if anything worthy of noting.  BE CAREFUL NOT TO DELETE DATA!!!

    Qualitative analysis – Shirley reported that NVivo is limited to help us with analysis of comments because of exports/imports issues. Shirley is still determining how we can use NVivo for our purposes. One main problem is the limit of 100 comments – Shirley will find out if we’re allowed to increase this limit beyond 100 comments.

    David and Shirley pointed out that it would be interesting to use mixed methodology method and to explore how we can coordinate qualitative and quantitative types of data.


    5. Timeline

    We are planning to report our results from the Undergraduate Survey in the Fall Semester.


    6. Assessment Committee Goals

    Rough draft of the survey (DF, SLS, LP) - updates for the group

    Impact Response (Lisa) - Lisa will post updates for the group

    ARL Stats -  Linda will post updates for the group

    Google Analytics – Dan will post updates for the group

    Mines -  Sarah and Konstantin will post updates for the group

    We will post documents for sharing and commenting on Google Drive.

    To add comments: highlight what you’d like to comment, right click, add comment




    Minutes for 2013-04-01


    Present: David, Linda (chair), Sarah, Kelly (minutes), Shirley, Steve, Dan


    Next meeting: 10:30-12, Thursday April 18, Mugar conference room


    Agenda items for next meeting: Discuss survey results, scope of report--just undergraduate survey v. all 3; continuing discussion of FY14 goals


    Action Items:


    Sarah will follow up with Lisa re: flyers in CGS


    Shirley will send email with link to brainstorming assessment page to bulibrariesstaff@bu.edu


    All: Read brainstorming page from top to bottom; send comments to/let Shirley know when you have read it (http://www.bu.edu/library/about/library-assessment/brainstorming/)


    Sarah, Lisa: work on mock-up of UNLV and UW graphic using our data

    Linda will send Sarah and Lisa a copy of the UPenn brochure


    Steve and Linda will examine documents re: collections and university programs and propose similar document (see Announcements)


    Linda will draw up standardized list of ARL stats and publish to library staff: this is what we collect; what are you doing with other stats? (to streamline assessment operations throughout the libraries)


    Konstantin: add language to Assessment Plan re: the role of the assessment committee in the libraries: how do we support, which things are our responsibility (e.g., conducting surveys and focus groups)


    Dan/Kelly: 2nd draft of Google Analytics goal, including info on working with web committee, updated timeline, proposed questions


    All: In drafts of goals outlines for FY14, tie goals to Assessment Plan


    Sarah: Add to MINES goals what questions we will ask during evaluation period


    Announcements


    Undergrad survey: Faculty will get email re: undergraduate survey on 4/2, students will get reminder email on 4/4


    Linda: Tim Barbari and Beth Loizeaux (Assoc. Provost for Graduate Affairs and Assoc. Provost for Undergraduate Affairs) want to evaluate collections in preparation for bringing on new faculty and starting new programs (chiefly grad. programs). Have requested a brief document on how this would work.

    This will be good for communication and raising the profile of the libraries. Has been influential in other schools on going ahead with programs (Georgetown, U. Maryland).


    Assessment Webpage


    Only issues actually being addressed should be on page (take out Stone)


    Changes agreed upon: Shirley change intro on action plan responses page to present tense; add to next to last paragraph “within their departments”; make sure names are formatted consistently; school libraries now “Professional Libraries”


    Undergrad Survey


    13% have responded so far (Survey Monkey documentation notes that you have to give answers to count as responding for statistics--does not count if someone just views the survey)


    Uptick in respondents after Dean Elmore retweeted link to survey


    Shirley has worked very hard on HTML for faculty email


    Publicity Work


    David--Mary F has covered Com, JD Eng, Brendan Astronomy


    Kelly--Flyers hung in CAS, FitRec, will take to Stone this afternoon


    Konstantin--Put some in CAS


    Steve--Put approx. 10 flyers around CFA, a few in SMG, including undergraduate lounge on 2nd floor


    Sarah--3 flyers in SMG; SHA--left large poster with staff in dean’s office for them to hang, left 12 to be placed on tables; MET--bulletin board near undergraduate services office


    Shirley--has placed 17 posters around dining services, excluding some Starbucks and Breadwinners


    Dan--poster in the lobby (will be up for 1 week) and 5-6 flyers in SED; digital displays in the Center for Career Services, 100 Bay State Rd.


    Linda--Bob Hudson on agenda for admins meeting; Bob Brown mentioned including in BU Today, and Linda has contacted IRB to see if this is okay--has not received response, will update group


    Next Steps


    2 emails on 4/2 and 4/4--faculty and students


    Close survey at end of business day on 4/16


    David will harvest data


    Comments: NVivo training in fall; tutorials available now at IMC in the SED


    Timetable for report: have ready to send to deans and undergrads in fall


    Goals for FY14


    Ongoing discussion: want to pause surveys for a year and do MINES before restarting the 3-year cycle and evaluate other tools; would give time to think about focus groups, door surveys, smaller targeted surveys on (e.g.) spaces, etc.


    ARL stats

    MINES: Will this tool work? Embraced by ARL

    Factsheets: Yes--good PR

    Revise Assessment Plan: 5 years old

    Overall survey: asses process, look at tool itself

    EResources: Yes


    Audience for Goals page on Assessment page: library staff, deans, Bob; not for assessment groups at other libraries (discussions of tools not important); present tense


    Google Analytics


    Can’t track outgoing links


    Primo search? Leave out for now; just look at site


    Questions: How can we change/improve site? Can we use GA to improve/inform collection development?


    How to interact with web committee? We analyze stats and then push to web committee?  What can we not control or study fruitfully? (Primo search)


    Look for base set of data (old GA data)

    Contact folks at IS&T after questions developed: Ron Yeany (mentioning Brendan Gannon); also bring Tim Lewontin into discussion; add these discussions to timeline


    MINES


    This year: 1. evaluate, 2. decide


    Administering tool itself--two options:

    Option 1: Do MINES through ARL and pay them for report; we would deal with IS&T; ARL delivers reports and peer comparisons

    Option 2: We use tool ourselves


    Questions for evaluation period:

    Do we work with U. Toronto? At what stages do we work with them?

    What is ARL’s timeframe?

    We have to decide every nth time questions appear, how long we run it.

    What info can we get on whom? (IS&T)



    Minutes for 3/21/2013

    Present: David, Linda (chair), Sarah (minutes), Kelly, Konstantin, Shirley, Steve. 

    Next meeting: 1:00-2:30 Monday, April 1st, Conference room 
    Agenda items for next meeting:
    Linda will cover new work to be undertaken, to get the libs involved when new programs proposed
    Discuss work to articulate goals for public website

    Action items:

    Sarah- send google groups on MOOCs and aside about hesitancy for the group registration

    Shirley- move old assessment plan to website from wiki, update paragraph on top page, add Konstantin's profile

    Steve- follow up with contact at Ithika

    Put posters up:

    • in small branch libs (Astro-David, Stone-Kelly, VAL-Sarah)
    • Steve will drop off ones for SED for either Dan or Yelena to post
    • Linda will follow up on other branches (SEL all set, Steve delivering to PERL)

    All - write up copy for website on our upcoming goals, each for their own area of responsibility

    Announcements:

    MOOCs- sign up if interested

    One page handouts- Lisa will do this (will be her goal) UW and UNLV examples

    Steve- Roger Schoenfeld from Ithika would like to communicate with us about our surveys, Steve will follow up

    Survey keeper- always do undergrads in spring semester to minimize underage students

    Survey Keeper- there are certificate and non-degree programs in several colleges/schools. This survey does not cover any of them

    Publicity work:

    David- Mary F wil cover com and JD hit Eng yesterday

    Kelly- dropped off poster and flyers at CAS (they have someone who posts them) and droped flyers at fitrec. Will follow up tomorrow

    Konstantin- CAS stuck a couple up

    Put posters up in small branch libs (Astro-David, Stone-Kelly, VAL-Sarah)

    Steve CFA and SMG- not yet, will hit today and tomorrow

    Sarah- SHA and MET not yet

    Shirley- starbucks refused, most others were fine (put takedown date and contact info on back

    Social media - mmcgowan and tom Kurland working on the display outside library. Error today, will be fixed tomoorw

    Denise putting up today in Mugar

    Dan got displays out to digital

    Steve will drop off ones for SED for either Dan or Yelena to post

    Linda will follow up on branches (SEL set, Steve delivering with PERL)

    Assessment Webpage

    Reword intro on top page

    Staff page- Shirley will add profile for Konstantin, add Lisa to list

    Added some visual interst (charts) on grad and fac survey pages

    Social media

    Continues apace- tweets going out (10 total), Dean Elmore will retweet, news piece being posted, blog posting with cc to facebook >all same wording

    Surveykeeper- be more open in IRB wording for social media next time?

    Goals

    have leaders for each goal- post to website along with objectives and timelines, a few well thought out paragraphs of description. Committee review before posting. Linda will work with anyone who wants help on what to write up.

    MINES- Sarah

    Factsheets- Lisa

    Google Analytics- Kelly

    Konstantin- revise assessment (plan form 2003 w/ Linda)

    Overall survey –David (quantitative and rpt, Linda admin inc IRB, Shirley-qualitative (comments)

    Eresources evaluation- Steve to lead (Linda and David and Tim also working on this)

    Dan?

    Other assessment collection work that we have been tasked with but not yet undertaken- Tim Barbari wants the get the libs involved when new programs proposed (future goal)

    LP will cover this in next mtg (add to agenda)

    Next Mtg

    Agenda- set regular mtg times


     




    Minutes for 3/11/2013

    Present: David , Linda (chair), Dan, Sarah, Kelly, Konstantin,Shirley (minutes) Steve. 

    Next meeting: 11:00am on Thursday, March 21st [is going to include lunch but Linda has to leave at 1:30pm]
    Agenda for next meeting: TBD.

    Action items

       Linda will send out a 'cheat sheet' of things to remember when asking permission to hang posters

       Linda will let us know when the 8 1/2 X 11 posters arrive at her office.  Everyone has to fetch their set of posters from Linda.

       Linda will send out the email about the upcoming survey to all library staff

       Linda will obtain a list of faculty email addresses  (faculty broadcast email to go out April 2)

       Linda to get Bob’s approval for the brainstorming page

       Linda to talk to Mary McGowan about a responses video.

       Dan to check the google doc prize entry spreadsheet to check that Linda's input appears in it

       Everyone - let Linda know guestimates for number of posters needed ASAP (8.5x11)

       Everyone  - prepare to put up posters in designated areas AFTER the broadcast email has been received by the undergrads(due March 19).

           David - Eng &  Comm

           Dan - SED & 100 Bay State

           Kelly - FitRec & CAS

           Konstantin - CAS

           Lisa - Sargent & CGS

           Sarah - Hospitality & MET

           Shirley - All dining halls

           Steve - CFA & SMG

       Shirley to create an Outlook Calendar for Assessment that can be shared

       Shirley to open a ticket for allowing bu.edu access only to the results of the brainstorming sessions

       Shirley to email all contributors to brainstorming page and ask them to edit their pages for the BU 'public'

       Sarah to contact Tom about reviewing the brainstorming solutions, particularly budget items

     

    Agenda

    1.       Minutes, announcements, changes to the agenda
    2.       Update on Undergrad Survey
    o   Publicity
      Digital displays
      Posters, assignments for schools, colleges, administrative units
    o   Communication
      Recruitment emails
             Four email webpages
             Test
             Registrar email list for undergrads
             Provost list for faculty
      Deans
      Staff
    o   Prizes
    o   Survey data
    3.       Feedback on Graduate student comments from departments & branches

    4.       Goals for FY14

     

    Discusson

    1.   No announcement

    2.    The survey will open on March 19 and be open for 4 weeks.

    3.   Dan reported that digital displays are all set for 13 locations around campus. 

    All of the colleges and schools listed in the survey drop down list will be covered, as well as dining services and 100 Bay State Rd.  

    Some have multiple locations and they will post in all of them.

    The digital images will only be sent once the survey opens so that the survey url remains inaccessible.

    The images will be sent in a variety of formats/sizes – as requested by the recipients (e.g. ppt, templates, webforms).

    Dan has established contacts and they have committed, only the time frame remains to be set. The images are ready.

    The digital images will only be sent once the survey opens so that the survey url remains inaccessible.

    The images will be sent in a variety of formats/sizes – as requested by the recipients (e.g. ppt, templates, webforms).

    Dan has established contacts and they have committed, only the time frame remains to be set. The images are ready.

    4.   Mary McGowan is in charge of the images and posters.

    5.   Digital images may not be available in Warren Towers – maybe posters could be used?

    6.   Mugar digital display will present a split screen: the right hand panel will display our poster, the left will ‘float’ pieces from the poster.

    7.    A large paper poster will be displayed inside the entrance of Mugar. 13x19s will be displayed in the two elevators.

    8.    The AC will hang and take down 8.5x11 posters in the schools and colleges.  Each committee member must make ready to hang the posters and ascertain the approx. number of posters they require.  Linda will let us know when the posters arrive.  Please fetch them from Linda’s office thereafter. Linda will provide a checklist to follow when requesting permission to hang posters. Use judgement - avoid posting in grad lounges.  Face to face contact is preferable when asking permission.          

    • David - Eng &  Comm·          
    • Dan - SED & 100 Bay State·          
    • Kelly - FitRec & CAS·          
    • Konstantin - CAS·          
    • Lisa - Sargent & CGS·          
    • Sarah - Hospitality & MET·          
    • Shirley - All dining halls·          
    • Steve - CFA & SMG

    9.        Resident Assistants will post the posters on every floor of the ug residences.
    10.    Warren Towers poster will go up 3/25 unless weather is inclement.
    11.    Mary McGown is working on the bus posters.
    12.    The tweet is ready – we are only asking Dean Elmore to retweet it.
    13.    Provosts first broadcast email will be left as is. It will be sent out as a broadcast to over 18 undergrads on March 19.
    14.    The emal address list of the undergrads is reay.  The groupings we wanted are possible.  Linda wll handle this.
    15.    Survey Monkey renewal occurs in the 15th but Linda has contacted them and the staff at SM will ensure coverage.
    16.    Ipad minis have arrived (wifi only).
    17.    Email to library staff is ready, Linda will send it out.
    18.    Linda has the faculty list of email addresses and the broadcast email to all faculty will be sent out on April 2.
    19.    Dan to check that Linda’s test of the google doc prize entry worked.
    20.    Shirley to open libfiles server as safe server, ticket.
    21.   Lunch will be March 21 Thursday) at 12:30, following an 11-12:30 meeting.(edited by Linda)
    22.    Shirley to create an Assessment Calendar to share with AC.

    23.    Goals for FY14

    ·      Survey : Leaders – LP, DF, SLS
    Could look across data for all 3 surveys

    ·      Collections:Leaders SDS, DF
    Will evaluate e-resources, starting with databases, will try Counter (Tim, Steve, David, Linda)

    ·      ARL Stats: Leader – LP
    (Joe H. Russ Sweet, Amy?,Ruth Malesky will join) will seek definitions for how we are getting our numbers

    ·      Assessment Plan 09 revision: Leader – KS
    Revise and update document for 2014

    ·      Google Analytics: Leader – KR
    Seek what to do with GA, how to get useful data.

    ·      MINES: leader –SRS
    exploratory.  ARL do analysis, will have peer group comparisons.  See who is using our resources.

    24.        Dan will do Qualtrix training
    25.        Linda, Sarah, Shirley, Konstantin will do NVivo training.
    26.        Brainstorming solution proposed web page was discussed and the responses reviewed:

    ·      Change layout to subject (name) problem/solution.

    ·      Email respondents to let them know it will be open to the public, so there’s a chance to rewrite. DF asks if we want to edit for public understanding (eg, use of words Alma and Primo)

    ·      Set a date when it will go public [March 31]

    ·      Responses as a PR tool –edit with an eye to public review

    ·      Add word “comments” to heading, and put it in perspective (actions based on comments as well as aggregate data)

    ·      Next step might be to push out through our existing PR avenues (Linda will run past Bob first – after the date to go public)

    ·      For security responses tweak wording to reflect that door checks etc are Mugar-specific

    ·      Pardee’s response as an example of response to user lack of knowledge about what is in our collections/databases

    ·      Look at what branches and law med theo can do ti share the followup on survey comments

    ·      Add dates to terms like “over intersession” and “by midterms” (eg, “by spring 2013 midterms”)

    ·      Ask all staff to review ALL TOPICS for things like policy changes to BLC cards, or need for another micro scanner

    ·      Add the fact we only asked for two topics to be identified and solutions posed

    ·      SRS will ask Tom to review (micro scanner as example)

    ·      LP will talk to Mary McGowan about a responses video

    ·      Shirley will password protect with BUlibraries

    ·      Shirley will open an IS&T ticket to get a response from Ron Yeany about  levels of password protection

    ·      Dan mentioned (somewhat unrelated) that libfiles as a place for image parking makes libfiles more open to the public. Does it need a content review?

    ·      Assessment committee can edit in google doc, Shirley will ask others to email her directly






    Minutes for 2/25/2013

    Present: David (minutes), Linda (chair), Dan, Sarah, Lisa, Kelly, Konstantin, Steve. Special guest: Dan O'Mahony

    Next meeting: 1:30pm on March 11 [was originally going to include lunch, but that has been postponed]
    Agenda for next meeting: Wrap up on survey, Shirley's presentation, goals for committee.

    Action items
    • Linda will write letter to library staff about undergraduate survey; share it with committee for suggestions before sending out.
    • David will periodically harvest survey data while it is happening for backup. Others should stay out of Survey Monkey
    • Dan will talk to Shirley to get URL of Thank You page 

    Discussion

    We introduced ourselves to Dan O., and he introduced himself: He is Director of Library Planning & Assessment, and Scholarly Resources Librarian in the Social Sciences at Brown. Currently finding out about what assessment means to various libraries. How it is divided up (Brown doesn’t have a group right now). How to handle qualitative stuff?

    Linda and Sarah described what Shirley did with comments from survey. General discussion. Dan O. said that they used lots of people in tagging; got more front-line and collection development librarians involved. Sarah asked how did they deal with privacy concerns? Dan O said it basically didn’t come up (they had stripped any identifying information).

    Talk about top-down vs. bottom-up assessment. Dan O talked about involvement in space planning.

    Linda asked about turnstiles added at Brown libraries (replaced guard desk checking IDs). Talked about how they handle data from turnstiles.

    Dan O. left.

    Linda: Update on ungrad survey. Provost weighed in at IRB, we are expedited, should hear by or before the 11th.

    Publicity: Linda met with Mary McGowan, she is taking on much of the responsibility. Mainly electronic, though there will be 100s of posters. Assessment committee members will be responsible for hanging (and taking down). Dan is investigating sizes of digital displays. Shirley is working on emails. Linda will write letter to library staff, we will see before it goes out. Gifts being worked on -- meeting decided we will go with Wifi only iPad Mini with 32Gig. David will periodically harvest survey while it is ongoing; everyone else (except Dan) should not go to SurveyMonkey while survey is on. Dan will talk to Shirley to get URL of thank you page (non-templated). Dan: Fitrec electronic displays come from cable channel they don’t control, so we will go with print posters there.



    Minutes 2/4/13

    Present: Linda (via skype), Shirley, Kelly, Lisa, David, Sarah, Konstantin (minutes), Dan 

    Next meeting Monday 2/25/13, 2:00pm--3:30pm, Mugar Conf Rm  (Dan O'Mahony from Brown will be joining our group)

    ACTION ITEMS:

    • Set up a spreadsheet and url for collecting names (Linda will include it in the IRB application)
    • Brainstorming comments 
    • Linda will break down by schools # of excluded students
    • submit comments regarding text in 2 image files (by Thursday)
    • proofread the survey - make sure we didn't missing anything, spelling, etc. 
    • change reference to # of prizes in the Introduction (Linda)
    • Linda will ask David Lazar to include students from the certification program.
    • Dan and David will take another look at the skip logic
    Announcements:

    Dan reported on SED survey review - recommendations weren't useful, N/A - some questions were raised. This was followed by a discussion in which we've concluded that our survey is in good shape.

    Linda talked about the timeline for submission to IRB. On Saturday Linda arranged with David Lazar, the Communication Specialist, Provost Office, a plan to encourage students to take the survey outside of the classes. Linda also asked to create a list of the undergraduates 18+ so that we could limit the number of invitations to those who are <18.

    Graduate certification programs (?)

    We need a break down by school of students that need to be excluded.

    Linda talked about the communications about the physical and the digital posters. Bethany and Mary's comments about colors, text, etc.

    David took us over the action items from last time (changes made by Dan, Kelly's communications with MET, skip logic, graphics).

    Shirley made suggestions about skip logic -  what if the student skips the primary library, is the second most frequently library going to be displayed?  Dan and David will take a look at it.

    Linda reported that she made changes to the Introduction to comply with the IRB - # of prizes and chance of winning. Some changes to the wording (f.e.x Individual libraries).  

    Discussion followed about search and discovery tools - we ask about importance and satisfaction but not about usage.

    Shirley -  do we want to ask specifically about our website. Discussion followed and we all agreed that for our purposes it would be too general.

    WE AGREED THAT OUR SURVEY IS FINAL EXCEPT PROOFREADING !!!!!!

    Linda will wait until Tuesday before saving the survey as a PDF file.

    How soon should we let other librarians know about the survey?  We agreed that this should be done after we get IRB approval.


    Agenda for next meeting:

    • lunch
    • communicating with staff
    • brainstorming comments
    • revisit our goals for this year
    • Steve will give a short presentation (ebooks and ejournals)
    • leadership roles



    Minutes 1/28/13

    Present: Linda (via skype), Shirley, Kelly, Lisa, David, Sarah, Konstantin, Dan (minutes)

    Next meeting Monday 2/4/13, 1:00pm--2:30pm, Mugar Conf Rm

    ACTION ITEMS:

    • Dan will make changes discussed to "draft1" of the survey, and notify Steve of 2 particular changes.
    • Dan will announce to the group that the draft1 version is ready to review.
    • Kelly will talk to a MET administrator to see if our two MET questions make sense.
    • Dan will contact and consult with the SED graduate student in an expeditious manner.
    • David will verify there is nothing we can do about skip logic for those who skip questions.
    • Everyone should proofread the new version as soon as Dan is able to make the changes. 
    • Everyone should review the graphics to be submitted soon by Mary McGowans group of students. Would you find this attractive / want to see it on the bus, etc. ?

    Agenda:

    1. Minutes, announcements
    2. Conflict of Interest forms
    3. Administrative support & budget
    4. SED grad student involvement
    5. 'Pilot results (implications/changes to survey instrument)
    6. Graphics
    7. IRB Application
    Discussion:

    1. No announcements

    2. Linda relayed that no conflict of interest form needed.

    3. We have administrative support from the Provost’s office for the broadcast email and the creation of the email distribution list, where we can hopefully exclude non-degree students. Bob Hudson has given the go-ahead for funding our incentives. We will not print poster or other materials until we have the final URL, QR code, and possible approval from the IRB.

    4. SED grad student
    Dan reports an SED grad student is interested in helping us. Her name is Meredith Knight and she is a post doc specializing in online survey writing. She will meet with Dan to review our questions and give her impressions of our work. She will not become a co-investigator or publish anything resulting from this survey, but this is a good opportunity for collaboration. We decided we may want her to meet with us to discuss types of analysis. She is likely not interested in our paid position but Dan will find out. As the survey is very close to IRB submission, time is of the essence. Dan will expedite the process by sending a PDF of the questions to her by email, and then review it with her this Friday. 

    5. Pilot changes to our survey
    We reviewed the Pilot results and comments and came up with these changes:

    Intro paragraph
    C
    hange "fifteen minutes" to "10-15 minutes". 

    Demographic section
    Kelly will talk to a MET administrator to see if our two MET questions make sense.
    We discussed CAS majors because of some confusion but in the end decided on not changing them at all.

    Services section
    On q8 in this section about "librarian-led instruction", we will add a "don't know" option.
    On q9, the "Borrowing / getting items not owned by the BU libraries" option will change to match the Interlibrary Borrowing option in q23 [i.e., "Interlibrary borrowing (Interlibrary Loan, BU WorldCat Local)"].

    Individual Libraries section
    To help students read our questions, we will capitalize MOST FREQUENTLY and NEXT MOST FREQUENTLY on q11 and q13.
    We discussed moving this section down because of some confusion on whether to answer for specific libraries (eg only for Mugar) but in the end decided on not changing the order at all.
    For anyone filling out the same library in both q11 and q13, we need to disregard their answers to q14.
    If a student skips q11 or q13 question, their answers on the next q should not be tallied in the data analysis.
    David will double check on what we can do about being unable to control skip logic for students who skip a question. Dan believes we are unable to control skip logic for those students that skip ahead - they simply go to the next page in our survey.

    Library Resources section
    In q16 we will get rid of the "other" option that is seemingly serving no purpose.
    We had a repeat of the concern about the ordering of sections during this discussion, but again chose not to change the order at all.

    Research Methods section
    We agreed to delete "to you" in q18 in this section.  We also agreed to pass this by Steve - which Dan will do.

    Overall Contributions Section
    We will delete the "planning for your future" option on q22, and again will pass this by Steve. 
    We will change the q23 scale label on very important as noted by several students from 1 to 5.
    On q23 and q24 we will change "librarian assistance with reference and research" to "research assistance from librarians" which matches the option on q9.

    Everyone should proofread the new version as soon as Dan is able to make the changes. 

    6. Graphics
    Everyone should review the graphics to be submitted soon by Mary McGowans group of students. Would you find this attractive / want to see it on the bus, etc. ?

    7. IRB Approval
    We should very nearly be ready to submit for IRB Approval after Monday of next week.

    _____________________________________________________________________________________________________


    Minutes 1/22/13

    Present: Linda (via skype), Shirley, Kelly, Lisa, Steve, David, Sarah (minutes), Dan

    Next meeting Monday 1/28/13, 2:30pm--4:00pm, Mugar Conf Rm

    ACTION ITEMS:

    • Linda will talk to Bob about extra funding for incentives
    • Linda will draft letter to pilot takers, send to assess-l
    • All review and comment on draft letter
    • Linda send link to conflict of interest form
    • All complete conflict of interest form
    • Dan creating a copy of the survey for the pilot, with modified comments boxes; Look for the BU icon to use; randomizing and alphabetizing as necessary. Fixing skip logic direction in 87 majors 

    Review wording/ question order:

    New order:

    1. Demographics
    2. Services
    3. Individual libraries
    4. Resources
    5. Research methods & Skills
    6. Overall contributions

    Top paragraph review for IRB:

    Mary M met with Linda re publicity and incentives

    • prizes would be best incentive, 1 or 2 big ones (suggested mini ipad) many smaller (e.g., convenience points)
    • Linda will talk to Bob about increased funding for that

    Publicity

    • We will be working with Bethany (an Anita Greene scholar from CFA) –looking at images like graphic novel covers. Colorful, eye catching, light
    • 9 files (including digital display, posters, possibly postcards) 
    • Promote with posters etc in GSU, BUshutle, new Deerfield St student services building, in departmental spaces> ask for broad permission.

    17 year olds

    • consensus to leave both mentions of ages in intro paragraph. 
    • Screen by age in all images (must say 18 years and older) 
    • ask Registrar to only put 18 year old and older in email list if possible

    Wording/phrasing:

    Changes made in real time

    Review order of responses on individual questions:

    Dan will randomize order of items in Q9, Q 12 (Q14 same random order as Q12), Q22, Q23 (Q24 same order as Q23 except overall at bottom); re-alphabetize Q11 and Q13

    Pilot:

    Cathy M coming up with list of pilotees

    Linda will send us an email to review by 2pm Wed 1/23, finalized for Cathy M by end of day Wed

    In email to pilotees, ask them to look for:

    • Confusing or misleading wording or options
    • Open up comments boxes for comments on the survey structure/wording itself (make it clear that there will be actual comment boxes in real survey)
    • Ask for time estimation
    • Encourage them to take the real survey when it comes out
    • 17 year olds can’t take the pilot either
    • Deadline (midnight X day)
    • Link to survey
    • Ask them not to share it
    • Will be eligible to win prizes for real prize

    ____________________________________________________________________________________________________________________________

    Assessment Committee meeting minutes 01/18/2013

    Present:  Dan, David, Kelly, Konstantin, Lisa, Sarah, Shirley, Steve (minutes)

    Next meeting:  Tuesday, January 22, 1:00-3:00 pm in the Mugar Conference Room

    Action Items:

    Dan: Complete edits discussed at today’s meeting in Survey Monkey

    Sarah: Scan the questions for continuity and consistency (specific examples listed below)
    All: Complete CITI Human Subjects training (see Linda’s email)

    For next meeting: we will begin planning the pilot.

     

    Agenda

    Review questions in Survey Monkey

    Specific issues identified:

    • Wording in introduction: “Library’s websites”—is this what we want to use?
    • Want to use “If other, please specify” throughout (except on Q20, which will stay as “Other (please specify)”)
    • Scan headings for ‘rate’ questions to make sure it matches the question
    • Where appropriate, “during the current academic year” should come at the end of the question.
    • Where appropriate, questions should begin “How important to you…” or “How satisfied are you…” (see Q8 in Survey Monkey for model)
    • Where examples are provided, they should read (e.g., [blah blah]); “etc.” should not be included
    • Make sure that N/A is provided as a response option where appropriate
    • On the list of primary library and ‘next most frequent’ library, need to re-alphabetize after splitting Law Annex, Law Library (Pappas), and re-baptizing the School of Theology Library

     

    ____________________________________________________________________________________________________________________________


    Assessment Committee meeting minutes 01/15/2013

    Present: Linda (Chair), 
    David, Kelly
    Steve, Konstantin (minutes), Shirley
    , Sarah, Dan, Lisa 

    Next meeting:     
    Friday, January 18th 2:00-3:30 pm in the Mugar Conference Room

    Action Items:
    All: Edit & add assigned survey questions to Google Drive by Thursday 12pm (or sooner!)  
    All: Complete CITI Human Subjects training (see Linda’s email) by January 17 by 5pm
    Dan: Scan the questions for continuity and consistency (i.e. tense, wording, for example "library" vs. "libraries"
    For next meeting: we should discuss the hiring of the graduate student assistant.

    Agenda

    1. Minutes, announcements
    2. 2nd round of question development--no more than 10 minutes per question
    1. Announcements:
    Linda (and the group) welcomed Lisa Philpotts (Mugar) who will be participating in the next 5-10 meetings.

     
    2. Question Development (2nd round)

    Participants discussed implemented changes. Discussion followed what further changes should be addressed next. Linda reassigned questions as follows:

    Demographics (Shirley--> David )

    Library Resources (Steve --> Sarah )

    Research Methods and Skills (David --> Konstantin)

    Individual Libraries (Konstantin  --->  Shirley)

    Services (Kelly -- > Steve)

    Contributions (Sarah --> Kelly)

    We've agreed that it's important to consider the following during our next meeting:  "worst to best" vs. "best to worst" order, "least to most" vs. most to least", etc. 

    Consistency in wording is very important! see Q 21 for example.

    Include n/a   (also test inclusion of n/a in a pilot)





    Assessment Committee meeting minutes 01/10/2013

    Present: Linda (Chair), 
    David, Kelly (minutes)
    Steve, Konstantin, Shirley
    , Sarah, Dan 

    Next meeting:     
    Monday, January 14th at 10:30am in the Mugar Conference Room
                                   Friday, January 18th 2:00-3:30 pm in the Mugar Conference Room

    Action Items:
    All: Complete CITI Human Subjects training (see Linda’s email) by January 16th.
    All: Continue to wordsmith the questions in your section (DF - Research Methods; KS - Individual Libraries; LP - Contributions; SLS - Demographics; KR - Library Services; SS - Library Resources)

    Agenda

    1. Minutes, announcements
    2. Question development--no more than 10 minutes per question
    1. Announcements:
    Linda : Dan O'Mahoney from Brown University will be visiting us in February to get input in order to create a more robust program at Brown.  Everyone agrees that both February 11th and February 25th (both Mondays) from 2:00 to 3:30 PM will work for us. Linda will submit both times to Dan as possibilities for his visit.
    Dan: Suggestion to use survey keepers as a checklist after questions are finalized, since some concern the IRB process.

    2. Question Development

    Demographics (Shirley)
    Change "Freshmen" to "First Year" (BU terminology)
    Check on wording/titling of College of Health and Rehabilitation Sciences/Sargent College
    Possibly list primary discipline for CAS and MET
    Move frequency questions to Services section

    Library Resources (Steve)
    Bold "library resources" and/or otherwise make clear that this question does not apply to textbooks or books/materials students have purchased
    Discussion on whether print and electronic books should be one option or two--would give use data but this is not a preference question; also not all books available as both print and electronic
    Include "during the current academic year" wording

    Research Methods and Skills (David)
    Use specific "databases" instead of the word "databases"--e.g., JSTOR, EBSCO, PsychINFO, etc.
    Add Y/N question: Have you used a bibliographic manager program to assist with citations (e.g., RefWorks, EasyBib, Zotero)?

    Individual Libraries (Konstantin)
    Change wording to "My primary BU library that I visit most frequently is..."
    Wording issues with "Order and cleanliness of the physical collection"--want to address signage, ease of finding items on shelves

    Services (Kelly)
    Now includes frequency questions
    Include "during the current academic year" on frequency questions
    Include scanners as a service

    Contributions (Sarah)
    Add "Decision to attend BU" as fourth option on question "What contributions do the BU libraries make to your:"
    Possibly use UW wording for importance and satisfaction questions
    Make sure overall importance question and satisfaction question (Q19, Q20) have same phrasing and "importance" and "satisfaction" in bold

    ___________________________________________________________________________________________________________________________


    Assessment Committee meeting minutes 01/07/2013

    Present: Linda (Chair), 
    David, Kelly
    Steve, Konstantin, Shirley
     (minutes)
    , Sarah. 

    Next meetings:   Thursday, January 10th 12:30-2:30pm in Mugar Conference Room
                                    Monday, January 14th at 10:30am in the Mugar Conference Room

    Action Items:
    All: Complete CITI Human Subjects training (see Linda’s email) by January 16th.
    All: Read Kelly's google doc in the Assessment folder on the google drive.
    All: Wordsmith the questions in your section (DF - Research Methods; KS - Individual Libraries; LP - Contributions; SLS - Demographics; KR - Library Services; SS - Library Resources)

    Agenda

    1. Minutes, announcements, changes to the agenda?
    2. Review process and timeline for development of the undergrad survey form
    3. Continue with question development for the undergrad survey (KR, SS,KS)
    4. Look at Survey Keepers for those relevant to the undergrad survey (DB)
    5. Schedule next meetings

    1. Announcements:
    Linda : Dan O'Mahoney from Brown University will be visiting us after mid-February sometime.  He won the poster session at the Baltimore Assessment Conference. He created a web form to train staff to categorize and tag comments.  Dan wants input in order to create a more robust program at Brown.  
    survey keeper - wait till Dan is back

    2. Review process and timeline for the undergraduate survey
    Linda would like the survey completed by mid February in order to submit it to the IRB. We should meet frequently over the next few weeks to finalize the undergraduate survey.

    3. Question development for the undergraduate survey
    a) David and Linda created a document combining all of the questions used in the UW and BU faculty, grad and undergrad surveys.  They selected  21 questions to be included in the current BU Undergraduate Survey. Linda handed out both documents.
    b) We reviewed the deleted questions, then examined the 21 questions which were suggested for inclusion.  Everyone agreed on the inclusion of the 21 questions.

    Discussion
     
    A)Modifications to the 21 Questions:

    Q1
    We need to add "Undecided" in here somehow.

    Q4, 5, 6
    We must specify "this academic year" in the question stem.

    Q6 
    Sarah suggested we get feedback from Tom C. on WR150 before we decide to include Q6 but Linda pointed out that WR150 was not mandatory for all, only for CAS students.

    Q7 
    We need to look at changing these categories.  For example textbooks may be important, but datasets may not be as important to undergrads. David wanted to distinguish between the use of a textbook or use of a library book. Linda suggested that maybe we should specify that they are "library resources" but then this would preclude mentioning textbooks.  This question needs to be reworded.
    David wanted to capture the use of Reserves - this should be added. 
    David would like these options to be shrunk down somehow.

    Q9
    Include 'BU Libraries Search' not just 'catalogs'
    Leave Google Scholar out as an option - it may be confusing for those who see it as a BU library tool.
    Change the wording of colleagues to indicate faculty and fellow students (peers and professors).

    Q10
    Perhaps focus on citation and plagiarism here (BU faculty was concerned with this). Faculty saw ethics and plagiarism as 2 diff topics in the fac survey.
    Maybe include a 'use' question here?

    Q12, 13, 14, 15
    We will ask them about their primary and secondary libraries here and then ask them specifics about that particular library.
    Sarah would like us to include hours here too.

    Q17
    Relook at wording order

    Q17, 18
    Could we include how libraries help info literacy skills/find resources/cite sources?(Or we could include this as a 'use' topic).

    Q19, 20
    Collections and Ref services may be combined, in order to facilitate comparisons to other survey data, once we are in the data analysis phase.
    These are 2 new questions - Linda referred to UW's 'Overall importance to work by group' chart.  We should use kind of chart in our results.
    May be useful to make the wording between Q19 , 20 different.  Instead use active voice - "you" in the question stem. e.g. How important are.... to you?

    B) Other:
    Sara would like us to incorporate some kind of time management question (ability to manage time)

    Sara pointed out that writing questions for undergrads is very different to what we have done before. We should simplify the wording but we should be concerned about the validity of comparisons across surveys/ survey groups and over time.  Linda replied that UW had made these kinds of comparisons.

    Sara - Distance ed is graduate only

    Sara mentioned UROP may be a resource.

    Sara is concerned that we are not collecting data about the libraries they don't use and why they don't use them.

    Sara would like to run focus groups to glean data needs etc..  Linda suggested individual libraries may carry these out.


    3. Survey Keepers
    We will discuss these when Dan gets back.

    4. Next Meetings
    Thursday, January 10th 12:30-2:30pm in Mugar Conference Room
     Monday, January 14th at 10:30am in the Mugar Conference Room

    ___________________________________________________________________________________________________________________________

    Assessment Committee meeting minutes 12/10/2012

    Present: Steve, Konstantin, Shirley, Dan, Sarah, Linda, David (minutes), Kelly

    Next meeting: January 7th 1:30pm in Mugar Conference

    Action Items:

    All: Complete CITI Human Subjects training (see Linda’s email) by January 16th.
    All: Look at links about project management in Konstantin’s email.
    All: Look at ARL letter attached to Linda’s email.
    Dan: Move or tag survey keepers that don’t apply to current survey.
    Kelly, Sarah, Konstantin: Work on wording for the first 5 undergrad questions before next meeting.

    Announcements:

    Linda: IS&T has bought 500 seats on NVivo (http://www.qsrinternational.com/products_nvivo.aspx) -- Qualitative data analysis software. We hope to use. Not sure what campuses will have access; hope it is “One BU.” 

    Linda: Have ok to interview and hire a graduate (or undergraduate) student for Assessment. Using Anita Greene funds. 5 to 10 hours. 

    Linda: All have to take CITI training, Linda will send it, including which modules. Get it done by mid January. 

    Undergrad Survey
    Looked at "Question & Wording Comparison for BU's Undergraduate Student Library Survey" document (in Assessment Google Docs collection) for planning for undergrad survey. Sarah took notes directly in document. 

    Linda will be focusing on IRB. Kelly, Sarah, and Konstantin will concentrate on wording of undergrad survey. They will go in depth on the 5 questions we covered in meeting.

    Assessment Committee Meeting minutes 11/30/12

    Present:  Linda (chair), Sarah, Konstantin, Kelly, David, Shirley, Dan (minutes)
    Next Meeting: 12/10/12
    Announcements: Steve Smith will join us at our next meeting. 
    We also decided to hold meetings every other Monday at 2:30pm. 

    Action items: 

    ·          Look at Shirley's Google Doc for collecting library improvements. 

    ·         Take a look at the Rationalizing ARL Annuals Statistics Collection – Draft,  for consideration by the Assessment Committee 11/8/12

    ·         Konstantin will investigate Microsoft Project vs Zoho and will report. 

    ·         Take a look at Kelly’s and Sarah’s document

    ·         Take a look at Survey Keepers


    Discussion:
    A) Projects - 
    Back Burner - Alma Analytics and ACRL Metrics
    Middle Burner - Google Analytics and MINES (The year after the undergrad survey might be good for MINES)
    Front Burner - undergraduate survey creation (lead by Linda) and Grad Survey feedback (lead by Shirley)

    New project: eResources Collection Assessment - 
    Steve Smith will be coming on board with this. 
    Probably will involve looking at high ticket items, packages.
    What is our preparedness for an uncertain budget?
    Project will involve 
    - looking at Counter compliant stats
    - SFX to see overlapping subscriptions 
    - careful consideration for departments with only a few very important resources
    - communication with faculty and students
    - dividing resources into 3 categories (can't live without, really want to keep, let's talk).

    B) Project Management (lead by Konstantin)
    Konstantin showed us three project management tools.  All would take some dedicated staff time to keeping the system up to date. 
    1.Basecamp - can control who gets each email notification; to dos only possible to assign to one person at a time; where is the Gantt vchart?
    2.Zoho - easy to spot task progress and milestones; convenient dashboard; like Facebook but for projects; prioritized tasks; task dependency; recognizes google APIs; $50 per month
    3.Clarizen - visual roadmap; milestones and tasks (with managers); documents discussion; time tracking; $25 per seat per month; visually reminds of SharePoint. 

    Discussion followed about Microsoft Project, Excel, and Outlook as to what extent we might use these as a PM tool. There was a concern that Project might not be cloud-based. Konstantin will look into Project to learn more, and confer with IS&T. The group generally favored Zoho. 

    C. Feedback on brainstorming library improvements (lead by Shirley)
    Feedback already received from Roger, Lou and Vika.  With the Alma implementation, some folks are not able to address this immediately. Shirley created a Google form to collect responses to our email scheduled in January.  

    https://docs.google.com/a/bu.edu/spreadsheet/viewform?fromEmail=true&formkey=dDA3eG83VkE5YWQtdGlXM3pQYUNrQmc6MQ


    _______________________________________________________________

    11/9/12

    Present:  Linda (chair), Sarah, Konstantin (minutes), Kelly, David, Shirley, Dan

    Action items for next meeting  (Next meeting: New date/time/location is Friday, 11/30, 2:30-4pm in the conference room, Administrative Offices, Mugar.

    ·         Dan will revise a draft email about brainstorming ideas for library improvements that could be released to Department and Branch Heads.

    ·         Linda will send Shirley a list of email contacts so that we can look forward to sending an email (that Dan has been working on) to librarians/department heads (around January)

    ·         Take a look at the Rationalizing ARL Annuals Statistics Collection – Draft,  for consideration by the Assessment Committee 11/8/12

    ·         Konstantin will compare Zoho, Basecamp and Microsoft Project for next meeting

    ·         Take a look at Kelly’s and Sarah’s document

    ·         Take a look at Survey Keepers

    ·         Take a look at Shirley’s document and try to think what else can be added


    Agenda:

    Linda’s feed back:

    ·                     MINES

    ·                      The poster sessions – “Using What You Collection, Library Data and Student Success” from the University of Minnesota

    ·                     Linda’s and David’s visit to the UMass

    Reports

    ·                     Project management software / timelines (Sarah and Konstantin)

    ·                     Document for facilitating brainstorming session with staff (Dan)

    ·                     ARL/BU stats documentation (Linda)

    ·                     UW’s undergraduate survey – wording, etc. (Kelly and Sarah)

    ·                     Alma Analytics (David)

    ·                       Document for comments processing (Shirley)

     

    _____

    Linda’s feedback about the conference

    ·         MINES  (Measured Impact of Networked Electronic Resources)

    o   Terry Plumb,  B. Franklin – web form for electronic resources

    o   York University – tied it to the “stream”  i.e. Ezproxy stream (for example, EBSO, Sage ebooks,  CSI, etc.)

    o   Linda suggested this since this kind of reporting is valuable for administration we can do it once a year or more

    §  We all agreed that it’s a great idea to go through ARL, although it is possible to do it on our own.

     

    The poster sessions – “Using What You Collection, Library Data and Student Success” from the University of Minnesota

    ·         GPA and retention rates, as tied to library data

    o   Web site login, circulation, instruction sections, reference sessions, etc.

     

    Linda’s and David’s visit to the UMass

    ·         Rachel’s feedback:  Comments sent out to department head

    o   Asked department head to give feedback after meeting with the staff

    “Brainstorming” sessions with staff

    ·         Dan prepared a document for brainstorming

    ·         We all agreed that it doesn’t have to be a “survey monkey”

    ·         We also agree that the timing has to be right

    ·         We can also do “save the date” message to department heads and send it out in January.

    Sarah and Konstantin

    ·         Basecamp, Zoho, Microsoft Project,  use of Excel sheets

    ·         Konstantin will compare these products for next meeting


    Dan

    ·         Dan  created a document  about brainstorming ideas

    o   For next meeting Dan will revise draft email about brainstorming ideas for library improvements that could be released to Department and Branch Heads. We are not going to post comments but we can offer to contact Shirley

    o   We should send out to ALC in January

    §  Linda will send Shirley a list so that these people can look forward to (January) a document that Dan has created (action item)

    §  See Rationalizing ARL Annuals Statistics Collection – Draft for consideration by the Assessment Committee 11/8/12

    Kelly and Sarah

    Question and Wording Comparison for BU’s undergraduate student library survey  (available on google sites)

    ·         Kelly and Sarah compared facilities, frequency, resources, sites, etc.

    o   Revisions that are needed 

    o   We need to be clear what questions we could group together and report on. This will affect our wording (action item)

    §  Consider defining core questions

    o   Is there an overall contribution question?  Yes.

    o   Sarah suggested to concatinate some questions (frequency and X)

    o   Linda – better use of skip service

    o   Next time:  take a look at Kelly’s and Sarah’s document (action item)

    David

    ·         ALMA analytics

    o   Reading documentation wasn’t helpful

    Shirley

    ·         Placed Grad Student Survey documentation in a Dropbox  (Documentation- Comments)

    o   Preparing the data spreadsheet

    o   Setting up the categories

    o   Tagging

    o   Filtering

    o   Sorting the comments on department contacts in the spreadsheet

     

    Discussion:

    ·         Surveys

    o   Follow up

    o   Development  ->  Analysis

    ·         Analytics

    o   Google

    o   Primo

    ·         Collections

     

    Administration

    ·         Charge

    o   What we do/how we do it/why we do it/resources

    ·         Documentation

    ·         Project Management

    ·         Planning

    ·         Investigation

    o   Analysis

    o   Collections

    o   Mines

    o   ACRL Metrics

    Take a look at Survey Keepers

    Take a look at Shirley’s document and try to think what else can be added


    ________________________________________________________________________________________________________________


    Minutes: Thursday 10/25 9am-10:30

    Present: Konstantin, Kelly, Shirley, Dan, David, Sarah (minutes), Linda

    Next meeting:

    Friday Nov 9, 2pm Agenda item: create timeline for tasks/goals

    Action items:

    Sarah and Konstantin will look for (free) software for project mgmt/timelines

    Dan will create a draft document for facilitating brainstorming sessions with staff

    Linda will look at ARL/BU stats documentation and draft note for folks she's working with

    Kelly and Sarah will get UWs undergrad survey and start thinking about changes vs keepers from their questions

    David will bond with Alma analytics

    Shirley will draft document for comments process

     

    upcoming year:

    timeline of our tasks/goals: project management approach to scheduling our work for the upcoming year(s) (Gantt chart?), rewrite charge

    Surveys:

    Timeframes - 2nd cycle right away?, Yearlong or multiyear schedule, review approach, pull in Gordon, ask UW how they did longitudinal work

    Document steps of process for data analysis (DF), comments (SL-S)

    Response approaches to finished surveys

    DB asked what if anything ever happened to bulleted followup suggestions from faculty survey? 

    *followup on grad and faculty

     getting feedback from staff re: how they are responding to data and comments

    -email to remind them that we want to know what they are doing in response to the data and comments

    -use our wiki to track/document the response?

    -documentation is crucual

    -facilitate INTERNAL brainstorming sessions with staff, get suggestions, escalate suggestions to folks with "policy chops" 

    -external focus groups or followup survey tools?

    -newsletter? time sink, indicitive of larger communications issue (internal and external)

    Develop undergrad survey 

    Fix up our existing Google wiki- move to BU google sites and clean up nav, etc

    Network with others on campus 

    *Analytics:

    Alma Analytics

    Google Analytics

    Primo Analytics

    Counter/Sushi


    Collections Assessment 

    -format and platform for e-materials? 

    Counter/sushi stats

    will fall to Steve

    ARL annual stats 

    LP has a charge to, in the next year, work with Law, Med, and Theo as well as Ruth Milesky from Institutional Research on rationalizing how ARL annual numbers are gathered and reported across the 4 BU libraries 

    ACRL Metrics 

    - we have purchased access to this tool 

    - includes access to ACRL's Counting opinions database (get it cheap b/c we submit ARL stats), which many institutions find useful for peer comparisons (despite the quality of the ARL data behind it)

    - Investigate ways to use?


    Discussion:

    DB and DF asked how much of the followup should fall to the assessment group. To what degree does this activity meet our group's charge? We were all in agreement about the need to track, document, and broadcast how change is happening based on data and survey results. SS pointed out that our group may be uniquely poised to facilitate communication of ideas from front end staff to policy makers who can institute that change. Suggestions about getting buy-in from department and branch heads.



    Minutes: Tuesday 10/2/2012 10am-11:30am

    Present:  Linda Plunket (chair), David Fristrom, Sarah Struble, Shirley Leslie-Smith, Kellie Riddle, Konstantin Starikov (minutes).

    Next meeting: Thursday 10/25 9am   Mugar

    • Make sure to let Dan know
    • Follow up to the 10/18 meeting

                Action Items:  

    • Sarah will let Linda/Bob know if there will be invited guests (faculty, non-librarians, etc.) at the All BU Library Staff mtg (10/18 2pm.) 
    • For the 10/18 event, Linda will check if there's wi-fi access (done - wi-fi available)
    • For the 10/18 event, Linda will let us know if we can use the room one hour earlier for setup (done -  access to Terrace is available at 1pm)
    • David, Linda and Shirley will prepare short presentations for 10/18 (Linda - introduction 5min, David - overview 20-30 min, Shirley - comments 10-15min)
    • Linda will lead the discussion after the presentations


    Announcements:  

    1.    All BU Library Staff Meeting:  Thursday, October 18, at 2pm in the Terrace Room, located on the second floor of the George Sherman Union (GSU).
    2.    We have access to the Terrace Room, GSU, on Thursday, October 18, from 1pm on so we can prepare for the all-staff meeting.

    Discussion:


               1.         Alma Analytics

     

                              ·         Reporting function

    o   Ebooks (!)

                              ·         Discussion:  

    o   The creation of lists functionality is very robust in Alma

    o   Need for analytics documentation

    o   Use of widgets

    §  We should have a widget

    o   We need to take a leadership role and/or identify staff in each library to explore analytics capability

    o   Is it possible to import Primo analytics data into Alma?

    o   Our goal – use ARL questions as a sandbox for creating stored logical sets

    o   Before Millennium goes away, Jack will run the stats as a backup.

     

                      2.       Google Analytics

                  o   Feedback:  Brendan Gannon’s presentation (9/21) was very helpful. Brendan emphasized the need to formulate questions first about our usage before delving into   

                                    analytics.

                 o   Should we set up the model for coordinating web analytics and Alma analytics teams?

                 o   Delegate the responsibilities to experts but have the group (committee) develop broad questions.

                 o      We need a technical lead

                      3.       Preparing for 10/18

                  o   Encourage your colleagues to come.

                  o   On The agenda:

           o   David will give overview of the report

       §  David suggested that we shouldn’t go over specific questions rather to remind everyone what the sections are

           o   Shirley will give overview of the comments

       §  Shirley will give examples

    o   Linda will lead the discussion but everyone will be encouraged to contribute.

    Other points:

                            ·         Bob Hudson will open up the meeting and will place it in the context

                            ·         New staff will be introduced by supervisors

                            ·         We should also do a general discussion about the conclusions.

                            ·         Bring up the Assessment page so that people will identify members

     

         §  Agenda for 10/18:

    o   Bob Hudson will introduce new staff (10 minutes)

    o   Linda will go over the report and agenda for the meeting and that people should feel comfortable contacting assessment members  (5 minutes)

    o   David will use Prezy to talk about the survey (facts, charts, graphs, reports)  (up to 30 minutes)

    o   Shirley will talk about the comments (15 minutes)

    §  Will talk about the process of identifying and tagging data, trends, etc.

    o   Questions from the audience

    §  If there are no questions/ or we can also open the discussion by talking about lost books, group study space, training patrons on finding e-journal articles and explore ideas how we can improve our services.

                      Remaining issues:

                     Discussion: what investigators will do to address questions from non-committee library staff

                                              ·         Linda: we are not going to announce what we will or will not do but at the same time as members of the Assessment Committee we will find the time to provide data that will help librarians with departmental decisions.

                                               ·         Sarah - librarians should come with specific inquiries in order for us to help them.

                                     Sarah -  Sarah will let Linda/Bob know if there will be guests who are not librarians.

     

                      Logistics -  bring a computer.

                                             ·         Linda will check if wi-fi access is available

                                             ·         Room access (set up, etc.) – we need to access the room 15-20 minutes before the event.

     

                       4.       Future goals:  preparing for the undergraduate survey

    ·         March of 2013  (?)

    ·         When should we start – last year, about right after Christmas

    ·         Take a look at the “things learned” on our website and also at the recommendations from Steve’s report (The University of Washington)

    ·         Pilot -  important to do for the next survey

    ____________________________________________________________________________________________________________________________________________________


    Minutes: Wednesday 9/5/2012 

    Present:  David, Linda (chair), Kelly, Sarah, Shirley(minutes), Young-Joo.

    Next meetings:    

    1. September 11, 8-4pm, PERL
    2. September 19, 3:00, Mugar Admin (Google Analytics meeting with Brendan)
    3. Last week of September?  Linda to send out a doodle


    Action items:

    • Dan will check the numbers in the Appendix
    • Linda to establish a timeline for release of the survey (Provost, Deans)
    • Linda to create a doodle for an assess-l meeting to discuss the staff presentation
    • Linda to ask Bob about including Nancy Coleman; and sending report to ALL grad students
    • Linda to look into date and venue for the open staff session
    • David will make the agreed changes to the library survey document and will create a Black&White version after the final version is confirmed
    • All – try to learn as much about Google Analytics as possible. Online tutorials, FAQ are available
    • Shirley will make changes to the comments spreadsheet

    Announcements:
    1. The ALMA Analytics all day training course will take place at PERL on Tuesday 9/11 from 8-4pm. Only 10 people will be allocated passwords (only Linda and David from assess-l).  Trainer will be Connie from Exlibris. Konstantin Starikov will join us.

    2. Farewell to Young-Joo.

     

    Discussion:

           1.       Numbers check.  David made a few changes based on Dan’s findings. Thanks to Dan for checking the numbers. Dan will check the numbers in the Appendix.

    2.       Bob Hudson’s name is no longer on the survey.  He felt that those who were responsible for the document should receive the credit for it.  The Library Assessment Committee is now credited.

    3.       Sarah suggested sharing the survey report with Nancy Coleman (Head of Dist Ed.).  Linda will confer with Bob about sharing the report with the Deans and Nancy.

    4.       Survey Report  Discussion

    4.1.    Pg 2 Place to study.  Dan wanted more info so David will add a sentence indicating the need for study space for grad students was the greatest need.

    4.2.    Pg 5 David will try to adjust figure 2.

    4.3.    Pg 6 Fig 4 - Linda requested that librarians take care to explain that the size of the circle relates to the number of respondents not total students.  Young-Joo and Kelly in particular should relay this information to Mary and Amy.  The response rates per school may be used to illustrate this. Sarah compared School of Social Work (+_ 60%) with the Dental School (+_ 14%).

    4.4.    Pg 10 The comment reflects that there is still a need for print resources in some areas.

    4.5.    Pg 14 Dan’s suggestion was considered. The footnote was deemed to be important and will be kept.

    4.6.    Pg 15 The title of fig 13 includes the word ’Use’ in reference to class presentations and online guides. We couldn’t think of a better word to replace it.

    5.       Next Steps:

    5.1.    Linda will give the report to Bob who will send it to the provost, and will ask for a meeting thereafter.  That day Linda will give the report to Mary, and then get it to the Deans.

    5.2.    Bob and Linda will look at sending a copy of the report to ALL of the graduate students.

    6.       Comments Discussion

    6.1.    Change Pappas to Law Libraries

    6.2.    Add filters and freeze panes to each tab

    6.3.    Complete comments as soon as possible

    6.4.    Ask Konstantin to take over Young-Joo’s med comments (no student assistance allowed)

    6.5.    Anonymity is paramount

    7.       Open Staff Meeting

    7.1.    We may have to have two meetings – one general meeting and then a meeting to discuss the comments spreadsheet and how to use it.  Try to complete the comments ASAP.

    7.2.    Venue may be in the Law tower or the PAL Study Lounge.  Where and when to be determined.

    8.       Sarah noted that we still need to keep working with the data, even though the exec summary is complete.

    9.       Appendix Discussion

    9.1.    The Appendix will now be called ‘Supplementary Data’ and this must be changed throughout the report.

    9.2.    Dan will check the numbers.

    9.3.    Linda will revise all of the key categories throughout the report (e.g. change’ Important’ to ‘4’) and will change the charts to reflect this.

    9.4.    The report and the supplementary data will appear on the Assessment Committee web page once complete. (Law, Med and Theo can post it on their sites as they prefer).

    10.   ALMA Training

    10.1.Linda mentioned the Tuesday (1:30-3:30) and Thursday (10:30-12:30) training sessions and recommended that assess-l attend. 

    10.2.The Assessment Committee should all attend next Tuesdays training session on Reporting and Analytics.  It starts at 8am and ends at 4.

    11.   Google Analytics

    11.1.Meeting with Brendan Gannon at 3pm on Sept. 19

    11.2.We must think about how we can use the data that we are collecting.

    11.3.We should try to complete all GA tutorials, read the FAQ’s etc before the meeting.





    Meeting Date: July 26, 2012
    Present: Linda, David, Jim (minutes), Dan, Sarah, Shirley, Young-Joo

    Next meetings:    
    1. August 22nd, 9:30, Mugar Admin
    2. August 28th, 2:30, Mugar Admin
    3. September 5th, 9:30, Mugar Admin
    Decisions/Actions:
    1. Group will look at the categories of students and think about alternatives to "power patrons" and perhaps "casual patrons" as well.
    2. David may make the current bulleted list of patron categories into a table.
    3. David will review numbers again and see about some chart changes. These include: revising figure 12, creating a chart with school and discipline for interest in prospective library services.
    4. Group will think about charts for section #1 to see if it makes sense to create separate charts for just in-person visits, just online visits, and then a combination.
    5. Linda will make sure that possible areas of confusion in reporting are clarified for Bob before he shares the report.
    6. Jim and Shirley will continue to tag and review comments and try to select representative comments that could be put into the report.
    7. Grad Survey report will go to Bob Hudson first, then to Jean Morrison. After that meeting, Bob and Mary Blanchard will discuss how to present the information to the Medical Campus Provost, Karen H. Antman. Following the sharing of the report with Provost Antman, we will present the report more widely via website postings and e-mails to staff and students,
    ___________________________________________________________________________________________________________________________________________

    Meeting Date: June 28, 2012

    Present: Shirley, Dan, David (minutes), Linda

    Next Meeting: Friday, July 13th, 2:00pm, Mugar Admin

    Decisions:

    • Decided that finishing report on recent survey has higher priority than data management survey

    Discussion:

    Linda described what is going on with Google Analytics for new website. There were some questions, so we called Brendan Gannon from IS&T to get some answers:
    • It's important to make sure the correct date range is set when looking at Analytics (explains why we weren't seeing recent searches)
    • “Site Search” under Content is giving Primo search terms
    • To tell where search is coming from, use Advanced Segments (we still have to figure out)
    • To tell whether it is advanced or basic search, use Secondary Dimensions:Site Search: Site Search Category
    • Confirmed that existing analytics (from old site) will not be lost

    • Every page, even those dynamically created, on the new site will have the GA script embedded. New pages going forward will also have the script automatically included. So all pages will be counted.
    • There are two user accounts on GA for our site-- one is for the website and the other is for Primo.
    • The counts in the Primo account, compile stats for both basic (basic script identified by string vlfreetext0) and advanced (advanced script identified by string vlfreetext1) searches. 
    • In the Primo account, GA will capture referring URLs (e.g., referrals from ...library/asl). Therefore, in the Advanced Segments, we can discover lists of search terms used by patrons coming from a particular referring URL such as ...library/asl.

    We looked at Appendix Draft.

    Question was raised of using “Satisfied” etc. as labels when actual label was 4. Unresolved.

    Should we do executive summary? Maybe wait until we have report.

    Next steps? Try to come up with narrative for report.

    Library centric report vs. grad student centric


    Meeting date: Thursday, May 24, 2012

    Present: Dan Benedetti, David Fristrom, Young-Joo Lee, Shirley Leslie-Smith, Linda Plunket (minutes), Sarah Struble

    Next meeting: Tuesday, June 12, 3:30-5PM

    Decisions
    • We will look at the overall data first and look for themes. How we present the data will be determined by the themes that emerge.
    • Appendix will include charts and graphs for most questions.  
    • Once we have a draft of the report, we will call a staff meeting to show what we have analyzed and ask what other breakdowns staff would find useful.
    • We will shoot to have a final report to Bob by mid-August.
    Action items
    • Everyone will look at the data in Survey Monkey, Excel, or JMP by June 12
    • Linda and Shirley will work in Excel with the data in June

    Reviewed Survey Monkey analysis by page and question (ignoring the comments for now)
    Since everyone is very busy with the website transition, Alma implementation, end-of-year spending, and L2324 evaluations, few have had a chance to look at the data. We reviewed the data together to get a sense of the data and what we need to do.

    PAGE: DEMOGRAPHICS
    1. Is your primary affiliation with a Distance Education or Online Learning program at BU?
    Comments:
    • Dan looked at respondents who identified as being primarily affiliated with Distance Ed and who visited one or more of the libraries on a regular basis. It was a small group.
    • We will want to exclude respondents who are primarily affiliated with a distance ed program from many questions.
    PAGE: DEMOGRAPHICS PAGE 2
    1. With which campus are you primarily affiliated?
    Comments:
    • Distance and overall combined in one chart
    2. What is your degree program?
    Comments:
    • David is using JMP to re-categorize those who are clearly in one of the degree choices given
    • When analyze doctoral might want to separate out med, law, etc.
    • If masters & doctoral they are being categorized by doctoral
    • Will want to compare SPH eoctoral and Medical doctoral, if there are enough
    3. As a graduate student at BU, have you:
    Comments:
    • Very interesting, drill down
    4. What is your primary discipline?
    Comments:
    • Nearly 1/10 skipped question
    • David is looking at departments instead of discipline in JMP
    • Could compare to faculty
    • Arts management, music education... not sure which discipline to choose from choices
    • Will use department
    • Multiple things going on... lack of understanding, terminology, interdisciplinary
    5. With which college/school are you primarily affiliated?
    Comments:
    • Won't graph the departmental breakdown for each college-- not interesting-- but we need this breakdown for disciplinary and other reasons
    PAGE: LIBRARY RESOURCES
    1. How frequently do you visit a BU library in person?
    Comments:
    • Take out dist ed
    2. How frequently do you access BU Libraries' resources, services or websites online?
    Comments:
    • Interesting to take out dist ed and compare
    • If look by discipline... look at both questions 1 & 2... could be interesting patterns

    3. Please rate the IMPORTANCE of the following resources to your work:
    Comments:
    • Compare to faculty and by discipline
    4. Please rate your SATISFACTION with the following BU library resources:
    Comments:
    • Dist ed look at that specifically with and without
    • Survey takers seemed to understand the difference bet. importance and satisfaction
    PAGE: RESEARCH METHODS
    1. How important have the following skills been to your success in your current academic program?
    Comments:
    • Compare to faculty, close enough, but different scales, so can put next to each other but not in the same chart
    2. Please rate the importance of the following for FINDING the resources you need for your work:
    Comments:
    • None
    PAGE: LIBRARY SERVICES
    1. Select the BU libraries you visit IN PERSON on a regular basis:
    Comments:
    • Look at various libraries, e.g., PERL-- who is using it? From what school?
    • Take out dist ed
    • What percent of respondents in a school are visiting the embedded libraries?
    • Published in a scholarly work-- this school 30% this school 10% OK independent of response rate
    2. Are BU Libraries open when you need them during periods listed below?
    Comments:
    • Strip out dis ed
    • Also look at take out NA
    • Look at by school
    3. Please rate the IMPORTANCE of the following BU library services to your work:
    Comments:
    • Don't use or not important 
    • Note important: ILL, access to computers in libraries, study or work areas
    4. Please rate your SATISFACTION with the following BU library services:
    Comments:
    • None
    5. Please rate how useful augmenting the following library services would be for your work:
    Comments:
    • In library questions-- compare to which libraries they use
    • Separate out distance ed
    • Linking questions... look at filter for those who teach
    • Archiving research questions (don't remember what this note refers to?)
    PAGE: OVERALL CONTRIBUTION AND SATISFACTION
    1. What contributions do the BU libraries make to your:
    Comments:
    • None
    2. How satisfied are you with the BU libraries':
    Comments:
    • None
    _____________________________________________________________________________________________________________________________
    Minutes: Monday 5/10/12
    Present: Linda, Dan (minutes), Young-Joo, Sarah, Shirley, David, Jenna, Mary F.
    Next Meeting: Thursday, May 24, 2012 @ 9:30. Mugar Conference Room
    Action items:
    • Everyone should look at the survey results, with no filters or crosstabs for schools (or departments) applied, for interesting charts.  Crosstabs yes, filters no!
    • For the meeting after, we will be looking for interesting results with filters applied (each person on the committee will take a different school for filters, like last time).
    • Linda will ask Steve Hiller where UW stores data.  Linda also suggested asking UW before we post our survey questions public.
    • David will continue translating survey to JMP.
    • Shirley will learn more about JMP and possibly help David.
    Discussion:
    Data Management Pilot Project with Assessment
    Jenna and Mary from the Data Management Group (DMG) joined us for a discussion of the pilot project for our data. Dataverse is an open source tool for organizing research data, although there is a cost associated with it. It is different from repository software because the focus is on data, whereas Dspace is more focused (or used) primarily for text documents.
    Data we could provide for public consumption, or sharing with other stakeholders on campus:
    • surveys themselves (fac and grad)
    • aggregate data from both surveys
    • resulting full and school reports
    • other peripherals like letter from provost, posters and promo material, and IRB submissions

    Excluded were comments and the raw data itself; unless the storage space has sufficient security safeguards.
    Linda will ask Steve Hiller where UW stores data.  Linda also suggested asking UW before we post our survey questions public.  Next steps include the DMG writing up a questionnaire asking for their bare needs for development of a plan.  After the Assessment Committee fills out the questionnaire, the DMG will present a number of plan templates (based on a streamlined version of the Purdue data plan tool) from which to choose, and then will write up the plan.
    Outcomes for the pilot:

    • DMG gains experience developing a written plan.
    • DMG develops a short presentation to pitch the service, first to all library staff and later to externals.
    • Assessment Committee has safe place for organized data storage.
    • Assessment Committee has a site for efficient data sharing.
    ARL Annual Stats Survey (notes from LP)
    A revised version of the ARL annual statistics survey will be finalized after this meeting. The Committee is refining the instructions. A few of the supplemental statistics have been incorporated into the survey. The rest have been deleted.
    The loss of the serials count and the addition of a title count regardless of format were the most controversial changes. The aggregators confound the issue. ARL libraries cannot agree how to count titles. Do we count based on what we steward, catalog, maintain, pay for, or make discoverable? If a collection has 8000 items, do we count this as one or as 8000? Should HathiTrust titles be counted? One member library jumped from a serials count of 9M to 14M in one year by adding in what they held in common with their statewide system. The instructions will attempt to clarify how we count.
    The value of the statistical data is in its depth and breadth, in the ability we have to compare institutions. The problem with the data is how we count.
    The survey was abbreviated so that:
    • Member libraries could spend less time on gathering and reporting on these statistics and more time on showing how libraries are making a difference to their institutions.
    • The most useful and meaningful statistics would be kept and the least useful and meaningful deleted.
    • The reports could be published in a more timely fashion
    Other Topics
    1. The Committee is considering developing an ongoing facilities inventory that would be online and could be updated as changes are made. This might include renovations, number of physical items and off-site storage. This might include space as service and narrative descriptions.
    2. The Committee is considering developing a survey about Special Collections. This is is an early discussion stage.


    Grad Survey Analysis
    David has been working on translating the survey into JMP for manipulation not possible in Survey Monkey.  We discussed manipulation of those who responded "other" to the discipline question.  Shirley will learn more about JMP and possibly help David translate schools/departments into a list that would filter "other" responses into the disciplines for analysis.  Next steps are for everyone to look for what charts we want to highlight in the main report.  By the next meeting, everyone should look at the survey results with no filters (no schools or departments) applied for interesting results.  For the meeting after, we will be looking for interesting results with filters applied (each person on the committee will take a different school for filters, like last time).  Our goal is to produce one full report.


    ____________________________________________________________________________________________________________________________________________

    Minutes: Monday 4/30/12

    Present: Linda, Jim (minutes), Young-Joo, Sarah, Dan, Shirley, David

    Next Meeting: Thursday, May 10, 2012 @ 2:30. Mugar Conference Room

    Action items:
    • Linda will share take-aways from Megan's presentation with the AULs.
    • We will create an assessment site to which we can link for publicity. This will include how we report our actions and what steps we are taking to act on findings of surveys/assessment activities.
    • Linda & David will delete the file containing names of survey respondents once the winners have been notified.
    • Linda will e-mail Sarah with the names of Sarah's volunteers. (LP-- Done)
    • Everyone will spend time looking at the survey and thinking of ways we might analyze the data. If folks find an area for analysis they'd like to pursue, please let the group know.
    •  NO ONE should delete anything except the files as noted above.
    • People will use assess-l list for questions about grad survey.
    • Linda will forward Megan's slides to the group. (LP-- this turns out not to be possible given restrictions by the BLC)
    • Make sure your volunteers have been thanked.
    • David will handle some issues, like people choosing "other" for disciplines, by assigning a discipline and using JMP for analysis.
    Discussion:
    BLC Presentation with Megan
    • Partner with someone else to do statistics: a grad student in Stats was suggested. Also some discussion of using library student workers (Mary McGowan's?)
    • New BU Libraries website should focus more on public relations/publicity.
    • A new Communications liaison between the library and the rest of BU as well as within the libraries was proposed by the AULs as a way to promote initiatives and provide information about assessment activities.
    • New site should have faculty and grad student reports (eventually the undergrad report as well).
    • We need to publicize our responses (activities) to survey findings. Perhpa this could be done with one-page snapshot documents.
    • Discussed creating executive summaries for various populations (administrators, faculty, students, etc.)
    • We can and perhaps should talk to constituent bodies about our findings rather than just report results. Perhaps we can hold focus groups dealing with particular issues.
    • Package services to discrete populations.  The AULs will have to facilitate this endeavor.

    Survey
    • Winners were selected via the random generator. Jim took down the numbers for 12 possible winners and forwarded those numbers to David. David will find the associated names and share them with Jim and Linda.
    • .Linda gave group a quick overview of survey results.
    • The following ideas were shared for future surveys and survey planning:
      • Better organization of advertising/promoting the surveys.
      • Better lead-up to the launch if possible. Can we promote the survey before the Provost send the first notice?
      • Make recruitment guidelines in IRB application more general: "distributed" rather than "handed out", e.g.
      • Have a better pilot for the undergrad survey.
      • Advertise on BU Shuttle.
      • Distribute materials at school entrances, not just dining halls.
      • Add a "what year are you?" question to undergrad survey and perhaps leave out the discipline question.
      • START earlier!
    • Mimic the faculty report but with changes. Perhaps make the kind of information in that report into an appendix and make the grad student report more visual and less word-heavy?  OR, make the grad report more narrative with comparisons between campuses, between grads and faculty, and other more descriptive information.  Take advantage of software for color charts.
    • Analyze key questions including campus breakdowns, distance education, and discipline.
    ____________________________________________________________________________________________________________________________________________

    Minutes: Thursday 3/29/2012 

    Present: Sarah, Shirley, Dan, David (minutes), Linda (chair), Jim

    Next meeting: Thursday, April 12th, 2pm. Mugar conference room.

    Action items:
    • All -- Keep track of all who volunteer to hand out postcards
    • David -- Will chart response over time (once survey finished)
    • All -- Think about best schedule for surveys going forward

    Discussion:

    Robert Hudson stopped by and told us about his contact with the Deans; he is pushing them to push the survey.

    Report on response to survey; currently at 21% overall.  

    We looked at response rates for the various schools and colleges.

    Sarah met with someone from distance ed to find out who to contact about getting out word. Will send email to Nancy Coleman and Eric Friedman.

    MET is going to be hard to reach.

    Talk about varying responses to postcards at varying venues.
    GSU will continue for another week, SMG will ramp up again for another week.

    Talk about keeping copies of response tracking so we can look at results over time. David pointed out that SurveyMonkey gives us date (and time) stamps of when people complete survey, so the response rate over time could be charted.

    There is a JMP presentation on April 4th, but it was decided no one needs to attend.

    Linda mentioned that Washington does all three (facutly, grad, undergrad) surveys together. Do we want to continue doing one a year, or switch to their model?

    Full week until next reminder goes out to grad students..

    Sarah talked about presentation she saw at Library Technology Conference on visual display of data.


    Minutes: Thursday 3/22/2012 1:30-2:30

    Present:Linda (chair), Dan, David, Jim, Sarah (minutes), Shirley

    Next meeting: Thursday, March 29, 2pm. Mugar large conf rm

    Action Items:

    1. JS to add link to survey on Theo Library news feed
    2. LP to alert assessment-l when Bob's email to deans is sent.
    3. All -work with "your" deans as appropriate to assist them after Bob's email goes out
    4. SS to work with Tom on drafting a version of Bob's email for Nancy & Eric at ODE
    5. All- send LP names & departments of volunteers for dining hall flyering for thank you notes
    6. DF will speak with Jenna about Data Management pilot with our Faculty Survey data
    7. SS to report on LibTech Data Presentation Session at next meeting

    Recruitment:
    •  Went over our recruitment strategies. Noted areas where we needed to act (see action items), and survey keepers about recruitment (IRB permission to poster in student lounges, general advice to our future selves to take more time to think through recruitment strategies and write more open-ended distribution parameters in next IRB plan).
    •  Bob Hudson has drafted a letter for the deans and sent it to be okayed before sending it out to the Deans Council members. LP shared his draft with assessment committee. 
    •  We decided against using a shared Google calendar for scheduling staff to hand out postcards in the dining halls for this survey cycle. May want to plan such tools for the larger push the undergrad recruitment will require.
    •  Looked at the numbers of surveys completed to incentive drawing entrants. Most who complete are signing up for the drawing. Majority of sign-ups are with BU email addresses. We will use random number generator to draw, and will verify winners are grad students before distributing prizes.
    • One of the Anita Greene students edited the the electronic version of the "postcard" to a narrower "header" version. We agreed to use this for the 3rd provost email (2nd reminder). 2nd email (1st reminder) will be in the same format as the first. It looked classy.
    • note that the survey will close 4/17, not 4/20

    Data Saving:

     No one but David Fristrom and a certain anxious but unnamed assessment committee chair should be going in to the survey in Survey Monkey while it is open.
     DF is doing daily loads in XLS of both survey results and drawing entrants list, saved to dropbox folder "2012 graduate survey backups" 
     

    Data Mangement:

    LP has spoken to Jenna about using our Faculty Survey data as a test case for the Data Management group. We agreed to work with them to create a data management plan. This will include only our fac survey material, not all of the assessment committee's work (at this time). As DF is on both committees, he is going to communicate with Jenna about moving forward with this.
      

    Minutes: Monday, 3/12/2012, 9:30-11AM

    Present: Linda (chair), Dan, David, Jim, Sarah, Shirley(minutes) and Young-Joo.

    Next meeting: Thursday, March 22, 1:30-3:00 pm

    Action items:

    Everyone: fetch your posters/postcards from Linda's office. The easels are in the coat room

    [Take another poster per school as per Linda's direction:
    Dan needs two more 13X19 (SED and SMG); Total for 13X19 is 6
    David needs two more 13X19 (ENG and CFA); Total for 13X19 is 6
    Sarah needs one more 13X19 (SAR); Total for 13X19 is 3
    Linda needs one more (COM); Total for 13X19 is 5
    Jim needs three more 13X19 (STH, MET, and SSW); Total for 13X19 is 5] 3/16/12

    Dan Benedetti  [1 large dining hall poster for the BU Pub (use your own easel); 2 large posters for PERL;2 large posters for PARDEE; 3 small posters for SED departments; Shirley has the 24 CAS posters; and a bunch of postcards]

    David Fristrom [1 large dining hall poster for the SMG Starbucks (plus 1 easel); 2 large posters for SEL; 2 large posters for ASTRO; 3 small posters for ENG departments; 3 small posters for CFA; and a bunch of postcards] 

    Shirley [1 large dining hall poster for the School of Law Cafe (plus 1 easel); 1 large poster for Pappas; 2 large posters for African Studies; 1 large poster for ASC; 24 small posters for CAS departments; 6 small posters for LAW departments; and a bunch of postcards] 

    Sarah Struble  [1 large dining hall poster for the F/S Dining Hall (plus 1 easel); 2 large posters for MUSIC; 4 small posters for MGMT departments; 3 small posters for SARGENT; and a bunch of postcards] 

    Linda Plunkett  [1 large dining hall poster for the GSU Food Court (plus 1 easel); 2 large posters for MUGAR; 2 large posters for STONE; 3 small posters for COMM departments for Diane D'Almeida; and a bunch of postcards] 

    Jim Skypeck  [2 large posters for THEO; 10 small posters for MET departments;  3 small posters for THEO departments; (zero posters for SSW)] 

    Young-Joo Lee  [1 large dining hall poster for the Med Campus); 2 large posters for MED library 4 large posters for each schools; 10 small posters for DENTAL departments; 8 small posters for MED departments; 8 small posters for SPH departments and 4 extra small posters and a bunch of postcards (1/4 from the box, I don't know how many -Young-Joo]

    Linda to send a test email, containing the proposed link to the graphic html page, to assess-l, in order to facilitate a decision on whether to include the graphic in emails 2,3 and 4 or stick with the text option.

    Linda to contact Diane d’Almeida regarding CAM posters

    Linda to ensure posters for Med are ready by Wed/Thurs and will give them to Joe to take back.

    Linda to contact ?Rubin Rossi? About reserving the GSU link ( GSU Demo Area  We have booked the Demo Area of the GSU (first floor, entrance to food court) all day, M-F, 4/2 through 4/17, but not 4/4. The dates between 3/20 and 3/30 were already booked.- Linda 3/12) (Thanks to Jim for the suggestion to try for the link. I think it's close enough that we can consider it the GSU. I called GSU Reservations and spoke with Jim. He said there are no tables left for 3/20-3/30, but he gave us permission to stand with an easel between the bottom of the staircase to the 2nd floor and the door to the outside courtyard. He said no one else should be this spot. He will let John Battaglino, the manager of GSU, and Mindy, from the Student Activities Office (SAO), know that library staff have permission to stand and hand out postcards. - Linda 3/13)

    Linda will let us know as soon as Bob’s email has been sent

    Linda to order an extra poster for the African Studies Center.

    Linda to send an email– giving staff basic info (start, end dates) and indicators on do’s and don’ts surrounding the survey. Linda will contact Marlene. Young-Joo and Jim will pass it on to their staff.

    Linda and Dan to add their SurveyKeepers to the site

    Dan and Linda to work on connecting the survey to the done page (prize entry page)

    Dan to create banner on top pages for all library sites (Library websites message/banner-- OK'd by Bob Hudson and Tim Lewontin. Dan  Benedetti will create a banner message and post it no sooner than 3/20 on the top page of www.bu.edu/library. He will ask each Mugar branch head if they would like a banner on their top page. If so, he will do this for them and post no sooner than 3/20. Dan will also let Marlene, Jim Skypeck, and Young-Joo know that we are doing this and offer to give them the HTML code if they would like to follow suit.- Linda 3/12)

    Dan to check if the collector/ existing survey results can be cleared (and should ensure this is done on the night March 19. (Dan was able to clear the one person who had taken the survey. So please do take the survey using this URL--surveymonkey.com/s/bugradstudents. Please take the survey all the way through including registering for the drawing. Please email me by Wednesday evening after you've done this and let me know if it went smoothly or if you ran into any glitches whatsoever. Then Dan or I will clear the survey and the drawing spreadsheet before 3/20.- Linda 3/1/2)

    Dan to check with Yelena to see if she can change her hours to help in the BU Pub.

    Sarah to call ODE – (look over Tom’s email beforehand)

    David to contact Brock for help handing out postcards at SMG Starbucks

    Sarah to remind Lisa to place posters

    Young-Joo to ask Constantine and Mary to place posters while she is away.


    Discussion Items:

     Survey recruitment strategies:

    ·         Broadcast emails:

    1.     Linda contacted the provosts dept and the IRB and received permission to include a link to a graphic html page in the 4 broadcast emails that are to be sent out.  Susan Wishinsky created the web page at www.bu.edu/library/about/gradsurvey/gradsurvey.html .  The link out to the web page is embedded in the email, not the graphic itself. No attachments are permitted in broadcast emails.

    2.     The smaller graphic is darker so Mary McGowan and students are to work on this

    3.     David noted that many browsers do not display graphics by default and suggested that the first email should be sent in a text only (universal) format.

    4.     Sarah suggested we send a test email to ourselves first.  Linda agreed and will send a test email to assess-l. 

    5.     A final decision on the graphic/text emails will be made at the next AC (assessment committee)meeting.

    PJ is lining up an analyst for our email list, it has not yet been created. (I have asked that the analyst who is developing the email list of graduate > students give us counts that will be helpful for the analysis of the data.  I've asked for the number of graduate students in each school or college who will be on the email list. From this we can figure out the breakdown of CRC vs. Med Campus by adding up the number of grad students in each of the three schools on the Medical Campus. I also asked, if possible, for the analyst to send us the number of Distance Education students who would receive the emails. I don't think they can do this, but I thought I'd ask. Is there any other information we want from the analyst creating the email list? – Linda 3 /12)

    The survey does not yet connect to the “done” page which collects the prize entry information.  Dan and Linda will work on this.

    Linda would like to test to ensure that the collector is working and that the spreadsheet is functioning before March 20. Dan will look into this.

    The existing survey results should be emptied/deleted pref. on March 19.  Dan will check to see how /if this is possible. (Dan was able to clear the one person who had taken the survey. So please do take the survey using this URL--surveymonkey.com/s/bugradstudents. Please take the survey all the way through including registering for the drawing. Please email me by Wednesday evening after you've done this and let me know if it went smoothly or if you ran into any glitches whatsoever. Then Dan or I will clear the survey and the drawing spreadsheet before 3/20.- Linda 3/1/2)

    Recruitment Posters:

    1.     There are

    a.     one 13x19 per school

    b.    two 13x19 per library (try to place in lobby)

    c.     one 8 1/2x11 per department

    2.     Locations and numbers of  8 1/2x11 posters:

    a.     Med ( 8 posters) – Young-Joo (will ask Constantine and Mary to place the posters at Med)

    b.  SPH  ( 8 posters) – Young-Joo 

    c.  Dental ( 10 posters) – Young-Joo 

    b.    ENG ( 3 posters)  – David Fristrom

    c.     Sargent ( 4 posters)  - Sarah (to remind Lisa; also find out no. of posters; place in grad lounges)

    d.    COMM ( 3 posters)  – Diane(Linda is to contact Diane D’Almeida and ask her to place posters)

    e.     CAS ( 24 posters)  – Dan and Shirley (24 depts, SED- Dan; CAS place 13x19 on grad bulletin board)

    f.     MET ( 10 posters)  – Jim Skypeck

    g.    CFA ( 3 posters)  – David Fristrom

    h.     LAW ( 6 posters) - Shirley L-S

    i.      MGMT ( 4 posters)  – Sarah Struble (Terry Crystal and Mary to help?)

    j.      School of Social Work ( 0 posters) and Thelogy ( 3 posters)  – Jim

    k.   SED ( 3 posters) – Dan 

    Use your good sense and ask re: best placement for posters

    3.     The orders for all of the graphics are in

    a.     3000 postcards

    b.    35 (13x19)

    c.     6 (24x30) – one for the Mugar lobby

    d.     (8 ½ x11) these will have a white border, Linda will have them printed by the end of the week.  Linda will send them to Young-Joo via Joe(wed/Thursday).

    4.     Hanging 13x19 posters in the libraries (2 posters each):

    ·         SEL and Astro – David Fristom

    ·         ASL – Shirley L-S

    ·         Pardee – Dan Benedetti

    ·         PERL – Dan Benedetti

    ·         Music – Sarah Struble

    ·         LAW – Shirley L-S

    ·         Med – Young-Joo Lee

    ·         Stone – Linda Plunkett

    ·         Theo - Jim Skypeck

    ·         Mugar - Linda Plunkett

    Distribution of responsibility for 6 dining halls/ postcard distribution:

    1.     3000 postcards have been ordered.  If we begin to run low it will take 4-7 days to reorder.  When our supply drops to below 1000 we need to communicate this back to Linda.

    2.     Responsibility for each of the 6 dining halls includes:

    §  Taking charge of postcards

    §  Taking charge of people handing out the postcards

    §  Easels

    3.     Allocation of dining halls:(1 large poster plus easel each)

    §  LAW School Café – Shirley and Jim(some hours)  [Marlene Alderman has graciously accepted responsiblity for the Law School Cafe) Shirley3/15/12]
    M-Th: 8a-3p; F: 8a-2p

    §  SMG Starbucks – David (with Brock; Danny Pierkarski? or Pardee night staff?)
    M-Th: 7:30a-9:30p; F: 7:30a-4p; Sa-Su: noon-6p

    §  Faculty Dining hall – Sarah
    M-F: 11:30a-2p

    §  GSU Food Court – Linda
    Sun-Th 11a-8p; F-Sat: 11a-7p

    §  BU Pub – Dan (Yelena?)
    M-W: 11:30a-11p; Th-F:11:30a-midnight

    David will contact Brock; Linda to ask Tom for help; Shirley to contact Marlene Alderman; Dan to ask Yelena if she can change hours; Everyone can contact Tom to get staff to help out.
    Scott has given us permission and will talk to the dining hall managers so they know what/when to expect us.  Scott will need to know when we expect to have our staff present.  The leader must introduce themselves to the manager/desk personell

    4.        Linda will contact Rubin Rossi(?) about reserving the GSU demo area/link. ( GSU Demo Area
    > We have booked the Demo Area of the GSU (first floor, entrance to food > court) all day, M-F, 4/2 through 4/17, but not 4/4. The dates between 3/20 > and 3/30 were already booked.- Linda 3/12)

    5.     Easels: there are 3 new easels(med already has one; Dan already has one for the pub).  One easel for each dining room (Linda, Sarah, David and Shirley each get one).  Kathy has them in the coat room (they must be returned).

    6.     Sarah to call ODE and should look over Tom’s initial email.

    Monitoring participation: David to help Linda to monitor participation once the survey is open.

    Survey recruitment strategies general:

    Bob will contact the deans and will ask to include digital postcard on schools smartboards,/ in lobbies/ entrances. (When Bob contacts the Deans next week about the survey he will ask them to > contact their graduate students and encourage them to take the survey. Bob will also suggest that they post a message on one or more centrally located smart boards or electronic monitors in their schools. He will include a couple of our recruitment files for their use on the monitors – Linda 3/12)
    Linda to get another poster for the African Studies Center. Centers and institutes must be included in future (surveykeeper).
    The library website will display a banner with a link to the survey. He will not mention the incentives.  Dan will investigate and implement this once we have Bob’s permission.
    (Library websites message/banner-- OK'd by Bob Hudson and Tim Lewontin. Dan > Benedetti will create a banner message and post it no sooner than 3/20 on > the top page of www.bu.edu/library. He will ask each Mugar branch head if > they would like a banner on their top page. If so, he will do this for them > and post no sooner than 3/20. Dan will also let Marlene, Jim Skypeck, and > Young-Joo know that we are doing this and offer to give them the HTML code > if they would like to follow suit.- Linda 3/12)

    ·                     Linda will let us know as soon as Bob’s email has been sent (anticipate sending 3/20 or 19)

    Wrap up of library wide meeting:

    Largest number ever to attend  A success!
    Dan asked that next time we highlight the changes that we made to the UW survey, and also that we do not hand out the URL to staff before the go live date (in future).

    Surveykeepers:

    All institutes and centers should be included (as well as school/colleges etc.)

    Use public terminals to advertise the undergrad survey

    Extra-large posters in the link


    Discussion Items for next meeting:

    Should we make the 2, 3, 4 emails plain text or include the graphic?



    Minutes: Thursday, 2/23/2012, 1:30-3PM


    Present:
    Linda (chair), Jim , Young-Joo (minutes),Dan, Shirley, and Sarah

    Next meeting: Monday, March 12, 9:30-11 am

    Action items:

    ·       Survey comments communication: Develop clear guideline on communicating survey comments to library staff

    ·       Survey comments & Privacy issue: Shirley will email to Linda the list of librarians who received the comments, Linda will send email to all librarians to remind them about confidentiality(done - Friday, February 24, 2012 3:56 PM)

    ·       Survey Recruitment: Jim will get the list of dining rooms (and their schedules – found: http://www.bu.edu/dining), information on grad students’ dining habit (where and when) from the Dining services (done: list (Friday, February 24, 2012 10:11 AM); communication with the contact person at Dining Services – permission to hang poster granted(Friday, February 24, 2012 12:01 PM))

    ·       Recruitment on Med Campus: Young-Joo will find out whether we could handout posters at Peets’ café in the Medical building, use electronic bulletin board (big TV screen on the Med building) and will arrange who and how will post/distribute the posters and postcards on the week of March 19 (She’s out of town March 20-22).

    Discussion Items:

    • Shirley: showed the Comments in excel (received applaud) : suggested having a column for each person for next survey comments
    • Survey comments:
      • All agreed that comments were useful;
      • How to communicate the survey comments INSIDE library. Should a supervisor pass the information to his/her staff YES; Shirley reported some problematic situations
      • Develop clear guideline on communication survey comments to library staff à Department affiliation should be noted for all comments; each library will get comments related to their own library? (example I have it Theology is only to Jim).
      • Privacy issue: Some librarians wanted to talk to faculty regarding their comments  absolutely NO (it’s violation of IRB policy, survey is to be anonymous, even when we can recognize the respondents) à Linda will send email to let staff know that they cannot address the comments (even to thank them) à for next survey, comments will be grouped in school level (?) à Dan: “Although we cannot communicate the comments, we need to address the issues”
    • Survey recruitment strategies (Linda):

    (IRB application says…)

    The University Provost will send out two reminder emails spaced approximately one week apart. A third recruitment email will be sent from the four Library Directors of the main libraries, if deemed necessary by the response rate.

     

    Attached please also find a two files we will use for recruitment—one is for a poster and one for postcards-- which we will use to encourage graduate students to take the survey.

    These files will be sent electronically to:

    §  All the schools and colleges that have graduate students asking them to distribute the flyer (or the information in the flyer) to graduate students while the survey is open

    §  The Office of Distance Education asking them to distribute the flyer (or the information in the flyer) to their graduate students while the survey is open

    §  The various BU library websites asking them to post the flyer (or the information in the file) while the survey is open

    §  These files will be printed as:

    §  Posters which will be hung in each BU Library

    §  Posters which will be hung in each department in each BU School or College that has graduate students

    §  A poster which will be hung on the outside of Warren Towers

    §  Postcards which will be handed out in the George Sherman Union and other dining halls on both campuses.

    §  Strategies to increase participation:

    o   Poster & postcard: on 3/20 – 4/20?

    §  Locations of posters:

    ·       Med (YJ –will be out all week on the week of 3/20, need to arrangement with other librarian at Med Lib)

    ·       End(David)

    ·       Sargent(Serah)

    ·       CAM(Diane)

    ·       CAS(Dan & Shirley)

    ·       MET(Jim)

    ·       Dining Rooms (may need permission – Jim will ask)

    o   Jim (will send poster to his contact person?)

    o   Linda (should track what we’re doing, will get permission for shuttle bus next time)


    ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    Minutes: 2/1/2012

    Present:  Dan, David, Jim, Linda (chair), Sarah, Shirley, and Young-Joo

    Minutes jointly taken by Jim, Sarah, and posted by Linda

    Action items

    Next meeting: TBD via a Doodle Sarah will send out

    • Pilot: Sarah will send out a Doodle scheduling the next AC meeting. The purpose of this meeting will be to plan and carry out a pilot of the survye with BU graduate students. No changes based on the pilot will be made to the survey until after 2/14. 
    • David and Shirley will review the survey and send an email to Linda on 2/1 with any changes that need to made. (DONE)
    • Linda will make any necessary changes and save the finalized survey as a pdf for the Provost's Office review. (DONE)
    • Linda will send the finalized survey, the recruitment flyer, and the five recruitment emails to the Provost's Office on 2/1 for review. (DONE)
    • After receiving the Provost's approval (or approval pending relatively minor changes which Linda will make), Linda will send the application for exemption to the IRB on 2/2.
    Topics

    • Committee reviewed what still needed to be done before submitting the application for exemption to the IRB.
    • Committee briefly discussed the pilot. Sarah will send out a Doodle scheduling the next meeting to plan the pilot.
    • The following changes were made to the survey:
      • Based on feedback solicited from the library directors and the subject librarians, we corrected some of the names of the departments and programs in the drop-down choices.
      • Based on feedback from graduate students in our pilot, we divided one question into two questions and changed the wording. 
        • The original question was: "How frequently do you use the following methods to access BU Libraries?". This question had two sub-parts which were transformed into the two new questions. The two sub-questions were "Visit in person" and "Access online". The two new questions are: "How frequently do you visit a BU library in person?" and "How frequently do you access BU Libraries' resources, services or websites online?"
        • We changed the display of the frequency choices from a horizontal display to a vertical display.
      • In the question "Please rate the IMPORTANCE of the following resources to your work:" We changed the order of the resources. The Director of the Medical Library had requested that we change the order of some of the resources. We were able to accomplish this by randomizing the order after ensuring that the two questions about books and the two questions about journals remained adjacent. Based on feedback from graduate students in our pilot, we added examples to one of the resource types. We changed "Audiovisual media" to "Audiovisual media (e.g., video, images, sound)"
      • In the following related question, "Please rate your SATISFACTION with the following BU library resources:", we made the exact same changes as above.
      • Based on feedback from graduate students in our pilot, we changed the wording of the question "Please rate the importance of the following to your success in coursework and research:" to "How important have the following skills been to your success in your current academic program?"
      • Based on feedback from graduate students in the pilot and from two Associate University Librarians, we changed the list of services in the question "Please rate how useful augmenting the following library services would be for your work:". The list of services is now:
        • Linking to library resources and services in course management software (e.g., Blackboard, Sakai)
        • Online tutorials for databases and research skills
        • Assistance with submitting papers and other publications to journals and digital archives
        • Support in managing and archiving your research data
        • Access to streaming media resources
        • More quiet work and study areas in libraries
        • More group work and presentation spaces in libraries
    • We added two ideas to survey keepers
      • 2/1/2012-- consider asking grad students how long they have been at BU
      • 2/1/2012-- consider asking grad students to self assess their info lit skills
    • We reviewed the two Human Subject Training Certificates. These will be faxes to the IRB on 2/2.
    • We reviewed all eight Conflict of Interest forms for completeness. These will be attached to the IRB application on 2/2.
    • Dan finalized the survey.

    Minutes: 1/23/12

    Present:
    Linda (chair),Jim (minutes), Young-Joo, David, Dan, Shirley, and Sarah

    Next meeting: 1/27/12@1:30pm

    Action items:

    Jim will continue to look at university departments to eliminate those without graduate students.
    Linda and David will meet with consultants on Thursday and share findings with group on Friday.
    Dan/Linda will change the wording on the "raffle" page from raffle to drawing.
    Dan will add "graduate student councils" to preamble line about sharing results once he hears from Linda.
    Dan will change the wording to UW's for #3 Library Resources.


    Discussion Items:

    • Welcomed Young-Joo to committee.
    • Added "distance education" question to survey.
    • Second and subsequent words in survey questions/answers will be lower-case. (Sarah will proof-read).
    • When discussing Boston University, survey team will use the initials "BU". When discussing the BU Libraries' as an entity, we will spell out "Boston University."
    • We will revisit the library hours page when the categories have been added.

    Agenda for next meeting:

    • Continue to go through charts and survey.
    • Decide on the method of the drawing.
    • Discuss pilot and review practices (Lisa and Meredith may serve as our staff reviewers).
    • Conflict of interest forms.

    ___________________________________________________________________________________________________________________________________________

    Minutes: 1/20/12

    Present: Linda (chair), Shirley(minutes), Jim, Dan B., David, Sarah

    Next Meetings: 1/23/12 @ 11:30am,  1/30/12 @ 10:00am (Location: conference room, Adminstrative Offices, Mugar)

    Action Items:

    • Linda:
      -Ensure that distance students are included in the email list
      -Check to see if we can exclude certificate students from the email list
      -
      Ensure that the first reminder is sent by the Deans (ensure PJ does this in the IRB)
      -
      Include the prize info web page in the IRB
      -Create a ‘Thank you” web page using google docs/forms to harvest information for the prizes.  Note that only those with BU email addresses will be included in the competition
      -Once Grad survey is complete send it to Mary for comments
      -Ask Steve his rationale for not including any questions on Open Access (regarding the “decision to publish” etc) for grad students.
    • Linda And Jim: check the BU preamble (Linda confirmed the preamble- no need for Jim to check)
    • Jim: check the college/school/departments names and exclude those with non-grad students; communicate these changes to Dan who will be sole editor
    • Dan: Change the scale options on Q21, and all the other questions, back to the Faculty Survey options.
    • Linda and David: Will discuss wording with a consultant.  Will gain clarity about the scale headers/labels regarding time intervals; whether to include N/A as a 6th option to time intervals, whether N/A is equivalent to never; how to designate various levels of satisfaction;  and methods to include failure to respond.

     

    Discussion Items:

    • News
      • Young-Joo Lee from the Medical library will be joining the Assessment Committee
    • Updates
      • Faculty Survey
        • Linda Carr to meet with Shirley on Monday to finalize assignments of comments. Shirley completed the human studies workshop
    • Discussion:
    • Graduate Student Survey
        • IRB is almost done.  The Associate Provost is on board and an analyst will be made available to help us
        • Mary McGowan and her students will complete the poster next week
        • We decide to include Distance Ed students and include the question about department affiliation.
        • 2010(?) there were approx. 5000 grad students that graduated
        • Raffle - a thank you web page (URL) to harvest email addresses and linking to a spreadsheet must be created; subject to IRB/Provost approval.
        • Edits to the grad survey were done on the fly by Dan who will be the sole editor.
        • Edits to note:
          • Q1 - 'Distance Education Programs' was added as a third option. Sarah felt the third option was necessary because many Distance Ed students were not sure of their campus affiliation. David confirmed that he could identify the Distance Education Students as affiliated to Charles River Campus or the Medical Campus by filtering them out according to their departmental designations. Sarah recommended including an explanation of the logic behind the inclusion of the third option in the IRB report
          • In order to gather comments under the Publication and Open Access section, we decided to make use of duplicate comments questions, one appearing in each section of the skip logic block. This solved the problem of the comments appearing alone on a single page.
          •  Changes that may impact the Faculty Survey next year:
            Q27 - we changed the wording to "No page or publication charges to author" to "Page charges or other costs to author"
        • Linda and David will meet with a consultant next week who will advise on the use Likert Scales, for example the correct use of labels for scales when designating time intervals or levels of satisfaction. Sarah queried the inclusion of N/A as a sixth button in time intervals and asked if it was the equivalent of 'never'.  Sarah wondered how a failure to respond to a question could be included. Linda requested that all scale headers be reverted to those used in the faculty survey to facilitate the discussion with the consultant.
        • Only our editor, Dan, will make changes to the survey monkey site.

     



      __________________________________________________________________________________________________________________


      Minutes: 1/12/12
      Present: Linda (chair), David (minutes), Jim, Dan B., Shirley, Sarah

      Next Meetings: 1/20/12 @ 1:30pm, 1/23/12 @ 10am (if needed), 1/30/12 @ 10am (Location: conference room, Adminstrative Offices, Mugar)

      Action Items:
      • Shirley: Analyze faculty survey comments, prepare them to share with staff
      • Linda: Write message to library staff, telling them they will receive comments, and what we have done to improve things.
      • Everybody, by next meeting: Sit down with survey -- orignal UW and as marked up by Dan, as well as our faculty survey -- and decide what changes need to be made
      • Dan: Put graduate student survey on Survey Monkey
      • Shirley: Ask Beth if she can create art for posters (LP- Note: No longer needed; Shirley notified. The Anita Greene Scholars will create the posters)
      • Shirley: Take human studies workshop
      • Sarah: Consider before next meeting whether or not we want to include Dist Ed grad studetns in our population for the survey and what the ramifications would be (LP added this action item on 1/19)
      Discussion Items:
      • News
        • Dan. P has resigned from committee
          • He will be sorely missed
          • We have an opening and should consider what skills we need
      • Linda proposed and everyone agreed on the following goals for committee for year:
        • Wrap up Faculty Survey
        • Conduct Graduate Student Survey
        • Support Alma Analytics
        • Prepare ACRL/ARL peer comparison for Bob
      • Faculty Survey
        • Shirley will take over analyzing comments, prepare them to share with staff
        • Linda will write message to library staff, telling them they will receive comments, what we have done to improve things.
        • Keep list of lessons learned up to date (Use the Survey Keepers link in the navigation bar)
      • Graduate Student Survey
        • Need to submit to IRB before Linda leaves on Feb. 3rd.
        • Need philosophical discussion about what we want to do differently.
        • Shirley distributed comments on what faculty thinks should be different in survey.
        • We will use Likert scale
        • We will use incentives
          • Could be complications with anonymity
          • 5 or 6 $50 Barnes and Nobles cards as prizes?
        • Everybody should sit down with survey -- orignal UW and as marked up by Dan, as well as our faculty survey -- and decide what changes need to be made
        • Discussed whether we need question about department affiliation
          • Decided we do need question (especially for breaking down CAS)
          • Will use skip logic in Survey Monkey again
        • Sarah asked, are we including distance services students in survey?
          • If conclusion was reached, I failed to record it. (LP note: Sarah will investigate this and we will continue this duscussion at our meeting on 1/20)
        • In campus affiliation question, need “Other” choice with “Please specify”
        • We will Get rid of question from UW survey about impact of reduced services.
        • We looked at what Dan changed, and generally approved
        • Dan will put survey into Survey Monkey
        • Possible complications with proposed date for survey (LP note: we're shooting for  March, 20, 2012)
          • New library website might be going live
          • Alma
        • Posters for publicity
          • Work with Mary McGowan? (LP note: all set. Mary M. and her Anita Greens Scholars will create the poster)
          • Shirley will ask Beth if she can create art (LP note: no longer needed)
        • Shirley will take human studies workshop
        • Should have at least a staff pilot to catch typos and oversights
          • Should use Grad student staff



      Minutes: 9/20/11
      Present: Linda, Sarah, David, Dan B (minutes), Dan P, Shirley, Jim

      Next Meeting: Jim Self visit 10/03/11 | 3-4:30 | Mugar Conference Room

      Action items:
      • Sarah will be meeting with David re charts for the SAR report.
      • David will transfer the punch list Word document into a Google Docs document and link to it from the wiki.
      • Everyone will add their punch list items to the Google document by 9/28/11.
      • We decided to highlight web related punch list items and forward that to the Web Committee for consideration as soon as possible.
      • Jim will speak with Tom Casserly re: positive feedback on wr100 and wr150 program.
      • Review Linda's action items on the punch list document.
      Discussion items:
      • Hello and welcome to Shirley Leslie-Smith.
      • Status overview of the customized reports.  Four remain:
        • Sarah will be meeting with David re charts for the SAR report.
        • Dan will revise the SED report.
        • The CAS and CFA customized reports are also finishing up.
      • Linda provided a Punch list Word document for Recommendations / action items from the 2010 Faculty Survey:
        • Jim asked about tracking comments that are critiques of the survey itself.  It was decided that those may fit well under the Survey Keepers section of this site.
        • We decided to transfer the Word Document into a Google Docs document.  David will do that and link to it from the wiki.
        • Everyone will add their punch list items to the Google document by 9/28/11.
        • Linda asked that the present tense be used for items on the punch list.
        • We noted that many comments or themes cut across the various areas of the surveys we worked on (Library Resources, Services etc.).  We decided to list items as we have them, and then possibly look at consolidation after everyone adds their items to the document.
        • Given that there is currently an ongoing web redesign project, we decided to highlight web related punch list items and forward that to the Web Committee for consideration as soon as possible.
        • At this point, we went through individual sections as they were assigned (see minutes of last meeting). Some brief highlights: Dan noted how staff are regarded as resources and if put that way to staff we may get some buy-in to better performance.  David relayed comments on high cost of Open Access publishing for authors. Jim will speak with Tom Casserly re: positive feedback on wr100 and wr150 program. Sarah spoke about the desire for online tutorials, and we discussed.
        • Danny will continue categorizing comments.
        • We all will review Linda's action items on the punch list document.
        • We did decide to survey graduate students next.
      Tentative agenda for next meeting:
      • "Punch list" document review.
      • Continue discussion on what we need to convey to the library staff: who to address with what action items.
      • Review of Dan P's tagged comments spreadsheet
      _______________________________________________________________________________________________



      Minutes: 8/29/11
      Present: Linda, Sarah, David, Dan B, Kate, Jim (minutes)

      Next Meeting: 9/20/11|3-4:30|Mugar Conference Room

      Action items:
      • Sarah will work with Linda to finish the CFA report.
      • Sarah will work with David to complete the SAR report.
      • Linda P & Danny (with some help from Linda Carr) will work on completing the CAS report.
      • Members will look at the final report and look at assigned sections with a focus on action items and comments. Please tag the comments in your section so we can compare them with Danny's.
        • Information Literacy-Jim
        • Library resources-Dan B.
        • Library Services-Sarah
        • Open Access-David
        • Overall Satisfaction-Linda
        • Comments--Danny P (please analyze and tag comments)
      • Linda will ask Bob Hudson about using a library-wide meeting to review the survey as well as his preference about the next survey: undergraduate or graduate students?
      • Bob H will let the Provost know that the customized reports exist but may not actually provide them all.
      • Jim will share survey results with the Theology staff in a staff meeting.
      Discussion items:
      • This was Kate's last meeting with the group. We wish her much luck at Tufts. Kate mentioned that we may get a Medical library replacement this year if the focus is on graduate students in the next survey. If the decision is made to survey undergraduates first, Med may delay sending a replacement.
      • Mary at Med has presented the MED report to the Provost and Kate has put the final copy in the Dropbox folder. There were two favorable results of this presentation:
        • We may get PLOS.
        • The Provost is willing to look at the Medical Library facilities as a result of some comments on the survey.
      • Will customized reports be done in the future? There was much discussion on this issue with the following possibilities presented by the members of the committtee:
        • Do the reports as a group.
        • Allow members to create reports if there are requests from their constituents.
        • A full report discussing BU-wide results with appendices that are school-specific (charts, graphs, tables).
        • Next report should highlight differences (campus, discipline).
        • We aren't sure we can determine how useful the individual reports are until we get feedback from the Deans of the schools and/or the faculty.
      • The committee will look at ways to distribute survey information to the library staff. One idea was to use a library-wide meeting with Bob Hudson. Another was to send e-mails to library heads with the reports.
      • The possibility of having a focus group with a select group of staff volunteers was suggested. We'd have someone else conduct the focus group, provide pizza (or food of some kind as an incentive), and ask them 3-4 questions about the results of the survey and next steps.
      Tentative agenda for next meeting:
      • Determine what we need to convey to the library staff:
        • Highlights of the survey
        • Things that we should be proud we've done (we=library staff, not the Assessment Committee)
        • Action items: what do we need to do based on survey results.
      • "Punch list" section by section.
      • Discuss new members to the committee.
      • Determine which group will be surveyed next.
      • Tagging comments: comparing what we've done with one another.
      ________________________________________________________________________________________________________________________________________


      Minutes: 8/9/11
      Present: Linda, Dan B., Danny P., Sarah, David, Kate and Alex (minutes)

      Next Meeting: 8/29 | 3-4:30 |  Mugar Conference Room

      Action Items:
      • Everyone: get reports into final folder
      • Alex, Danny and Kate will share their reports to library heads (if they haven't already).
      • Alex will do the Sargent graph
      • Linda will track down final CAS Report
      • Everyone to look at UW's student surveys (Grad and Undergrad)
      Discussion:
      • Alex's last meeting... we are sad. Kate is also leaving... we are sad again. How on earth will these lovely ladies be replaced :) 
      • Customized reports: Bob Hudson may want customized reports for future surveys
      • Implementation of changes suggested by report: 
        • Assessment committee could hold open meetings with staff to get recommendations. Or schedule meetings by departments.
        • Baseline numbers necessary for eBooks (Roger Brisson?). See how the collection grows over the next few years.
        • Identify faculty misconceptions in the survey and try to use them as education opportunities. Address them in instruction sessions. 
        • Provide the assistance with open access issues and course management sites as promised in the proposed services question
      • Next steps: 
        • Possibly survey grad and undergrad together in one? 
      • Jim Self to meet with Assessment Committee on October 3rd.
      ____________________________________________________________________________________________________________________

      Minutes 7/28/11

      Present: Dan B, Kate, David, Dan P, Linda, Jim (minutes), Alex, and Sarah

      Next meeting:  


      Tuesday 8/9 2:30-4:00, PERL E-lab

      Action items:

      • Kate will look at the UW survey for phrases like, "Faculty are more likely to...", to see if said phrases imply a judgment or project a finding into the future.
      • Folks will continue to edit reports in hopes of having them ready by August 15th. Jim will work with Sarah to fine-tune the Sargent report.
      • Danny, Alex, and Kate will check with Arlyne, Holly, and Mary respectively about including their names on the reports they've done.
      • Everyone will look at the scales in their charts to make sure that any charts doing a comparison (e.g., Sargent vs. non-Sargent schools) are using the same scale.
      • Dan B will check to see if we need to be re-certified by NIH for our work.
      • Future meetings will  involve a discussion of:
        • membership
        • skills (what skills do we need to have on the committee to do our work?)
        • the future of customized reports
        • our next steps.
      Discussion:
      • For next survey, we need to clarify what we mean by "staff assistance" and "subject librarians" to differentiate what kind of help we are discussing and who is providing it.  There were questions whether or not respondents made this distinction in their responses to the survey.
      • How will the libraries assist with courseware integration? Cross-library task force? Another committee? Should we have an IS&T contact or point person for this process?

      ________________________________________________________________________________________________________________________________

      Minutes 7/14/11

      Present: Dan B., Dan P (minutes), Jim, Linda (chair), Kate, Alex, David

      Next Meeting:

      Thursday, 7/28 2-3:30, Mugar Admin Office

      Action items:  
      Finish your personalized reports
      Error correct your partners report (see below for pairings)

      Decisions:
      • We decided to place names on reports in the following way:
      MED - Mary's name on the report
      SMG - Ask Arlyne
      Education - Dan B. will place his name on the front of the report
      SAR - Assessment committee
      CAS - Assessment committee
      LAW - Will ask
      CFA - Offer to Holly

      • We will use David's style and formatting for the reports
      • We have set a August 15th deadline for editing
      • On our next survey we will ask about usage of services so we don't have to extrapolate from the data
      • We will send the Provost copies of all the reporst
      • We may not do customized reports next survey, instead focusing on one main report
      Discussion:
      We are currently working on 8 customized reports. We have setup proofreaders for the reports. The groupings are as follows:
      Alex & Linda
      Sarah will proofread SAR for Jim. Kate will work on the formatting.
      Dan B & Alex
      David and Jim will work together on the numbers. Linda will format the report with help from Sydney.
      Kate and Dan P. will work together.

      Our consultants are waiting on BU Works to be functional before they can start helping us.

      ________________________________________________________________________________________________________________________
      Minutes 4/26/11

      Present: Dan B., Dan P, Jim, Linda (chair), Kate, Alex, David (minutes), Sarah

      Next Meeting:

      Monday, 5/2 3-4:30, PERL, webinar with LibPAS

      Action items:
      All: Think of questions for LibPAS
      David: Help Jim with his report as needed


      Decisions:
      Aim for June for completing all reports.
      Bob will send CAS/GRS report, heads of branches will send their reports, Mary will send medical reports, etc.

      Discussion:
      Dan P. mentioned need for error checking on reports.

      Linda asked for opinions on LibPAS, but no one had any at this time. All need to think of questions for webinar.

      David showed completed first draft of report for ENG; had some feedback. Discussed who report should be “to”; decided dean of ENG.

      Linda suggested having partners for completing reports; having Alex help her was extremely useful. David volunteered to help Jim.

      Some talk of deadline. Linda reports Bob thinks it would be great to have them to Deans by end of semester. Sarah can not make it her highest priority. Linda pointed out Deans don't disappear in summer, June might be reasonable. All should probably go out at around the same time (within reason). Bob will send CAS/GRS report, heads of branches will send their reports, Mary will sent medical reports, etc.

      Linda reports consultants willing to work with us, not too expensive.  One condition, which we felft was reasonable: if we are dissatisfied with their work, talk to them rather tan gossiping with others.

      Status of reports:

      • David (ENG): First draft complete
      • Jim (CAS/GRS): Almost completed charts that don’t involve disciplines, hasn't started discipline charts yet.
      • Dan P (SMG):: Did six hours yesterday, getting there. Doesn't know if he needs Numbers.
      • Dan B (SED):: Worked a lot on SED report, mainly comparing to not-SED faculty, may change that.
      • Sarah (SAR):: Got started.
      • Alex (CFA):: Doing analysis of data, has lots of tables, created one chart, looks okay.  Needs help on meaning.
      • Kate (Med campus reports): Has all data broken down in Excel tables, has outline of words, needs to make words more cautious. Problem making complicated Excel and then brining into Numbers.  May make single report for whole medical campus -- see what Mary wants.
      • Linda (Law): Charts for Law are in good shape (though not finished); hasn't written words yet.  They love law library.

      Down the road, we might do a comparison of embedded libraries to non-embedded.

      Kate would like help with error checking.

      Dan P. is working on tool for error checking.

      _______________________________________________________________________________________________________________________

      Minutes 4/14/11

      Present: Dan P, Dan B, Linda (chair), David, Jim, Kate (minutes)

      Next Meetings:
      • Tuesday 4/26 3-4:30
      • Monday 5/2, 3-4:30
      Location: Mugar admin, unless otherwise notified

      Action Items:
      Dan B will make a list of possible tags for comment analysis.
      Linda will set up a webinar with Counting Opinions, ask about possibility of 2 different locations
      Continue with customized reports (all)



      Discussion:
      Survey Monkey Gold

      Up and ready to use. Costs $300/year. Dan B briefly discussed his exploration of SMGold, saying it did interesting things with comments, like makes tag clouds and lists frequencies of certain words. For any comment we can add multiple (5-10?) tags.
      Linda likes the negative/positive categories that Dan O'Mahoney did.
      Discussion of using library names as tags. Even though filters are helpful in organizing the school/dept origin of the commenter, David points out that tags denoting library names are still worthwhile for when specific libraries are mentioned in comments.
      Linda asks how we should start a list of tags, important to have a complete so we don’t have to re-tag. Dan B volunteered to make a list, then we can all review and tackle at next meeting.

      Counting Opinions LibPAS
      Linda talked about ACRL vendors. The program Roger Brisson recommended (?XML) was too expensive ($25,000/year). But she found Counting Opinions which is a lot less expensive ($7000/yr with discount for a 3 year contract). They have also done impressive things with Rutgers and Cornell. Their product can pull data from different systems (like III, gate counts, surveys). Includes LibSAT and LibPAS. LibSAT more qualitative. More interested in LibPAS. There is no example site to look at, but Linda will set up a (free) webinar so we can see the examples provided at the conference.
      A big issue associated with LibPAS is the amount of time involved in setting up the data collection and organization. The reps estimated it would take one person 75% of their time for 3 months to have it set up. Linda thinks the time-investment might be worthwhile, because once it’s set up the data collection can be automatic, or at least easier to submit, and what the programs can do with the data is impressive.
      The company reps claimed they are familiar with III. There could be an issue with ezproxy stats - the company hasn’t worked with it before but wants to.
      Linda will talk to people at Temple and possible University of Arizona as they have the product.
      David thinks this looks like a good product but will take a lot of staff commitment to make worthwhile. Assessment is only one part of it. He is also concerned that the company is still in the pilot stage (based on their advertising material). Linda says from what she saw of the Rutgers and Cornell programs, they seem to be able to do a lot of what they promise.
      Also said it contains a survey tool that some users (according to reps) have described as “survey monkey on steroids.”

      All agreed we should at least look at the product via a webinar. Linda will set up to coincide with one of the next two meetings. If we can get the webinar simultaneously on two campuses, this would be better as Kate wants to invite people from med lib who deal with more with stats.

      Consultant Updates
      Bob OK’d taking on Gordon Fretwell as a consultant on retainer. Gordon is a retired UMass assesment coordinator and a consultant for ARL stats. Gordon asked that we also take on Rachel Lewellen, a part time assessment coordinator at UMass Amherst.

      Progress on Customized Survey Reports
      Jim has written a lot of the CAS report and thus has a lot of questions from this experience.
      1. Is it important to mention when a department is under reported?
      We think no, because we carefully describe the data in the charts (as in, “respondents who answered this question).
      2. The plan for reports - will they first be given out by Bob?
      The library director will have discretion here. For example, Arlene can give to School of MGMT. The Provost wants to see all the customized reports at some point, but she doesn’t need to see them first. For CAS, Bob will give this one out, and then the dept liaisons can work with the reports.
      Ultimately these reports will be public, like the original report is.
      3. How do we create these reports with respect to the original? Should they be addenda, or should they stand on their own?
      Discussion about the merits of either approach. Decide we want the report to stand on its own, but we shouldn’t repeat too much of the original report. David pointed out that readers looking carefully at the first document will be forced to go through the same information twice.
      We want to report to stand alone, but not be the same length as the original. As Linda put it, the customized report should be like a short story; the original report, a book.

      Discussed the need for an example, or a rough template, but the reports really should reflect the specific needs of each college.
      Kate will summarize parts of the original document.
      We are encouraged to steal from each other.
      Jim will send out his report as an example and get feedback.

      In the customized report tweak the acknowledgements section, so the person who wrote it should be first, and the “submitted to” area will change based on who is getting the document (library director, for example).

      Linda thinks it would be interesting to look at the differences of embedded libraries on the charles river campus (example look at branch libraries - Med library)

      Things we need to remember next survey:

      Ask about discipline (humanites, social science, sciene)
      Provide NA option for each question

      _______________________________________________________________________________________________________________________________________
      Minutes 3/24/11

      Present: Dan P, Alex, Kate, Linda (chair), David (minutes), Jim, Sarah, Dan

      Next Meeting: Tuesday, March 29th, 3pm.  Location: Mugar admin

      Action Items:
      Sarah: Find resources on minimum number of survey respondents needed for adequate privacy.
      Linda: Ask Bob about hiring Gordon Fretwell for consulting.
      David: Will do ENG report
      Jim: Will do CAS report
      Dan P: Look into open source analysis tools
      Alex and Dan B: Look into using NVivo.
      Dan P: Create an Excel spreadsheet with all comments.
      Linda: Ask Bob to buy SurveyMonkey Gold.
      Dan P: Create secure area of Assessment Wiki to store final versions of documents.
      Kate: Go on vacation.


      Discussion:

      Linda reported that Numbers is ordered for the iPads; Tom expects it RSN.  People gave back their iPads (labeled with their names) so that Numbers could be installed on them.

      Dan P uploaded survey data to DropBox in mySQL, not normalized.  Deciding on interface and how to make it available.

      David brings up problem of simultaneous users of SurveyMonkey applying competing filters -- there is no way to have duplicate copies of survey with data.  Kate mentioned you could quickly take screenshot with Zotero.

      Kate noticed differences between SurveyMonkey and JMP when filtering.  David suggested that it may be because of the questionable way SM does percents.

      Dan P showed snapshot of data in mySQL.  Showed report with SQL.  Can create interface so anyone can extract data.

      Kate again brought up discrepancies.  Will double check her figures, and will check with David if there is still a problem. [Kate has checked, and everything is now fine.]

      Problem with small number of responses for theology.  We decided there aren't enough people to do report.  Alex asked what about when subset (department) is too small?

      Sarah will look to find resources of minimum number needed for adequate privacy.

      Linda raised possibility of getting consulting from Gordon Fretwell..  Sarah suggests deciding what questions we would want to ask consultant.  Now may be good time to ask.

      Jim asked “What is vision for report?”

      Linda’s vision was to start with overall report, and rewrite it to reflect data and interests of school.

      David’s vision was to assume everyone would read overall report; additional reports would be supplements with additional information relevant to school.

      No decision was made on which vision to go with, or even if we need to stick to one vision.

      Jim asked if we should break down ENG by college?

      Alex asked if there is a problem comparing subset (e.g. ENG) to overall (which includes ENG)?  Decided there was no problem as long as comparison was adequately described.

      Jim and David swap ENG and CAS reports.

      According to Linda, Bob wants these reports before deans leave for summer.

      Discussion of how to analyze comments.

      The options suggested were:
      Betsy's Excel spreadsheet.  Linda isn't sure it is right tool.
      Atlas TI (sp?)  has a day or two learning curve, but is good.  Expensive. $1K a seat.
      NVivo, School of Ed has 7 copies, may only be student license. $650 a seat.

      David asked what the goal of analysis was.  Kate suggested it would provide more data for eventual publication, Dan P. that is would make searching comments easier, Sarah that there is information hidden in comments we need to analyze to find.  Linda says it allows
      quantitative analysis of comments, and from Betsy's article, benefits include: provide meat for liaisons, find low-cost action items. Sarah said comments can help improve next survey, drive actions.

      Dan P said WordPress supports tags, so it is possible to create analysis tool with php and SQL.  

      Dan P will look into open source analysis tools,  Alex and Dan B will look into NVivo.

      Dan P can create an Excel spreadsheet with all comments.

      Linda will ask Bob to buy SurveyMonkey Gold.

      Dan P brought up need for place for final documents (like survey report).  He will create secure part of Assessment Wiki.

      Kate's job is to go on vacation.
      ______________________________________________


      Minutes 03/16/2011

      Present:  Bob, Linda, Dan B, Dan P, Alex, Sarah, David, Kate

      Next Meeting:  Thursday, March 24, 1pm.  Location:  TBD

      Action Items:
      Dan B to make a link under "about us" to the faculty library survey report
      David will look into setting up a Dropbox account for the committee
      All will start thinking about creating our individual customized reports (see list below)
      Dan P will look at both Google charting software, and the possibility of using MySQL as a text analysis tool.

      Discussion:

      A. Feedback from Bob
      The Provost's comments on our report were very positive.  Our report went out to faculty by email under Bob's name a few days later.  The Provost is leaning toward having us survey graduate students next.

      B. Creating customized reports are now the focus.  The low number of respondents in some areas rules out several groups such as Theology, but we came up with the following to guide our initial work.

      1st group
      We will start with one customized report for MED.
      Medical Campus - Kate
      Law - Linda

      2nd group
      CFA - Alex;
      SED - Dan B;
      CAS - David;
      Sargent - Sarah;
      Management - Dan P;
      and by group decree, Engineering and everything else - Jim.

      C. How To Create Reports
      We started out thinking that we will use iPads for the "numbers" program that David used for charts, but decided several other tools should be considered as well.  If we will use "Numbers", we need to contact Jack to make sure that app is installed on our University iPads.  David will do some training on numbers at our next meeting. Dan P will look at Google charting software.

      D. Comment analysis / Mining comments
      Survey Monkey has a text analysis in a subscription level above SED's.  There's a tab for coding text comments.  In the future, we want our own survey tool anyway, not SED's subscription.  Various methods of mining comments were discussed.  Betsy from Northeastern created an Excel tool.  Dan O Mahoney from Brown created a web form that had a dual purpose - to analyze the comments and to get staff buy-in on the whole assessment project.  Linda P. may also follow-up with Gordon Fretwell.
      Dan P will look at the possibility of using MySQL as a text analysis tool.

      E. Other Assessment activities
      Sarah pointed out the need to create a bigger scale game plan.

      _______________________________________________


      Minutes 03/01/2011

      Present: 
      Bob Hudson, Linda (chair), David, Kate, Danny P(Minutes), Dan B, Jim, Sarah

      Next Meeting:  Monday, March 14, 3:15pm.  Location:  Mugar Conf. Room

      Action Items:

      Add assessment committee names to the executive summary (acknowledgments)

      David will go into the pdf and change some of the smaller bar charts to make values visible

      Think about what other assessment we can do

      Begin planning for customized reports


      Agenda/Decisions:

      Bob will meet with the provost on 3/8/11.  Once we have approval from the provost Linda will send out the report to the head of Law/Theo/Med.  Step 3 will involve Linda sending the report to the head of the branches and from their the Deans of the individual schools.

      We discussed adding color to the charts.  Bob leaned towards not adding color and this was the agreement of the group.

      Sarah brought up the fact that certain schools (SHA) don't have enough responses to do breakdown reports.  In this case we will most likely need to give the overall summary.

      At some point in the near future (after he meets with the provost), Bob has asked the group to prepare a PowerPoint presentation of the survey results.  He mentioned some key things to keep in mind for the PowerPoints, namely not to make it too dense, and to show some self-awareness and humor.

      Bob encouraged the group to add their names to the executive summary.

      Linda has asked the group to start thinking about what other assessments we can do and to start to consider how we will do breakdowns for the schools.



      Minutes 2/16/11

      Present: Linda P (chair), David, Kate, Danny P, Dan B, Alex, Jim, Sarah
      S (minutes)

      Next meeting: Thursday 3/3/11 1:30-3pm LOCATION TBD (Mugar conf rm is
      booked. Dan or Danny, is there room at Pardee or PERL?)

      Agenda:

      1) Review draft of full report including executive summary
      -review comments
      -address the importance/satisfaction issue
      -additional changes?

      2) Next steps with survey

      3) Balanced Scorecard


      Action items:

      KB makes changes as note from review of draft
      SS and LP wordsmith executive summary
      DF adds "Figure no.X" text to front of text below each chart

      Discussion:

      REVIEW DRAFT
      DB mentioned and SS seconded interest in seeing the report in color. DF
      said he can do it- in March. Default color scheme from the charts ap
      he's using isn't bad. We agreed we will not hold up presenting report
      based on availability of color charts, but it would be a nice added
      bonus. Also agreed that we will present report in paper and PDF formats
      to avoid formatting issues from Microsoft Word version conversions.

      As a group, we walked through the suggested changes and comments from
      Kate, Linda and Jim. KB will make the changes in the current draft and
      resend them. Needs to get them to DF by Friday so he can make changes
      to charts (adding figure no.s) before he leaves for beautiful sunny
      California.

      Sarah and Linda will wordsmith the executive summary, paying particular
      attention to adopting changes we made in the full report.

      NEXT STEPS
      After Bob Hudson meets with the provost to present the report...

      -timeframe for customized reports? (we did not set one, but JS mentioned it)
      -analyze and tag comments
      -how many customized reports will we need? At least 3, for Law, Med & Theo; probabaly also one for Pardee. DB thinks he has what he needs from Survey Monkey for PERL. Didn't decide/know about SEL, Music, ASL. Either way, Law, Med and Theo would be next in line for custom reports.

      We discussed the process for creating these customized reports. SS expressed concern that Assessment group has the best handle on the limitations of the data based on how we asked the questions, and was concerned about possible unintentional misrepresentation of the data by those less familiar with the entire survey process. LP suggested that we create drafts, present directors/heads of other libraries/ departments with some possible charts and comments from which to choose, and then work together on creating a finished product to their satisfaction, but that we craft initial drafts. This plan seemed to be generally agreeable to everyone.

      BALANCED SCORECARD
      Linda reported on the 2/14/11 ARL Balanced Scorecard Webcast. SS also mentioned a Balanced Scorecard postconference workshop that she attended as part of the 2010 Assessment Conference. Both LP and SS are in favor of BU Libraries using this tool. JS and others want to understand it better. LP suggested looking at McMaster's documentation of the process.

      McMaster:

      ARL "Research Library Issues" on Balanced Scorecard
      http://www.arl.org/bm~doc/rli271-scorecard.pdf

      PPT Slides from the workshop SS attended:
      http://libraryassessment.org/bm~doc/lewis_vivian.pdf


      _____________________________________________________________________________________________________

      Minutes 2/9/11

      Present: Jim, Linda, Sarah, Danny, Dan B., Alex (Minutes), David and Ella!

      Next Meeting: 2/16/11, 1:15pm-2:45pm, Mugar Conference Room

      Action Items:
      1. EVERYONE will: Re-Read Report esp. the last three sections
      2. David will: Update graphs; Insert graphs and quotes into report and executive summary using MS Word; Resolve percentages between report/executive summary and graphs data (Survey Monkey numbers vs. JMP numbers); 
      3. David and Danny will check SMG numbers
      4. Sarah will: Read through the report for flow and grammar; Check APA style guidelines for partial quotes and ellipses; Send edits to Linda 
      5. Alex will: Double check selected comments for accuracy; Forward the comments selection to the group (primarily David); investigate text formatting in publisher in case Word proves unwieldy 

      Discussion/Decisions
      1. We talked about Linda's proposed changes to the charts. We decided to keep the medical campus chart and go ahead with the other changes.
      2. We discussed needing to clarify what our percentages are based on. (Did we clarify? My notes are a little fuzzy on this.)
      3. We discussed the usage of "faculty" in the report/executive summary. We decided as long as we are consistent the current usage is fine.
      4. We decided not to include the 5 colleges charts in the provost's report, but that they will be very useful in the customized reports.   
      5. We discussed the comments selections and where they could fit into the report/ executive summary.
        
      ________________________________________________________________________________________________________________________
      Minutes 01/26/2011

      Present:
      Linda, Sarah, Kate, Alex, David, Danny, Dan B, Jim (minutes)

      Next meeting: February 2, 1pm, Mugar Conference Room

      Agenda/Decisions:
      1. We went through David's charts as a group and made decisions about layout and verbiage. 
      2. We gathered volunteers to handle various aspects of the process of creating a final report in time for Bob Hudson's meeting with the Provost.
      3. We agreed to look at the way we ask about proposed services in future surveys. We think we should either ask respondents to prioritize (which is most important to them) or choose one or two of the suggestions.
      Action items:
      1. Sarah will look through the Full Report and make suggestions.
      2. Linda will review the Executive Summaries created and formulate a final draft.
      3. Jim will create a Google doc version of his list of comments and Alex will review them and make selections for a group vote at the next meeting.
      4. Danny will check the numbers David used in his charts.
      5. Kate will check Linda's numbers in the Full Report.
      6. We will look at the scales for information literacy and the charts. Do they ask what we want to know?
      7. David will make some changes in his charts from having the categories on the bottom to having them on the left side, resulting in horizontal bars rather than vertical on the graphs.
      ________________________________________________________________________________________________________________________

      Minutes 01/11/2011

      Present: Linda, Jim, David, Alex, Kate, Dan B (minutes)

      Next meeting: Jan 26th, 1pm, Mugar conf. rm

      Announcements: Linda said we may have a bit more time than expected to finish our report to the provost, as she is not yet accepting meetings.

      Decisions
      :
      The Executive Summary needs to tell more of a "story"
      We will list *summary percentages* for each question as an appendix to the full report, at least for Bob

      Action Items:
      Kate will investigate presenting some data either in tables, charts, or bullet points.
      Jim will look into the comments to gather relevant material.
      Alex will check the total tally of comments we received.
      David will produce charts Linda has laid out in the draft of the full report.
      Dan B will look for other interesting charts to possibly include.
      Kate will check validity of two statements; one about Med's digital policy on books and the other about Open Access- Med vs. CRC.
      All will continue to scrutinize the two drafts we now have, our charts as they get created (see a-d below), and also any other charts or comments that may be worth reporting.

      Discussion:
      Executive Summary and Full Report of the 2010 Faculty Survey -

      Linda brought drafts of these to the meeting.  The Executive Summary should present what we want the Provost to know, but some on the committee thought the it was a bit too long and perhaps needs to tell more of a "story".  Kate will investigate presenting some data either in tables, charts, or bullet points - although Linda stated Executive Summaries usually do not contain those. 

      In order to help tell a story Dan suggested possibly singling out one or two telling comments in the Executive Summary.  Jim will look into the comments to gather relevant material for the Summary as well as the full report, and also look into if revealing a bit of demographic info about any particular commenter, without breaking privacy, would help tell the story.  Alex will check the total tally of comments we received.

      David posited that we should feel free to draw conclusions, especially in the Summary, *about* the faculty, rather than just communicate facts as revealed in the survey.  In other words, we can draw out those conclusions, and then mention the survey results in the full report that back up those statements.  David also suggests listing *summary percentages* for each question (not the full data itself) as an appendix to the full report, and we agreed to do that, at least for our submission to Bob, who may then choose to withdraw it.

      Linda has laid out the needs for most graphics in the full report, and David will produce those charts.  Dan B will look for other interesting charts to possibly include, and a number of other charts will need close inspection after they are created, such as:
      a) attitudes toward ebooks in SEL vs Med (David's suggestion)
      b) something about finding aids, such as: is the importance of the catalog in the humanities as opposed to the same in the sciences (or something like this) interesting? (Dan's suggestion)
      c) can we somehow highlight schools vs something?  Either standing by themselves or some combination thereof? (Linda's suggestion)
      d) ILL on the Med campus vs the Charles River Campus (Kate's suggestion)
      Further, two charts that Linda mentioned in the draft of the full report will also need to be checked to make sure their statements are valid, and Kate will check these:
      1) Regarding ebooks, that the Medical Library "has already adopted a digital strategy for books"; this is in the Executive Summary as well as the Full Report; and
      2) regarding Open Access, that "More respondents on the Medical Campus as compared to those on the Charles River Campus have submitted to a journal allowing Open Access...."; this is only in the Full Report.

      Next Steps -
      Linda mentioned a possible consultant for statistical expertise named Gordon Fretwell, but she remains open to the idea of finding someone on campus to fulfill that role.  After finishing these reports, we will work to create ones for individual schools.



      ________________________________________________________________________________________________________________________

      Minutes 12/20/2010

      Present: Linda, Sarah, Jim, David, Alex, Dan B, Dan P, Kate (minutes)

      Next meeting: Jan 11th, 2pm, Mugar? Note that Jan 4th meeting cancelled

      Text highlighted in green note survey results flagged for possible inclusion in report to provost

      Updates:
      David sent out survey data in a JMP file (FacultySurvey.jmp), shared on individual google doc accounts.
      Report for the provost needs to be ready by late January

      Decisions:
      Group will furnish reports for different schools and will handle requests for data when asked.
      The next student survey will go out in Spring 2012


      Action Items:
      Linda will put up rough draft of report/story Jan 5th or 6th
      David will send out spreadsheet of number of respondents vs. # of sent email invitations
      Sarah will let us know about the free graphics program
      David and Dan P. will send out invitations to DropBox (free program for storing, syncing and sharing files)
      Everyone will do analyses in JMP and consider interesting points to pull out for report.
      Email the assess-l list with interesting survey results, Linda will compile list


      Discussion:
      JMP
      David reviewed JMP capabilities. Note that JMP is good for analysis but for our graphics it may be best to use another program like Excel or the free one Susan uses (JMP isn’t very pretty, and the graphics aren’t displayed on the same scale across compared distributions).  Because our data is ordinal/nominal, our safest form of analysis is distributions.
      To get to the distribution tool in JMP, go to Analyze -> Distributions.
      If you want to look at, for example, Ebook,ejournal, etc use by departments, choose Ebook, ejournal questions and put it on the Y columns, then select “By” and choose the second factor (for example campus, department)
      David notes that there is an interesting difference between print book use between MED and science (science likes print books more). This may affect our Sci/SocSci/Humanities breakdown.
      In our analyses on JMP, we should look for other interesting things to flag
      You can save your analyses to a location outside of JMP to share Also, the labels in JMP are shortened to fit into one row. If you need to see the full question, look in SurveyMonkey.

      Report
      David gave a brief outline of notable results we should include in our report to the provost.
      • Importance of the library to the faculty - although only 11% visit the physical library, 65% visit online library resources weekly or often. Sarah pointed out that if we talk about low foot traffic, we should make it clear that we expect different responses from the students.

      Linda said we should look at the stats for MGMT - might have higher rating for foot traffic         and could speak to facility differences.

      • New services: want more ebooks
      • ILL - service people are most interested in
      • Info Lit - importance to faculty (esp for undergrad), less satisfied with performance
      Linda pointed out that we should make a list of areas we DON’T have to include
      • Open access - brief Bob about this in case there the provost has questions, but don’t need to highlight
      • Pie chart of schools - even if there are areas of concern about under representation, this could just be included in a footnote.
      Sarah asked that we decide what story we want to tell in the report. Suggests we frame the report in terms of services vs. resources. Perhaps bundle stats or rates of use (like, x amount of faculty use ILL, next to stats for use of resources. Give representative numbers for each.
      The outline of the report can be divided into 3 parts:
      1. What we are doing right - numbers and quotes
      2. Areas to work on (could use comments about people using Harvard/other area libraries here)
      3. New directions the library is taking and support from the survey (ex. proposed services)
      In reference to a question Jim asked, at this point comment analysis will be in the form of pulling out illustrative quotes to go along with the highlighted survey findings we decide to go into the report.

      Discussion about the structure of the report. David thinks it should be organized by themes. Dan B suggested we pull out schools mentioned in the Strategic Plan (MGMT, Med, Law, CFA). David can do breakdowns. After discussion about importance of other schools, we decided that we should make sure to include examples that focus on the strategic plan schools, but not to put too much focus on just these four.

      David pointed out that in the survey, those that received external funding rated the library as more important to them.

      Other items to explore for the report
      • Web searching - breakdown by dept (ex heavy use in COM understandable). May point to ways we need to deliver and communicate library services to faculty. Also juxtapose database searching (still high)
      • Physical spaces - isolate libraries for comparison (ex SEL vs MGMT). Pull out comments about space.
      Next Survey
      Linda wondered whether we should have had survey takers select there type of discipline (sci, soc sci, humanities) instead of (David) doing all the work to code the departments after the fact. Some disagreement as to whether this is a good idea or not. Keep in mind for next survey.

      Discussion about timing of next survey  - agreed spring is best time to survey students. Will wait until 2012 to do it. Cycle may go faster with subsequent surveys.

      Data Analysis
      Sarah recommended getting some outside statistical expertise to back up our findings. David points out that JMP won’t let us take means, but we do need to investigate the numbers behind the analyses it provides. Linda suggested putting out an ad for a work study grad student in statistics, or asking a faculty contact to recommend a student. Social science would be the best area.

      As far as other schools/depts, decided we would analyze for them - tell them the variables we looked at, and we can do the data analysis for them. Perhaps do a report for each school.
      _____________________________________________________________________________________
      Minutes 12/14/10
      Present: Linda, David, Kate, Alex (Minutes)

      Next Meeting: Monday 12/20 2pm @ Mugar Conference Room

      Updates: Bob Hudson would like Mary Blanchard to hold off on reporting the Med. Campus data to the Med. Provost till after Bob's meeting with the new CRC provost.

      Action Items:
       -David will send out cleaned up/complete JMP file with the data.
       -EVERYONE to play with data in JMP or Survey Monkey.
       -EVERYONE to brainstorm the type of analysis we'd like for the Provost's report. What kind of a story do we want to tell about the library? Look            at data in the context of the University's Strategic Plan. 
       -Kate and Sarah will reconcile the MED/Dental/Public Health subject breakdowns (Science, Social Science, Humanities)

      Decisions: 
       -Keep a running list of changes and additions to Faculty Survey for future iterations
       -Take note of misleading/ambiguous comments in the survey.

      Pending Decisions:
       -Next user survey: Who? When? Estimated time to develop? Should we be spending our time on this now? 
       -Other projects besides user survey? 
       -Will the different schools/branches analyze their own data from the faculty survey or will the Assessment Committee do the analysis.  

      Discussion:
      -Linda mentioned that we should drop the word "representative" from our comments selections because representative comments are ascertained through a form of analysis that we're not doing. Instead we will use "selected" or "illustrative" comments. 
      -David mentioned using the comments as a way to illustrate the quantitative data for the provost's report. 
      -JMP overview: JMP good for analysis, not visualization. We can use excel for nice graphs/charts. There are slight formatting issues when data is downloaded into JMP. David is working on cleaning up the data. David will send clean workable file to committee. 
      -Excel overview: We can use pivot tables for graphing/charting. Lots of options for visualization. Colors can be customized and saved. Graphs can be saved to any format: JPEG, TIFF etc. 

      ____________________________________________________________________________________
      Minutes 12.7.10
      Present: Linda (chair), Alex, Sarah, Dan B, Dan P, Jim, Kate, David

      Next Meetings:
      Tue 12/4, 1pm; Mon 12/20, 2pm; Tue 1/4 2pm; Tue 1/11, 2pm

      The Mugar Conference room is reserved for all these dates

      Action Items:
      1. JS and SS - break down departments to discipline groupings
      2. LP, AS, and KB - look at comments, pull out examples to illustrate initial report
      3. KB/LP - determine if Marlene should present a report to Med Campus provost to accompany Bob's mtg w/ CRC provost
      4. Everyone will try to do some of the JMP online training:https://www.bu.edu/software/stats/jmp/
      5. DF -keep working on data in JMP
      Discussion
      • We will write a report for the new provost, to be delivered (by Bob, who will need additional supporting information) in January. May ask Mary Blanchard to present this report to Med provost - Dr.  Antman - simultaneously.
      • None of the individual schools or departments will be given reports or data until after this initial report is presented.
      • This is only a brief report of PRELIMINARY results
      • DF has put the data into JMP. Anyone of us, as co-investigators (remember to protect the data and our surveyee's privacy!), may download the data from Survey Monkey.
      • We all looked at the preliminary report DF put together in Survey Monkey
        • mean was used at times when it should not have been; scales not always displaying correctly to compare 2 charts; in general Survey Monkey completely inadequate for what we need to do with the data
        • for demographics by campus/status, note % of survey takers who responded?
        • for teaching clinical/grad/undergrad draw out a separate chart showing grant funding by teaching status?
        • primary college/dept affiliation: groups smaller depts together? group by discipline? highlight 4 Programs mentioned by name in university planning docs.
        • many of the importance/satisfaction paired questions will benefit from being placed on the same graph or chart. Survey Monkey cannot do this, DF will work on making it work in JMP
        • we all want to start looking at cross tabulation by campus, whom they are teaching, and discipline
        • brief conversation about the "not sure" answers - to separate them out or group them with negative responses (will vary by question)
        • know we want filters (by cross tabs and other groupings, such as discipline, which we create), but may not know which ones we want to use until we have applied them and seen what they look like
        • may want to get advice or assistance from someone with a more robust statistics background than anyone in our group has
      _____________________________________________________________________________________________
      Minutes 08.30.10

      Present: Linda (chair), Alex, Sara, Dan B., James, Kate, David (minutes), Dan P.

      Next Meeting: TBD

       

      Action Items:


      1. Need to get an e-mail list for the survey (Linda)
      2. Make changes in survey based on discussion (Alex)
      3. Check out and respond to Alex's edits (everyone)
      4. Add "you can stop anytime" wording to survey intro (Linda)
      5. Submit completed application to IRB (Linda)
      6. Create html files of survey for IRB (David) [After meeting it was discovered we don't need this.]


      Discussion:

      • Linda is still trying to get a contact to talk to about using Survey Monkey for analysis.
      • People turned in certificates and conflict of interest forms -- 100% participation!
      • Linda reported we need to have all written communication finalized to submit with the IRB request.
      • Linda reports that in his meeting with Bob, the Provost was extremely supportive.
      • Considered Mary Blanchard's request for changes to wording of a few questions.  Everyone agreed to change.
      • Sara and Linda made minor change to scale names in question about contribution to work.
      • "HeinOnline" was added as example in question 27.
      • Agreed to combine CAS and GRS. 
      • Discussion of high-level list of schools.  Kate will work with Alex on how to handle Graduate School of Medicine.  Will write out names of schools, not using TLA.
      • David raised concern that we may not want to be as fine-grained as department.  After some discussion, we decided to leave it as it is, but may reconsider for future surveys.
      • Dan B.: mentioned we are probably missing a few people in our list of departments.  But we do have the "other" choice, so that's okay.
      • IRB is going in tomorrow.  Turn around in week?  Contingent upon e-mail list.
      • Can we show survey to interested staff?  Yes, but don't give copy.
      • David gave presentation on what Survey Monkey can and can't do: we will be able to use it for some analysis, but there are some things we will want to do that will require a more powerful tool.  Link to his presentation: https://docs.google.com/present/edit?id=0ARIN2pT8txcGZGNuanJncnRfMjQ4Zms5ZjhwY2o&hl=en&authkey=CNeHrIIC
      • Linda showed IRB exemption doc.  We read through it and suggested changes.

      ________________________________________________________________________________________________________________


      Minutes 08.04.10

      Present: Linda (phone), Danny P., Alex, Dan B., David, Jim, Kate (minutes)
      Next meeting: Monday, August 30th, Mugar Admin Conference Room


      Action Items:
      1. Create list of departments for school and update the departmental list page (everyone - see assignments below)
      2. Linda will send out links to the COI form and training we need to complete.
      3. Sign conflict of interest form and complete online human research training, send COI form and certificate from training to Linda (everyone)
      4. David will show us the options and limitations of the analysis tools SurveyMonkey
      5. Linda will contact Pamela Bagely at Dartmouth regarding SurveyMonkey analysis
      6. Alex will make a copy of the faculty survey and experiment with Skip Logic.
      7. Jim and Kate will email Linda with suggested changes for follow-up letter to faculty pre-testers
      8. Send list of faculty pre-test contacts/email addresses to Alex (everyone); Alex send list to Linda
      9. Danny P will talk about website statistics for the next meeting (Kate for medical campus).

      Discussion:
      A. Discussion of Departments
      List of departments will have to come from contact with each school. There is no list for the complete university; the registrar’s office reports that there is no exhaustive list that includes all the departments for each school.
      There is a question of whether we really need to have a break down of all the departments, but  Linda points out that a list would be helpful for Tom Casserly’s replacement, as he or she will develop a liaison program.
      As for SurveyMonkey, Linda looked into Skip Logic and reports that we will have to have the department options on a separate page (since the list of options will vary based on which school a user selects). Alex will make a copy of the survey and experiment with SkipLogic.
      We will work collectively to make a list of departments, and have split up the schools in the following manner:
      Alex: SED/CGS/MET/SUM
      Dan: CAS (includes GRS), EOP
      Jim: STH/CFA/COM
      Kate: Everything on MED
      Danny: S&G
      David: ENG
      Sarah: SAR/SHA/SSW
      Send your list to Kate and she’ll compile a master list for Alex.  

      B)Timing of Survey
      Mary Blanchard likes the Nov 1st date for the med campus, but any later that week and the survey would conflict with a AAMC conference most med faculty members will be attending.
      Linda says that Bob Hudson will talk to the provost about the timing in the year, and will discuss any other surveys going out (to possibly improve the timing of ours). Will also show provost the links to the preliminary results of the UW study to give them an idea of ours.

      C)IRB Exemption
      Linda talked with Mary Banks (head of IRB) and they agreed we should formally apply for an exemption. We will need this if we want to publish a paper using the data we collect. We will try for a B2 exemption. Linda is going to be the PI. Everyone approved of Linda being the PI (for real). Everyone else on the committee will be co-investigators. That means everyone has to fill out a conflict of interest (COI) form and complete an online training regarding human research subjects. Linda will send out links to these forms and we will send the signed COI form and training certificate back to her.
      We have Bob’s approval to go ahead. Linda pointed out that IRB has to see all of our recruitment material we use (this includes any emails, and possible brochures, flyers, and website). We can’t do any ad hoc recruiting. We can’t change the survey, text of emails. We have to go over EVERYTHING. If we have to change something we’ll need to reapply. Linda thinks the fastest we can get an exemption is in 2-3 weeks.

      D)Follow-up Email
      Jim and Kate made small suggestions, will email Linda with changes.
      Question as to whether we have a master list of faculty pre-testers - we don’t because everyone sent their emails separately to the contacts. Everyone should send their list of contacts/email addresses to Alex and she will sent to Linda.

      E) SurveyMonkey analysis
      Linda has decided to contact Pamela Bageley at Dartmouth about analysis in SurveyMonkey because she has used the program extensively. Linda thinks SM has good enough analysis tools for our first survey, but probably won’t be good enough for the types of analysis we want to do in the future. David thinks that SM’s tools aren’t even good enough for a lot of basic analysis (like for combining departments into larger sections like humanities - difficult to sort users and responses by variables we don’t explicitly ask about). We will have to export the data to another tool. Linda will ask Pamela about other tools to use.

      We discussed the analysis tools further and decided we need more guidance on how to use them. In the next meeting David will explain the tools and show what we can and can’t do with SM, using the UW preliminary results as a guide to the sort of analysis we’d like to do.

      Jim brought up a discrepancy in the pdf version of the UW survey results (9a + 9b), but the discrepancy does not exist in the html version, which accounts for 80% of responses.

      Linda also mentioned that in the next meeting we should talk about website statistics and how some pages use Google Analytics data and others use NIS numbers generated by IS&T. Danny will talk about this in the next meeting; Kate will talk about how the medical library website stats are collected.
      ________________________________________________________________________________________________________________

      Minutes 07.14.10

      Present: Linda by phone (Chair), Dan B. (minutes), Sarah, Kate, David, Alex, Jim
      Next meeting: Wednesday August 4th, Mugar Admin Conference Room

      Announcements:
      1) EOP is "English Orientation Program", which we do not need, and UNI is also a category to delete.
      2) Steve has put up preliminary results from their 2010 Faculty Survey (http://www.lib.washington.edu/assessment/surveys/survey2010/default.html)
      3) Danny has put up the latest (7/1/09-6/30/10) Google Analytics on the Welcome page.

      Action Items:

      1) Kate will contact the Registrar (possibly both campuses) to request a list of academic departments. Make shift list available here
      2) Kate will likewise check with Mary about the timing of the Fall survey.
      3) Linda will check with Bob about coordinating the timing of the Fall survey with the Provost.
      4) Linda will draft a follow-up email to pretest participants.
      5) Linda will contact a colleague she believes is at Haverford College about whether to use Survey Monkey alone as our analysis tool.
      6) Sarah will look at the comments about the introduction to the pretest.
      7) Sarah will work on a draft message to send out to the Fall survey participants.
      8) Everyone should try a bit of analysis with the pretest results for our next meeting.
      9) Everyone should take a look at UW's 2010 preliminary results at http://www.lib.washington.edu/assessment/surveys/survey2010/default.html

      Decisions:
      1) We will administer the actual survey in November if possible.  We planned November 1st as the launch, November 8th for the first reminder email, November 15th for the second reminder email, and November 30th as the deadline.

      Discussion:
      A) We went through the entire survey question by question, approving most for the actual Faculty Survey, and changing others on the fly. 
      Question 3 - Based on the pretest, we will have trouble rationalizing what we receive as text, so we should consider changing this to a drop-down or other menu.  Kate will contact the Registrar (possibly both campuses) to request a list.
      Question 5 - Changed wording to "during the past academic year at BU, have you...."
      Question 7 - Expanded "Journals (electronic or print)" to two items, "Print Journals" and "E-Journals"
      Question 8 - We decided to leave this the way it is after a lengthy discussion of alternatives to "library pages".
      Questions 13-16 - Approved each of these questions, and also decided to treat non-answers as "not applicable" answers.
      Questions 18-19 - Changed final item to "Subject librarian(s) in your academic area".
      We also added the new question as number 22.  The wording from UW did not meet our needs, so we changed the wording.  After much discussion, "What contribution does the BU Libraries make to your..." changed to "What contributions do the BU Libraries make to your..."; and the first item was changed to now read "Ability to keep current in your field".  The final page of the survey (page 6) was changed to reflect the addition of that question, so that page is now titled "Overall Contribution and Satisfaction".

      B) Other considerations:
      Sarah will look at the comments about the introduction to the pretest.
      We decided to administer the actual survey in November if possible.  We planned November 1st as the launch, November 8th for the first reminder email, November 15th for the second reminder email, and November 30th as the deadline.  Linda will check with Bob about coordinating this timing with the Provost, to ensure we are not competing with other surveys that may be taking place.  Kate will likewise check with Mary.
      Linda will contact a colleague she believes is at Haverford College about whether to use Survey Monkey alone as our analysis tool.
      Linda will draft a follow-up email to pretest participants, thanking them yet making it clear they should also fill out the actual survey in the Fall.  Alex reports details regarding this email can be found in our minutes of 6-9.
      Sarah will also work on a draft message to send out to the Fall survey participants.
      Everyone should try a bit of analysis with the pretest results for our next meeting.
      Everyone should take a look at UW's 2010 preliminary results at http://www.lib.washington.edu/assessment/surveys/survey2010/default.html



      __________________________________________________________


      Minutes
      6/22/10

      Present: Alex, Kate, Danny, Sarah, Jim (minutes)
      Next Meeting: 7/14 2:30-4:00 OR 7/15 2:00-4:00  Alex will send out a Doodle.

      Action items:
      1. Alex will make changes to the survey after we've studied current responses on pre-test.
      2. Danny will look into the schools of EOP and UNI to see if we can eliminate them as categories.
      3. Alex will add the UW definition of datasets to the survey to assist people with understanding what they are.
      4. Sarah will work on the introductory paragraph to be used on the survey and the e-mail request for folks to take it.
      5. The committee will make suggestions on re-wording of questions 13-16 to allow for respondents to skip irrelevant questions or choose a simple one-time (N/A) response.
      Discussion:
       The group discussed the wide variety of responses to the department question and wondered if there was a need for a more standardized list. We were not sure a drop-down menu would work but we were concerned about ambiguity of the pre-test responses.

      We decided to keep "other" in question #8 but drop listservs. We also decided to leave questions 10-11 alone as (N/A) seems an adequate response to a set of publishing questions.

      For question #20, we decided to reduce the list to 4 services"
      • Online tutorials
      • Integration of library services
      • E-books
      • Open Access assistance.

      Agenda for next meeting:

      1. Discuss wording of questions 13-16.
      2. Discuss question #8. Is leaving "other" okay?
      3. Review re-wording of survey.



      ______________________________________________________________________________________




      Minutes 6/9/2010

      Present: Linda (Chair), Dan B., Sarah, Kate, David, Danny, Alex (minutes)
      Next meeting: Tuesday June 22nd, Pardee

      Announcements: UW changed a few of their question based on our survey (we think). Woohoo!

      Action Items:
      1)Add "Answer only if relevant to you" (exact phrasing?) to Questions #10, #11 and #13 - #16 - Alex
      2)Change "Subject/Branch librarian" in Question #18 to "Subject librarian(s) for your academic area" - Alex
      3)Question #8: Delete "Online Groups and list serves" and "Other" from list of options - Alex
      4)Question #22: Add "Libraries physical spaces/facilities" as an option -Alex 
      5)Re-write intro paragraph to survey and draft email to faculty from provost (or whoever sends out Fall survey) - Sarah
              Include the following in intro: reasoning behind survey; survey will be publicly available; emphasize confidentiality;
      6)Draft follow-up email thanking pre-test participants - Linda
              Include the following: Sincere thanks; importance of their comments; changes in survey based on comments; reminder to please
              re-take in the fall     
      7)Contact Steve Hiller about UW 2010 Question #7: how do they interpret the results, what do they use the data for? - Linda  

      Discussion:
      With great valiance and tenacity we discussed all of the Pretest comments in one sitting. For the most part the pretest comments were constructive and resulted in positive changes such as action items #s 1 and 2.
        
      Sarah suggested we start tracking problematic questions for revision in later iterations of the survey

      The Fall survey should go out to an all faculty list-serve from the provosts of both the Charles River Campus and the Medical Campus 

      Linda proposed deleting Question #8 from the BU survey and adding question #7 from the UW 2010 survey. There is some hesitation about adding any more untested question after the pretest. A final decision was not reached. Linda will contact Steve Hiller for clarification and talk to Bob Hudson about the necessity of this type of question. After some discussion we decided to drop several options from BU's Question #8 (action item #3) and to add an option to BU's Question #22 (action item #4).

      Agenda for next meeting:
      -Steves response on UW 2010 Q#7
      -Bob's response to UW 2010 Q#7
      -Datasets?
      -Proposed new services question
      -Survey Responses (not just pretest)
      -nuts and bolts of getting the survey out (contacting the provost etc.)
      -other projects for Assessment Committee to handle

      ________________________________________________________________________________________________________________
      Minutes 5/11/10

      Present: Linda (chair), Dan B., Jim, Kate, Sarah, David, Danny (minutes), Alex!
      Next Meeting:  June 9 - 2pm - Pardee Library

      Decisions:
      The pretest will be sent out on 5/17
      A reminder will be sent out on 5/24
      The end date for the pretest will be 6/4
      The title of the survey will be:  "2010 Faculty Library Survey"
      We will ask the pretest survey takes how long the test took them.
      The pretest will be sent out individually to the faculty members by the person who selected them.

      Action Items:

      Sarah's Questions:
      There is a "#1" before the section title, it should be removed.
      Sarah S. will write an intro paragraph and Alex will put it up on the wiki.
      Alex will send out an URL (containing the survey) by the end of the week for the assessment group to test.
      Sarah will email everyone two texts, the original message to be sent out to the pretest group, and a reminder message to be sent out on 5/24
      Alex will add a box for faculty members to voluntarily place their email address when they take the pretest.

      Alex's redone Survey (
      2.0.9376532):
      #6 -> Switched Scale, No N/A needed
      #7 -> add N/A
      #8 -> Same scale as #7, Add N/A
      #10 -> add N/A
      #12-15 -> Remove the question "familiarity with research materials", combine two of the questions to produce "providing correct citations and using information in an ethical manner.
      #17&18 -> will have the option "don't use" instead of N/A
      #21 -> Typo: oline should be online.  Add N/A

      Kate's Questions:
      Add a comment box after Q11 on the 2.0 survey. - "Comments on publishing or open access?"
      Remove the bolding and all caps from questions 12-15, 17-18.





      Minutes 5/6/10

      Present: Linda (chair), Dan B., Jim, Kate, Sarah, David (minutes)

      Next meeting scheduled for 5/11 at 2pm-4pm at Pardee(?)

      Announcments:
      News about poor Alex; glad to hear she is recovering
      Sarah is taking LYRASIS class on Basic Surveys for Librarians, Dan will also take class

      Decisions:
      We will push to get survey ready for pilot ASAP, so we don't lose volunteers to summer

      Action Items:
      Everyone will find additional faculty volunteers for pilot, and email all their volunteer names and email addresses to David
      David will collect volunteer names and addresses and put them someplace, but not post them to public wiki.
      Linda will contact Holly, Beth, Marlene, to see if they can suggest faculty volunteers for pilot.
      Dan will work with Arlyne to line up SMG faculty for pilot (Linda asked Dan Wednesday after meeting)
      Kate will make all the comment boxes in survey a consistent size.
      Kate will change scale on question 8 to importance rather than frequency.
      Someone (Kate?) needs to remove "Merged Survey to Reflect UW 2010" title from survey.

      Discussion:

      Went over agenda.

      So far we have about 20 possible faculty for pilot (not all have been contacted).  Sarah brought up problem that we know these people, they probably have pro-library biases.  Linda replied (quoting Steve): this is convenience sampling.

      Discussion of who (besides Alex) can edit Survey Monkey.  Current survey is 2.0; anyone can play with previous one; Kate will play with it.

      Began to go through staff feedback, which Kate did such a great job of organizing in spreadsheet.  Made changes in survey as we went along, so they are not listed here.

      Didn't get through all staff recommendations (we started with the easy stuff); will continue at next meeting.



      Minutes 4/13/10

      Present: Linda, David, Jim, Alex, Sarah S., Dan b., Kate (minutes)

      Next meeting scheduled for 5/5/10 at 2:30pm at Mugar (most likely)

      Announcement:
      Linda announced that we will not have to go through the IRB process; though this might be a problem for freshman who are >=18
      Sarah mentioned the ACRL News has a whole issue dealing with IRB --http://crln.acrl.org/content/71/4/190.full

      Decisions:
      Staff comments will go out the 21st. Staff will have a week to comment
      The Assessment committee will convene (5/5) to make changes due to the comments
      The Faculty pilot will go out after graduation

      Action Items:
      Alex will edit the survey on digilib for staff viewing
      Linda will write email up going out to staff about getting feedback for entire survey
      Linda will send out survey to Mary, Marlene and Jim, asking to send it out to staff
      Sarah will talk to Rhoda (Mugar) and Terri (Law), and Kate to Joe&Konstantin regarding ILL and this question
      Kate will collect comments and summarize them for our 5/5 meeting
      Each of us will have list of ~3 faculty names/contact info for pilot survey
      Sarah will tweak UW letter to faculty re: pilot survey for use at BU
      Sarah will look at pretest/pilot wording from UW if available, for use at BU


      Discussion:
      Proposed service questions:
      Discussion of the proposed new services questions got them down to 9 (see the list here). Most of the questions that were eliminated were done so because we were either already offering the service (in-class instruction) and it's covered in other survey questions, or the service is already being planned (federated searching/metalib).
      The list will further be edited after comments from the staff. The number of new services on the final survey will be 5.




      Intercampus transportation service -
      Problems about copyright if we offer article service between campuses. Do those copies made count towards copyright limit?
      Chrls river already doing this for free on ILL/charge when through medical library
      Changed wording of question to Dan's suggestion of "Expedited delivery..." for both article and book question to deal with differences between medical and charles river campuses
      Kate will talk to Joe, and Sarah to Rhoda and Terri regarding ILL and this question
      As for the question about delivering ILL via email, some discussion over whether to ask this considering we will already be doing this. Decided to leave it in to see response.

      Curriculum-
      Discussion over whether we are promising more than we can offer with tutorials. Some have software for it, but it's just a matter of time available. Left in to see interest.

      Blackboard/CMS question -
      Changed to reflect a wider range of services we can offer for BB (instead of just helping to embed links)

      Open access-
      Dan thinks it would be better to cover this with other means of assessment (focus group, for example). Doesn't think it will be one of the top 5 most important questions. 2 initial questions merged into 1, will leave to gauge any interest, and also make sure the survey includes OA question
       
      Linda will write email up going out to staff about getting feedback for entire survey
       
      Holes in survey
      Dealing with the survey in surveymonkey. Will put in the new services questions.
      OA question (Have you ever submitted an article to OA journal) to after question about where to publish

      One comment about the survey going from high to low (5->1) - commenter thought it should go from low to high. But gets awkward with "Don't use"/N/A button
      Will leave range the way it is.
      Have to make it clear about 5 being high b/c of digilib formatting (won't be a problem on survey monkey)
       
      Alex will try to change the right side menu "comments on paragraph 1" to "comments on question 1". Change weekly/monthly to week/month b/c of space issue
      "If Other Please Specify" to be changed
       
      LP will send out survey to Mary Marlene Jim, ask to send it out to staff
       
      Need to do some trouble shooting with comments, but wait until Alex puts in edits
       
      Should we send out a link to survey monkey to the staff giving feedback? No, because can't fill it out.
       
      Will be doing a faculty pilot, should suffice for getting opinions about layout (if it's confusing, for instance) want staff to focus on wording.


      SURVEY TIMELINE

      Staff comments
      Alex edits digilib, Linda writes email out to staff (give staff a week to comment - try to get it out on the 21st, then give them to 27th). Kate organizes staff comment, assessment committee edits, make adjustments due to comments. then out to library heads.
       
      Give time estimate as to how long it takes to complete the survey. Have some staff not involved in committee take it (we are all too familiar with it).
       
      Meet 2:30pm Wed, May 5th
       
      Linda asks whether we should introduce the  over staff meeting - won't have but will discuss results with them.
       

      Faculty Pilot
      Currently recruit for faculty pilot. Need about 20-25. 2-3 from science, 2-3 education, 2-3 med (maybe more?), etc.
      Have a letter (UW's letter that we can tweak)- Sarah will look for this letter and tweak it

      Pilot might go out at a difficult time b/c of grading.
       
      Questions over whether it should go out from a central email or if individual committee members should send. More likely to get response if ind members, BUT harder to remember to send out reminders.
       
      Do survey after graduation
       
      For faculty - how to leave comments? Comment boxes in survey monkey

      From UW-Steve's advice about the pilot:
      Should be timing the pre-test and it is Important to have open ended questions on the pretest. (will get staff to do)
      Put something about importance of our knowing the timing in the faculty letter - also, in survey, ask meta-questions about how long it took them to complete the survey, if there was anything that was confusing, suggestions about better wording.


      ______________________________________________________________________________________________________________________________

      Minutes 3/31/10

      Present: Linda, David, Dan P., Jim, Alex, Sarah S., Kate, Dan b. (minutes)

      Next meeting scheduled for 4/13/2010 at 2:30pm at Pardee(?)

      Announcement:
      BLC annual meeting Assessment COI 4/1/10 1:30--2:30 having balanced scorecard presentation.

      Decisions:

      Limit the number of "potential new services" to 5 on the survey itself, but ask staff for comments on all. 
      Include a note in the email that goes out to teams asking for comments that asks them to pay particular attention to question 15 (this is the potential new services question).
      Theo, Med, and Law will also be asked to send Digilib faculty survey to their staff for comments.

      Action Items:
      Kate will work on proposed services question
      Dan P will work on formatting digilib comments, and also enable anonymous comments.
      LP will speak to other AULs about assessment/planning documents
      Staff for comment, IRB application, then pilot; Or- pilot then IRB application? LP will ask Prof Frasier
      SS looks for holes in overall survey

      Discussion:
      A. Look at strategic plan for assessment wording
          David led us through his document at http://docs.google.com/Doc?docid=0ARIN2pT8txcGZGNuanJncnRfMjE3dHdxZ3oyZ2s&hl=en that discusses the implications of the strategic plan for our faculty survey.  At the end there are five conclusions, quoted here:
         1. Although we ask about satisfaction with several services, we don't necessarily ask about every service (e.g. circulation or storage retrieval).
         2. We don't ask about mobile or other means of access (beyond a question about print vs. electronic books).
         3. We ask nothing about whether they consider the library a "top destination."
         4. We don't ask about "awareness and responsiveness" of library staff.
         5. We don't ask about the faculty's awareness of instruction services, or their likelihood of using them.
      In addition, we talked about the need to be strategic in what we assess, and how we might enable other staff to take on assessment roles for themselves (eg - Helen/Circulation).  We then talked about the possibility of crafting advice to the teams on setting priorities, setting reasonable goals, understanding just what needs to be done and why to do it, getting baseline data, and identifying where in the faculty survey relevant data may be.

      B. Discuss ideas for "potential new services" question
          We had quite a few ideas for potential new services to ask about on the faculty survey:
      1. Alumni database access:  IST does not have a way to authenticate alumni, and Linda suggested this may be more expensive than we can afford.  This will not appear on the faculty survey at this time.
      2. Mobile devices (eg iPhone)
      3. Intercampus "paging" (ie delivery) of books.  Sarah suggested we might ask Jack about turning on this button in III, but (apparently) there is currently not a way around the problem of someone in Mugar requesting a book be delivered from Mugar.
      4. Intercampus "paging" (ie delivery) of photocopies. 
      5. help for fulfilling article deposit mandates (eg NIH)
      6. help with library links in courseware systems (eg Blackboard)
      7. course-specific (research) guides
      8. article desktop delivery (electronic) to faculty for ILL documents (already happening at THEO)
      9. more eBooks in your field of study
      We then talked about how we should probably limit the number of "potential new services" to 5 on the survey itself, but that we might ask staff for comments on all of these.  Also, we decided to include a note in the email that goes out to teams asking for comments that asks them to pay particular attention to question 15 (this is the potential new services question).

      C. New wording for collection satisfaction question

          This discussion, led by Alex, resulted in editing question 21 on the fly, to phrase it in the manner we all accepted: "How satisfied are you with the BU Libraries?"

      D. Discuss getting survey up on digilib site (same style as strategic planning) to elicit staff comments
          The faculty survey is being copied question by question into a Digilib blog at http://digilib.bu.edu/blogs/buassessment/ - this is where staff will be able to make comments on each question.  Dan P will work on formatting digilib comments, and also enable anonymous comments.  Theo, Med, and Law will also be asked to send Digilib faculty survey to their staff for comments once the version is formatted correctly.  LP will bring this up with the AULs.  The process going forward will be to get the faculty survey to staff for comment, then the IRB application will be submitted, and then we will run the pilot: Or should it be, rather, this way: pilot, then IRB application?  LP will ask Prof Frasier about this.
      UW recruits faculty through their liaison system.  LP suggested that if each of us convinced three faculty members to join the pilot, then we would have a good pilot group.  But it is not quite time to do this yet.

      Next meeting's agenda
      Look at any holes in what we are asking/survey in its entirety - discussion led by Sarah S
      Finalize Digilib?
      ________________________________________________________________________________

      Minutes 3/22/10

      Present: Linda, David, Dan P., Jim, Alex, Sarah S. (minutes), Kate

      Next meeting scheduled for Wed 3/31 at 2pm at Pardee

      Announcement:
      LP spoke with SED faculty who is on the CRC IRB. He will review our survey before we formally submit it. CRC approval should clear us for both campuses.

      Decisions
      1) Edited Section 2, Website questions, and removed it as a separate section. List of edits below in "Discussion" section.
      2) Looked at and accepted changes to Information Literacy questions

      Action Items
      1) Alex will make sure all the (high)s and (low)s are consistent in survey
      2) David will look at strategic plan for assessment wording (begun at http://spreadsheets.google.com/ccc?key=0AhIN2pT8txcGdDB0U0o1Ri0yUW0zR3dXZGp1UTZFOEE&hl=en)
      3) Sarah S will look for holes in what we are asking
      4) All members will brainstorm ideas for "potential new services" question
      5) Alex will work on new wording for "overall satisfaction" question (UW#16) -collection component does not differentiate e from print

      Discussion

      Section 2 Website
       Question 1- "library resources" wording made it a question about resources, not about access, and we decided the info we really want to know is how often they access the library. Changed it to: How frequently do you use the following methods to access the Boston University Libraries? The scale (frequent to infrequent 1-5 likert with no actual amounts listed and an N/A option) did not make sense. We noted that Steve had mentioned if we use time periods (day week month) will not be able to find a mean/median, but were not convinced that would actually be useful/necessary. Scale is: now more often    weekly    monthly    once per semester    less often, with no N/A (will interpret less often as including nevers or N/As). Ranges were chosen with idea in mind that we may want to compare across user groups once we do grad/undergrad surveys, too.

       Question 2- unanimously agreed to delete this question. (which library do you usually begin with w/drop-down list). We know what we really want to know behind this- who is using the branch pages, are they starting there, etc, but we could not come up with a clear way to ask this. The wording implied that they start at a lib page when we acknowledged that probably they start with Google or a bookmarked link. Will have to use another method to get this data.

       Questions 3 and 4- We discussed the reason we would want to ask #3, since we already know the answer. Considered moving 3 & 4 up to importance/satisfaction services questions and including them there, but decided #3 doesn't make sense to ask and that #4 is primarily historical- we don't have this problem nearly as much post ezproxy. Also were concerned about how to phrase it to stress that it was about access, not collections/content. Decided to drop them both.

       Question 5 - deleted. Decided it was too open ended, and would turn into a laundry list of e-resources faculty want. Perhaps this is a focus group question?

       At this point there was only one question in the "web site" section, it it asks about visits to both the virtual and physical library. So we moved it up under the demographic questions, currently it is Q #6
       
       
      In the course of discussing these questions, we segued into the topic of collections, and looked at where and how we asked collections questions. We noted that the UW question #16, which we have adopted, asks a collections satisfaction question but does not differentiate between e-resources and print holdings. We decided we wanted to make that distinction . Alex will work on the wording.

      Information Literacy Questions
      We looked over these questions, and also looked at the Strategic Plan http://digilib.bu.edu/blogs/buLib/ , to discuss how these questions met and were informed by goals 4 and 5. Linda pointed out the way that UW reports their information literacy survey question results always as a block (never just one question, always as a set) and suggested that we also do that. Sarah mentioned SAILS and suggested that we could use that as a baseline to compare the numbers we get from the survey to (eg, do the numbers from SAILS match with what skills faculty believe students do/don't have). We accepted these questions as they stand, and made no edits except to the spacing of the high and low in the response scale, which Alex will look for and fix throughout the survey.

      Next meeting's agenda
      Look at strategic plan for assessment wording- discussion led by David
      Look at any holes in what we are asking/survey in its entirety - discussion led by Sarah S
      Discuss ideas for "potential new services" question (all)
      New wording for collection satisfaction question- discussion led by Alex
      Discuss getting survey up on digilib site (same style as strategic planning) to elicit staff comments (all)



      ______________________________________________________________________________________________________________________________

      Minutes 3/15/10

      Present: Linda, David, Dan B., Dan P., Jim, Alex

      Next meeting scheduled for Monday 3/22 at 2pm. Location TBD

      Announcement: BLC Assessment to discuss Balanced Score Card at BLC's Annual Celebration
      Location: BPL 
      Date: 4/1 
      Time: 1:30pm-2:30pm 

      Decisions:
      1) Committee will continue to consider sending paper version of survey. UW found a higher response rate for paper surveys but work load may be prohibitive
      2) An overall satisfaction question like UW's question 16 from their 2010 survey will be added. Wording and categories TBD.
      3) David will make proposed edits to the Service Questions section in 2.0 version of survey. List of edits below in "Discussion" section.
      4) Dan B. will make proposed changes to Open Access questions in version 2.0 of survey. List of edits below in "Discussion" section.
      5) Jim made proposed edits to Demographics questions to version 2.0 of survey in real time. List of edits below in "Discussion" section.
      6) Alex will add "Which school are you primarily affiliated with" back to the 2.0 version of survey. It will be Q# 2


      Action Items:
      1) Danny P. will take over the website questions section of survey. Analysis based on criteria described in 1/21/10 minutes.
      2) Linda to discuss posting the survey to digilib with Jack Ammerman. Survey will appear in the same format as Jacks version of the Strategic Plan for staff comments.
      3) David will look over the strategic plan to align its assessment goals with the work of the assessment committee.
      4) Committee must look into how to interpret non-respondents for each question. 
      5) Alex will investigate color coding in survey monkey.
      7) Alex to send email to Tracy Thrasher-Hybl about usability testing on "Find @ BU"

      Discussion:

      David's Service Questions: 
          
      David presented his analysis of the service questions section (now Q#16 and Q#17 in the 2.0 version of the survey). Click here for David's analysis.

      We discussed the use of service questions as a public relations tool and a guide for resource allocation. The issue of non-respondents came up. It makes sense to lump non-respondents in with "don't use" to create an average for the "importance" section of the service questions. In other sections non-response/don't use can be interpreted as "don't know" or "hate." A final decision on how to treat non-respondents has to be made for each section.  

      We discussed color coding question in Survey Monkey as a way of demarcating related questions. Color coding may not be an option.

      Should we ask specific questions about the types of services the library can/should provide. UW does this on their 2010 survey in Question 14. Committee should look to Strategic Plan for purposed library services currently in the works as a starting point for formulating this type of question.

      The list of resources for Q#16 and Q#17 is too long. David will shorten the list to six resources. "Requesting books or journal items not owned by the library" will be changed to "Borrowing books or getting journals not owned by the library." Eg ILL" will be added in parenthesis will also be added. David will delete "Assisting your students in finding material for their assignments" from the list. It is covered in other areas.


      Dan B.'s Open Access Questions:

      Dan B's analysis of the Open Access questions can be found here.
      Institutional Repository group has asked to keep "timeliness" in the response section of Q#10 in the 2.0 version of the survey (How important are the following factors in your decision on where to publish journal articles?). This question to remain as is.
      Open access Q2 and Q3 will be merged. A decision must be made on the placement of this question in the survey. 


      Jim's Demographics Questions:

      Q5 moved to Q6, disrupted flow of demographics questions.
      Q1 "What is your primary department" may be tricky for people with multiple associations. Committee reached a decision to accept multiple associations.    
      "What is your primary school" to be added in the Q2 spot.
      An "other" section with no comment space to be added to "With which campus are you primarily affiliated?"
      The full time vs. part time status is an important question to know since the survey will go to both typed of faculty.
      Q4 "Which BU library is your primary library" was deleted. This question is too problematic eg. multiple library associations, distance education, library top page as location.

      * Unassigned actions and pending decisions appear in bold italics   
       



      Minutes 3/3/10

      Next meeting: TBD: Linda has sent out a Doodle request for scheduling purposes and will notify the group of the preferred dates.
      Present:  Linda, Alex, Jim, Dan B, David, Megan
      Action items:
      • Linda will like the group to review the assessment portion of the GRS strategic plan located here: http://sites.google.com/site/graduateresearchservices/goals-objectives-3rd-draft.  Please also look at the plans for DIOA and U&DS located here: http://sites.google.com/site/strategicplanningmugarbranches/ . (The acronyms provided under the objectives designate which group is responsible for implementation.)  Please comment using red font and initial even if it's just to say "okay" or even if something is just a little "funny".
      • Continue to work on your sections of the survey using the criteria described in the 1/21/10 minutes below.
      • Megan will revise her section based on group discussion and post it to the wiki before she leaves next week.

      The meeting began with recognition of Megan's departure and her valuable work with the committee. She will be missed. Megan suggested adding Kate Bronstad from Medical as her replacement and Linda added her to the assessment distribution list and the committee.

       The rest of the meeting consisted of a review of Megan's work on the Library Resources section of the survey and group discussion of changes to the wording of our survey. Megan graciously agreed to post the revision of her work on the wiki.

      The group questioned the need for an SFX question in this section or even in the survey itself. Further discussion of this subject is planned once the entire committee meets again.

      In question #1, Megan and Jim raised the possibility of a comments section to answer the "why" of the questions. If faculty rank personal collections as high but the library as low, don't we want to know why that is? The group decided that these questions might be better asked in other settings.

      The group categorized the 3 questions in this way:
      • Question #1 is about sources, or "where" one gets information.
      • Question #2 is about frequency using these sources.
      • Question #3 is about resources or formats--"what" do you want to get.

      _________________________________________________________________________________________________



      Minutes 1/21/10

      Next Meeting:  Wednesday, Feb. 10th at 2pm, Pardee (if available) or PERL (backup)

      Present: Linda, Alex, Sarah, Dan B., Dan P. Tim, Jim, Megan, David

      Action Items:

      David will learn something about statistics.

      Sarah will look for holes in survey (important information we aren't collecting)

      For each section of the survey, the following people will go over the questions, ask why we are asking each of them, think about what results we want (including possible charts/graphs), and go over the points raised about the section by Steve:

      Tim: Website questions (though they may ultimately end up in other sections).

      Megan: Resources

      Dan B: Open access

      Jim: Demographics

      David: Services

      Danny and Linda: Fight over information literacy

      Decisions:

      Faculty survey will be pushed to November.

      Announcements

      Linda reported on assessment forum at ALA.  Big name researcher Carol Tenpir from U of Tennessee gave talk -- she has a big grant to look at value of academic libraries to universities, based on work at U of Illionois looking at flow of grant money, but taking it much further.  Linda presented our work, was well recieved.  Power point is up on assessment wiki (Alex helped with design).  She spent all day at LibQual session, as reported in e-mail.

      Talked about upcoming Assessment Conference in Baltimore (Oct. 24-27).  Linda is going and doing poster session on our survey work, Sarah is thinking of going.  Linda says the conference is very, very, good -- live and breath assessment for three days.  Anyone wants to go, talk to Linda.

      Long Discussion of What Next for Survey

      We discussed what we learned from Steve Hiller's visit.  Sarah emphasized the importance of knowing what we are going to do with the data before creating questions.  Linda described having lunch with Steve after meeting, and his opinion that we are not really ready to do the survey this Spring.  We all agreed to move the survey to November

      Some discussion of whether we should include part-time faculty (Steve's survey doesn't, there are reasons pro and con).  No decision was reached.

      Sarah asked if we are getting ahead of ourselves in working on the detailed wording of the survey, should we step back and concentrate more on what we want out of it?  Megan suggested we investigate the recent Sustainability survey to see if we should use the same tool.  Sarah raised the concern that a BU homegrown survey tool may not be maintained.  No decision was reached.

      Discussion of statistics tools, and recent workshop on SPSS a couple of people attended.  JMP (http://www.jmp.com/) has been suggested as alternative to SPSS (BU will be providing it).  Sarah emphasized importance of either having one of us come up to speed on statistics, or getting outside help; Linda argued for growing out own expertise ("how hard can it be?").  David stressed importance of some actual knowledge of statistics, e.g. not just knowing the difference between a mean and a median, but knowing which one to use in a particular situation.  David volunteered to learn something about statistics.

      Linda suggests everyone take a section and mock up the kind of report/chart we would want to create using data, while Sarah wanted to figure out something closer to the raw data we want.  

      Sarah brought up section of Steve's article "Assessing User Needs, Satisfaction, and Library Performance at the University of Washington Libraries" (Library Trends 49.4 Spring 2001 p605) that talked about their survey goals:

      • determine who users and potential users are;
      • how and why the library is used (or isn't used);
      • what sources are used for library-related information;
      • what faculty and students' library-related needs are; and
      • how satisfied faculty and students are with the libraries.

      Should we have a similar list of goals?  We decided we should, and Sarah and Jim would independently come up with them (but later on we appeared to change their charges).

      David mentioned that we had already done some looking at what kinds of questions we wanted answered before creating our current questions.  Sarah wondered if we should find out from faculty what they want to be asked; Dan. B. suggested that we will be asking what is missing during pre-testing, and Linda reminded us we could include open-ended questions in pre-testing.

      Dan. P. didn't think we should tear apart the current survey and start over.  Sarah thought we didn't need to throw it completely our, but must make sure we are collecting data we can use and manipulate.

      Linda suggested that we break down the survey by sections, and for each one someone should come up with what information we want, and what it might look like (charts, graphs, etc.).  Jim suggested we don't get hung up looking at Steve's survey while we do this.

      We arrived at the following breakdown of assignments:

      Sara will look for holes (questions we aren't asking)

      Tim will do website questions (though they may ultimately end up in other sections).

      Megan will do library resources section

      Dan B will do open access section

      Jim will do demographics section

      David will do services section

      Danny and Linda will fight for information literacy

      For all, we will look at why we are asking each question, the possible graphical representation of the results, and the issues raised about the section in the minutes of meeting with Steve

      Megan started taking us through her minutes of meeting with Steve.

      Some discussion of IRB, will do Charles River.  We want to be "exempt"

      More discussion of faculty feedback.  Megan suggested if we have a few questions we can't word correctly, we might have focus group with faculty. 

      Yet more discussion of Primary Library question.  We want to use it with other questions (so we can break down answers per library).  Should we have multiple check-offs?  What about people who never use physical library?


      Minutes 1/15/10

      Next Meeting: Thursday Jan. 21 at 3pm, School of Education Library

      Present: Linda (chair), Megan (minutes), Dan B., Dan P., Jim, David, Sarah and Steve Hiller

      Action Items: Update survey questions based on Steve's feedback (See question #9 below).

      1)  Can you speak about the UW's intent in regard to using responses for Public Relations versus using them to improve internal work flows?   

      PR has a big role in how we communicate the value that the library is providing the community.  UW's survey allowed them to look at program improvement but also to get a sense of the importance for our various user communities.  For example, UW put up a report last spring during budget cuts that contained a lot of the data from the survey.  This is currently available on their website.  Data that relates to resources and services is very useful for communications. However, Steve stressed that the survey itself is not a marketing tool.  Sarah asked how much do he finds it ends up being this way anyway.  Steve answered that faculty may become more aware of a service, but we can't push something overtly in the survey.  

      Sarah asked which of the questions on the survey are asking questions that you want to know about the library , and which of the questions are asking questions for information about the survey (for example, demographics).  Steve answered that it is less the latter because of the IRB. Also, respondents many not be as forthcoming.  For example, gender has been dropped.    

       

      2)  In what general or specific ways has the UW faculty survey changed since the first time it was given?  

      UW tried to move the survey to include more impact questions or demographics that allow UW to do specific types of demographics questions- ex: have you taught this year, did you get grant funding. They also added the budget reduction question  and the importance questions.  

      David F- how much rewording have you done and how has this affected your longitudinal analysis? UW survey question #4- 2007 asked both using a campus computer/ using an off-campus computer- this has been eliminated due to changes in wireless service,use of mobile devices, etc.  

      Sarah- do you write the questions for longitudinal use?  Yes, if there are things that they want to keep track of.  

      Sarah- concern of whether faculty understand the difference between subscription resources and the open web.  Steve answers that it depends on the local environment, helps that UW does not have federated searching.  Faculty have seemed to be able to differentiate between relatively well, but it is messy.  

       

      3)  How would you recommend we do the pilot for the survey:  

           a) have faculty take the actual survey, and we interpret the results

      or, b) have faculty give direct feedback about the questions we ask, and those we're maybe leaving out?

      Sarah is concerned that we have not used a model where we talk to the group that we are surveying about what we are asking, partly because it has been modeled so heavily on the UW survey.  Is this an issue?  Steve- Best if we can do this, but may be able to use the pretest to gather some of this information by asking additional questions about what was confusing for participants or if there is anything that they expected to be there that wasn't? Steve has agreed to send pretest questions that UW asked.  They meet with the faculty senate council in the library and talk to them about what they see as important- generally this has not been extremely helpful. Also, met with student groups and tend to get the best feedback from this group.

      Dan asked if faculty who take the pretest also take the regular test.  Steve said that they do and that we need to be explicit about asking faculty to take it twice.  

      UW recruits faculty through the liaisons.  20-25 faculty for the pre-test. Steve also agreed to send us the message that UW liaisons used for recruitment.  

      We asked Steve whether we need both an email pretest and a focus group?  Is this too much?  He suggested that if we do both, just target specific questions.  We should get as much feedback as possible on the survey because it can give the perspective of the respondent in a qualitative way and allow us to look for patterns in the responses.  

      Steve stresses that for the analysis piece, we should use something other than survey monkey, such as SPSS, SAS.  We will need more flexibility than survey monkey can provide.  As a result, we need to set up the survey in a way that it can feed into SPSS and think about what we want to use the data.   For example, one might look at frequency of use and might pull out the group who says they use something weekly.  Then you can evaluate the rest of that group's responses in this context and track trends over time.

      Should be timing the pre-test and it is Important to have open ended questions on the pretest.

      4)  What shortcomings are there with web-only surveys?  Are there any significant advantages with print?

      Logistically, it is the easiest for distribution, financially, and data capture.  They went to web-based in 2004 but decided to include a print survey in one of the reminder notices. UW distributed the survey with the same URL, but with reminder notices, they mapped where responses were coming from by department and specific groups can be targeted by liaisons.  When response rate was particularly low, would target specific group because it is important to get a representative sample.    

      Steve recommends that we wait a week to 10 days to send reminder.  We should expect to get half as many responses for each reminder.  

      Sarah brings up the issue of survey fatigue.  Because UW partners with the Office of educational assessment, they are able to identify samples of students and controls who will be over-surveyed. Helps to mention upfront that we only do this once every three years and faculty will be less irritated. 

      Linda asks if spring is the best time to do this? With students is a definite wait, faculty it is better to do so, but if not possible wait to do until late in the fall semester, say Nov. 

       

      5)  How did UW encourage faculty response?  How does UW separate the faculty identities from the responses, and how easy is that to move past the Institutional Review Board?

      UW emails come from the Library Dean.  Liaisons are really important in the reminder emails.  Also, offered incentives for the bookstore but with the faculty this didn't really seem to make a difference.  UW will probably drop the incentive for faculty in the near future.

      The survey was used for organizational improvement only and was anonymous. The incentive does not effect this because it goes to a different agency.  Participants finish the survey, their anonymous data is submitted and then they have the opportunity to sign up for a drawing.  However, different with the print.  Still in 2007, 28% of responses came back in print. Showed that there are still people who prefer to do a print survey.  It is extremely important to make the print survey mirror the online version as much as possible so we can compare data.  

      Sarah asked Steve his opinion on LibQual.  He said that he felt that the results are not as useful, especially outside of the comments field.  

       

      6)  We see that you collected faculty rank.  Is that merely to ensure you do not get skewed results toward any singular group?  Are there other questions on the survey where the concern is simply to make sure results haven't been skewed toward any singular group?

      We asked Steve what he considered important in terms of demographics because we ask primary academic department, but don't identify rank.  Steve does not think that rank is very important.  Originally, UW used rank in place of an age questions to evaluate the adoption of the new technologies.  Currently, UW does not survey part-time faculty who are not tenure track because the response rate is much lower and it pulls down the response rate.  This group also tended to be a little more negative and less thorough because they did not have as much experience with library services.  Thus, we have to decide how we want to define faculty? UW's definition has included tenure track, research faculty, full time lectures/instructors, post-docs.  Sarah suggests that it would be important to get data about these ratios from the schools.  Steve said that post-doc response seems to be equivalent to assistant professors.  UW does not survey affiliate faculty, i.e. people who do not get paid by them, but researchers who get paid by grants are included.  

      UW also did a survey of non-faculty professional staff


      7)  Will your faculty have to log in to fill out the survey?  Does making them do so add a legitimacy to the survey that makes it worthwhile to require?

      They don't log in.  They only submit identifiable information about themselves if they want to sign up for the incentives once the participants have completed the survey. 


      8)  Asking about Frequency / Importance / Satisfaction, or some combination of those three.  We have had many discussions about the need to ask each of these for various services and resources.  We have only one single question about frequency of use (our website), so are we missing something here?

      Steve said that UW asks a lot fewer frequency questions than they did before.  Qualitative data has show that frequency is not a predictor of value.  The only frequency question left is #4.  Also, if you specify time periods (weekly, monthly, etc.), you can't do a statistical mean.  With frequency questions, we have to think in advance of how we are going to use this and what sort of statistical analysis we will be doing.  UW ditched the library website for the frequency question- a lot of confusion about what is the library website.

      Jim asked about the new survey questions #10 and #11.  Specifically, these questions don't talk about student performance in the graduate category, only undergraduate.  We plan to do both in our survey.  Steve says it is muddy with graducate students if it is not tied to a specific tract or specific class.  Graduate students are spread around.  The undergraduate question is used almost as a gap analysis.  Also, UW took out finding info on the web and in the library because they can be the same thing.  Currently, they are discussing adding a question to the student survey that asks "What makes you anxious about using the library", but they are working on the wording.

      The difference between Frequency, importance, satisfaction questions also comes into play when you think about how you treat missing responses.  On satisfaction, it doesn't matter.  In terms of importance, UW found that there are times when people just don't answer for that question- 23% don't answer this question.  We have to decide how we would rate this- does this mean it is low importance?  Need to be consistent with how we deal with this.  For example,  with a frequency question is a "no" response an "I don't use this"?  

      Sarah- how do you use the I don't knows"? Steve- can be a good indicator of good visability.  Satisfaction- 41.4% no response for the last question in 12 about liaisons.    

      Too confusing include all of these types of questions, would never ask all three.

       

      9)  Would you please preview our survey with us and give some feedback as we go?

      Comments about our survey:

      Demographics

      #1- may need an "other" option.  People get upset if they don't have a way to identify this.

      #2- Drop academic department

      #3- UW asks them just to do the departments, and then have a mapping feature that auto maps the department to school, so we are not asking them twice.  

      UW does to require answers to any question.  Tends to be some people that don't answer the demographics.

      #4- Get rid of this questions- can just do a separate question

      #5- Which is your primary one?  How will we interpret this information- faculty would need to choose more than one.  Are we looking at which library they visit, or which library's staff, resources they use?  Worth explaining this in the survey. Perhaps list all of the libraries and select which ones they use on a regular basis.  UW only does this for students but doesn't ask this for faculty anymore because they don't come to the library anymore.  

      #6- Do we want to split the funding question? federal vs. external funding- may not be that meaningful for us. Change to "Federal or other external funding".


      Web Site Questions:

      #1- only frequency questions- what are we measuring against here?  Especially if they can get to a range of library resource without going through the library. Thinking about how we will use this information.  Jim suggests that instead we use UW question #4.  

      Steve likes questions #3 and #4-- UW added a satisfaction question on the remote access to questions and services.  

      #5- open ended questions might be better to ask in pre-test and then construct a list question.  Work on revising this one.  

       

      Open Access: 

      #2 have you submitted to an open access journal?  Sticks out on a standardized survey, we need to include examples.  

      #3 Have you ever deposited an item to: -- ok, but we may want to look at whether we ask the DCommon question


      Find @ BU:

      With one-offs, Steve recommends that we fit this into another category- for example, put this in with satisfaction with library resources.  

      Library Resources:

      Maybe we could lump the SFX question into this section?  Saves space and gives the participant context.  

      #1 was dropped at UW for undergraduates and they are considering dropping it for faculty because they are not sure how separate those categories are now.  

      #2 Wikipedia is out on the new one.

      #3 UW stopped separating print and electronic- not about the format, about the source.  The old vs. new journals question comes from decision whether or not to purchase back files. 

      Information Literacy:

      #1 Need to provide a NA option.  Also think about whether to include the "in the library" "on the web" distinction. UW changed to "scholarly information".  

      Service Questions:

      Need to add NA on satisfaction questions.

      We may want to drop library instruction option because it is asked about in other areas of the survey.

      Requesting books or journals articles- UW got rid of this for real estate.  

      Overall satisfaction questions?  Do we want to add this?  An overall importance question to different areas is useful in the context of looking at resources and services.  

      Steve stressed the importance of thinking about how we can make it shorter and he thinks that we are trying to do too much.  Some of the satisfaction questions could be answered in an overall satisfaction questions.  



      Minutes 1/13/10

      Next meetings: Friday, Jan. 15th at 10am Mugar conf. rm w/ Steve Hiller; Thursday, Jan. 21st at 3pm Mugar conf rm (probably)
      Present: Linda (chair), Dan B. (Minutes), Dan P., Tim, David, Megan, Sarah

      Action Items: EDIT THE QUESTIONS FOR STEVE (all)
      Decisions:  We changed some questions in surveymonkey on the fly....

      Announcements:  None
      Discussion about the faculty survey:
      There was some concern that we may be moving too fast to try to release the survey this Spring.  It was decided to forge ahead and revisit this issue after the meeting with Steve, when we might have a better handle on just how much work our current survey iteration might need.  We changed some questions in surveymonkey on the fly, such as incorporating the GoogleScholar wording in the FIND@BU question.

      Questions for Steve:
      1)  Can you speak about the UW's intent in regard to using responses for Public Relations versus using them to improve internal work flows? 
      2)  In what general or specific ways has the UW faculty survey changed since the first time it was given? 
      3)  How would you recommend we do the pilot for the survey: 
           a) have faculty take the actual survey, and we interpret the results
      or, b) have faculty give direct feedback about the questions we ask, and those we're maybe leaving out?
      4)  What shortcomings are there with web-only surveys?  Are there any significant advantages with print?
      5)  How did UW encourage faculty response?  How does UW separate the faculty identities from the responses, and how easy is that to move past the Institutional Review Board?
      6)  We see that you collected faculty rank.  Is that merely to ensure you do not get skewed results toward any singular group?  Are there other questions on the survey where the concern is simply to make sure results haven't been skewed toward any singular group?
      7)  Will your faculty have to log in to fill out the survey?  Does making them do so add a legitimacy to the survey that makes it worthwhile to require?
      8)  Asking about Frequency / Importance / Satisfaction, or some combination of those three.  We have had many discussions about the need to ask each of these for various services and resources.  We have only one single question about frequency of use (our website), so are we missing something here?
      9)  Would you please preview our survey with us and give some feedback as we go?

      -Dan B.

      ___________________________________________________________________________________

      Minutes 1/6/10

      Next meeting:  Wed. January 13th at 2pm (location: Mugar Admin Conference Room unless otherwise notified).

      Present: Jim Skypeck (minutes), Dan Benedetti, Dan Piekarski, Linda Plunket (chair), Megan Bresnahan, David Fristrom, Tim Lewontin, and Alex Solodkaya.

      Announcements:

      1. Steve Hiller from UW will meet with us on Jan. 15, 2010 from 10-12 in the Mugar Conference Room.
      Tasks
      1. Alex will try to get a BU logo onto the survey to customize the theme of the survey.
      2. Members should edit their own sections based on discussion at this meeting and forward those edits/make those edits known to Alex. Alex will send an update to the list noting edits that were done, i.e., Survey Monkey w/ Megan's edits. Alex will "stabilize" the survey after the members' edits have been made.
      3. If possible, edit your questions and get them to Alex as soon as you can so we can forward the survey to Steve Hiller in advance of our meeting with him next week.
      4. Prepare questions for Steve so our discussion next Friday can provide us with the information we think we need.


      ____________________________________________________________________________________________________________

      Minutes 12/21/09

      Next Meetings: Wed Jan @ 2pm, Wed Jan 13 @ 2pm (location Pardee unless we hear otherwise)

      Present: Dan Piekarski, Sarah Struble (minutes), Tim Lewontin, Dan Benedetti, Jim Skypeck, Linda Plunket (chair), David Fristrom, Megan Bresnahan

      Announcements:

      • SED okayed our use of their copy of survey monkey professional
      • Alex Solodkaya, from PERL, will translate our questions onto Survey Monkey (LP will invite Alex to 1/13/10 mtg)
      • We can continue to edit them there *until* we send the link out and it starts gathering data (this has repercussions for testing the questions with staff)
      • In January LP will begin the IRB process. She may have to do it for both campuses. MB has worked with the IRB on the med campus before.
      • next step after/alongside IRB is to solicit and work with a focus group of library staff to review the questions, wording, order
      • after that LP will show to an SED prof with survey creation experience
      • LP will email Steve to see if he has any availability to meet with us around  ALA

      IRB and survey distribution questions

       Are we asking survey-takers to log in? If we do, that will make IRB process much harder. Discussion seemed to conclude that anonymous, no login was fine. Will send the link in an email to faculty only. Should note in email that it is a FACULTY survey, to please not forward it, and note approximate time it will take to complete (since recent techqual survey was so long).

       Will send email reminder(s) to entire group. LP wil ask Bob Hudson if he can nudge deans to get them to encourage faculty participation. 

      Reviewed question development

      each of us will make updates to our section before Christmas, so Alex can start moving them to Survey Monkey

       Services (DF):

      • David does not think the frequency question is redundant to the importance, but allowed that it was far less important for us to ask, and conceded to remove it. DB noted that UW does include frequency in their survey.
      • Wording changes -DF will make changes that were suggested at last meeting
      • ILL questions stay in services, not covered well in any other section
      • in info literacy questions, DF will make grad/undergrad questions more uniform. Some of what we were looking to gather about students at different points in ther academic careers will hopefully come in on the student surveys when we do those.
      • make sure there is a comments section for the info literacy question
       Demographics (LP)
      • discussed making school/college and departmental affiliation into 2 questions. May be able to make it clear with survey monkey as a 2 part question
       Website (TL)
      •  Many of the questions Tim had were also on Megan and Sarah's Resources section. We looked at the ones unique to the libraries websites Use/Navigation, rather than services on the web, and focused on those.
      • Keep directions/hours info question, with an ease of use scale
      • importance/satisfaction scales for off-campus access question
      • keep "home library website" question, add library catalog, work on wording of question. MB suggested tiny screenshots, DP suggested they could be onrollovers. May or may not be possible in Survey Monkey
      • add comments box

       Resources (MS and SS)
      •  Did not go over this section of questions as a group, should probably do that at next meeting?
      •  will add question modified from TLs list, about when/how faculty direct students to library services/tools
         


      Minutes 12/3/09

      Next Meeting: Dec. 21st at Pardee

      Present: Danny Piekarski, Sarah Struble, Tim Lewontin, Dan Benedetti, Jim Skypeck, Linda Plunket

      The group worked on editing recent updates made to the faculty survey, as highllighted in red on the survey itself by Sarah Struble.

      Under Services, it was decided to drop frequency questions, as it would make evaliuation of survey results easier.

      Under Importance of Services:
        There were some issues with wording that would need to be tested in a pilot.
        Do questions relating to the Web Site belong here? Probably not.
        There ain't no "Virtual Catalog," but will we be looking at surveying on WorldCat Local when the survey is ready to go out?
        The word "held" was compared disfavorable with the word "owned".
        Do ILL related questions belong here or under a section for ILL?
        Under "subject collector in your area", do we also need demographics on the survey takers departmental affiliation?

      On the U Washington Survey, we had a question for Steve (lastname?) as to whether or not  #6 (frequency) and #13(satisfaction) seem to ask the same thing  viz use of online/web services and sites

      Under "Other Questions":

         Should questions relating to print vs digital  all a "no preference" option?
         On use of peer reviewed Open Access, there should be a "Don't Know" option?
         Question 3  on "consideration" of Publication in OA journal should really ask, have you ever "submiitted for publication to" an OA journal?
         Question 4 on discussing OA with peers, seemed unclear and should be dropped.
         Question 6 question relating to SFX linking, we asked to add a comment box so we can diagnose what the source of  potential disatisfaction might be..

      Under "Demographic Questions":

         a pull-down should be added asking for primary affiliation with  college/school and department
         Probably should drop the comments box ffrom this section..

      Under "Information Literacy"

        We wondered how listed categories correlated with  ACRL standards.
        Do we actually need dual responses to these categories for both "importance" and "performance"?

      Under Rating of "Graduate/Professional Student Perfomance":

        Should we be framing these questions as an evaluation over time? Does performance improve/decline during these student's BU career?
        Should we be asking different questions about undergraduates as opposed to graduate students? Probably not.

      Other:

        We also wondered whether or not we should ask for a "primary library affiliation?"
        Should we have a generic non-subject specific  comments box at the end of the survey?


      Sarah and Meghan will prepare the survey section on Library Resources, and Tim will prepare the section on  Library Websites for the next meeting.  


      Minutes 11/17/09

      Next Meeting: Dec. 3rd at Mugar

      Present: Linda (chair), Dan B., Dan P., Tim, David, Megan (minutes), Jim, and Sarah


      Logistics:

      The group decided to primarily concentrate our discussion on the faculty survey for this meeting.  The group also agreed to meet every other week in order to meet spring goals.

      David suggested that Google Wave may be a useful tool for virtual meetings- David will send invites and links to tutorials for GoogleWave to everyone in the group.  

      We decided to meet every other Thursday.  Next two meeting dates will be Dec. 3rd and 17th at 2 pm.

       

      Faculty Survey:

      We began by looking at each of the sections/categories that we defined in the previous meeting.  Each group member posted their write-up on the wiki (http://sites.google.com/site/buassessment/faculty-survey---goals-process-categories-of-questions)

       

      Demographics:

      Linda asked the group to identify any holes in her list. Megan suggested that we might add something about the faculty members’ status as either an adjunct, assistant, associate, or full professor, or more importantly, whether the faculty member is clinical or research faculty.  Sarah agreed that asking the faculty member to define whether they do research, classroom teaching, or clinical teaching would be useful at the CRC campus.  However, the group decided that it would be unnecessary to ask the participant to define their faculty status.  As mentioned last week, we may have a difficult time distributing this survey to adjunct faculty because it is not clear whether there is a comprehensive listserv for CRC adjuncts.  Linda said that she thinks there are 2 lists- one for all faculty and another for CRC including adjuncts.  Linda will check into whether and how we can contact adjuncts.

      We discussed whether or not (and if so how) we should ask faculty about their primary BU library affiliation. We discussed how we would ask this question.  Tim suggested that we ask what library they use most often.  Linda stressed that we would need to have an “NA” option because this question presumes that they use a library.  Then we can state something like “the following questions refer to your primary library.”   David suggested that we provide a list of librarians and ask the faculty member “which of these locations have you used in the last year?” Sarah and Tim both stressed that we will need to make the distinction between virtual and physical visits to the library. Sarah also mentioned that she thinks that library space is not very important to faculty.  The group decided that this question should be moved to the facilities section.

      Linda and Jim agree that asking whether a person is a full or part time is valuable.  We also decided that we do not need to know how many years a faculty member has been employed at BU.

      There is unanimous agreement among group members that our survey should not be as long as TechQual!

       

      Library Resources.

      We began by reviewing the list of items that we would define as “Library Resources”.  Everyone agreed that we can get rid of reserve and reference in the books section.  We can just ask if they use print or electronic books. Jim suggested that instead of old or new journals, maybe define by storage or stacks, but the group decided that we should keep it simple and just ask about print or electronic journals.   Sara mentioned that we need to give specific examples of what we would consider a database or index. Same with “Archives”, we will need to define this and give specific examples so it is clear what we are asking faculty.  We decided to get rid of newspapers because, other than access through Lexis/Nexis we don’t really subscribe to individual newspapers at either campus.  Audio and Visual media can be lumped into a single source (this would mostly be for the music collections).

      Megan asked if we move library research/subject guides and tutorials from Library Resources to Services.  The group agreed that this would be appropriate as long as it is mentioned somewhere.  Megan also mentioned that we will not be listing these questions by category on the survey.

      “Software” is included as library resource.  The group discussed whether faculty use software and decided that we can probably get rid of this resource.  Megan noted that we will want to add software back in for our survey of students.  We also decided to get rid of dissertations, reports and scores from the library resources list.

      Next we discussed the information we want to learn about the list of library resources.

      Sarah mentioned that it might be difficult to figure out how faculty find library resources? How do they track stuff down?  How are they accessing these resources?  Maybe we can use the UW questions for this section . . . hard to phrase without putting words in their mouths. 

      Most people agreed that it is important to ask open ended questions and/or include a comments field.  Even if we cannot analyze all of this qualitative data, we still have it available to us so we can pull golden responses from faculty to support our quantitative findings.

       

      Library Services:

      David- are there services that they want from us that we are not providing? 

      Sarah- what services do we have do they not know that we have?

      David said he broke his section into three categories:

      ·      How often do they use services
      ·      How important is the service to them
      ·      How satisfied they are with services

      Sarah suggested that if we ask about course management software, we need to give examples and specifically mention Blackboard or CoureInfo.  Sarah said that we need to be careful about asking “how important is this service?”  She thinks they will say it is important even though they don’t use it.  David added that it is difficult to ask why they are not using something?

      Linda and David agree that we will need to provide several opportunities for faculty to make comments in this section.  

       

      Library Websites:

      Tim said that we need to ask if they are using the website and if not how are they accessing our resources.  Megan and Linda mentioned that the focus of these questions should not only relate to the Mugar website but should also account for the branch libraries such as Medical, Law and Theology.  We will need to list different library websites, including catalog, so that the faculty comments can be associated with a particular library site.

      Dave mentioned that he really likes the way that the UW survey does not ask about the website specifically, but instead asks about information seeking behavior.

      Linda suggestions that our plan should be to take our discussion from the meeting, write our questions, and really look at the UW survey.   If we decide not to use any of their questions we must have a strong reason.

       

      Library Facilities:

      Dan P.  suggests cutting the hours questions, but how frequently do you use the library should be left.   The group also decided to cut questions about how knowledgeable the staff are because we won’t know if the faculty are referring to a student worker or a professional staff member.  Comments related to staff services will likely be captured through the questions on library services. Dan P. (may have been another group member?) suggested that we ask something like “How do you rate the quality of the facilities? Please elaborate.”   Group agreed that it would be useful to ask faculty to elaborate on many of their answers.

      Sarah suggested that as some point in the future we might want to run a door survey to capture additional information in this category about how people who use the physical space in the library are using that space. In this survey, we might ask “what did you do today, how long did you stay, did you interact with library staff, did you find what you are looking for” etc. 

      We agreed to eliminate the frequency and hours questions.


      Other:

      Dan suggested that we might try to capture faculty’s preferences for print or digital (which was somewhat covered under the library resources category.  We noted that preference would be more valuable to know than what faculty member’s actually use- they would use the format that the library provides.

      The group agreed that Open Access should be addressed in the survey.  At the least, this may help raise awareness of the issue (though the survey should not be instructive) and provide a metric for comparison when the next faculty survey in conducted in 3 years when OA awareness at the University will likely be much greater.  Megan mentioned that she thinks we should make a distinction between faculty who are using open access resources and those who are seeking out OA resources for publication. 

      Dan also noted that we will need to find a way to address SFX services and the group agreed that we should refer to the service as “Find at BU”.  Tim also reiterated that we should ask about faculty members’ experiences with remote access. 


      For the Next Meeting on Nov. 3rd: Action Items

      Since David noted that he liked the questions about information literacy on UW survey, he agreed to come up some questions related to information literacy. 

      Each group member is now responsible for writing up questions related to their categories and lining them up with the UW survey questions when possible.  

      Sarah will make a Google doc for listing our questions and share with everyone. 

      Tim will post his questions to the wiki.

      Megan will post minutes. 

      Dan P. will add several group members as Gmail users to the wiki.



      Quick Note (11/16/09):

      I happened to notice that the Google Sites storage limit is 100MB per site.  I know this is something that had concerned us in the past.  We are currently using 6% (6MB for you liberal arts majors).

      -DP



      Minutes 10/26/09
      Next meeting: TBD
      Present: Linda (chair), Dan B. (Minutes), Dan P., Tim, David, Megan Bresnahan, Jim Skypeck

      Action Items:
       - Linda will add Megan, Mary, and Jim to our assessment listserve, and also query Marlene to see if she would like to be added.  Jim will also forward select emails to Vika.
       - Committee members will decide what the data might look like and create the questions that would result for each of the following:
      Member:                                        Category:
      Tim                                                Library Websites
      David                                             Library Service
      Sarah  & Megan                             Library Resources
      Dan P.                                            Library Facilities
      Dan B.                                            Other
      We agreed that these reports should be posted to the wiki.


      Decisions:
       - We decided that for demographic purposes (the first bullet under Categories), we would like the first bulleted item underneath that begins "Primary School, College, Institute..." to be reflected on the survey as multiple-select boxes, so that faculty can select as many as are appropriate. 
       - We also decided that we do not need to collect the fifth bullet item, that begins "Status - full, associate, assistant..." but that we do wish to learn how many years someone has been here.
       - Ask, under the demographics portion, if the faculty member strictly does only research, teaching, or a combination of both.


      Announcements:
      Tim L. has looked more closely at the ezproxy logs MySQL database that Scott from IT had compiled one time.  There is a long version, with around 250 million entries, which is probably too large for us to handle in any way, and a short version, which has about 250 thousand entries, which Tim investigated.  The short version gives us info on how many times a certain domain was accessed, user affiliations, dates, etc.  It would be possible that the database could be seeded with info from the EZproxy server on a set schedule, but pulling info out (vendor names, for example) will probably have to be tailored each time because of URL changes to databases and the like.  Tim estimated that perhaps about 1,200 separate domains exist in the log.


      Faculty Survey:
      Linda P provided an outline of what the next steps for the faculty survey should be.  It is posted on the wiki at: http://sites.google.com/site/buassessment/faculty-survey---goals-process-categories-of-questions.
      Megan worried that respondents are backing into our resources through Google and the survey might not get to that.  Sarah seconded that concern, confirming that it has been her experience that users definitely back into some, and don't connect with it as a library-provided resource.  Dan B. suggested that if we wanted to see how much this is happening, we need to be careful in the way the question is worded on the survey, since knowing users start at Google is not quite the same as knowing that users actually do back into library resources that way.
      There is a LibQual session happening at MidWinter, coming up on January the 18th.  Linda, Dan P., and David expressed interest in attending.  Linda will add Megan, Mary, and Jim to our assessment listserve, and also query Marlene to see if she would like to be added.  Jim will also forward select emails to Vika.
      In general for the survey, we are looking to use Survey Monkey only (no paper) and are hoping that it will be a short survey.  Permission will have to be obtained through the IRB and the Provost.  An email would be sent out to faculty (adjuncts included?) and a link provided.
      Much of the discussion focused on how we will proceed to create the questions and then run the survey.  Jim pointed out that faculty probably shouldn't have input on drafting the questions that they will then be asked to complete (under #2 on the outline).  Dan suggested perhaps some faculty expert in running surveys might be consulted (particularly, Linda had previously mentioned a Professor Stefanakis at SED).  David added that perhaps faculty liaisons might serve a role as a possible pilot group, as they are naturally interested in library issues.  Sarah noted that department secretaries might be better able to provide lists of adjunct faculty than deans, if cannot the email lists we can get from the Provost does not adequately cover them.
      To proceed to creating questions, Linda has broken out the types of information we may wish to gather into categories on the outline.  Tim seconded this approach.  Dan B.  suggested we may just want to go through the Univ. of Washington's survey, and possibly other surveys, to cherry-pick questions we think apply to us, and then review those to see if anything is missing.
      We decided that for demographic purposes (the first bullet under Categories), we would like the first bulleted item underneath that begins "Primary School, College, Institute..." to be reflected on the survey as multiple-select boxes, so that faculty can select as many as are appropriate.  We also decided that we do not need to collect the fifth bullet item, that begins "Status - full, associate, assistant..." but that we do wish to learn how many years someone has been here.
      A couple of clarifications for survey purposes were suggested.  Megan stated that it is important to be sure to include ebooks as a clear question on the survey, as the Med. School now has quite a collection.  And David contributed that under "library services" (bullet three in the categories), we need to be clear when we ask faculty about reference, whether we are asking them to give feedback on their own reference service experience, or whether we are asking them to give feedback on how they feel their students have been helped by reference services.  Distinctions like this in other services and other categories are likewise to be duly noted.  This lead to the decision to ask, under the demographics portion, if the faculty member strictly does only research, teaching, or a combination of both.

      Sarah suggested and we decided to draft the questions after deciding what data we would like to receive.  Committee members will decide what the data might look like and create the questions that would result for each of the following:

      Member:                                        Category:

      Tim                                                Library Websites
      David                                             Library Service
      Sarah  & Megan                             Library Resources
      Dan P.                                            Library Facilities
      Dan B.                                            Other

      We agreed that these reports should be posted to the wiki.

      ______________________________________________________________________________________________________________________

      Minutes 08/06/09

       

      Next meeting: September 21st 2:00pm.


      Present: Linda (chair), Dan P. (Minutes), Tim, David

       

      The ARL Mapping report that David has worked on can be found under Survery Manipulations.


      ARL Mapping:  Next Steps:

      - A column should be added that would include an example (such as a number that has been collected in the past)

      - Next to the example column should be a column of "steps".  This would be the process that whoever is collecting the data has to go through to get the number.  All queries that are used in Millennium should be written down.  Also important is the fields that the queries use.

      -This is important because, as Linda said, "Bob will not be here forever"  <--- Threat?

       

      EZProxy:

      -Tim informed the group that IT has scripts for ezproxy data and keeps the data in a MySQL database that needs to be parsed.

      -David noted that we should look more closely at the EZProxy data when/if we get counter data.


      Library Learning Wiki:

      -Tim has placed a report up on the library learning wiki.


      Worldcat Local Comparative Stats:

      -The group decided to look at the data collected after we have worldcat up and running.  This has been decided as a sixth initiative for the assessment group to tackle.

      -The group will look at ILL & Circulation Numbers.  Did our numbers go up or down with worldcat and do we think there is causation?


      Six People, Six Projects:

      -The six initiatives of the assessment group have been split among the group members.

      -At the next meeting each group member will give a small presentation about their area and what we can do to tackle the initiative (see our charge for more details)

      Member:                                        Project:

      Tim & David                            Counter & EZProxy

      Sarah                                     Worldcat Local

      Dan P.                                    Google Analytics

      Dan B.                                    Data Website / farm

      Linda                                      Faculty Survey

      Tim                                        Data Collection Survey



      Minutes 7/14/09

       

      Next meeting: August 6th 2:30pm.


      Present: Linda (chair), Dan P., Dan B., David (minutes)

       

      David presented his work on mapping from data collection survey to ARL report (up on wiki).  A few corrections were made.  We will think about what to do next and discuss it at next meeting.

       

      Next we discussed Linda's proposed Assessment Plan; overall the reaction was positive.

      David wondered if there was too much to do?  Dan suggested extending time window.  

      Danny wondered how we would keep data farm going?  Do it in steps?  Start small?

      Linda suggested not a full-blown data farm, just web site for reporting data.  Simplify.

      Linda also suggested changing it to five year plan.

      On doing survey, we were still waiting to hear if we can use Steve's survey [we later heard that we could].  Should we use Survey Monkey?  General opinion was favorable.

      On COUNTER: Need Tim input.

      Analytics: Danny's done fair amount.



      Minutes 6/30/09 - Pardee Conference Room, 2:30
       
      Next Meeting  7/14/09, Pardee Conference Room, 2:30
       
      Present: Dan Benedetti, David Fristrom,  Dan Piekarski, Linda Plunket (chair, brief minutes)

      Danny and Dan reported glorious weather on their vacations!

      David gave an update on the mapping. It's somewhat tricky mapping from ARL statistics back to various branch and department statistics, but it's coming along. David should have a first draft by 7/14. Linda mentioned a tool that might (or might not) come in handy for the mapping. It's called Vue, is open source, and was developed at Tufts.

      Most of the meeting was fleshing out an Assessment Plan. Linda will write a first draft and post on the wiki before 7/14. The Assessment Group will be asked to review and comment first, and then ALC and Refnews. We agreed to propose five projects over the course of the next two years:

      Improve Infrastructure

      • Data Collection Survey
      • Data Farm
      Better Understand Users
      • User Survey, perhaps surveying 3 primary user groups (faculty, graduate students, undergraduate students), one group each year. This survey would be adapted from the UWA Libraries Triennial Surveys, assuming Steve Hiller OKs the use of the surveys. Linda will contact Steve after the draft plan is finalized by the Assessment Group. The Group looked over the questions on both the undergraduate and faculty surveys and discussed the feasibility of conducting a web survey and the importance of ensuring an adequate response rate. We agreed on the usefulness of the data for improving services and promoting the library.
      Assess the Use of Resources & Services
      • Project Counter
      • Google Analytics
      At our next meeting we'll finalize a Plan and decide how we want to organize the work, who might champion each project. The primary topic will be David's mapping of the ARL statistics to the branch and departmental data.



      Minutes 6/16/09 - Pardee Conference Room
       
      Next Meeting  6/30/09
       
      Present: Dan Benedetti, David Fristrom, Tim Lewontin (minutae), Dan Piekarski, Linda Plunket (chair)
      --Linda tabled an agenda motion to re-select chair until a meeting when all members were present.
       
      --Dan B reviewed his revision of the survey. Several future edits were considered, but tabled for the time being.:
        1)Re-organize columns so that like categories for each library or department matched like category as one reads across the columns.
        2)Rationalize items by scope, so that all libraries or departments to which a statistic collection process applied are included for that category.
        3)Combine ARL statistical report with Annual Report stats.
        4)Review Dan's list of "Questionables" in order to ask for clarification for each list item from survey participants.
       
      It was decided that it would be better for now, to restrict ourselves to mapping survey results to the ARL Report.
       
      Linda distributed a chart on Goals and Outcomes for the group.
       
      Action Items:
       
      --Tim will post ARL Statistics Report in the Wiki.
      --David will map survey results to ARL Statistics Report.
      --Group will work on filling out the Goals and Outcomes chart.

      Minutes - 5/27/09 - Pardee Conference Room

      Next Meeting - 6/16/09 - 2:30pm - Pardee Conference Room

       Present: Dan Benedetti , David Fristrom, Sarah Struble, Dan Piekarski (minutes) , Linda Plunket (chair)

      -Linda added items to the environmental scan section of the wiki: 
      a.)  Sarah Laughlin's presentation (2 parts).  Linda noted  that Sarah had broken down the 6 points that customers want for assessment.
      b.)  NERCOM data
      -David added the Webinar page with slides and Q&A under the COUNTER section of the wiki.

      -Dan P. briefly showed the mySQL work he had done with the collected data from the survey.  It was decided that work on this should wait until later in the process.

      -We then debated what to do with the data we have collected from the survey.
      a.)  Linda brought up that we could sit with the five branch heads and standardize the data collected.  Once we learn how the data is actually being collected we hope that we can centralize some functions that are now being done redundantly.
      b.)  Sarah pointed out that we should allow the data to direct us rather than trying to anticipate where it is going.  We all nodded sagely.
      c.) Linda hopes that we can use the data that we have collected to create a standardized report form.  This form will be shown to the branches in the hope that we will get feedback and buy-in.
      d.)  It was decided that we will separate Mugar as if it were a branch when we sort the collected data.  We will also add the categories "Mugar & Branches (Totals)" and "All BU".

      -On the white board we planned out how we will sort the collected data:
      The column on the left hand side listed: SEL, PERL, Pardee, Mugar, Mugar & Branches (Totals), ALL BU
      The Row across the top listed:  Circ., Physical Usage (gate counts), Reference, Instruction, Collection, ILL, Reserve, Cataloging, Serials, Eresources

      -Dan B. volunteered to go through the seven non-branch items and break down the data collected into the table we created on the whiteboard.

      - We ended by working on an Assessment Group plan, namely what questions to ask.  Some of the items that came up:
      efficiencies/time, what users don't want, what users want, data to support decisions
      Ask about Phyiscal space in the library (user commons)
      Who are our users?  Broken down in table form:
             
                             |    Virtual                 |    Physical
      -----------------|------------------------ |------------------------
      Use data      | Google Analytics |  Annual Reports
      Users            | Usability                |   

      Brainstorm Our Path:
        -entrance survey (collected as students leave the library)
        - general survey (for PR purposes?)
          -Include those not currently using the library
          - e.g. lib qual
          - e.g. SMG's survey
          - knowledge of what lib. offers
        -Online Usage data
          -outbound links
          -COUNTER/SUSHI

        -Work Processes
           -processing time
          -use over time
           -continuous improvement




      Assessment Committee Minutes 5-11-2009


       Present: Dan Benedetti (minutes), David Fristrom, Tim Lewontin, Sarah Struble, Dan Piekarski , Linda Plunket (chair)

       Next Meeting
      : Wednesday, May 27th, 2:30pm, Pardee Conference Room

       Action Items:
       * David will add his presentation to the Assessment Wiki.

       * Sarah may follow up with David Snyder about certain III stats.

       * Linda will post the revised charge to the wiki.  NOTE: If anyone would like to edit this document, be sure to turn on the "track changes" function (tools - track changes) within your version of Word.


       Decisions:
       * Linda's revised charge was accepted.
       * The group generally agreed to continue analyzing survey results as a whole while simultaneously trying to create an assessment plan as a whole, rather than break into subgroups to accomplish this, at least for the time being.
       * Everyone agreed that our plan should focus on what we can reasonably hope to accomplish; on practical things such as identifying how much time it takes a book to go from ordered to placement in the stacks.


       Agenda
       1. Minutes, announcements, reports on assessment programs/webinars
          A> David reported on the Counter webinar.  He used a free presentation software available at prezi.com.  Counter is not a standard but rather a code of practice, explicitly defining items, such as requests of full text articles per journal, which are then presented in XML of Excel format by counter-compliant vendors.  Currently, compliance for things like journals or databases is fairly common, not so much compliance from ebook vendors yet.  This info is desirable as something complementary to journal impact factors for collection development use.  Part of the webinar called "Using Counter Reports" was a presentation of how one librarian manipulates spreadsheets to present data efficiently.  David speculated that a relational database would be even better, but would take a lot of work up front.  Another part of the webinar focused on SUSHI, a way to automate counter report downloads from different vendors.  Serials Solutions offers a product (360) that does just that.  ERM could be our depositing place for such reports but there are tough technical issues to resolve.  David will add his presentation to the Assessment Wiki.
          B> Linda reported on a NERCOMP SIG workshop entitled "Assessment into Action" that she attended.  There were two presenters, including an excellent one with a focus on quantitative methods of research.  The gist will likely be repeated in an upcoming BLC Assessment COI meeting.  She will post the url to this presentation as soon as it's posted on the NERCOMP site.

       2. Data Collection Survey
       Bob's group collects no unique data, and security will likewise not provide a finished survey.  David Snyder may have more a thorough response soon, and Sarah may get in touch with him about some particular items after this Wednesday.
       We looked at the Cataloging response to the survey, and talked in general as to how we will use the results.  Sarah suggested the survey results should inform policy-making, but noted that it would be difficult to really change historical problems such as Law's separate bibliographic records in our catalog due originally to their RLIN membership.  Tim backed that up by saying the other libraries at BU often have good reason to maintain separate bib. records.  Linda hoped to be able to compare branches, or other like units, in order to streamline stat gathering - eliminating duplication of effort.  David thought we might also be able to see holes in what is current gathered - opportunities to collect other statistics that might be more meaningful to staff.  Dan B. suggested that at least the survey could serve as a means to try to centralize what is currently gathered to an online form (data farm).  Dan P. noted that during the process of filling out the data collection survey, Pardee staff were able to identify and eliminate the collection of a few types of data that is no longer needed.

       3. Planning
       Linda made several changes to the charge, which we all accepted gratefully.  Some minor revisions were suggested, and Linda will post it to the Wiki.  NOTE: If anyone would like to edit this document, be sure to turn on the "track changes" function (tools - track changes) within your version of Word.
       We spoke about the ARL Spec Kit at http://www.arl.org/bm~doc/spec303book.pdf.zip in order to view sample charges and plans.  Dan B. liked the idea of red-flagging items (it is USC that does this, page 89) such as "data not stored centrally", and "self-reporting where automation is possible".  The group generally agreed to continue analyzing survey results as a whole while simultaneously trying to create an assessment plan as a whole, rather than break into subgroups to accomplish this, at least for the time being.  The reason to take this approach was well-put by David when he noted that there are assessment actions we can take which won't be evident from the survey results, like user surveys, counter reports, and the recommendations from the consultants.  Everyone agreed that our plan should focus on what we can reasonably hope to accomplish, such as identifying how much time it takes a book to go from ordered to placement in the stacks.  At the same time, some policy decisions clearly must be made from a top-down perspective - such as a clear policy on the addition of electronic journals into the catalog.  For those types of problems, the assessment process must be carried out, so that focus groups and the like can inform policy-making, as was stated at the beginning of the meeting by Sarah.



      Assessment Committee Minutes 4/7/2009

      Present: Dan Benedetti, Tim Lewontin, Dan Piekarski , Linda Plunket (chair), David Fristrom (minutes)

      Next Meeting: Monday, May 11th, 2:30pm, Pardee Conference Room(?)

      Action Items:

      * David will come up with a new wording for survey question 5.

      * David will reformat survey worksheet to wrap text and align it at the top.

      * David will edit survey e-mail to include reference to using Alt-Enter for carriage returns, and to make explicit that staff should be involved in answering survey.

      * Dav

      Comments